My gripe with an approach like this is the lack of any grounding to these generated topics. Hallucination accumulates like error in this case so every generation that is conditioned by a previous one (the recursive "hierarchical topic exploration" in TFA).
The question is: Is it like jpeg compression where the errors do not accumulate but the image comverges to a self inverse compressed image or does the data set converge to a single point which is meaningless?
The transformation function in jpeg (DCT) is generally well defined math. While lossy, most of the information is reprocudable.
An LLM is layers and layers of non-linear transformations. It's hard to say exactly how information is accumulated. You can inspect activations from tokens but it's really not clear how to define what the function is exactly doing. Therefore error is poorly understood.
JPEG is similar actually. The DCT is invertible, but the result of the DCT is quantized, which is where some of the compression happens (DCT -> quantization -> IDCT), so the end to end process is not truly invertible. Maybe an analogy to the non-linearities in between the linear steps in deep learning
I think it would be interesting to deflate out to a huge dataset and see where this happens.
Certainly it will occur as the generated data exceeds the original, eg after 1-10T tokens.
I think you could also do this faster by moving down the tree in a depth first manner.
Typically I use this for knowledge transfer, style transfer, catastrophic forgetting mitigation, etc and so I don’t go very far. I usually manually review the data samples before using it.
It can be a description by a shorter bit length. Think Shannon Entropy and the measure of information content. The information is still in the weights but it is reorganized and the reconstructed sentences (or lists of tokens) will not provide the same exact bits but the information is still there.
“Decompression” is a metaphor, not a fact claim to be proved; it is a description of an approach to generating a dataset from an LLM where most of the potential utility is still fairly explicitly speculative, a jumping off point for further work.
> Wouldn’t this method be good if applied on humans in job interviews?
Uhm, no? I mean, some firms do abuse job interviews to pump candidates for usable information, and some have gotten a notable bad reputation for that which impacts their funnel of candidates, but from the article: “Generating comprehensive datasets requires thousands of model calls per topic”—you aren’t going to get a candidate to hang around for that...
There are some fun early theoretical ML papers on this topic.
They prove that it is possible to fully clone a brain based on this method.
I think one could theoretically estimate how many queries you would need to make to do it. The worst case is proportional to the number of parameters of the model, i.e. at least 10^15 for a human. At one minute per spoken sample, that comes out to about 2 billion years to clone one human.
I suspect it is not practical without advancements in neural link to increase the bandwidth by billions of times.
I personally like playing around with empirical methods like this blog post to understand the practical efficiency of our learning algorithms like back prop on transformers.
I also try not to invest too much effort into this topic given the ethical issues.
Think about all the times in llm gets it wrong, the fact that would have helped to get it right is something that was lost. I suppose this isn't proof it's lossy just maybe we don't know how to get the data out.
Or look at it another way LLMs or just text prediction machines, whatever information doesn't help them predict the next token or conflicts with the likelihood of the next token is something that gets dropped.
Or look at it another way these things are often trained on the many terabytes of the internet yet even a 200 billion parameter network is 100 or 200 GB in size. So something is missing, and that is a way better compression ratio then the best known algorithms for lossless compression.
Or we can look at it another way, these things were never built to be lossless compression systems. We can know by looking at how these things are implemented that they don't retain everything they're trained on, they extract a bunch of statistics.
I think extraction from the model itself is a bad idea. But extraction from external sources, such as the deep research reports LLMs generate, or solving problems where we have validation of correctness is a good idea. The model is not validating its outputs by simply doing another inference, but consults external sources or gets feedback from code execution. Humans in chat rooms could also provide lots of learning signal, especially when actions are judged against the outcomes they cause down the line, using hindsight.
So in short what works is a model + a way to know its good outputs from bad ones.
Not sure if you mean in general, but I'll answer both branches of the question.
In general:
Depending on the method of compression, you can have lossy or non-lossy compression. Using 7zip on a bunch of text files can lossless-ly compress that data. Briefly, you calculate the statistics of the data you want to compress (the dictionary), and then make the commonly re-occuring chunks describable with fewer bits (encoding). The compressed file basically contains the dictionary and the encoding.
For LLMs:
There are ways to use an LLM (or any statistical model of text) to compress text data. But the techniques use similar settings as the above, with a dictionary and an encoding, with the LLM taking the function of a dictionary. When "extracting" data from the dictionary alone, you're basically sampling from the dictionary distribution.
Quantitatively, the "loss" in "lossy" being described is literally the number of bits used for the encoding.
it's exactly the same as JPEG images being lossy, while you can see image as a whole (and it is enough for 99% of people), you are obviously missing some details
and the more you rely on those details (professional photography, scientific data) the more obvious it is (to the point of image being useless in some cases)
same with LLMs, we are currently testing how far we can go before we seeing obvious issues
Lossy compression vs lossless compression is the difference of whether you can get a 1:1 copy of the original data if you compress and then decompress it.
A simple example of this is if you have 4 bits of data and have a compression algorithm that turns it into 2 bits of data. If your dataset only contains 0000, 0011, 1100, and 1111; then this can technically be considered lossless compression because we can always reconstruct the exact original data (e.g. 0011 compresses to 01 and can decompress back to 0011, 1100 compresses to 10 and can decompress back to 1100, etc). However, if our dataset later included 1101 and got compressed to 10, this is now “lossy” because it would decompress to 1100, that last bit was “lost”.
An LLM is lossy compression because it lacks the capacity to 1:1 replicate all its input data 100% of the time. It can get quite close in some cases, sure, but it is not perfect every time. So it is considered “lossy”.
How good can you recreate an image that is described by words? Obviously not bit by bit and pixel by pixel. You get something that resembles the original but not an exact copy.
Yes. For example, you could always say "give me a jpeg image file that is encoded as the bytes 255, 216, 255, 224, 0, 16, 74, ...". But that's just pointing out that the input to your "LLM" function includes the prompt. It's f(model, prompt) = response.
It's not straightforward to prove that models have to be lossy. Sure, the training data is much larger than the model, but there is a huge amount of redundancy in the training data. You have to compare a hypothetically optimal compression of the training data to the size of the model to prove that it must be lossy. And yet, it's intuitively obvious that even the best lossless compression (measured in Kolmogorov complexity) of the training data is going to be vastly larger than the biggest models we have today.
You can always construct toy examples where this isn't the case. For example, you could just store all of the training data in your model, and train another part of the model to read it out. But that's not an LLM anymore. Similarly, you could make an LLM out of synthetic redundant data and it could achieve perfect recall. (Unless you're clever with how you generate it, though, any off the shelf compression algorithm is likely to produce something much much smaller.)
My gripe with an approach like this is the lack of any grounding to these generated topics. Hallucination accumulates like error in this case so every generation that is conditioned by a previous one (the recursive "hierarchical topic exploration" in TFA).
I suspect most of the "leafs" are unusable.
The question is: Is it like jpeg compression where the errors do not accumulate but the image comverges to a self inverse compressed image or does the data set converge to a single point which is meaningless?
The transformation function in jpeg (DCT) is generally well defined math. While lossy, most of the information is reprocudable.
An LLM is layers and layers of non-linear transformations. It's hard to say exactly how information is accumulated. You can inspect activations from tokens but it's really not clear how to define what the function is exactly doing. Therefore error is poorly understood.
JPEG is similar actually. The DCT is invertible, but the result of the DCT is quantized, which is where some of the compression happens (DCT -> quantization -> IDCT), so the end to end process is not truly invertible. Maybe an analogy to the non-linearities in between the linear steps in deep learning
I think it would be interesting to deflate out to a huge dataset and see where this happens.
Certainly it will occur as the generated data exceeds the original, eg after 1-10T tokens.
I think you could also do this faster by moving down the tree in a depth first manner.
Typically I use this for knowledge transfer, style transfer, catastrophic forgetting mitigation, etc and so I don’t go very far. I usually manually review the data samples before using it.
Huh. I wonder what good output would look like at extremes. Hallucinations that just happen to be true or something more interesting?
Not different for inference... Just saying.
I wonder how many cycles of train->extract->train->extract->... you can do before most of your output will be hallucinations.
It would be an expensive experiment to perform.
I wonder how you could do it more efficiently?
Learning == Compression of information.
It can be a description by a shorter bit length. Think Shannon Entropy and the measure of information content. The information is still in the weights but it is reorganized and the reconstructed sentences (or lists of tokens) will not provide the same exact bits but the information is still there.
The compression is lossy.
The claims in this paper don't make sense. There is no proof that anything has been decompressed
“Decompression” is a metaphor, not a fact claim to be proved; it is a description of an approach to generating a dataset from an LLM where most of the potential utility is still fairly explicitly speculative, a jumping off point for further work.
Nope http://arxiv.org/abs/2509.11208
About that http://arxiv.org/abs/2509.11208
Remember those programming books that were like "1000+1 tips for C++", making those with llms would be trivial now.
My understanding was that the Alpaca data was a distillation from text-davinci-003
Wouldn’t this method be good if applied on humans in job interviews?
> Wouldn’t this method be good if applied on humans in job interviews?
Uhm, no? I mean, some firms do abuse job interviews to pump candidates for usable information, and some have gotten a notable bad reputation for that which impacts their funnel of candidates, but from the article: “Generating comprehensive datasets requires thousands of model calls per topic”—you aren’t going to get a candidate to hang around for that...
That is evil, no, I was thinking more about selective knowledge exploration to see if a candidate is fit for the position.
how long would it take to do a complete memory dump of your brain by voice stream? days? months? years?
this is more like writing one's autobiography.
There are some fun early theoretical ML papers on this topic.
They prove that it is possible to fully clone a brain based on this method.
I think one could theoretically estimate how many queries you would need to make to do it. The worst case is proportional to the number of parameters of the model, i.e. at least 10^15 for a human. At one minute per spoken sample, that comes out to about 2 billion years to clone one human.
I suspect it is not practical without advancements in neural link to increase the bandwidth by billions of times.
I personally like playing around with empirical methods like this blog post to understand the practical efficiency of our learning algorithms like back prop on transformers.
I also try not to invest too much effort into this topic given the ethical issues.
I was thinking about selective knowledge exploration to see if the candidate is fit for the offered position. No need to dump everything
What open source code do you use to pull synthetic data from LLMs?
> This compression is lossy
Is compression really lossy? What is an example of lost knowledge?
Think about all the times in llm gets it wrong, the fact that would have helped to get it right is something that was lost. I suppose this isn't proof it's lossy just maybe we don't know how to get the data out.
Or look at it another way LLMs or just text prediction machines, whatever information doesn't help them predict the next token or conflicts with the likelihood of the next token is something that gets dropped.
Or look at it another way these things are often trained on the many terabytes of the internet yet even a 200 billion parameter network is 100 or 200 GB in size. So something is missing, and that is a way better compression ratio then the best known algorithms for lossless compression.
Or we can look at it another way, these things were never built to be lossless compression systems. We can know by looking at how these things are implemented that they don't retain everything they're trained on, they extract a bunch of statistics.
I think extraction from the model itself is a bad idea. But extraction from external sources, such as the deep research reports LLMs generate, or solving problems where we have validation of correctness is a good idea. The model is not validating its outputs by simply doing another inference, but consults external sources or gets feedback from code execution. Humans in chat rooms could also provide lots of learning signal, especially when actions are judged against the outcomes they cause down the line, using hindsight.
So in short what works is a model + a way to know its good outputs from bad ones.
Not sure if you mean in general, but I'll answer both branches of the question.
In general: Depending on the method of compression, you can have lossy or non-lossy compression. Using 7zip on a bunch of text files can lossless-ly compress that data. Briefly, you calculate the statistics of the data you want to compress (the dictionary), and then make the commonly re-occuring chunks describable with fewer bits (encoding). The compressed file basically contains the dictionary and the encoding.
For LLMs: There are ways to use an LLM (or any statistical model of text) to compress text data. But the techniques use similar settings as the above, with a dictionary and an encoding, with the LLM taking the function of a dictionary. When "extracting" data from the dictionary alone, you're basically sampling from the dictionary distribution.
Quantitatively, the "loss" in "lossy" being described is literally the number of bits used for the encoding.
I wrote a brief description here of techniques from an undergrad CS course that can be used: https://blog.wtf.sg/posts/2023-06-05-yes-its-just-doing-comp...
LLMs have finite entropy (it is related to their training loss) and training typically doesn’t store the residuals.
Some compression methods use LLMs internally and also store the residuals, making them lossless.
It is at least as lossy as jpeg compression. Details get lost and artifacts are generated.
https://en.wikipedia.org/wiki/Lossy_compression
The law of thermodynamics would require it to be lossy
it's exactly the same as JPEG images being lossy, while you can see image as a whole (and it is enough for 99% of people), you are obviously missing some details
and the more you rely on those details (professional photography, scientific data) the more obvious it is (to the point of image being useless in some cases)
same with LLMs, we are currently testing how far we can go before we seeing obvious issues
whats an example of loss?
Lossy compression vs lossless compression is the difference of whether you can get a 1:1 copy of the original data if you compress and then decompress it.
A simple example of this is if you have 4 bits of data and have a compression algorithm that turns it into 2 bits of data. If your dataset only contains 0000, 0011, 1100, and 1111; then this can technically be considered lossless compression because we can always reconstruct the exact original data (e.g. 0011 compresses to 01 and can decompress back to 0011, 1100 compresses to 10 and can decompress back to 1100, etc). However, if our dataset later included 1101 and got compressed to 10, this is now “lossy” because it would decompress to 1100, that last bit was “lost”.
An LLM is lossy compression because it lacks the capacity to 1:1 replicate all its input data 100% of the time. It can get quite close in some cases, sure, but it is not perfect every time. So it is considered “lossy”.
How good can you recreate an image that is described by words? Obviously not bit by bit and pixel by pixel. You get something that resembles the original but not an exact copy.
you can create original exactly with right prompt
Yes. For example, you could always say "give me a jpeg image file that is encoded as the bytes 255, 216, 255, 224, 0, 16, 74, ...". But that's just pointing out that the input to your "LLM" function includes the prompt. It's f(model, prompt) = response.
It's not straightforward to prove that models have to be lossy. Sure, the training data is much larger than the model, but there is a huge amount of redundancy in the training data. You have to compare a hypothetically optimal compression of the training data to the size of the model to prove that it must be lossy. And yet, it's intuitively obvious that even the best lossless compression (measured in Kolmogorov complexity) of the training data is going to be vastly larger than the biggest models we have today.
You can always construct toy examples where this isn't the case. For example, you could just store all of the training data in your model, and train another part of the model to read it out. But that's not an LLM anymore. Similarly, you could make an LLM out of synthetic redundant data and it could achieve perfect recall. (Unless you're clever with how you generate it, though, any off the shelf compression algorithm is likely to produce something much much smaller.)