Language Modeling Is Compression
Open accessAbstract
It has long been established that predictive models can be transformed into lossless compressors and vice versa. Incidentally, in recent years, the machine learning community has focused on training increasingly large and powerful self-supervised (language) models. Since these large language models exhibit impressive predictive capabilities, they are well-positioned to be strong compressors. In this work, we advocate for viewing the prediction problem through the lens of compression and evaluate the compression capabilities of large (foundation) models. We show that large language models are powerful general-purpose predictors and that the compression viewpoint provides novel insights into scaling laws, tokenization, and in-context learning. For example, Chinchilla 70B, while trained primarily on text, compresses ImageNet patches to 43.4% and LibriSpeech samples to 16.4% of their raw size, beating domain-specific compressors like PNG (58.5%) or FLAC (30.3%), respectively. Finally, we show that the prediction-compression equivalence allows us to use any compressor (like gzip) to build a conditional generative model.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Predicting subsequent words is akin to losslessly compressing significant information.
“Besides, recent research [72, 291] suggests that predicting subsequent words is akin to losslessly compressing significant information.”
From A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions (2025) · cites this paper as [72]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 54
- Léonard Blier and Yann Ollivier. The description length of deep learning models. In NeurIPS, 2018.
- Rishi Bommasani et al. On the opportunities and risks of foundation models. arXiv:2108.07258, 2021. · arXiv
- Tom B. Brown, Benjamin Mannand Nick Ryder, Melanie Subbiah, et al. Language models are few-shot learners. In NeurIPS, 2020.
- Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott M. Lundberg, Harsha Nori, Hamid Palangi, Marco Túlio Ribeiro, and Yi Zhang. Sparks of artificial general intelligence: Early experiments with GPT-4. arXiv:2303.12712, 2023. · arXiv
- Aydar Bulatov, Yuri Kuratov, and Mikhail S. Burtsev. Scaling transformer to 1m tokens and beyond with RMT. arXiv:2304.11062, 2023. · arXiv
- Yu Cheng, Duo Wang, Pan Zhou, and Tao Zhang. A survey of model compression and acceleration for deep neural networks. arXiv:1710.09282, 2017. · arXiv
- John G. Cleary and Ian H. Witten. Data compression using adaptive coding and partial string matching. IEEE Trans. Commun., 1984.
- David Cox. Syntactically informed text compression with recurrent neural networks. arXiv:1608.02893, 2016. · arXiv
- Grégoire Delétang, Anian Ruoss, Jordi Grau-Moya, Tim Genewein, Li Kevin Wenliang, Elliot Catt, Chris Cundy, Marcus Hutter, Shane Legg, Joel Veness, and Pedro A. Ortega. Neural networks and the chomsky hierarchy. In ICLR, 2023.
- Jarek Duda. Asymmetric numeral systems. arXiv:0902.0271, 2009. · arXiv
- Eibe Frank, Chang Chui, and Ian H. Witten. Text categorization using compression models. In Data Compression Conference, 2000.
- Tao Ge, Jing Hu, Xun Wang, Si-Qing Chen, and Furu Wei. In-context autoencoder for context compression in a large language model. arXiv:2307.06945, 2023. · arXiv
- Tim Genewein, Grégoire Delétang, Anian Ruoss, Li Kevin Wenliang, Elliot Catt, Vincent Dutordoir, Jordi Grau-Moya, Laurent Orseau, Marcus Hutter, and Joel Veness. Memory-based meta-learning on non-stationary distributions. In ICML, 2023.
- Mandy Guo, Joshua Ainslie, David C. Uthus, Santiago Ontañón, Jianmo Ni, Yun-Hsuan Sung, and Yinfei Yang. Longt5: Efficient text-to-text transformer for long sequences. In NAACL-HLT (Findings), 2022.
- Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, et al. Training compute-optimal large language models. arXiv:2203.15556, 2022. · arXiv
- Emiel Hoogeboom, Jorn W. T. Peters, Rianne van den Berg, and Max Welling. Integer discrete flows and lossless compression. In NeurIPS, 2019.
- Paul G. Howard and Jeffrey Scott Vitter. Analysis of arithmetic coding for data compression. In Data Compression Conference, 1991.
- David A. Huffman. A method for the construction of minimum-redundancy codes. Proceedings of the IRE, 1952.
- Marcus Hutter. Universal Artificial Intellegence -Sequential Decisions Based on Algorithmic Probability. Springer, 2005.
- Zhiying Jiang, Yiqin Dai, Ji Xin, Ming Li, and Jimmy Lin. Few-shot non-parametric learning with deep latent variable model. In NeurIPS, 2022.
- Zhiying Jiang, Matthew Y. R. Yang, Mikhail Tsirlin, Raphael Tang, Yiqin Dai, and Jimmy Lin. "low-resource" text classification: A parameter-free classification method with compressors. In ACL (Findings), 2023.
- Friso H. Kingma, Pieter Abbeel, and Jonathan Ho. Bit-swap: Recursive bits-back coding for lossless compression with hierarchical latent variables. In ICML, 2019.
- Taku Kudo. Subword regularization: Improving neural network translation models with multiple subword candidates. In ACL (1), 2018.
- Taku Kudo and John Richardson. Sentencepiece: A simple and language independent subword tokenizer and detokenizer for neural text processing. In EMNLP (Demonstration), 2018.
- Michael Laskin, Luyu Wang, et al. In-context reinforcement learning with algorithm distillation. In ICLR. OpenReview.net, 2023.
- Ming Li and Paul M. B. Vitányi. An Introduction to Kolmogorov Complexity and Its Applications, 4th Edition. Springer, 2019.
- David J. C. MacKay. Information theory, inference, and learning algorithms. Cambridge University Press, 2003.
- Fabian Mentzer, Eirikur Agustsson, Michael Tschannen, Radu Timofte, and Luc Van Gool. Practical full resolution learned lossless image compression. In CVPR, 2019.
- Fabian Mentzer, Luc Van Gool, and Michael Tschannen. Learning better lossless compression using lossy compression. In CVPR, 2020.
- Tomas Mikolov. Statistical Language Models Based on Neural Networks. PhD thesis, Brno Universtiy of Technology, 2012.
- Suvir Mirchandani, Fei Xia, Pete Florence, Brian Ichter, Danny Driess, Montserrat Gonzalez Arenas, Kanishka Rao, Dorsa Sadigh, and Andy Zeng. Large language models as general pattern machines. arXiv:2307.04721, 2023. · arXiv
- Behnam Neyshabur, Srinadh Bhojanapalli, David McAllester, and Nati Srebro. Exploring generaliza- tion in deep learning. In NIPS, 2017.
- Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. Librispeech: An ASR corpus based on public domain audio books. In ICASSP, 2015.
- Richard C. Pasco. Source coding algorithms for fast data compression (ph.d. thesis abstr.). IEEE Trans. Inf. Theory, 1977.
- Ivan Provilkov, Dmitrii Emelianenko, and Elena Voita. Bpe-dropout: Simple and effective subword regularization. In ACL, 2020.
- Jack W. Rae et al. Scaling language models: Methods, analysis & insights from training gopher. arXiv:2112.11446, 2021. · arXiv
- Hochang Rhee, Yeong Il Jang, Seyun Kim, and Nam Ik Cho. LC-FDNet: Learned lossless image compression with frequency decomposition network. In CVPR, 2022.
- Anian Ruoss, Grégoire Delétang, Tim Genewein, Jordi Grau-Moya, Róbert Csordás, Mehdi Bennani, Shane Legg, and Joel Veness. Randomized positional encodings boost length generalization of transformers. In ACL (2), 2023.
- Ionut Schiopu and Adrian Munteanu. Deep-learning-based lossless image coding. IEEE Trans. Circuits Syst. Video Technol., 2020.
- Jürgen Schmidhuber and Stefan Heil. Predictive coding with neural nets: Application to text compression. In NIPS, pp. 1047-1054. MIT Press, 1994.
- Jürgen Schmidhuber and Stefan Heil. Sequential neural text compression. IEEE Trans. Neural Networks, 1996.
- Rico Sennrich, Barry Haddow, and Alexandra Birch. Neural machine translation of rare words with subword units. In ACL (1), 2016.
- Chaofan Tao, Lu Hou, Wei Zhang, Lifeng Shang, Xin Jiang, Qun Liu, Ping Luo, and Ngai Wong. Compression of generative pre-trained language models via quantization. In ACL (1), 2022.
- William J. Teahan and David J. Harper. Using Compression-Based Language Models for Text Categorization, pp. 141-165. Springer Netherlands, 2003.
- Hugo Touvron, Thibaut Lavril, Gautier Izacard, et al. Llama: Open and efficient foundation language models. arXiv:2302.13971, 2023a. · arXiv
- Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton-Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurélien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open foundation and fine-tuned chat models. arXiv:2307.09288, 2023b. · arXiv
- James Townsend, Thomas Bird, and David Barber. Practical lossless compression with latent variables using bits back coding. In ICLR (Poster), 2019.
- Chandra Shekhara Kaushik Valmeekam, Krishna Narayanan, Dileep Kalathil, Jean-François Cham- berland, and Srinivas Shakkottai. Llmzip: Lossless text compression using large language models. arXiv:2306.04050, 2023. · arXiv
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In NIPS, 2017.
- Joel Veness, Marc G. Bellemare, Marcus Hutter, Alvin Chua, and Guillaume Desjardins. Compress and control. In AAAI, 2015.
- Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In NeurIPS, 2022.
- Frans M. J. Willems, Yuri M. Shtarkov, and Tjalling J. Tjalkens. The context-tree weighting method: basic properties. IEEE Trans. Inf. Theory, 1995.
- Ian H. Witten, Radford M. Neal, and John G. Cleary. Arithmetic coding for data compression. Commun. ACM, 1987.
- Manzil Zaheer, Guru Guruganesh, Kumar Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontañón, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, and Amr Ahmed. Big bird: Transformers for longer sequences. In NeurIPS, 2020.
Cited by
Sorted by citations
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025 · see citing works
Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., et al. (2025) · ACM Transactions on Information Systems · doi:10.1145/3703155 · Cited by 2,142 · see citing works
LLMLingua: Compressing Prompts for Accelerated Inference of Large Language Models
Jiang, H., Wu, Q., Lin, C.-Y., Yang, Y., & Qiu, L. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.825 · Cited by 168 · see citing works
Wang, S., Hu, T., Huang, X., Li, Y., Zhang, C., Ning, H., Zhu, R., Li, Z., & Ye, X. (2024) · International Journal of Digital Earth · doi:10.1080/17538947.2024.2353122 · Cited by 121 · see citing works
LongLLMLingua: Accelerating and Enhancing LLMs in Long Context Scenarios via Prompt Compression
Jiang, H., Wu, Q., Luo, X., Li, D., Lin, C.-Y., Yang, Y., & Qiu, L. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.91 · Cited by 96 · see citing works
Will we run out of data? Limits of LLM scaling based on human-generated data
Villalobos, P., Ho, A., Sevilla, J., Besiroglu, T., Heim, L., & Hobbhahn, M. (2022) · arXiv · doi:10.48550/arxiv.2211.04325 · Cited by 80 · see citing works
Hallucination of Multimodal Large Language Models: A Survey
Bai, Z., Wang, P., Xiao, T., He, T., Han, Z., Zhang, Z., & Shou, M. Z. (2024) · arXiv · doi:10.48550/arxiv.2404.18930 · Cited by 38 · see citing works
“Neural networks, especially large language models, possess an intrinsic tendency to memorize training data, as noted in [23].”
Generative AI for Immersive Communication: The Next Frontier in Internet-of-Senses Through 6G
Sehad, N., Bariah, L., Hamidouche, W., Hellaoui, H., Jäntti, R., & Debbah, M. A. (2024) · IEEE Communications Magazine · doi:10.1109/mcom.001.2400199 · Cited by 27 · see citing works
Sharma, M. K., Farhat, I., Liu, C., Sehad, N., Hamidouche, W., & Debbah, M. A. (2024) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2024.3455763 · Cited by 26 · see citing works
Lossless data compression by large models
Li, Z., Huang, C., Wang, X., Hu, H., Wyeth, C., Bu, D., Yu, Q., Gao, W., Liu, X., & Li, M. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01033-7 · Cited by 21 · see citing works
Desbordes, T., King, J.-R., & Dehaene, S. (2024) · Cell Reports · doi:10.1016/j.celrep.2024.113847 · Cited by 18 · see citing works
He, Y., Wang, E., Rong, Y., Cheng, Z., & Chen, H. (2025) · 2025 IEEE/ACM International Workshop on Responsible AI Engineering (RAIE) · doi:10.1109/raie66699.2025.00013 · Cited by 19 · see citing works
Tailored Visions: Enhancing Text-to-Image Generation with Personalized Prompt Rewriting
Chen, Z., Zhang, L., Weng, F., Pan, L., & Lan, Z. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00738 · Cited by 22 · see citing works
GPTArticleExtractor: An automated workflow for magnetic material database construction
Zhang, Y., Itani, S., Khanal, K., Okyere, E., Smith, G., Takahashi, K., & Zang, J. (2024) · Journal of Magnetism and Magnetic Materials · doi:10.1016/j.jmmm.2024.172001 · Cited by 16 · see citing works
Yang, J. (2024) · International Journal of Chinese Linguistics · doi:10.1075/ijchl.00023.yan · Cited by 22 · see citing works
Deep Generative Modeling Reshapes Compression and Transmission: From Efficiency to Resiliency
Dai, J., Qin, X., Wang, S., Xu, L., Niu, K., & Zhang, P. (2024) · IEEE Wireless Communications · doi:10.1109/mwc.005.2300574 · Cited by 16 · see citing works
Kim, J., Yi, S., & Wang, Z. (2025) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2025.111474 · Cited by 13 · see citing works
Akrout, M., Mezghani, A., Hossain, E., Bellili, F., & Heath, R. W. (2024) · IEEE Communications Magazine · doi:10.1109/mcom.001.2300456 · Cited by 10 · see citing works
A Clinician’s Guide to Understanding Bias in Critical Clinical Prediction Models
de Matos, J. C. R. G., Gallifant, J., Chowdhury, A., Economou-Zavlanos, N., Charpignon, M., Gichoya, J. W., Celi, L. A., Nazer, L. H., King, H., & Wong, A.-K. I. (2024) · Critical Care Clinics · doi:10.1016/j.ccc.2024.05.011 · Cited by 9 · see citing works
Bennett, M. T. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-65572-2_2 · Cited by 9 · see citing works
Unifying Generation and Compression: Ultra-low bitrate Image Coding Via Multi-stage Transformer
Xue, N., Mao, Q., Wang, Z., Zhang, Y., & Ma, S. (2024) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme57554.2024.10687549 · Cited by 9 · see citing works
Simplicity bias, algorithmic probability, and the random logistic map
Hamzi, B., & Dingle, K. (2024) · Physica D Nonlinear Phenomena · doi:10.1016/j.physd.2024.134160 · Cited by 8 · see citing works
Quayle, M. (2025) · European Review of Social Psychology · doi:10.1080/10463283.2025.2514433 · Cited by 8 · see citing works
In-context Autoencoder for Context Compression in a Large Language Model
Ge, T., Hu, J., Wang, L., Wang, X., Chen, S.-Q., & Wei, F. (2023) · arXiv · doi:10.48550/arxiv.2307.06945 · Cited by 11 · see citing works
“Delétang et al. (2023) extensively investigates the compression abilities of LLMs, uncovering their potential as versatile predictors, which also provides insights into recent developments in scaling laws and tokenization.”
BackgroundCoelho Mollo, D., & Millière, R. (2026) · Philosophy and the Mind Sciences · doi:10.33735/phimisci.2026.12307 · Cited by 48 · see citing works
LMM-driven Semantic Image-Text Coding for Ultra Low-bitrate Learned Image Compression
Murai, S., Sun, H., & Katto, J. (2024) · IEEE International Conference on Visual Communications and Image Processing (VCIP) · doi:10.1109/vcip63160.2024.10849863 · Cited by 6 · see citing works
Li, C., Jin, Z., Wang, F., Zhang, Z., Liu, B., & Guo, Y. (2025) · JMIR Rehabilitation and Assistive Technologies · doi:10.2196/69230 · Cited by 6 · see citing works
Bridging Information-Theoretic and Geometric Compression in Language Models
Cheng, E., Kervadec, C., & Baroni, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.762 · Cited by 10 · see citing works
BatFix: Repairing language model-based transpilation
Ramos, D., Lynce, I., Manquinho, V. M., Martins, R., & Goues, C. L. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3658668 · Cited by 5 · see citing works
FINCH: Prompt-guided Key-Value Cache Compression for Large Language Models
Corallo, G., & Papotti, P. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00716 · Cited by 5 · see citing works
Exploring lightweight language models for materials informatics with AlchemBERT
Wang, Y., Liu, X., Chen, H., Yang, T., Liu, X., & Wen, X. (2025) · Cell Reports Physical Science · doi:10.1016/j.xcrp.2025.102724 · Cited by 4 · see citing works
Human languages trade off complexity against efficiency
Koplenig, A., Wolfer, S., Rüdiger, J. O., & Meyer, P. (2025) · PLOS complex systems. · doi:10.1371/journal.pcsy.0000032 · Cited by 3 · see citing works
Selective fine-tuning for large language models via matrix nuclear norm
Xia, T., Li, Y., Wu, Y., & Chang, Y. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104259 · Cited by 3 · see citing works
Is Complexity an Illusion?Preprint
Bennett, M. T. (2024) · doi:10.36227/techrxiv.171207379.99385056/v4 · Cited by 3 · see citing works
Yi, X., Li, Y., Shi, D., Wang, L., Wang, X., & He, L. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.129101 · Cited by 4 · see citing works
Identifying Outliers in AI-based Image Compression
Ashraf, R. A., Guo, L., Lee, H., & Tallent, N. R. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825013 · Cited by 1 · see citing works
Improving the Contextual Understanding of LLMs Through Multi-Teacher Knowledge Distillation and RAG
Ippili, Y., Jain, V. K., Dixit, Y., Saraf, G., & Prasad, V. R. B. (2025) · International Conference on Electronics, Computers and Artificial Intelligence (ECAI) · doi:10.1109/ecai65401.2025.11095547 · Cited by 1 · see citing works
Is Complexity an Illusion?Preprint
Bennett, M. T. (2024) · doi:10.36227/techrxiv.171207379.99385056/v1 · Cited by 1 · see citing works
Yi, X., Li, Y., Wang, L., Wang, X., & He, L. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5169050 · Cited by 1 · see citing works
Is Complexity an Illusion?Preprint
Bennett, M. T. (2024) · doi:10.36227/techrxiv.171207379.99385056/v2
Guiding ChatGPT to Generate Salient Domain Summaries
Gao, J. H., Cao, Z., Huang, S., Qin, L., & Ai, C. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650616 · Cited by 1 · see citing works
Enhancing LLM-Based Text Compression with Context-Aware Frequency Adaptation
Kinger, R., Murali, R., Krishna, P. R., Goru, M., & Upadhyaya, S. R. (2025) · International Conference on Electronics, Computers and Artificial Intelligence (ECAI) · doi:10.1109/ecai65401.2025.11095536
The cost of language: tokenization as a metric of labor
Caffoni, P. (2026) · AI & Society · doi:10.1007/s00146-026-02990-2
Enabling high-volume data processing of space-based data via generative AI compression
Holguin, P., Toy, V., Barham, S., Villafañe-Delgado, M., Mayorga, L., & Stephens, D. (2026) · Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications · doi:10.1117/12.3094705
Bennett, M. T. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-33010-9_4
Understanding, prediction, and compression
Poldrack, R. A. (2026) · Synthese · doi:10.1007/s11229-026-05801-9
Artificial Intelligence as an Emergent Order
Berg, C. (2026) · Palgrave studies in classical liberalism · doi:10.1007/978-3-032-22310-4_3
- Authors
- Delétang, G., Ruoss, A., Duquenne, P.-A., Catt, E., Genewein, T., Mattern, C., Grau-Moya, J., Wenliang, L. K., Aitchison, M., Orseau, L., Hutter, M., & Veness, J.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2309.10668
- OpenAlex
- W4386908184
- Cited by
- 39
- References
- 0
- Citations by year
- 2026: 4 · 2025: 12 · 2024: 21 · 2023: 2
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Algorithms and Data Compression
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Data Compression Techniques
- Keywords
- lossless compression, large language models, foundation models, scaling laws, tokenization, in-context learning, conditional generative models, image compression, speech coding, chinchilla
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
