Inverse Scaling: When Bigger Isn't Better
Open accessAbstract
Work on scaling laws has found that large language models (LMs) show predictable improvements to overall loss with increased scale (model size, training data, and compute). Here, we present evidence for the claim that LMs may show inverse scaling, or worse task performance with increased scale, e.g., due to flaws in the training objective and data. We present empirical evidence of inverse scaling on 11 datasets collected by running a public contest, the Inverse Scaling Prize, with a substantial prize pool. Through analysis of the datasets, along with other examples found in the literature, we identify four potential causes of inverse scaling: (i) preference to repeat memorized sequences over following in-context instructions, (ii) imitation of undesirable patterns in the training data, (iii) tasks containing an easy distractor task which LMs could focus on, rather than the harder real task, and (iv) correct but misleading few-shot demonstrations of the task. We release the winning datasets at https://inversescaling.com/data to allow for further investigation of inverse scaling. Our tasks have helped drive the discovery of U-shaped and inverted-U scaling trends, where an initial trend reverses, suggesting that scaling trends are less reliable at predicting the behavior of larger-scale models than previously understood. Overall, our results suggest that there are tasks for which increased model scale alone may not lead to progress, and that more careful thought needs to go into the data and objectives for training language models.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
More capable models tend to be easier to attack, a form of inverse scaling law.
“We find that more capable models tend to be easier to attack, a form of inverse scaling law [38] (a similar observation had been made in [37]).”
From AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents (2024) · cites this paper as [38]
The Scaling Paradox at L3 mirrors the Inverse Scaling Prize findings where larger models perform worse on tasks involving negation or counter-intuitive truths.
“The 'Scaling Paradox' we observe at L3 mirrors the 'Inverse Scaling Prize' findings (McKenzie et al., 2023), where larger models perform worse on tasks involving negation or counter-intuitive truths.”
From Diagnosing and Mitigating Sycophancy and Skepticism in LLM Causal Judgment (2026) · cites this paper as McKenzie et al., 2023
A higher-capability model can be more sycophantic under adversarial hints, because rationalizing a wrong hint requires the cognitive capacity to construct a plausible bridge from a derivation to the hinted value.
“Table 12 replicates the inverse scaling finding of McKenzie et al. (2023): a higher-capability model can be more sycophantic under adversarial hints, because rationalizing a wrong hint requires the cognitive capacity to construct a plausible bridge from a derivation to the hinted value.”
From Diagnosing and Mitigating Sycophancy and Skepticism in LLM Causal Judgment (2026) · cites this paper as McKenzie et al. (2023)
The paper’s Modus Tollens task description discusses larger models making reasoning errors, and §3.2 discusses unwanted imitation of mistakes. Neither addresses sycophancy under adversarial hints or the proposed mechanism of constructing a plausible bridge to a hinted value. Searches for sycophancy, hints, and rationalization found no passage supporting that specific claim. (LLM, conf=0.74)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 41
- Amanda Askell, Yuntao Bai, Anna Chen, Dawn Drain, Deep Ganguli, Tom Henighan, Andy Jones, Nicholas Joseph, Ben Mann, Nova DasSarma, et al. A General Language Assistant as a Laboratory for Alignment, December 2021. URL http://arxiv.org/abs/2112.00861. arXiv:2112.00861 [cs. · arXiv
- Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, et al. Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback. arXiv preprint 2204.05862, 2022. URL https://arxiv.org/abs/2204. 05862. · arXiv
- Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S. Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the Opportunities and Risks of Foundation Models, July 2022. URL http://arxiv.org/abs/2108.07258. arXiv:2108.07258 [cs. · arXiv
- Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Nee- lakantan, Pranav Shyam, Girish Sastry, et al. Language Models are Few-Shot Learners. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Sys- tems, volume 33, pp. 1877-1901.
- Curran Associates, Inc., 2020. URL https://proceedings.neurips. cc/paper_files/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf.
- Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramèr, and Chiyuan Zhang. Quantifying Memorization Across Neural Language Models. arXiv preprint 2202.07646, 2022. URL https: //arxiv.org/abs/2202.07646. · arXiv
- Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harrison Edwards, Yuri Burda, Nicholas Joseph, et al. Evaluating Large Language Models Trained on Code. arXiv preprint 2107.03374, 2021. URL https://arxiv.org/abs/2107.03374. · arXiv
- Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, et al. PaLM: Scaling Language Modeling with Pathways. arXiv preprint 2204.02311, 2022. URL https://arxiv.org/abs/2204.02311. · arXiv
- Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep Rein- forcement Learning from Human Preferences. In Advances in Neural Information Processing Systems, volume 30.
- Curran Associates, Inc., 2017. URL https://proceedings.neurips.cc/paper/2017/file/ d5e2c0adad503c91f91df240d0cd4e49-Paper.pdf.
- Deep Ganguli, Amanda Askell, Nicholas Schiefer, Thomas I. Liao, Kamilė Lukošiūtė, Anna Chen, Anna Goldie, Azalia Mirhoseini, Catherine Olsson, Danny Hernandez, et al. The Capacity for Moral Self- Correction in Large Language Models, 2023. URL https://arxiv.org/abs/2302.07459. · arXiv
- Leo Gao, John Schulman, and Jacob Hilton. Scaling Laws for Reward Model Overoptimization, October 2022. URL http://arxiv.org/abs/2210.10760. arXiv:2210.10760 [cs, stat]. · arXiv
- Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring Massive Multitask Language Understanding, January 2021. URL http://arxiv.org/abs/ 2009.03300. arXiv:2009.03300 [cs. · arXiv
- Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, et al. Training Compute-Optimal Large Language Models. arXiv preprint 2203.15556, 2022. URL https://arxiv.org/abs/2203.15556. · arXiv
- Evan Hubinger, Chris van Merwijk, Vladimir Mikulik, Joar Skalse, and Scott Garrabrant. Risks from Learned Optimization in Advanced Machine Learning Systems. arXiv preprint 1906.01820, 2021. URL https://arxiv.org/abs/1906.01820. · arXiv
- Nikhil Kandpal, Eric Wallace, and Colin Raffel. Deduplicating Training Data Mitigates Privacy Risks in Lan- guage Models. In ICML, 2022. URL https://proceedings.mlr.press/v162/kandpal22a/kandpal22a. pdf.
- Zachary Kenton, Tom Everitt, Laura Weidinger, Iason Gabriel, Vladimir Mikulik, and Geoffrey Irving. Alignment of Language Agents, March 2021. URL http://arxiv.org/abs/2103.14659. arXiv:2103.14659 · arXiv
- Najoung Kim, Tal Linzen, and Paul Smolensky. Uncontrolled Lexical Exposure Leads to Overestimation of Compositional Generalization in Pretrained Models, December 2022. URL http://arxiv.org/abs/ 2212.10769. arXiv:2212.10769 [cs]. · arXiv
- Tomasz Korbak, Kejian Shi, Angelica Chen, Rasika Bhalerao, Christopher L. Buckley, Jason Phang, Samuel R. Bowman, and Ethan Perez. Pretraining Language Models with Human Preferences, Febru- ary 2023. URL http://arxiv.org/abs/2302.08582. arXiv:2302.08582 [cs. · arXiv
- Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, et al. Holistic Evaluation of Language Models, November 2022. URL http://arxiv.org/abs/2211.09110. arXiv:2211.09110 [cs. · arXiv
- Antonio Valerio Miceli-Barone, Fazl Barez, Ioannis Konstas, and Shay B. Cohen. The Larger They Are, the Harder They Fail: Language Models do not Recognize Identifier Swaps in Python, May 2023. URL http://arxiv.org/abs/2305.15507. arXiv:2305.15507 [cs. · arXiv
- Kanishka Misra, Julia Taylor Rayz, and Allyson Ettinger. COMPS: Conceptual Minimal Pair Sentences for testing Robust Property Knowledge and its Inheritance in Pre-trained Language Models, February 2023. URL http://arxiv.org/abs/2210.01963. arXiv:2210.01963 [cs. · arXiv
- Richard Ngo, Lawrence Chan, and Sören Mindermann. The Alignment Problem From a Deep Learning Perspective, 2023. URL https://arxiv.org/abs/2209.00626. · arXiv
- Maxwell I. Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, et al. Show Your Work: Scratchpads for Intermediate Computation with Language Models. CoRR, abs/2112.00114, 2021. URL https://arxiv. org/abs/2112.00114. · arXiv
- OpenAI. GPT-4 Technical Report, 2023. URL https://arxiv.org/abs/2303.08774. · arXiv
- Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback, March 2022. URL http://arxiv.org/abs/2203.02155. arXiv:2203.02155 [cs. · arXiv
- Ethan Perez, Douwe Kiela, and Kyunghyun Cho. True Few-Shot Learning with Language Models. NeurIPS, 2021. URL https://arxiv.org/abs/2105.11447. · arXiv
- Ethan Perez, Sam Ringer, Kamilė Lukošiūtė, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, et al. Discovering Language Model Behaviors with Model-Written Evaluations, 2022. URL https://arxiv.org/abs/2212.09251. · arXiv
- Jack W. Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, et al. Scaling Language Models: Methods, Analysis & Insights from Training Gopher. arXiv preprint 2112.11446, 2021. URL https://arxiv.org/abs/2112.11446. · arXiv
- Rohin Shah, Vikrant Varma, Ramana Kumar, Mary Phuong, Victoria Krakovna, Jonathan Uesato, and Zac Kenton. Goal Misgeneralization: Why Correct Specifications Aren't Enough for Correct Goals, 2022. URL https://arxiv.org/abs/2210.01790. version · arXiv
- Irene Solaiman and Christy Dennison. Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasets. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P. S. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, volume 34, pp. 5861-5873.
- Curran Associates, Inc., 2021. URL https://proceedings.neurips.cc/paper/2021/file/ 2e855f9489df0712b4bd8ea9e2848c5a-Paper.pdf.
- Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, et al. Beyond the Imitation Game: Quantifying and Extrapolating the Capabilities of Language Models. arXiv preprint 2206.04615, 2022. URL https://arxiv.org/abs/2206.04615. · arXiv
- Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. Learning to Summarize with Human Feedback. In H. Larochelle, M. Ran- zato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 3008-3021.
- Curran Associates, Inc., 2020. URL https://proceedings.neurips.cc/ paper_files/paper/2020/file/1f89885d556929e98d3ef9b86448f951-Paper.pdf.
- Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. SuperGLUE: A Stickier Benchmark for General-Purpose Lan- guage Understanding Systems. In Advances in Neural Information Processing Systems, volume 32.
- Curran Associates, Inc., 2019. URL https://papers.nips.cc/paper_files/paper/2019/hash/ 4496bf24afe7fab6f046bf4923da8de6-Abstract.html.
- Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. Emergent Abilities of Large Language Models, October 2022b. URL http://arxiv.org/abs/2206.07682. arXiv:2206.07682 [cs. · arXiv
- Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed H. Chi, Quoc V Le, and Denny Zhou. Chain of Thought Prompting Elicits Reasoning in Large Language Models. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022c. URL https://openreview.net/forum?id=_VjQlMeSB_J.
- Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, et al. OPT: Open Pre-trained Transformer Language Models. arXiv preprint 2205.01068, 2022. URL https://arxiv.org/abs/2205.01068. · arXiv
- Yuhui Zhang, Michihiro Yasunaga, Zhengping Zhou, Jeff Z. HaoChen, James Zou, Percy Liang, and Serena Yeung. Beyond Positive Scaling: How Negation Impacts Scaling Trends of Language Models. In Findings of the Association for Computational Linguistics (ACL Findings), 2023.
Cited by
Sorted by citations
Do LLMs Exhibit Human-like Response Biases? A Case Study in Survey Design
Tjuatja, L., Chen, V., Wu, T.-S., Talwalkwar, A., & Neubig, G. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00685 · Cited by 262 · see citing works
Findings of the BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora
Warstadt, A., Mueller, A., Choshen, L., Wilcox, E., Zhuang, C., Ciro, J., Mosquera, R., Paranjabe, B., Williams, A., Linzen, T., et al. (2023) · Proceedings of the BabyLM Challenge at the 27th Conference on Computational Natural Language Learning · doi:10.18653/v1/2023.conll-babylm.1 · Cited by 74 · see citing works
“Scaling is undoubtedly central to building deployable models (though see McKenzie et al. 2023 for counterexamples) and raises its own set of scientific questions, such as quantitative scaling laws (Kaplan et al., 2020) and the emergence of new abilities (Wei et al., 2022).”
Self-citationThe Limitations of Large Language Models for Understanding Human Language and Cognition
Cuskley, C. F., Woods, R. J. H., & Flaherty, M. (2024) · Open Mind · doi:10.1162/opmi_a_00160 · Cited by 77 · see citing works
Hagendorff, T., Dasgupta, I., Binz, M., Chan, S. C. Y., Lampinen, A., Wang, J. X., Akata, Z., & Schulz, E. (2023) · arXiv · doi:10.48550/arxiv.2303.13988 · Cited by 68 · see citing works
Language in Brains, Minds, and Machines
Tuckute, G., Kanwisher, N., & Fedorenko, E. G. (2024) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-120623-101142 · Cited by 69 · see citing works
Introduction to large language models (LLMs) for dementia care and research
Treder, M. S., Lee, S., & Tsvetanov, K. A. (2024) · Frontiers in Dementia · doi:10.3389/frdem.2024.1385303 · Cited by 63 · see citing works
Wu, Z., Qiu, L., Ross, A., Akyürek, E., Chen, B., Wang, B., Kim, N., Andreas, J., & Kim, Y. (2024) · Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2024.naacl-long.102 · Cited by 58 · see citing works
Self-citationConversing with business process-aware large language models: the BPLLM framework
Bernardi, M. L., Casciani, A., Cimitile, M., & Marrella, A. (2024) · Journal of Intelligent Information Systems · doi:10.1007/s10844-024-00898-1 · Cited by 55 · see citing works
Inverse Scaling Can Become U-Shaped
Wei, J., Kim, N., Tay, Y., & Le, Q. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.963 · Cited by 25 · see citing works
Self-citationDai, H., Mir, T. H., Chen, C.-T., Chen, C.-C., Yang, H., Lee, C.-H., Chou, Y.-Y., Teng, Y.-C., Gupta, S., Panchal, O., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01921-7 · Cited by 21 · see citing works
Pitfalls of large language models in medical ethics reasoning
Soffer, S., Sorin, V., Nadkarni, G. N., & Klang, E. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01792-y · Cited by 11 · see citing works
Language-Models-as-a-Service: Overview of a New Paradigm and its Challenges
La Malfa, E., Petrov, A., Frieder, S., Weinhuber, C., Burnell, R., Nazar, R., Cohn, A., Shadbolt, N., & Wooldridge, M. (2024) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.15865 · Cited by 9 · see citing works
Into the Unknown: Self-Learning Large Language Models
Ferdinan, T., Kocoń, J., & Kazienko, P. (2024) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw65004.2024.00060 · Cited by 8 · see citing works
AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents
Debenedetti, E., Zhang, J., Balunović, M., Beurer-Kellner, L., Fischer, M., & Tramèr, F. (2024) · arXiv · doi:10.48550/arxiv.2406.13352 · Cited by 7 · see citing works
“We find that more capable models tend to be easier to attack, a form of inverse scaling law [36] (a similar observation had been made in [35]).”
BackgroundDerivational morphology reveals analogical generalization in large language models
Hofmann, V., Weissweiler, L., Mortensen, D. R., Schütze, H., & Pierrehumbert, J. B. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2423232122 · Cited by 13 · see citing works
Assessing the Emergent Symbolic Reasoning Abilities of Llama Large Language Models
Petruzzellis, F., Testolin, A., & Sperduti, A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72344-5_18 · Cited by 5 · see citing works
Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress?
Ren, R., Basart, S., Khoja, A., Gatti, A., Phan, L., Yin, X., Mazeika, M., Pan, A., Mukobi, G., Kim, R. H., et al. (2024) · arXiv · doi:10.48550/arxiv.2407.21792 · Cited by 10 · see citing works
Target-driven Attack for Large Language Models
Zhang, C., Jin, M., Shu, D., Wang, T., Liu, D., & Jin, X. (2024) · Frontiers in artificial intelligence and applications · doi:10.3233/faia240685 · Cited by 4 · see citing works
An Investigation of LLMs’ Inefficacy in Understanding Converse Relations
Qi, C., Li, B., Hui, B., Wang, B., Li, J., Wu, J., & Laili, Y. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.429 · Cited by 5 · see citing works
Do Large Language Models Align with Core Mental Health Counseling Competencies?
Nguyen, V. C., Taher, M., Hong, D., Possobom, V. K., Gopalakrishnan, V. T., Raj, E., Li, Z., Soled, H. J., Birnbaum, M. L., Kumar, S., et al. (2024) · arXiv · doi:10.48550/arxiv.2410.22446 · Cited by 3 · see citing works
LLMs can Design Sustainable Concrete – a Systematic BenchmarkPreprint
Völker, C., Rug, T., Jablonka, K. M., & Kruschwitz, S. (2024) · Research Square · doi:10.21203/rs.3.rs-3913272/v1 · Cited by 4 · see citing works
Diagnosing and Mitigating Sycophancy and Skepticism in LLM Causal Judgment
Chang, E. Y. (2026) · Findings of the Association for Computational Linguistics: ACL 2026 · doi:10.18653/v1/2026.findings-acl.427 · Cited by 2 · see citing works
“2023): a higher-capability model can be more sycophantic under adversarial hints, because rationalizing a wrong hint requires the cognitive capacity to construct a plausible bridge from a derivation to the hinted value.”
Probing Quantifier Comprehension in Large Language Models: Another Example of Inverse Scaling
Gupta, A. (2023) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2023.blackboxnlp-1.4 · Cited by 2 · see citing works
Iterative improvements from feedback for language modelsPreprint
Li, Y. (2023) · doi:10.14293/pr2199.000220.v1 · Cited by 2 · see citing works
Assessing the nature of large language models: A caution against anthropocentrismPreprint
Speed, A. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.07683 · Cited by 2 · see citing works
Sartori, C. C., Blum, C., & Bistaffa, F. (2025) · IEEE Access · doi:10.1109/access.2025.3535837 · Cited by 4 · see citing works
A Conversational Framework for Faithful Multi-perspective Analysis of Production Systems
Casciani, A., Lestingi, L., Marrella, A., & Matta, A. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-94569-4_10 · Cited by 1 · see citing works
The Impact of Multilingual Model Scaling on Seen and Unseen Language Performance
Pokharel, R., Nezhad, S. B., Agrawal, A., & Singh, S. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-8912-5_13 · Cited by 2 · see citing works
Kagan, B. J., & Kitchen, A. C. (2025) · Springer eBooks · doi:10.1007/978-3-031-93623-4_12 · Cited by 1 · see citing works
ChapGTP, ILLC’s Attempt at Raising a BabyLM: Improving Data Efficiency by Automatic Task Formation
Jumelet, J., Hanna, M., Kloots, M. D. H., Langedijk, A., Pouw, C., & van der Wal, O. (2023) · BabyLM Challenge · doi:10.18653/v1/2023.conll-babylm.6 · Cited by 1 · see citing works
Look-Ahead-Bench: a Standardized Benchmark of Look-ahead Bias in Point-in-Time LLMs for Finance
Benhenda, M. (2026) · arXiv
Ai, Q., Zhan, J., & Liu, Y. (2025) · arXiv · doi:10.48550/arxiv.2501.02842
“McKen-zie et al. [33] even found that scaling results in worse metric scores.”
Emergent Inabilities? Inverse Scaling Over the Course of Pretraining
Michaelov, J. A., & Bergen, B. K. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.973
Wuli-Shili-Renli-Jili System Approach and Its Application in Large Model Value Alignment
Qi, J., Jia, K., Zhang, Y., Feng, X., & Yan, H. (2024) · IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C) · doi:10.1109/qrs-c63300.2024.00069
Self-citationMapping Errors to Corrections: Promoting Self-Correcting LLM in Speech Summarization
Zhao, X., Qi, X., Jia, H., Song, Z., He, X., & He, L. (2026) · Journal of Shanghai Jiaotong University (Science) · doi:10.1007/s12204-026-2915-z
Chang, E. Y. (2026) · ACM eBooks · doi:10.1145/3822380.3822386
LLM-Based Orchestration of Business Process Management Toolchains
Ulbricht, O., Blaschke, M., Seidel, A., & Weske, M. (2026) · Lecture notes in business information processing · doi:10.1007/978-3-032-36710-5_16
- Authors
- McKenzie, I. R., Lyzhov, A., Pieler, M., Parrish, A., Mueller, A., Prabhu, A., McLean, E., Kirtland, A., Ross, A., Liu, A., Gritsevskiy, A., Wurgaft, D., Kauffman, D., Recchia, G., Liu, J., Cavanagh, J., Weiss, M., Huang, S., Droid, T. F., Tseng, T., Korbak, T., Shen, X., Zhang, Y., Zhou, Z., Kim, N., Bowman, S. R., & Perez, E.
- Affiliations
- New York University (US)
- Google (United States) (US)
- University of Oxford (GB)
- Johns Hopkins University (US)
- Boston University (US)
- Anthropic (United States) (US)
- ORCID
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2306.09479
- OpenAlex
- W4381245716
- Cited by
- 29
- References
- 0
- Citations by year
- 2026: 3 · 2025: 7 · 2024: 11 · 2023: 8
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- National Science Foundation (2046556, 1922658)
- Open Philanthropy Project
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- large language models, scaling laws, model size, training data, training objectives, memorization, in-context learning, distractor task, benchmark dataset
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
