The DOI matches its DataCite record; 6 of the 6 references it cites were found in a registry or an index.
Llama 2: Open Foundation and Fine-Tuned Chat Models
Open accessAbstract
In this work, we develop and release Llama 2, a collection of pretrained and fine-tuned large language models (LLMs) ranging in scale from 7 billion to 70 billion parameters. Our fine-tuned LLMs, called Llama 2-Chat, are optimized for dialogue use cases. Our models outperform open-source chat models on most benchmarks we tested, and based on our human evaluations for helpfulness and safety, may be a suitable substitute for closed-source models. We provide a detailed description of our approach to fine-tuning and safety improvements of Llama 2-Chat in order to enable the community to build on our work and contribute to the responsible development of LLMs.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“Llama 2's total training data was 1.8T tokens (Touvron et al., 2023), while Llama 3 used 3.7T tokens for mathematical reasoning (Grattafiori et al., 2024).”
From LIMO: Less is More for Reasoning (2025) · cites this paper as Touvron et al., 2023
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For example, the most widely used open-source LLMs can only support a few dozen back-and-forth messages or reason about a short document before exceeding their maximum input length (Touvron et al., 2023).”
From MemGPT: Towards LLMs as Operating Systems (2023) · cites this paper as Touvron et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“current knowledge injection methods often suffer from computational overhead [13, 10], catastrophic forgetting [63], and overfitting [64].”
From WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models (2024) · cites this paper as [13]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Similarly, Llama2 [254] demonstrates that post-RLHF, the model consistently generated factual responses to prompts on factual information.”
From Survey of Hallucination in Natural Language Generation (2023) · cites this paper as [254]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“contemporary widely used LMs such as Llama 2 (Touvron et al., 2023) are trained on trillions of words (Figure 1).”
From Findings of the BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora (2023) · cites this paper as Touvron et al., 2023
Experimental — accuracy not yet measured
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Based on the GPT-3.5 results, it is unlikely that STOP would consistently work with any open-source LM at the time of writing (Touvron et al., 2023; Jiang et al., 2023).”
From Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation (2023) · cites this paper as Touvron et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“We evaluate our OS-inspired design in two domains where the limited context windows of modern LLMs severely handicaps their performance: document analysis, where MemGPT is able to analyze large documents that far exceed the underlying LLM's context window, and multi-session chat, where MemGPT can create conversational agents that remember, reflect, and evolve dynamically through long-term inter”
From MemGPT: Towards LLMs as Operating Systems (2023) · cites this paper as Touvron et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“A limiting factor for in-context learning (ICL) in LLMs is the context window, restricting prior research to the few-shot ICL regime.”
From Many-Shot In-Context Learning (2024) · cites this paper as [57]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“existing foundation models for vision and language such as CLIP [8], SigLIP [9], and Llama 2 [10] are capable of these types of generalization and more, stemming from the priors captured by their Internet-scale pretraining datasets.”
From OpenVLA: An Open-Source Vision-Language-Action Model (2024) · cites this paper as [10]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“recent Large Language Models (LLMs) (Touvron et al., 2023; OpenAI, 2023; Anil et al., 2023) have shown impressive capabilities in processing and generating text with remarkable accuracy, even outperforming human experts across diverse specialized domains including math, physics, history, law, medicine, and ethics.”
From ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Models (2025) · cites this paper as Touvron et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This model uses 32,000 tokens and 4,096 dimensions for its token embeddings.”
From The Linear Representation Hypothesis and the Geometry of Large Language Models (2023) · cites this paper as Touvron et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Its seven billion parameters are pre-trained on two trillion sentencepiece (Kudo & Richardson, 2018) tokens, 90% of which is in English.”
From The Linear Representation Hypothesis and the Geometry of Large Language Models (2023) · cites this paper as Touvron et al., 2023
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“However, finetuning and pretraining are computation-expensive [13, 10] and usually suffer from catastrophic forgetting [63] and overfitting [64].”
From WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models (2024) · cites this paper as [13]
The cited source says what the claim attributes to it.
In the citing paper
“A representative example of such data-driven methods can be found in the development of Llama-2 [254], where the most factual sources within the pretraining data were up-sampled in order to reduce hallucinations.”
From Survey of Hallucination in Natural Language Generation (2023) · cites this paper as [254]
The cited source says what the claim attributes to it.
In the citing paper
“The winners of the challenge's various tracks made contributions that led to impressive improvements in our evaluation over not just the BabyLM baselines, but also the massively pretrained Llama 2 model (Touvron et al., 2023).”
From Findings of the BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora (2023) · cites this paper as Touvron et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“As instruction tuning does not require so much data (Zhou et al., 2023a;Touvron et al., 2023)”
From OctoPack: Instruction Tuning Code Large Language Models (2023) · cites this paper as Touvron et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“we note that LLaMA-2-Chat-7B also undergoes RLHF [60]”
From Laboratory-Scale AI: Open-Weight Models are Competitive with ChatGPT Even in Low-Resource Settings (2024) · cites this paper as [60]
The cited source says what the claim attributes to it.
In the citing paper
“Large language models (LLMs) have revolutionized AI, but are constrained by limited context windows, hindering their utility in tasks like extended conversations and document analysis.”
From MemGPT: Towards LLMs as Operating Systems (2023) · cites this paper as Touvron et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“State-of-the-art LLMs [66, 67, 7, 52] are generally implemented using the Transformer architecture, which is a multi-layer feedforward network equipped with attention mechanism [71].”
From Hallucination is Inevitable: An Innate Limitation of Large Language Models (2024) · cites this paper as [67]
The cited source says what the claim attributes to it.
In the citing paper
“We have seen similar positive performance from other recent models, like Llama 2 (7B and 13B) chat variants (Touvron et al., 2023).”
From NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Rails (2023) · cites this paper as Touvron et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“Human feedback is commonly utilized to finetune AI assistants.”
From AI Supported Degradation of the Self Concept: A Theoretical Framework Grounded in Established Cognitive and Computational Mechanisms (2023) · cites this paper as Touvron et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“Llama 2 is pre-trained on 2 trillion tokens of data using the autoregressive language modeling task.”
From Exploring the Capabilities of LLMs for Code-Change-Related Tasks (2025) · cites this paper as [73]
The cited source says what the claim attributes to it.
In the citing paper
“Prior studies showed that within the same LLM family, larger models are associated with better performance [59, 67, 73].”
From Exploring the Capabilities of LLMs for Code-Change-Related Tasks (2025) · cites this paper as [73]
The cited source says what the claim attributes to it.
In the citing paper
“Exploring many-shot ICL is now feasible, given the recent increase in context windows of publicly available LLMs by at least 100 × : from only a few thousand tokens in GPT-3 [8] and Llama 2 [57] to 1M tokens in Gemini 1.5 Pro [16].”
From Many-Shot In-Context Learning (2024) · cites this paper as [57]
The cited source says what the claim attributes to it.
In the citing paper
“In particular, SELF-RAG outperforms retrieval-augmented ChatGPT on four tasks, Llama2-chat (Touvron et al., 2023) and Alpaca (Dubois et al., 2023) on all tasks.”
From CareerX: A Retrieval-Augmented Generation Framework for Personalized AI-Driven Career Guidance (2023) · cites this paper as Touvron et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“Intriguingly, our program with the 13b variant of the model is competitive with their 34b-based results even though we don't use human reasoning chains in our program.”
From DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines (2023) · cites this paper as Touvron et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“The Llama2 authors (Touvron et al., 2023) presents 28.7% for llama2-13b , 42.2% for llama2-34b , and 56.8% for llama2-70b .”
From DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines (2023) · cites this paper as Touvron et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“A model whose weights are published33,34,35,36 can, in principle, be retrieved and rerun by others later, giving the runtime a snapshottable character more like that of conventional software environments and narrowing the shelf-life gap that closed-weight models create.”
From Designing reproducible large-language-model-assisted scientific analyses (2026) · cites this paper as [33]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“some game environments are procedurally generated and game states grow exponentially, making games more robust against evaluation dataset contamination as observed in recent works (Touvron et al., 2023).”
From SmartPlay: A Benchmark for LLMs as Intelligent Agents (2023) · cites this paper as Touvron et al., 2023
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“generalization to other architectures should be verified [Touvron et al., 2023, Achiam et al., 2023].”
From Accurate and Efficient Long-Term Memory for LLM Agents (2026) · cites this paper as Touvron et al., 2023
Experimental — accuracy not yet measuredIn the citing paper
“Errors in retrieving recent information may be intrinsic to the current LLMs because the pretraining procedures weight all text in the training data equally, without specific emphasis on more recent or high-quality information.27”
From Performance of Large Language Models on Medical Oncology Examination Questions (2024) · cites this paper as [27]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Scaling Laws for Neural Language Models
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · doi:10.48550/arxiv.2001.08361 · Cited by 9,550
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
Self-citationKnow What You Don’t Know: Unanswerable Questions for SQuAD
Rajpurkar, P., Jia, R., & Liang, P. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p18-2124 · Cited by 2,240
Toolformer: Language Models Can Teach Themselves to Use Tools
Schick, T., Dwivedi-Yu, J., Dessì, R., Raileanu, R., Lomeli, M., Zettlemoyer, L., Cancedda, N., & Scialom, T. (2023) · arXiv · doi:10.48550/arxiv.2302.04761 · Cited by 1,150
Self-citationEthical and social risks of harm from Language Models
Weidinger, L., Mellor, J., Rauh, M., Griffin, C., Uesato, J., Huang, P.-S., Cheng, M., Glaese, M., Balle, B., Kasirzadeh, A., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.04359 · Cited by 477
AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models
Zhong, W., Cui, R., Guo, Y., Liang, Y., Lu, S., Wang, Y., Saied, A., Chen, W., & Duan, N. (2023) · arXiv · doi:10.48550/arxiv.2304.06364 · Cited by 211 · cited 4×
“In Table 24, we present fine-grained results from the English part of the AGI Eval (Zhong et al., 2023) benchmark.”
BackgroundInfluential
Research works printed but not linked: 98
- Daron Acemoglu and Pascual Restrepo. Artificial intelligence, automation, and work. In The economics of artificial intelligence: An agenda , pages 197-236. University of Chicago Press, 2018.
- Amanda Askell, Yuntao Bai, Anna Chen, Dawn Drain, Deep Ganguli, Tom Henighan, Andy Jones, Nicholas Joseph, Ben Mann, Nova DasSarma, et al. A general language assistant as a laboratory for alignment. arXiv preprint arXiv:2112.00861 , 2021b. · arXiv
- Amanda Askell, Yuntao Bai, Anna Chen, Dawn Drain, Deep Ganguli, Tom Henighan, Andy Jones, Nicholas Joseph, Ben Mann, Nova DasSarma, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Jackson Kernion, Kamal Ndousse, Catherine Olsson, Dario Amodei, Tom Brown, Jack Clark, Sam McCandlish, and Chris Olah. A general language assistant as a laboratory for alignment. arXiv preprint arXiv:2112.00861 , 2021a. · arXiv
- YuntaoBai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862 , 2022a. · arXiv
- Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. Constitutional ai: Harmlessness from ai feedback. arXiv preprint arXiv:2212.08073 , 2022b. · arXiv
- Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Margaret Mitchell. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency , pages 610-623, 2021a.
- Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency , pages 610-623, 2021b.
- A Stevie Bergman, Gavin Abercrombie, Shannon L Spruit, Dirk Hovy, Emily Dinan, Y-Lan Boureau, and Verena Rieser. Guiding the release of safer e2e conversational ai through value sensitive design. In Proceedings of the 23rd Annual Meeting of the Special Interest Group on Discourse and Dialogue , pages 39-52, 2022.
- Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. Piqa: Reasoning about physical commonsense in natural language. In Proceedings of the AAAI conference on artificial intelligence , pages 7432-7439, 2020.
- Su Lin Blodgett, Gilsinia Lopez, Alexandra Olteanu, Robert Sim, and Hanna Wallach. Stereotyping norwegian salmon: An inventory of pitfalls in fairness benchmark datasets. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) , pages 1004-1015, 2021.
- Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems , volume 33, pages 1877-1901. Curran Associates, Inc., 2020. URL https://proceedings. neurips.cc/paper\_files/paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf .
- Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomás Mikolov. Enriching word vectors with subword information. CoRR , abs/1607.04606, 2016. URL http://arxiv.org/abs/1607.04606 . · arXiv
- Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, Ilya Sutskever, and Wojciech Zaremba. Evaluating large language models trained on code, 2021.
- Eunsol Choi, He He, Mohit Iyyer, Mark Yatskar, Wen-tau Yih, Yejin Choi, Percy Liang, and Luke Zettlemoyer. Quac: Question answering in context. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing , pages 2174-2184, 2018.
- Paul F Christiano, Jan Leike, Tom Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. Advances in neural information processing systems , 30, 2017.
- Adam Roberts, Denny Zhou, Quoc V. Le, and Jason Wei. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416 , 2022. · arXiv
- Elizabeth Clark, Tal August, Sofia Serrano, Nikita Haduong, Suchin Gururangan, and Noah A. Smith. All that's 'human' is not gold: Evaluating human evaluation of generated text. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) , pages 7282-7296, Online, August 2021. Association for Computational Linguistics. doi: 10.18653/v1/2021.acl-long.565. URL https://aclanthology.org/2021.acl-long.565 . · DOI · Paper page
- Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. Boolq: Exploring the surprising difficulty of natural yes/no questions. arXiv preprint arXiv:1905.10044 , 2019. · arXiv
- Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457 , 2018. · arXiv
- Jiawen Deng, Hao Sun, Zhexin Zhang, Jiale Cheng, and Minlie Huang. Recent advances towards safe, responsible, and moral dialogue systems: A survey. arXiv preprint arXiv:2302.09270 , 2023. · arXiv
- Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168 , 2021. · arXiv
- Yuntian Deng, Anton Bakhtin, Myle Ott, Arthur Szlam, and Marc'Aurelio Ranzato. Residual energy-based models for text generation. In International Conference on Learning Representations , 2019.
- Emily Dinan, Gavin Abercrombie, A Stevie Bergman, Shannon Spruit, Dirk Hovy, Y-Lan Boureau, and Verena Rieser. Anticipating safety issues in e2e conversational ai: Framework and tooling. arXiv preprint arXiv:2107.03451 , 2021. · arXiv
- Jwala Dhamala, Tony Sun, Varun Kumar, Satyapriya Krishna, Yada Pruksachatkun, Kai-Wei Chang, and Rahul Gupta. BOLD: Dataset and metrics for measuring biases in open-ended language generation. In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency , pages 862-872, 2021.
- Jesse Dodge, Maarten Sap, Ana Marasović, William Agnew, Gabriel Ilharco, Dirk Groeneveld, Margaret Mitchell, and Matt Gardner. Documenting large webtext corpora: A case study on the colossal clean crawled corpus. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , pages 1286-1305, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. doi: 10.18653/v1/2021.emnlp-main.98. URL https://aclanthology.org/2021.emnlp-main. 98 . · DOI · Paper page
- Nan Du, Yanping Huang, Andrew M Dai, Simon Tong, Dmitry Lepikhin, Yuanzhong Xu, Maxim Krikun, Yanqi Zhou, Adams Wei Yu, Orhan Firat, Barret Zoph, Liam Fedus, Maarten P Bosma, Zongwei Zhou, Tao Wang, Emma Wang, Kellie Webster, Marie Pellat, Kevin Robinson, Kathleen Meier-Hellstern, Toju Duke, Lucas Dixon, Kun Zhang, Quoc Le, Yonghui Wu, Zhifeng Chen, and Claire Cui. GLaM: Efficient scaling of language models with mixture-of-experts. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors, Proceedings of the 39th International Conference on Machine Learning , volume 162 of Proceedings of Machine Learning Research , pages 5547-5569. PMLR, 17-23 Jul 2022. URL https://proceedings.mlr.press/v162/du22c.html .
- Jesse Dodge, Taylor Prewitt, Remi Tachet Des Combes, Erika Odmark, Roy Schwartz, Emma Strubell, Alexandra Sasha Luccioni, Noah A Smith, Nicole DeCario, and Will Buchanan. Measuring the carbon intensity of ai in cloud instances. arXiv preprint arXiv:2206.05229 , 2022. · arXiv
- Kawin Ethayarajh, Yejin Choi, and Swabha Swayamdipta. Understanding dataset difficulty with V -usable information. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato, editors, Proceedings of the 39th International Conference on Machine Learning , volume 162 of Proceedings of Machine Learning Research , pages 5988-6008. PMLR, 17-23 Jul 2022.
- Deep Ganguli, Liane Lovitt, Jackson Kernion, Amanda Askell, Yuntao Bai, Saurav Kadavath, Ben Mann, Ethan Perez, Nicholas Schiefer, Kamal Ndousse, et al. Red teaming language models to reduce harms: Methods, scaling behaviors, and lessons learned. arXiv preprint arXiv:2209.07858 , 2022. · arXiv
- Prakhar Ganesh, Hongyan Chang, Martin Strobel, and Reza Shokri. On the impact of machine learning randomness on group fairness. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency , pages 1789-1800, 2023.
- Deep Ganguli, Amanda Askell, Nicholas Schiefer, Thomas Liao, Kamil˙ e Lukoši¯ ut˙ e, Anna Chen, Anna Goldie, Azalia Mirhoseini, Catherine Olsson, Danny Hernandez, et al. The capacity for moral self-correction in large language models. arXiv preprint arXiv:2302.07459 , 2023. · arXiv
- Sebastian Gehrmann, Elizabeth Clark, and Thibault Sellam. Repairing the cracked foundation: A survey of obstacles in evaluation practices for generated text. Journal of Artificial Intelligence Research , 77:103-166, 2023.
- Fabrizio Gilardi, Meysam Alizadeh, and Maël Kubli. Chatgpt outperforms crowd-workers for text-annotation tasks. arXiv preprint arXiv:2303.15056 , 2023. · arXiv
- Udit Gupta, Mariam Elgamal, Gage Hills, Gu-Yeon Wei, Hsien-Hsin S Lee, David Brooks, and Carole-Jean Wu. Act: designing sustainable computer systems with an architectural carbon modeling tool. In Proceedings of the 49th Annual International Symposium on Computer Architecture , pages 784-799, 2022a.
- Arnav Gudibande, Eric Wallace, Charlie Snell, Xinyang Geng, Hao Liu, Pieter Abbeel, Sergey Levine, and Dawn Song. The false promise of imitating proprietary llms. arXiv preprint arXiv:2305.15717 , 2023. · arXiv
- Udit Gupta, Young Guen Kim, Sylvia Lee, Jordan Tse, Hsien-Hsin Sean Lee, Gu-Yeon Wei, David Brooks, and Carole-Jean Wu. Chasing carbon: The elusive environmental footprint of computing. IEEE Micro , 2022b.
- Kilem Li Gwet. Computing inter-rater reliability and its variance in the presence of high agreement. British Journal of Mathematical and Statistical Psychology , 61(1):29-48, 2008.
- Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarten Sap, Dipankar Ray, and Ece Kamar. Toxigen: A large-scale machine-generated dataset for adversarial and implicit hate speech detection. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages 3309-3326, 2022.
- Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. Deberta: Decoding-enhanced bert with disentangled attention. arXiv preprint arXiv:2006.03654 , 2020. · arXiv
- Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Xiaodong Song, and Jacob Steinhardt. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300 , 2020. · arXiv
- Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. Training compute-optimal large language models. arXiv preprint arXiv:2203.15556 , 2022. · arXiv
- Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874 , 2021. · arXiv
- Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. The curious case of neural text degeneration. In International Conference on Learning Representations , 2020. URL https://openreview.net/forum?id= rygGQyrFvH .
- Saghar Hosseini, Hamid Palangi, and Ahmed Hassan Awadallah. An empirical study of metrics to measure representational harms in pre-trained language models. arXiv preprint arXiv:2301.09211 , 2023. · arXiv
- Or Honovich, Thomas Scialom, Omer Levy, and Timo Schick. Unnatural instructions: Tuning language models with (almost) no human labor. arXiv preprint arXiv:2212.09689 , 2022. · arXiv
- Fan Huang, Haewoon Kwak, and Jisun An. Is chatgpt better than human annotators? potential and limitations of chatgpt in explaining implicit hate speech. arXiv preprint arXiv:2302.07736 , 2023. · arXiv
- Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. arXiv preprint arXiv:1705.03551 , 2017. · arXiv
- Clayton Hutto and Eric Gilbert. Vader: A parsimonious rule-based model for sentiment analysis of social media text. In Proceedings of the international AAAI conference on web and social media , volume 8, pages 216-225, 2014.
- Andreas Köpf, Yannic Kilcher, Dimitri von Rütte, Sotiris Anagnostidis, Zhi-Rui Tam, Keith Stevens, Abdullah Barhoum, Nguyen Minh Duc, Oliver Stanley, Richárd Nagyfi, et al. Openassistant conversationsdemocratizing large language model alignment. arXiv preprint arXiv:2304.07327 , 2023. · arXiv
- James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences , 114(13):3521-3526, 2017.
- Tomasz Korbak, Kejian Shi, Angelica Chen, Rasika Bhalerao, Christopher L Buckley, Jason Phang, Samuel R Bowman, and Ethan Perez. Pretraining language models with human preferences. arXiv preprint arXiv:2302.08582 , 2023. · arXiv
- Sachin Kumar, Vidhisha Balachandran, Lucille Njoo, Antonios Anastasopoulos, and Yulia Tsvetkov. Language generation models can cause harm: So what can we do about it? an actionable survey. arXiv preprint arXiv:2210.07700 , 2022. · arXiv
- Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. Natural questions: a benchmark for question answering research. Transactions of the Association for Computational Linguistics , 7:453-466, 2019.
- Katherine Lee, Daphne Ippolito, Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison-Burch, and Nicholas Carlini. Deduplicating training data makes language models better. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics . Association for Computational Linguistics, 2022.
- Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 , 2019. · arXiv
- Stephanie Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Measuring how models mimic human falsehoods. arXiv preprint arXiv:2109.07958 , 2021. · arXiv
- Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V Le, Barret Zoph, Jason Wei, et al. The flan collection: Designing data and methods for effective instruction tuning. arXiv preprint arXiv:2301.13688 , 2023. · arXiv
- Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. Self-refine: Iterative refinement with self-feedback. arXiv preprint arXiv:2303.17651 , 2023. · arXiv
- Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 , 2017. · arXiv
- Grégoire Mialon, Roberto Dessì, Maria Lomeli, Christoforos Nalmpantis, Ram Pasunuru, Roberta Raileanu, Baptiste Rozière, Timo Schick, Jane Dwivedi-Yu, Asli Celikyilmaz, et al. Augmented language models: a survey. arXiv preprint arXiv:2302.07842 , 2023. · arXiv
- Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. Model cards for model reporting. CoRR , abs/1810.03993, 2018. URL http://arxiv.org/abs/1810.03993 . · arXiv
- Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. Can a suit of armor conduct electricity? a new dataset for open book question answering. arXiv preprint arXiv:1809.02789 , 2018. · arXiv
- Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Lonbrown Ouyanbrown, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, Xu Jiang, Karl Cobbe, Tyna Eloundou, Gretchen Krueger, Kevin Button, Matthew Knight, Benjamin Chess, and John Schulman. Webgpt: Browser-assisted question-answering with human feedback. In arXiv , 2021.
- Cuong V. Nguyen, Alessandro Achille, Michael Lam, Tal Hassner, Vijay Mahadevan, and Stefano Soatto. Toward understanding catastrophic forgetting in continual learning. arXiv preprint arXiv:1908.01091 , 2019. · arXiv
- OpenAI. GPT-4 technical report. CoRR , abs/2303.08774, 2023. doi: 10.48550/arXiv.2303.08774. URL https://doi.org/10.48550/arXiv.2303.08774 . · DOI · Paper page
- Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems , 35:27730-27744, 2022.
- David Patterson, Joseph Gonzalez, Quoc Le, Chen Liang, Lluis-Miquel Munguia, Daniel Rothchild, David So, Maud Texier, and Jeff Dean. Carbon emissions and large neural network training. arXiv preprint arXiv:2104.10350 , 2021. · arXiv
- Vinay Venkatesh Ramasesh, Aitor Lewkowycz, and Ethan Dyer. Effect of scale on catastrophic forgetting in neural networks. In International Conference on Learning Representations , 2021.
- Stephen Roller, Y-Lan Boureau, Jason Weston, Antoine Bordes, Emily Dinan, Angela Fan, David Gunning, Da Ju, Margaret Li, Spencer Poff, et al. Open-domain conversational agents: Current progress, open problems, and future directions. arXiv preprint arXiv:2006.12442 , 2020. · arXiv
- Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM , 64(9):99-106, 2021.
- Maarten Sap, Hannah Rashkin, Derek Chen, Ronan LeBras, and Yejin Choi. Socialiqa: Commonsense reasoning about social interactions. arXiv preprint arXiv:1904.09728 , 2019. · arXiv
- Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilić, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, et al. Bloom: A 176b-parameter open-access multilingual language model. arXiv preprint arXiv:2211.05100 , 2022. · arXiv
- John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 , 2017. · arXiv
- Thomas Scialom, Paul-Alexis Dray, Sylvain Lamprier, Benjamin Piwowarski, and Jacopo Staiano. Discriminative adversarial search for abstractive summarization. In Hal Daumé III and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learning , volume 119 of Proceedings of Machine Learning Research , pages 8555-8564. PMLR, 13-18 Jul 2020a. URL https://proceedings.mlr.press/v119/ scialom20a.html .
- Thomas Scialom, Paul-Alexis Dray, Sylvain Lamprier, Benjamin Piwowarski, and Jacopo Staiano. Coldgans: Taming language gans with cautious sampling strategies. Advances in Neural Information Processing Systems , 33:18978-18989, 2020b.
- Uri Shaham, Elad Segal, Maor Ivgi, Avia Efrat, Ori Yoran, Adi Haviv, Ankit Gupta, Wenhan Xiong, Mor Geva, Jonathan Berant, and Omer Levy. SCROLLS: Standardized CompaRison over long language sequences. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pages 12007-12021, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. URL https://aclanthology.org/2022.emnlp-main.823 .
- Eric Michael Smith and Adina Williams. Hi, my name is martha: Using names to measure and mitigate bias in generative dialogue models. arXiv preprint arXiv:2109.03300 , 2021. · arXiv
- Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Yarin Gal, Nicolas Papernot, and Ross Anderson. The curse of recursion: Training on generated data makes models forget. arXiv preprint arxiv:2305.17493 , 2023. · arXiv
- Eric Michael Smith, Melissa Hall, Melanie Kambadur, Eleonora Presani, and Adina Williams. 'i'm sorry to hear that': Finding new biases in language models with a holistic descriptor dataset. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , pages 9180-9211, 2022.
- Nisan Stiennon, Long Ouyang, Jeff Wu, Daniel M. Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul Christiano. Learning to summarize from human feedback. In NeurIPS , 2020.
- Irene Solaiman, Zeerak Talat, William Agnew, Lama Ahmad, Dylan Baker, Su Lin Blodgett, Hal Daumé III, Jesse Dodge, Ellie Evans, Sara Hooker, et al. Evaluating the social impact of generative ai systems in systems and society. arXiv preprint arXiv:2306.05949 , 2023. · arXiv
- Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V Le, Ed H Chi, Denny Zhou, et al. Challenging big-bench tasks and whether chain-ofthought can solve them. arXiv preprint arXiv:2210.09261 , 2022. · arXiv
- Yarden Tal, Inbal Magar, and Roy Schwartz. Fewer errors, but more stereotypes? the effect of model size on gender bias. In Proceedings of the 4th Workshop on Gender Bias in Natural Language Processing (GeBNLP) , pages 112-120, Seattle, Washington, July 2022. Association for Computational Linguistics. doi: 10.18653/v1/2022.gebnlp-1.13. URL https://aclanthology.org/2022.gebnlp-1.13 . · DOI
- AlonTalmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. Commonsenseqa: A question answering challenge targeting commonsense knowledge. arXiv preprint arXiv:1811.00937 , 2018. · arXiv
- Ross Taylor, Marcin Kardas, Guillem Cucurull, Thomas Scialom, Anthony Hartshorn, Elvis Saravia, Andrew Poulton, Viktor Kerkez, and Robert Stojnic. Galactica: A large language model for science. arXiv preprint arXiv:2211.09085 , 2022. · arXiv
- Oriol Vinyals, Igor Babuschkin, Wojciech M Czarnecki, Michaël Mathieu, Andrew Dudzik, Junyoung Chung, David H Choi, Richard Powell, Timo Ewalds, Petko Georgiev, et al. Grandmaster level in starcraft ii using multi-agent reinforcement learning. Nature , 575(7782):350-354, 2019.
- Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language model with self generated instructions. arXiv preprint arXiv:2212.10560 , 2022. · arXiv
- Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. Finetuned language models are zero-shot learners. In International Conference on Learning Representations , 2021.
- Michael Webb. The impact of artificial intelligence on the labor market. Available at SSRN 3482150 , 2019.
- Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V Le. Finetuned language models are zero-shot learners. In International Conference on Learning Representations , 2022a. URL https://openreview.net/forum?id=gEZrGCozdqR .
- Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems , 35:24824-24837, 2022b.
- Carole-Jean Wu, Ramya Raghavendra, Udit Gupta, Bilge Acun, Newsha Ardalani, Kiwan Maeng, Gloria Chang, Fiona Aga, Jinshi Huang, Charles Bai, et al. Sustainable ai: Environmental implications, challenges and opportunities. Proceedings of Machine Learning and Systems , 4:795-813, 2022.
- Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. Hellaswag: Can a machine really finish your sentence? arXiv preprint arXiv:1905.07830 , 2019a. · arXiv
- Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. Defending against neural fake news. Advances in neural information processing systems , 32, 2019b.
- Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068 , 2022. · arXiv
- Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, Lili Yu, Susan Zhang, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer, and Omer Levy. Lima: Less is more for alignment. arXiv preprint arXiv:2305.11206 , 2023. · arXiv
- Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. Large language models are human-level prompt engineers. In The Eleventh International Conference on Learning Representations , 2022.
- Terry Yue Zhuo, Yujin Huang, Chunyang Chen, and Zhenchang Xing. Exploring ai ethics of chatgpt: A diagnostic analysis. arXiv preprint arXiv:2301.12867 , 2023. · arXiv
Cited by
Sorted by citations
Items 301–400 of 417
UniMTS: Unified Pre-training for Motion Time Series
Zhang, X., Teng, D., Chowdhury, R., Li, S., Hong, D., Gupta, R., & Shang, J. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-3414 · Cited by 8
Bejan, C. A., Reed, A. M., Mikula, M., Zhang, S., Xu, Y., Fabbri, D., Embí, P. J., & Hsi, R. S. (2025) · Scientific Reports · doi:10.1038/s41598-025-86632-5 · Cited by 8
README: Bridging Medical Jargon and Lay Understanding for Patient Education through Data-Centric NLP
Yao, Z., Kantu, N. S., Wei, G., Tran, H., Duan, Z., Kwon, S., Yang, Z., & Yu, H. (2024) · Findings of the Association for Computational Linguistics: EMNLP 2024 · doi:10.18653/v1/2024.findings-emnlp.737 · Cited by 8
EcoAssistant: Using LLM Assistant More Affordably and Accurately
Zhang, J., Krishna, R., Awadallah, A. H., & Wang, C. (2023) · arXiv · doi:10.48550/arxiv.2310.03046 · Cited by 8
“5-turbo and GPT-4) and one open-source model (LLAMA-2-13B-chat (Touvron et al., 2023)).”
BackgroundMethodIn-context Autoencoder for Context Compression in a Large Language Model
Ge, T., Hu, J., Wang, L., Wang, X., Chen, S.-Q., & Wei, F. (2023) · arXiv · doi:10.48550/arxiv.2307.06945 · Cited by 11
“Listing 1: Prompt used by GPT4 API to generate the P W C dataset.”
MethodPrismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Models
Karamcheti, S., Nair, S., Balakrishna, A., Liang, P., Kollar, T., & Sadigh, D. (2024) · arXiv · doi:10.48550/arxiv.2402.07865 · Cited by 7
“This “patch-as-token” approach enables training with a simple objective – next-token prediction – and allows us to harness the ecosystem of powerful LMs such as Llama-2 and Mistral (Touvron et al., 2023; Jiang et al., 2023), along with the tools to efficiently train them (e.g., FSDP; Zhao et al., 2023).”
BackgroundMethodInfluentialTaskBench: Benchmarking Large Language Models for Task Automation
Shen, Y., Song, K., Tan, X., Zhang, W., Ren, K., Yuan, S., Lu, W., Li, D., & Zhuang, Y. (2023) · arXiv · doi:10.48550/arxiv.2311.18760 · Cited by 24
Self-citationGao, Y., Myers, S., Chen, S., Dligach, D., Miller, T. A., Bitterman, D., Churpek, M., & Afshar, M. (2024) · Findings of the Association for Computational Linguistics: EMNLP 2024 · doi:10.18653/v1/2024.findings-emnlp.311 · Cited by 8
Evaluating the World Model Implicit in a Generative Model
Vafa, K., Chen, J. Y., Rambachan, A., Kleinberg, J., & Mullainathan, S. (2024) · arXiv · Cited by 22
Chia, Y. K., Toh, V., Ghosal, D., Bing, L., & Poria, S. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.962 · Cited by 10
Risks from Language Models for Automated Mental Healthcare: Ethics and Structure for Implementation
Grabb, D., Lamparth, M., & Vasan, N. (2024) · arXiv · doi:10.48550/arxiv.2406.11852 · Cited by 39
AnyMAL: An Efficient and Scalable Any-Modality Augmented Language Model
Moon, S., Madotto, A., Lin, Z., Nagarajan, T., Smith, M., Jain, S., Yeh, C.-F., Murugesan, P., Heidari, P., Liu, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2309.16058 · Cited by 7
Enabling Efficient Batch Serving for LMaaS via Generation Length Prediction
Cheng, K., Hu, W., Wang, Z., Du, P., Li, J., & Zhang, S. (2024) · arXiv · doi:10.48550/arxiv.2406.04785 · Cited by 7
MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Use
Huang, Y., Shi, J., Li, Y., Fan, C., Wu, S., Zhang, Q., Liu, Y., Zhou, P., Wan, Y., Gong, N. Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.03128 · Cited by 7
L-Eval: Instituting Standardized Evaluation for Long Context Language Models
An, C., Gong, S., Zhong, M., Zhao, X., Li, M., Zhang, J., Kong, L., & Qiu, X. (2023) · arXiv · doi:10.48550/arxiv.2307.11088 · Cited by 6
“Meanwhile, almost all previous long-sequence text generation benchmarks relied primarily on n-gram matching metrics (Zhang et al., 2023; Shaham et al., 2022), such as ROUGE (Lin, 2004).”
BackgroundMethodBenchmarking Benchmark Leakage in Large Language Models
Xu, R., Wang, Z., Fan, R.-Z., & Liu, P. (2024) · arXiv · doi:10.48550/arxiv.2404.18824 · Cited by 6
AI Hospital: Benchmarking Large Language Models in a Multi-agent Medical Interaction Simulator
Fan, Z., Tang, J., Chen, W., Wang, S., Wei, Z., Xi, J., Huang, F., & Zhou, J. (2024) · arXiv · doi:10.48550/arxiv.2402.09742 · Cited by 84
Shankar, R., Lim, A., & Xu, Q. (2026) · Healthcare · doi:10.3390/healthcare14101267 · Cited by 9
Sun, D., Hadjiiski, L., Bruno, G., Gormley, J., Chan, H.-P., Caoili, E. M., Cohan, R. H., Alva, A., Mihalcea, R., Zhou, C., et al. (2025) · Scientific Reports · doi:10.1038/s41598-025-27593-7 · Cited by 9
Mitigating Sycophancy in Large Language Models via Direct Preference Optimization
Khan, A. A., Alam, S., Wang, X., Khan, A. F., Neog, D. R., & Anwar, A. (2024) · 2024 IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825538 · Cited by 6
Do Vision and Language Models Share Concepts? A Vector Space Alignment Study
Li, J., Kementchedjhieva, Y., Fierro, C., & Søgaard, A. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00698 · Cited by 6
“CLIP results are presented in Appendix C. (Zhang et al., 2022), and LLaMA-2 (Touvron et al., 2023).”
ResultNaumov, V., Zagirova, D., Lin, S., Xie, Y., Gou, W., Urban, A., Tikhonova, N., Alawi, K., Durymanov, M., Galkin, F., et al. (2025) · doi:10.1101/2025.03.06.641840 · Cited by 5
Large Language Models as Biomedical Hypothesis Generators: A Comprehensive Evaluation
Qi, B., Zhang, K., Tian, K., Li, H., Chen, Z.-R., Zeng, S., Hua, E., Jinfang, H., & Zhou, B. (2024) · arXiv · doi:10.48550/arxiv.2407.08940 · Cited by 5
From Yes-Men to Truth-Tellers: Addressing Sycophancy in Large Language Models with Pinpoint Tuning
Chen, W., Huang, Z., Xie, L., Lin, B., Li, H., Lu, L., Tian, X., Cai, D., Zhang, Y., Wang, W., et al. (2024) · arXiv · doi:10.48550/arxiv.2409.01658 · Cited by 5
Self-citationKG-Rank: Enhancing Large Language Models for Medical QA with Knowledge Graphs and Ranking Techniques
Yang, R., Liu, H., Marrese-Taylor, E., Zeng, Q., Ke, Y. H., Li, W., Cheng, L., Chen, Q., Caverlee, J., Matsuo, Y., et al. (2024) · arXiv · doi:10.48550/arxiv.2403.05881 · Cited by 5
Lessons from the Trenches on Reproducible Evaluation of Language Models
Biderman, S., Schoelkopf, H., Sutawika, L., Gao, L., Tow, J., Abbasi, B., Aji, A. F., Ammanamanchi, P. S., Black, S., Clive, J., et al. (2024) · arXiv · doi:10.48550/arxiv.2405.14782 · Cited by 9
FINCH: Prompt-guided Key-Value Cache Compression for Large Language Models
Corallo, G., & Papotti, P. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00716 · Cited by 5
Performance Implications of Pipelining the Data Transfer in CPU-GPU Heterogeneous Systems
Li, R., Hanindhito, B., Yadav, S., Wu, Q., Kavi, K., Mehta, G., Yadwadkar, N. J., & John, L. K. (2025) · ACM Transactions on Architecture and Code Optimization · doi:10.1145/3746231 · Cited by 5
Experience-guided multi-agent interpretable framework for radiology report summarization
Li, J., Zhou, T., Zhou, Z., Wei, X., Song, H., Wang, Z., Chen, Y., & Lv, H. (2026) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2025.109078 · Cited by 4
ChunkKV: Semantic-Preserving KV Cache Compression for Efficient Long-Context LLM Inference
Liu, X., Tang, Z., Dong, P., Li, Z., Liu, Y., Li, B., Hu, X., & Chu, X. (2025) · arXiv · doi:10.48550/arxiv.2502.00299 · Cited by 4
“…become essential for addressing various downstream tasks of natural language processing (NLP), including summarization and question answering, which require the interpretation of a long context from sources such as books, reports, and documents, often encompassing tens of thousands of tokens [1–5].”
An Evaluation Mechanism of LLM-based Agents on Manipulating APIs
Liu, B., Jianxiang, Z., Meng, D., & Lu, H. (2024) · Findings of the Association for Computational Linguistics: EMNLP 2024 · doi:10.18653/v1/2024.findings-emnlp.267 · Cited by 4
Deciphering the Impact of Pretraining Data on Large Language Models through Machine Unlearning
Zhao, Y., Du, L., Ding, X., Xiong, K., Sun, Z., Jun, S., Liu, T., & Qin, B. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.559 · Cited by 5
Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces
Yang, J., Yang, S., Gupta, A. W., Han, R., Fei-Fei, L., & Xie, S. (2024) · arXiv · Cited by 120
WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models
Wang, P., Li, Z., Zhang, N., Xu, Z., Yao, Y., Jiang, Y., Xie, P., Huang, F., & Chen, H. (2024) · arXiv · doi:10.48550/arxiv.2405.14768 · Cited by 19
Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress?
Ren, R., Basart, S., Khoja, A., Gatti, A., Phan, L., Yin, X., Mazeika, M., Pan, A., Mukobi, G., Kim, R. H., et al. (2024) · arXiv · doi:10.48550/arxiv.2407.21792 · Cited by 10
LIMO: Less is More for Reasoning
Ye, Y., Huang, Z., Xiao, Y., Chern, E., Xia, S., & Liu, P. (2025) · arXiv · doi:10.48550/arxiv.2502.03387 · Cited by 4
ASVD: Activation-aware Singular Value Decomposition for Compressing Large Language Models
Yuan, Z., Shang, Y., Song, Y., Yang, D., Wu, Q., Yan, Y., & Sun, G. (2023) · arXiv · doi:10.48550/arxiv.2312.05821 · Cited by 4
“We applied our method to open-access LLMs, specifically LLaMA [Touvron et al., 2023a] and LLaMA-2 [Touvron et al., 2023b].”
MethodSiegel, N. Y., Camburu, O.-M., Heess, N., & Perez-Ortiz, M. (2024) · arXiv · doi:10.48550/arxiv.2404.03189 · Cited by 5
“However, deep neural networks like LLMs are often difficult to interpret (Fan et al., 2020).”
MethodInfluentialLet’s learn step by step: Enhancing in-context learning ability with curriculum learning
Liu, Y., Liu, J., Shi, X., Cheng, Q., Huang, Y., & Lu, W. (2024) · arXiv · doi:10.48550/arxiv.2402.10738 · Cited by 4
“We utilize a range of open-source performant LLMs, including LlaMA2 (Touvron et al., 2023), Mixtral-8x7B (Jiang et al., 2024), Qwen1.”
BackgroundMethodAI Control: Improving Safety Despite Intentional Subversion
Greenblatt, R., Shlegeris, B., Sachan, K., & Roger, F. (2023) · arXiv · doi:10.48550/arxiv.2312.06942 · Cited by 3
“The usual way AI developers attempt to make models safe is by fine-tuning them to avoid the generation of harmful content (Ouyang et al., 2022; OpenAI, 2023a; Touvron et al., 2023).”
BackgroundDong, Z., Tang, T., Li, J., Zhao, W. X., & Wen, J.-R. (2023) · arXiv · doi:10.48550/arxiv.2309.13345 · Cited by 7
“The most popular LLMs, e.g., ChatGPT 5 and Llama2 (Touvron et al., 2023b), only have a context window of 4k tokens while some of them have been extended to contain 16k or more tokens , including ChatGPT-16k 5 and ChatGLM2-6b-32k (Zeng et al., 2023).”
MethodEvaluating Very Long-Term Conversational Memory of LLM Agents
Maharana, A., Lee, D.-H., Tulyakov, S., Bansal, M., Barbieri, F., & Fang, Y. (2024) · arXiv · doi:10.48550/arxiv.2402.17753 · Cited by 3
“2 (Jiang et al., 2023), Llama-2-70b-chat (Touvron et al., 2023), and Llama-3-70B-Instruct 5 ; (2) Long-context LLMs with an extended context window i.e., gpt-3.”
Zhou, X., Weyssow, M., Widyasari, R., Zhang, T., He, J., Lyu, Y., Chang, J., Zhang, B., Huang, D., & Lo, D. (2025) · arXiv · doi:10.48550/arxiv.2502.06215 · Cited by 3
WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?
Drouin, A., Gasse, M., Caccia, M., Laradji, I. H., Del Verme, M., Marty, T., Boisvert, L., Thakkar, M., Cappart, Q., Vazquez, D., et al. (2024) · arXiv · doi:10.48550/arxiv.2403.07718 · Cited by 3
“These results also indicate that web agent benchmarks, and WorkArena in particular, offer a great tool for evaluating emerging capabilities in LLMs, with larger performance gaps and more room for improvement compared to regular benchmarks (Touvron et al., 2023).”
BackgroundMulti-Modal generative AI systems: Bridging text, vision and speech with advanced LLM Architectures
John, D. (2023) · International Journal of Science and Research Archive · doi:10.30574/ijsra.2023.9.2.0619 · Cited by 3
KIWI: A Dataset of Knowledge-Intensive Writing Instructions for Answering Research Questions
Xu, F., Lo, K., Soldaini, L., Kuehl, B., Choi, E., & Wadden, D. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.770 · Cited by 3
Do Large Language Models Align with Core Mental Health Counseling Competencies?
Nguyen, V. C., Taher, M., Hong, D., Possobom, V. K., Gopalakrishnan, V. T., Raj, E., Li, Z., Soled, H. J., Birnbaum, M. L., Kumar, S., et al. (2024) · arXiv · doi:10.48550/arxiv.2410.22446 · Cited by 3
Zhang, Y., & He, Z. (2024) · Findings of the Association for Computational Linguistics: EMNLP 2024 · doi:10.18653/v1/2024.findings-emnlp.691 · Cited by 3
Self-citationAdversarial Paraphrasing: A Universal Attack for Humanizing AI-Generated Text
Cheng, Y., Sadasivan, V. S., Saberi, M., Saha, S., & Feizi, S. (2025) · Advances in Neural Information Processing Systems 38 · doi:10.52202/085713-1588 · Cited by 3
Lu, H., Li, H., Shahani, P. S., Herbers, S., & Scheutz, M. (2025) · 2025 IEEE International Conference on AI and Data Analytics (ICAD) · doi:10.1109/icad65464.2025.11114062 · Cited by 13
LLMs as Meta-Reviewers’ Assistants: A Case Study
Hossain, E., Sinha, S. K., Bansal, N., Knipper, R. A., Sarkar, S., Salvador, J., Mahajan, Y., Guttikonda, S. R. P. K., Akter, M., Hassan, M. M., et al. (2025) · Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2025.naacl-long.395 · Cited by 3
HelloBench: Evaluating Long Text Generation Capabilities of Large Language Models
Que, H., Duan, F., He, L., Mou, Y., Zhou, W., Liu, J., Rong, W., Wang, Z. M., Yang, J., Zhang, G., et al. (2024) · arXiv · doi:10.48550/arxiv.2409.16191 · Cited by 3
On Measuring Faithfulness or Self-consistency of Natural Language Explanations
Parcalabescu, L., & Frank, A. (2023) · arXiv · doi:10.48550/arxiv.2311.07466 · Cited by 27
“Large language models (LLMs) generate answers in various tasks of increasing difficulty, acting as chatbots (OpenAI, 2023; Touvron et al., 2023), as programming (Chen et al., 2021) or scientific writing assistants (Taylor et al., 2022).”
Self-Critique Guided Iterative Reasoning for Multi-hop Question Answering
Chu, Z., Fan, H., Chen, J., Wang, Q., Yang, M., Liang, J., Wang, Z., Li, H., Tang, G., Liu, M., et al. (2025) · Findings of the Association for Computational Linguistics: ACL 2025 · doi:10.18653/v1/2025.findings-acl.123 · Cited by 2
PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling
Cai, Z., Zhang, Y., Gao, B., Liu, Y., Li, Y., Liu, T., Lu, K., Xiong, W., Dong, Y., Hu, J., et al. (2024) · arXiv · doi:10.48550/arxiv.2406.02069 · Cited by 2
“In particular, we focus on our analysis of the LLaMa [27, 28] and visualize the distribution and behavior of attention scores over layers.”
BackgroundMethodSelf-citationFrom Human Memory to AI Memory: A Survey on Memory Mechanisms in the Era of LLMs
Wu, Y., Liang, S., Zhang, C., Wang, Y., Zhang, Y., Guo, H., Tang, R., & Liu, Y. (2025) · arXiv · doi:10.48550/arxiv.2504.15965 · Cited by 3
“ChatGPT [26], DeepSeek-Chat [27], Claude [28], QWEN-CHAT [29], Llama 2-Chat [30], Gemini [31], PANGU-BOT [32], ChatGLM [33], OpenAssistant [34] II Personal Non-Parametric Long-Term ChatGPT Memory [18], Apple Intelligence [19], Microsoft Recall [35], Me.bot [36] Open-SourceFramework MemoryScope…”
Self-citationSelf-Taught Optimizer (STOP): Recursively Self-Improving Code Generation
Zelikman, E., Lorch, E., Mackey, L., & Kalai, A. T. (2023) · arXiv · doi:10.48550/arxiv.2310.02304 · Cited by 4
“Understanding how LMs can improve their scaffoldings can help researchers understand and mitigate the potential for misuse of more powerful LMs. Lastly, STOP may allow researchers to investigate the effectiveness of techniques for mitigating undesirable self-improvement strategies.”
BackgroundQuantifying Variance in Evaluation Benchmarks
Madaan, L., Singh, A. K., Schaeffer, R., Poulton, A., Koyejo, S., Stenetorp, P., Narang, S., & Hupkes, D. (2024) · arXiv · doi:10.48550/arxiv.2406.10229 · Cited by 2
Self-citationAre Retrials All You Need? Enhancing Large Language Model Reasoning Without Verbalized Feedback
Potamitis, N., & Arora, A. (2025) · arXiv · doi:10.48550/arxiv.2504.12951 · Cited by 2
Observational Scaling Laws and the Predictability of Language Model Performance
Ruan, Y., Maddison, C. J., & Hashimoto, T. (2024) · arXiv · doi:10.48550/arxiv.2405.10938 · Cited by 3
R-Judge: Benchmarking Safety Risk Awareness for LLM Agents
Yuan, T., He, Z., Dong, L., Wang, Y., Zhao, R., Xia, T., Xu, L., Zhou, B., Li, F., Zhang, Z., et al. (2024) · arXiv · doi:10.48550/arxiv.2401.10019 · Cited by 2
“The open-source models include Llama-2 (Touvron et al., 2023), Llama-3, Mistral (Jiang et al., 2023), and Vicuna (Chiang et al., 2023) series.”
MethodInfluentialBootstrapping Language-Audio Pre-training for Music Captioning
Lanzendörfer, L. A., Pinkl, C., Perraudin, N., & Wattenhofer, R. (2025) · ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp49660.2025.10887618 · Cited by 3
Less is More: Making Smaller Language Models Competent Subgraph Retrievers for Multi-hop KGQA
Huang, W., Zhou, G., Wang, H., Vougiouklis, P., Lapata, M., & Pan, J. Z. (2024) · Findings of the Association for Computational Linguistics: EMNLP 2024 · doi:10.18653/v1/2024.findings-emnlp.927 · Cited by 2
Turning Trash into Treasure: Accelerating Inference of Large Language Models with Token Recycling
Luo, X., Wang, Y., Zhu, Q., Zhang, Z., Zhang, X., Yang, Q., & Xu, D. (2025) · Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2025.acl-long.338 · Cited by 2
GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLM
Kang, H., Zhang, Q., Kundu, S., Jeong, G., Liu, Z., Krishna, T., & Zhao, T. (2024) · arXiv · doi:10.48550/arxiv.2403.05527 · Cited by 2
“We use GEAR as a plug-and-play KV cache compression for generative inference with various LLM models (including LLaMA2-7B/13B Implementation optimization and details.”
BackgroundMethodOh, G., Kim, S., Park, S., & Kim, B.-H. (2026) · Journal of Medical Internet Research · doi:10.2196/90693 · Cited by 7
MiMo: Unlocking the Reasoning Potential of Language Model - From Pretraining to Posttraining
Xiaomi, L.-C., :, Xia, B., Shen, B., Cici, Zhu, D., Zhang, D., Wang, G., Zhang, H., Liu, H., et al. (2025) · arXiv · doi:10.48550/arxiv.2505.07608 · Cited by 1
“…consists of Grouped-Query Attention (GQA, Ainslie et al. 2023), pre-RMSNorm (Zhang and Sennrich, 2019), SwiGLU activation (Dauphin et al., 2017) and Rotary Positional Embedding (RoPE, Su et al. 2024), similar to Llama (Grattafiori et al., 2024; Touvron et al., 2023) and Qwen (Yang et al., 2024).”
SmartPlay: A Benchmark for LLMs as Intelligent Agents
Wu, Y., Tang, X., Mitchell, T. M., & Li, Y. (2023) · arXiv · doi:10.48550/arxiv.2310.01557 · Cited by 6
“…been one of the core objectives of A.I. (Laird et al., 1987; Russell, 2010) Recently, large language models (LLMs) (Smith et al., 2022; Chowdhery et al., 2022; Ope-nAI, 2023; Manyika; Driess et al., 2023; Touvron et al., 2023) have made remarkable progress in various tasks (Bubeck et al., 2023).”
BackgroundUniMTS: Unified Pre-training for Motion Time Series
Zhang, X., Teng, D., Chowdhury, R. R., Li, S., Hong, D., Gupta, R. K., & Shang, J. (2024) · arXiv · Cited by 9
“In other fields such as natural language processing [39, 56] and computer vision [44, 30], pre-trained foundation models have shown remarkable performance in such settings with insufficient data.”
Mechanistic interpretability for steering vision-language-action models
Häon, B., Stocking, K., Chuang, I., & Tomlin, C. (2025) · arXiv · doi:10.48550/arxiv.2509.00328 · Cited by 1
FormalAlign: Automated Alignment Evaluation for Autoformalization
Lu, J., Wan, Y., Huang, Y., Xiong, J., Liu, Z., & Guo, Z. (2024) · arXiv · doi:10.48550/arxiv.2410.10135 · Cited by 1
Testing the limits of large language models in debating humans
Flamino, J., Modi, M. S., Szymanski, B. K., Cross, B., & Mikolajczyk, C. (2025) · Scientific Reports · doi:10.1038/s41598-025-98378-1 · Cited by 1
DebateQA: Evaluating Question Answering on Debatable Knowledge
Xu, R., Qi, X., Qi, Z., Xu, W., & Guo, Z. (2026) · Findings of the Association for Computational Linguistics: EACL 2026 · doi:10.18653/v1/2026.findings-eacl.44 · Cited by 1
RewardBench 2: Advancing Reward Model Evaluation
Malik, S., Pyatkin, V., Land, S., Morrison, J., Smith, N. A., Hajishirzi, H., & Lambert, N. (2025) · arXiv · doi:10.48550/arxiv.2506.01937 · Cited by 1
“Then, we aggregate the judgments for each set of 4 (or more, for ties) and score those ratings as though they were rewards—by giving the model a point for rating the correct response highest, and scoring two-way ties as partial credit of 0.5, three-way ties as 0.33, and four-way ties as 0.25…”
InfluentialAllaberdiev, S., Chen, X., Mamatov, D., Abdusalomov, A., Ubaydullaev, U., & Khan, A. (2026) · IET Software · doi:10.1049/sfw2/5529107 · Cited by 2
Benchmark on Peer Review Toxic Detection: A Challenging Task with a New Dataset
Luo, M., Peterson, B., Gan, R., Ramalingame, H., Gangrade, N., Dimarogona, A., Banerjee, I., & Howard, P. (2025) · arXiv · doi:10.48550/arxiv.2502.01676 · Cited by 1
AdvAgent: Controllable Blackbox Red-teaming on Web Agents
Xu, C., Kang, M., Zhang, J., Liao, Z., Mo, L., Yuan, M., Sun, H., & Li, B. (2024) · arXiv · doi:10.48550/arxiv.2410.17401 · Cited by 1
VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks
Koh, J. Y., Lo, R., Jang, L., Duvvur, V., Lim, M. C., Huang, P.-Y., Neubig, G., Zhou, S., Salakhutdinov, R., & Fried, D. (2024) · arXiv · doi:10.48550/arxiv.2401.13649 · Cited by 1
Forgetting Private Textual Sequences in Language Models Via Leave-One-Out Ensemble
Liu, Z., & Kalinli, O. (2024) · ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp48485.2024.10446299 · Cited by 1
SmoothAgent: Efficient Long-Horizon LLM-Based Agent Serving with Lookahead Context Engineering
Pan, Z., Wang, Q., Hu, Z., Chen, C., Guan, Y., Zhou, Y., Swanson, S., & Ding, Y. (2026) · arXiv · doi:10.48550/arxiv.2607.00151
“Both models use grouped-query attention (GQA) [62] and dense feed-forward layers, with a maximum context length of 32K tokens.”
xLSTM 7B: A Recurrent LLM for Fast and Efficient Inference
Beck, M., Pöppel, K., Lippe, P., Kurle, R., Blies, P. M., Klambauer, G., Böck, S., & Hochreiter, S. (2025) · arXiv · doi:10.48550/arxiv.2503.13427 · Cited by 9
“Many works report that re-placing the LayerNorm by RMSNorm at the input of each layer (e.g. in the pre-norm setting (Xiong et al., 2020)) improves training stability for Transformers (OLMo et al., 2025; Touvron et al., 2023; Gemma Team, 2024a; Yang et al., 2024a) and Mamba models (Zuo et al., 2024).”
ConceptFormer: Towards Efficient Use of Knowledge-Graph Embeddings in Large Language Models
Barmettler, J., Bernstein, A., & Rossetto, L. (2025) · arXiv · doi:10.48550/arxiv.2504.07624 · Cited by 3
An, Y., Lu, C., Zhu, K., Yu, T., Zhao, C., Wu, H., Tang, M., & Wang, J. (2026) · arXiv · doi:10.48550/arxiv.2605.08840
“We evaluate ReST-KV on five open-source LLMs spanning two mainstream attention architectures: (1) Multi-head attention , Llama2-Chat (Touvron et al., 2023) and Gemma-Instruct (Team et al., 2024); (2) Grouped-query attention , Llama3-Instruct (Dubey et al., 2024), Mistral-Instruct-v0.”
Hu, C., Cao, P., Chen, Y., Liu, K., & Zhao, J. (2024) · arXiv · doi:10.48550/arxiv.2408.07413 · Cited by 3
“In large language models (LLMs), outdated or incorrect knowledge may persist (Radford et al. 2019; Wang and Komatsuzaki 2022; Biderman et al. 2023; Touvron et al. 2023).”
Knowledge Decoupling via Orthogonal Projection for Lifelong Editing of Large Language Models
Xu, H., Lan, P., Yang, E., Guo, G., Zhao, J., Jiang, L., & Wang, X. (2025) · Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2025.acl-long.646
Back into Plato's Cave: Examining Cross-modal Representational Convergence at Scale
Koepke, A. S., Zverev, D., Ginosar, S., & Efros, A. A. (2026) · arXiv · doi:10.48550/arxiv.2604.18572
“For the trend analysis in Section D, we extend this set with 36 recent models: LLaMA-2 (7B– 70B) [94], Llama-3/3.1 [29], OLMo-2/3 [72], Ministral-3 (3B–14B) [59], Gemma-2 (2B–27B)[24], Gemma-3 (270M–27B) [25], DeepSeek-R1-Distill (1.5B–70B) [15], Qwen3 (1.7B–32B) [77], Fal-con3 (7B, 10B) [90],…”
InfluentialRethinking Mixture-of-Agents: Is Mixing Different Large Language Models Beneficial?
Li, W., Lin, Y., Xia, M., & Jin, C. (2025) · arXiv · doi:10.48550/arxiv.2502.00674 · Cited by 62
“5-72B-Chat (Bai et al., 2023), WizardLM-8x22B (Xu et al., 2023), LLaMA-3-70B-Instruct (Touvron et al., 2023), Mixtral-8x22B-Instruct-v0.”
Forouzandeh, S., Peng, W., Moradi, P., Yu, X., & Jalili, M. (2025) · arXiv · doi:10.48550/arxiv.2512.18950 · Cited by 16
Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization
Hsieh, C.-Y., Chuang, Y.-S., Li, C.-L., Wang, Z., Le, L. T., Kumar, A., Glass, J., Ratner, A., Lee, C.-Y., Krishna, R., et al. (2024) · arXiv · doi:10.48550/arxiv.2406.16008
“OpenAI, 2022; Touvron et al., 2023), their knowledge is inherently limited in its pretraining data, and they are observed to struggle in handling knowledge intensive tasks (Petroni et al., 2020).”
Designing reproducible large-language-model-assisted scientific analyses
Dunn, C. W., Schultz, D. T., & Musser, J. M. (2026) · Patterns · doi:10.1016/j.patter.2026.101644
“A model whose weights are published 33–36 can, in principle, be retrieved and rerun by others later, giving the runtime a snap-shottable character more like that of conventional software environments and narrowing the shelf-life gap that closed-weight models create.”
Accurate and Efficient Long-Term Memory for LLM Agents
Zhao, Z., Guo, X., Lv, L., Wang, M., Li, M., & Li, S. (2026) · arXiv · doi:10.48550/arxiv.2607.16211
“5-plus-2026-02-15); generalization to other architectures should be verified [Touvron et al., 2023, Achiam et al., 2023].”
MesaNet: Sequence Modeling by Locally Optimal Test-Time Training
von Oswald, J., Scherrer, N., Kobayashi, S., Versari, L., Yang, S., Mittal, S., Schlegel, M., Maile, K., Schimpf, Y., Sieberling, O., et al. (2025) · arXiv
Training Compute-Optimal Transformer Models: A Review of Scaling Laws, Allocation, and Practice
Memming, A. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7142719
Kummara, H. (2026) · Cambridge Open Engage (preprint) · doi:10.33774/coe-2026-xpkvb
Aruna, K. B., Kaliraj, V., Sudha, I., & Sureka, V. (2026) · EAI Endorsed Transactions on Internet of Things · doi:10.4108/eetiot.13349
Post-Incorporating Code Structural Knowledge Into Pretrained Models via ICL for Code Translation
Du, Y., Sun, H., & Li, M. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3605768
Exploring Optimal Prefetching Sizes in RaLMSpec to Enhance Retrieval-Augmented Generation Efficiency
Liu, S. (2025) · Highlights in Science, Engineering and Technology · doi:10.54097/hhff9g78
Adaptive hybrid speculative decoding for accelerating large language model inference
Yong, Y., Yang, T. T., Gao, S. S., & Yin, Y. H. (2026) · Neurocomputing · doi:10.1016/j.neucom.2026.133760
Exploring the Capabilities of LLMs for Code Change Related Tasks
Fan, L., Liu, J., Liu, Z., Lo, D., Xia, X., & Li, S. (2024) · arXiv · Cited by 41
Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation
Heineman, D., Hofmann, V., Magnusson, I., Gu, Y., Smith, N. A., Hajishirzi, H., Lo, K., & Dodge, J. (2025) · arXiv · doi:10.48550/arxiv.2508.13144
Metadata
- Authors
- Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., Bikel, D., Blecher, L., Ferrer, C. C., Chen, M., Cucurull, G., Esiobu, D., Fernandes, J., Fu, J., Fu, W., Fuller, B., Gao, C., Goswami, V., Goyal, N., Hartshorn, A., Hosseini, S., Hou, R., Inan, H., Kardas, M., Kerkez, V., Khabsa, M., Kloumann, I., Korenev, A., Koura, P. S., Lachaux, M.-A., Lavril, T., Lee, J., Liskovich, D., Lu, Y., Mao, Y., Martinet, X., Mihaylov, T., Mishra, P., Molybog, I., Nie, Y., Poulton, A., Reizenstein, J., Rungta, R., Saladi, K., Schelten, A., Silva, R., Smith, E. M., Subramanian, R., Tan, X. E., Tang, B., Taylor, R., Williams, A., Kuan, J. X., Xu, P., Yan, Z., Zarov, I., Zhang, Y., Fan, A., Kambadur, M., Narang, S., Rodriguez, A., Stojnic, R., Edunov, S., & Scialom, T.
- ORCID
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2307.09288
- OpenAlex
- W4384918448
- Cited by
- 2,604
- References
- 6
- Citations by year
- 2026: 173 · 2025: 808 · 2024: 1373 · 2023: 248
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Keywords
- large language models, clinical decision support, offline deployment, context retention, allergy documentation, discrepancy detection, treatment escalation, open-weight language models, resource-constrained environments, synthetic patient data
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
