Densing law of LLMs
Open accessAbstract
Abstract Large language models (LLMs) have emerged as a milestone in artificial intelligence. The scaling law indicates that the performance of LLMs can continually improve as the model size increases, which poses challenges for training and deployment. Despite numerous efforts to improve LLM efficiency, there is no general consensus on development trends and evaluation metrics for efficiency of LLMs with different scales. To address this tension between model performance and efficiency, we introduce the concept of capability density as a metric to evaluate the quality of the LLMs and describe the trend of LLMs in terms of both effectiveness and efficiency. Intuitively, capability density can be understood as the capability contained within each unit of model parameters. Capability density provides a unified framework for assessing both model performance and efficiency. Here we show an empirical observation, called the ‘densing law’, that the capability density of LLMs grows exponentially over time. More specifically, using widely used benchmarks for evaluation, the maximum capability density of open-source LLMs doubles approximately every 3.5 months. This reveals that both parameter requirements and inference costs of LLMs for achieving equivalent performance decrease exponentially, offering insights for efficient LLM development strategies.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
These findings are consistent with broader trends in LLM development, where model performance has improved with increases in model scale, quality of training data, or information availability.
“These findings are consistent with broader trends in LLM development, where model performance has improved with increases in model scale, quality of training data, or information availability.117”
From Large Language Models in Pharmacist-Led Direct Patient Care: A Rapid Review of Published Evaluations (2026) · cites this paper as [117]
The Abstract says LLM performance improves as model parameters and the amount of training data increase. It does not establish that improvements result from higher-quality training data or greater information availability; those are distinct from data volume. Page numbers are not available in our copy. (LLM, conf=0.87)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486 · see citing works
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602 · see citing works
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676 · cited 1× · see citing works
“In recent years, large language models (LLMs) have garnered wide-spread attention in the field of artificial intelligence (AI), demonstrating remarkable improvements across various tasks 1–5 .”
Llama 2: Open Foundation and Fine-Tuned Chat Models
Touvron, H., Martin, L., Stone, K., Albert, P., Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S., Bhargava, P., Bhosale, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.09288 · Cited by 2,604 · cited 1× · see citing works
“Specifically, we evaluated the density of the following models: Llama series of models 4,11,63 , Falcon 64 , MPT 65 , Phi series of models 66–68 , Mistral series of models 69 , StableLM 70 , TinyLlama 71 , MiniCPM series of models 17 , DeepSeek series of models 72 and Gemma series of models 45,73 .”
Efficient Memory Management for Large Language Model Serving with PagedAttention
Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J., Zhang, H., & Stoica, I. (2023) · Proceedings of the 29th Symposium on Operating Systems Principles · doi:10.1145/3600006.3613165 · Cited by 1,817 · see citing works
Efficient Memory Management for Large Language Model Serving with PagedAttention
Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J. E., Zhang, H., & Stoica, I. (2023) · arXiv · doi:10.48550/arxiv.2309.06180 · Cited by 1,802 · see citing works
Scaling Laws for Neural Language Models
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · Cited by 1,749 · cited 4× · see citing works
“The scaling law for LLMs further reveals that model performance continues to improve as model parameters and training data increase 6–8 .”
InfluentialDeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning
Guo, D., Yang, D., Zhang, H., Song, J., Wang, P., Zhu, Q., Xu, R., Zhang, R., Ma, S., Bi, X., et al. (2025) · Nature · doi:10.1038/s41586-025-09422-z · Cited by 1,701 · see citing works
Pre-trained models for natural language processing: A survey
Qiu, X., Sun, T., Xu, Y., Shao, Y., Dai, N., & Huang, X. (2020) · Science China Technological Sciences · doi:10.1007/s11431-020-1647-3 · Cited by 1,591 · see citing works
Evaluating Large Language Models Trained on Code
Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. D. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al. (2021) · arXiv · doi:10.48550/arxiv.2107.03374 · Cited by 1,436 · cited 4× · see citing works
“Following widely used settings, MMLU, BBH, MATH, HumanEval and MBPP were evaluated under the 5-shot, 3-shot, 4-shot, 0-shot and 3-shot settings, respectively.”
InfluentialPre-trained models: Past, present and future
Han, X., Zhang, Z., Ding, N., Gu, Y., Liu, X., Huo, Y., Qiu, J., Yao, Y., Zhang, A., Zhang, L., et al. (2021) · AI Open · doi:10.1016/j.aiopen.2021.08.002 · Cited by 994 · see citing works
The Falcon Series of Open Language ModelsPreprint
Almazrouei, E., Alobeidli, H., AlShamsi, A., Cappelli, A., Cojocaru, R.-A., Debbah, M., Goffinet, É., Hesslow, D., Launay, J., Malartic, Q., et al. (2023) · arXiv.org · doi:10.48550/arxiv.2311.16867 · Cited by 813 · see citing works
Textbooks Are All You Need II: phi-1.5 technical reportPreprint
Li, Y.-F., Bubeck, S., Eldan, R., Giorno, A. D., Gunasekar, S., & Lee, Y. (2023) · arXiv.org · doi:10.48550/arxiv.2309.05463 · Cited by 708 · see citing works
Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Vaughan, A., et al. (2024) · arXiv · doi:10.48550/arxiv.2407.21783 · Cited by 505 · cited 2× · see citing works
“Specifically, we evaluated the density of the following models: Llama series of models 4,11,63 , Falcon 64 , MPT 65 , Phi series of models 66–68 , Mistral series of models 69 , StableLM 70 , TinyLlama 71 , MiniCPM series of models 17 , DeepSeek series of models 72 and Gemma series of models 45,73 .”
GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
Ainslie, J., Lee-Thorp, J., de Jong, M., Zemlyanskiy, Y., Lebron, F., & Sanghai, S. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.298 · Cited by 454 · see citing works
GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
Ainslie, J., Lee-Thorp, J., de Jong, M., Zemlyanskiy, Y., Lebrón, F., & Sanghai, S. (2023) · arXiv · doi:10.48550/arxiv.2305.13245 · Cited by 447 · see citing works
Program Synthesis with Large Language Models
Austin, J., Odena, A., Nye, M., Bosma, M., Michalewski, H., Dohan, D., Jiang, E., Cai, C., Terry, M., Le, Q., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07732 · Cited by 388 · cited 5× · see citing works
“Following widely used settings, MMLU, BBH, MATH, HumanEval and MBPP were evaluated under the 5-shot, 3-shot, 4-shot, 0-shot and 3-shot settings, respectively.”
InfluentialGemma: Open Models Based on Gemini Research and Technology
Gemma Team, Mesnard, T., Hardin, C., Dadashi, R., Bhupatiraju, S., Pathak, S., Sifre, L., Rivière, M., Kale, M. S., Love, J., et al. (2024) · arXiv · doi:10.48550/arxiv.2403.08295 · Cited by 284 · see citing works
Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them
Suzgun, M., Scales, N., Schärli, N., Gehrmann, S., Tay, Y., Chung, H. W., Chowdhery, A., Le, Q., Chi, E., Zhou, D., et al. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.824 · Cited by 253 · see citing works
Scaling Language Models: Methods, Analysis & Insights from Training Gopher
Rae, J. W., Borgeaud, S., Cai, T., Millican, K., Hoffmann, J., Song, F., Aslanides, J., Henderson, S., Ring, R., Young, S., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.11446 · Cited by 235 · cited 1× · see citing works
“Guided by this trend, PaLM 540B 10 , Gopher-280B 47 and Llama-3-405B 11 achieve great improvements on various natural language processing tasks.”
Scaling Laws for Autoregressive Generative Modeling
Henighan, T., Kaplan, J., Katz, M., Chen, M., Hesse, C., Jackson, J., Jun, H., Brown, T. B., Dhariwal, P., Gray, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.14701 · Cited by 150 · see citing works
Gemma 2: Improving Open Language Models at a Practical Size
Gemma Team, Riviere, M., Pathak, S., Sessa, P. G., Hardin, C., Bhupatiraju, S., Hussenot, L., Mesnard, T., Shahriari, B., Ramé, A., et al. (2024) · arXiv · doi:10.48550/arxiv.2408.00118 · Cited by 149 · cited 2× · see citing works
“Specifically, we evaluated the density of the following models: Llama series of models 4,11,63 , Falcon 64 , MPT 65 , Phi series of models 66–68 , Mistral series of models 69 , StableLM 70 , TinyLlama 71 , MiniCPM series of models 17 , DeepSeek series of models 72 and Gemma series of models 45,73 .”
Self-citationPowerInfer: Fast Large Language Model Serving with a Consumer-grade GPU
Song, Y., Mi, Z., Xie, H., & Chen, H. (2024) · ACM SIGOPS Symposium on Operating Systems Principles (SOSP) · doi:10.1145/3694715.3695964 · Cited by 146 · see citing works
Gunasekar, S., Zhang, Y., Aneja, J., Mendes, C. C. T., Del Giorno, A., Gopi, S., Javaheripi, M., Kauffmann, P., de Rosa, G., Saarikivi, O., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.11644 · Cited by 138 · cited 1× · see citing works
“Specifically, we evaluated the density of the following models: Llama series of models 4,11,63 , Falcon 64 , MPT 65 , Phi series of models 66–68 , Mistral series of models 69 , StableLM 70 , TinyLlama 71 , MiniCPM series of models 17 , DeepSeek series of models 72 and Gemma series of models 45,73 .”
DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
DeepSeek-AI, Liu, A., Feng, B., Wang, B., Wang, B., Liu, B., Zhao, C., Dengr, C., Ruan, C., Dai, D., et al. (2024) · arXiv · doi:10.48550/arxiv.2405.04434 · Cited by 109 · cited 1× · see citing works
“Specifically, we evaluated the density of the following models: Llama series of models 4,11,63 , Falcon 64 , MPT 65 , Phi series of models 66–68 , Mistral series of models 69 , StableLM 70 , TinyLlama 71 , MiniCPM series of models 17 , DeepSeek series of models 72 and Gemma series of models 45,73 .”
Self-citationStable LM 2 1.6B Technical ReportPreprint
Bellagente, M., Tow, J., Mahan, D., Phung, D., Zhuravinskyi, M., Adithyan, R., Baicoianu, J., Brooks, B., Cooper, N., Datta, A., et al. (2024) · arXiv.org · doi:10.48550/arxiv.2402.17834 · Cited by 97 · see citing works
Survey on Knowledge Distillation for Large Language Models: Methods, Evaluation, and Application
Yang, C., Zhu, Y. T., Wang, L., Wang, Y., Chen, Q., Gao, C., Yan, B., & Chen, Y. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3699518 · Cited by 93 · see citing works
MiniCPM4: Ultra-Efficient LLMs on End DevicesPreprint
Xiao, C.-J., Li, Y., Han, X., et al. (2025) · arXiv.org · doi:10.48550/arxiv.2506.07900 · Cited by 60 · cited 1× · see citing works
“…devices (for example, personal computers and smartphones) to serve as personal assistants, which requires models to be more efficient and compact 15–19 ; and (3) the inference scaling law indicates that allowing LLMs to generate more tokens for ‘thinking’ during the inference stage is crucial for…”
Self-citationA Survey on Knowledge Distillation of Large Language ModelsPreprint
Xu, X., Li, M., Tao, C., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2402.13116 · Cited by 55 · cited 1× · see citing works
“LLMs are often constrained by high inference costs, making it challenging to run them on consumer devices 40–43 .”
Skywork: A More Open Bilingual Foundation ModelPreprint
Wei, T., Zhao, L., Zhang, L., Zhu, B., Wang, L., Yang, H., Li, B., Cheng, C., Lü, W., Hu, R., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.19341 · Cited by 53 · cited 1× · see citing works
“Notably, many pretrained models also introduce supervised fine-tuning data-sets in the pretraining phase, leading to the test set contamination issue 74,75 .”
Compact Language Models via Pruning and Knowledge Distillation
Muralidharan, S., Sreenivas, S. T., Joshi, R., Chochowski, M., Patwary, M. A., Shoeybi, M., Catanzaro, B., Kautz, J., & Molchanov, P. (2024) · neural information processing systems · doi:10.52202/079017-1299 · Cited by 53 · see citing works
Efficient GPT-4V level multimodal large language model for deployment on edge devices
Yao, Y., Yu, T., Zhang, A., Wang, C., Cui, J., Zhu, H., Cai, T., Chen, C., Li, H., Zhao, W., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-61040-5 · Cited by 50 · see citing works
OpenAI, :, Jaech, A., Kalai, A., Lerer, A., Richardson, A., El-Kishky, A., Low, A., Helyar, A., Madry, A., et al. (2024) · arXiv · Cited by 50 · see citing works
PowerInfer-2: Fast Large Language Model Inference on a SmartphonePreprint
Xue, Z., Song, Y., Mi, Z., Zheng, X., Xia, Y., & Chen, H. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2406.06282 · Cited by 48 · cited 1× · see citing works
“In addition, because the parameter size of MoE models often exceeds the storage capacity of a single end device chip, end-side MoE inference typically employs offloading strategies 16,56 , which substantially increases memory access time during inference.”
Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters
Snell, C., Lee, J., Xu, K., & Kumar, A. (2024) · arXiv · doi:10.48550/arxiv.2408.03314 · Cited by 27 · cited 1× · see citing works
“…to be more efficient and compact 15–19 ; and (3) the inference scaling law indicates that allowing LLMs to generate more tokens for ‘thinking’ during the inference stage is crucial for improving performance in complex reasoning tasks 20–22 , further increasing the demand for efficient inference.”
Apple Intelligence Foundation Language ModelsPreprint
Apple (2024) · arXiv.org · doi:10.48550/arxiv.2407.21075 · Cited by 17 · see citing works
Toward Inference-optimal Mixture-of-Expert Large Language ModelsPreprint
Yun, L., Zhuang, Y., Fu, Y., et al. (2024) · arXiv.org · doi:10.48550/arxiv.2404.02852 · Cited by 17 · see citing works
Large Language Monkeys: Scaling Inference Compute with Repeated Sampling
Brown, B., Juravsky, J., Ehrlich, R., Clark, R., Le, Q. V., Ré, C., & Mirhoseini, A. (2024) · arXiv · doi:10.48550/arxiv.2407.21787 · Cited by 14 · cited 1× · see citing works
“…to be more efficient and compact 15–19 ; and (3) the inference scaling law indicates that allowing LLMs to generate more tokens for ‘thinking’ during the inference stage is crucial for improving performance in complex reasoning tasks 20–22 , further increasing the demand for efficient inference.”
MMLU-CF: A Contamination-free Multi-task Language Understanding Benchmark
Zhao, Q., Huang, Y., Lv, T., Cui, L., Sun, Q., Mao, S., Zhang, X., Xin, Y., Yin, Q., Li, S., et al. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.656 · Cited by 11 · see citing works
Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?
Yue, Y., Chen, Z., Lu, R., Zhao, A., Wang, Z., Yue, Y., Song, S., & Huang, G. (2025) · arXiv · doi:10.48550/arxiv.2504.13837 · Cited by 3 · cited 1× · see citing works
“Existing research 54 has shown that the capability ceiling of LLMs after reinforcement learning is often constrained by the underlying base model’s capabilities.”
OpenAI, Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.08774 · cited 1×
“In recent years, large language models (LLMs) have garnered wide-spread attention in the field of artificial intelligence (AI), demonstrating remarkable improvements across various tasks 1–5 .”
Flagged first, then by citations
Research works printed but not linked: 27
- Hoffmann, J. et al. Training compute-optimal large language models. In Advances in Neural Information Processing Systems (eds Mohamed, S. et al.) 30016-30030 (NeurIPS, 2022).
- Brown, T. et al. Language models are few-shot learners. In Advances in Neural Information Processing Systems (eds Larochelle, H. et al.) 1877-1901 (NeurIPS, 2020).
- Sardana, N., Portes, J., Doubov, S. & Frankle, J. Beyond Chinchilla-optimal: accounting for inference in language model scaling laws. In International Conference on Machine Learning (eds Salakhutdinov, R. et al.) 43445-43460 (PMLR, 2024).
- Hu, S. et al. MiniCPM: unveiling the potential of small language models with scalable training strategies. In First Conference on Language Modeling (eds Choi, Y. et al.) (COLM, 2024).
- Hendrycks, D. et al. Measuring massive multitask language understanding. In International Conference on Learning Representations (eds Mohamed, S. et al.) (ICLR, 2021).
- Hendrycks, D. et al. Measuring mathematical problem solving with the MATH dataset. In Neural Information Processing Systems Track on Datasets and Benchmarks (eds Vanschoren, J. & Yeung, S.) (NeurIPS, 2021).
- Dao, T., Fu, D. Y., Ermon, S., Rudra, A. & Ré, C. Flashattention: fast and memory-efficient exact attention with io-awareness. In Advances in Neural Information Processing Systems (eds Koyejo, S. et al.) 16344-16359 (NeurIPS, 2022).
- Dao, T. Flashattention-2: faster attention with better parallelism and work partitioning. In International Conference on Learning Representations (eds Kim, B. et al.) (ICLR, 2024).
- Liu, Z. et al. Deja vu: contextual sparsity for efficient LLMs at inference time. In International Conference on Machine Learning (eds Krause, A. et al.) 22137-22176 (PMLR, 2023).
- Xiao, C. et al. Configurable foundation models: building LLMs from a modular perspective. Preprint at https://arxiv.org/abs/ 2409.02877 (2024).
- Ma, X., Fang, G. & Wang, X. LLM-Pruner: on the structural pruning of large language models. In Advances in Neural Information Processing Systems (eds Oh, A. et al.) 21702-21720 (NeurIPS, 2023).
- Sun, M., Liu, Z., Bair, A. & Kolter, J. Z. A simple and effective pruning approach for large language models. In International Conference on Learning Representations (eds Kim, B. et al.) (ICLR, 2024).
- Frantar, E., Ashkboos, S., Hoefler, T. & Alistarh, D. OPTQ: accurate quantization for generative pre-trained transformers. In International Conference on Learning Representations (eds Liu, Y. et al.) (ICLR, 2023).
- Vaswani, A. et al. Attention is all you need. In Advances in Neural Information Processing Systems (eds Guyon, I. et al.) 5998-6008 (NeurIPS, 2017).
- Liu, A. et al. Deepseek-v3 technical report. Preprint at https://arxiv. org/abs/2412.19437 (2024).
- Fedus, W., Zoph, B. & Shazeer, N. Switch transformers: scaling to trillion parameter models with simple and efficient sparsity. J. Mach. Learn. Res. 23, 120 (2022).
- Zelikman, E. et al. Quiet-star: language models can teach themselves to think before speaking. In First Conference on Language Modeling (eds Choi, Y. et al.) (COLM, 2024).
- Wei, J. et al. Emergent abilities of large language models. Trans. Mach. Learn. Res. (2022).
- Dettmers, T., Pagnoni, A., Holtzman, A. & Zettlemoyer, L. QLORA: efficient finetuning of quantized LLMs. In Advances in Neural Information Processing Systems (eds Oh, A. et al.) 10088-10115 (NeurIPS, 2023).
- Xiao, G. et al. SmoothQuant: accurate and efficient post-training quantization for large language models. In International Conference on Machine Learning (eds Krause, A. et al.) 38087-38099 (PMLR, 2023).
- Liu, J., Xia, C. S., Wang, Y. & Zhang, L. Is your code generated by ChatGPT really correct? Rigorous evaluation of large language models for code generation. In Advances in Neural Information Processing Systems (eds Oh, A. et al.) 21558-21572 (NeurIPS, 2023).
- Wei, J. et al. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems (eds Koyejo, S. et al.) 24824-24837 (NeurIPS, 2022).
- Abdin, M. et al. Phi-3 technical report: a highly capable language model locally on your phone. Preprint at https://arxiv.org/abs/ 2404.14219 (2024).
- Jiang, A. Q. et al. Mistral 7B. Preprint at https://arxiv.org/abs/ 2310.06825 (2023).
- Zhang, P., Zeng, G., Wang, T. & Lu, W. Tinyllama: an open-source small language model. Preprint at https://arxiv.org/abs/ 2401.02385 (2024).
- Dominguez-Olmedo, R., Dorner, F. E. & Hardt, M. Training on the test task confounds evaluation and emergence. In International Conference on Learning Representations (eds Yue, Y. et al.) (ICLR, 2025).
- Lewis, P. et al. Retrieval-augmented generation for knowledge- intensive NLP tasks. In Advances in Neural Information Processing Systems (eds Larochelle, H. et al.) 9459-9474 (2020).
Cited by
Sorted by citations
LAIGNELOT, F., Martin, G. L., OSSMAN, M., Pingeon, O., BOUBAKER, A., PICOVSCHI, E., Jia, K. I. M., Tannier, X., Cohen, J. F., & Dechartres, A. (2026) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2026.112221 · Cited by 25 · see citing works
The Cybernetic Teammate: A Field Experiment on Generative AI and Teamwork
Dell’Acqua, F., Ayoubi, C., Lifshitz‐Assaf, H., Sadun, R., Mollick, E., Mollick, L., Han, Y., Goldman, J., Nair, H., Taub, S., et al. (2026) · Organization Science · doi:10.1287/orsc.2025.20702 · Cited by 23 · see citing works
Li, P., Patel, A., Vallamchetla, S. K., Heninger, H., Contractor, H., Tao, C., & Cheung, J. (2026) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocag056 · Cited by 3 · see citing works
Semantic Retention and Extreme Compression in LLMs: Can We Have Both?
Laborde, S., Cousseau, M., Yaacoub, A., & Prévost, L. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11227279 · Cited by 1 · see citing works
Yu, Y., Ren, L., Feng, J., Zhang, Y., & Liu, T. (2026) · Genome biology · doi:10.1186/s13059-026-04134-7 · Cited by 1 · see citing works
“RNABERT performed comparatively poorly, likely reflecting its smaller pretraining corpus, reduced model capacity, and more limited sequence diversity, highlighting the importance of data scale and model size in the development of nucleic acid foundation models [48].”
Gemmi, G., Polese, M., & Melodia, T. (2026) · International Conference on Computer Communications and Networks (ICCCN) · doi:10.1109/icccn69946.2026.11662985 · Cited by 1 · see citing works
“Following Xiao et al. [15], we introduce a density scaling parameter ρ dens (annual capability density growth factor) that captures the growth of LLM capability density over time: the same hardware delivers more capable output as models improve.”
InfluentialLarge language model research: Knowledge structure, thematic evolution, and future research agendaPreprint
Cao, X., Yang, B., Yang, X., Wang, Q., & Cao, N. (2026) · Research Square · doi:10.21203/rs.3.rs-8876513/v1 · Cited by 1 · see citing works
Large Language Models in Pharmacist-Led Direct Patient Care: A Rapid Review of Published Evaluations
Yakob, R. A., Bamgboje-Ayodele, A., Collins, J. C., & Aslani, P. (2026) · Research in Social and Administrative Pharmacy · doi:10.1016/j.sapharm.2026.09.007
Shi, T., Li, Y., Li, Z., Liu, Q., Zhou, J., Xu, W., Li, Y., Dai, D., He, R., Zhou, W., et al. (2026) · Nature Machine Intelligence · doi:10.1038/s42256-026-01298-6
“Yet, such large-scale models involve substantial training overhead and deployment costs, restricting their accessibility to a few well-funded institutions and hindering real-time, interactive workflows 30,32,33 .”
Ploskonka, M. (2026) · Acta Informatica Pragensia · doi:10.18267/j.aip.313
On-device large language model-powered intelligent copilot for on-site construction tutorials
Yan, X., Yang, X., Tang, Z., Pu, H., & Zhang, S. (2026) · Advanced Engineering Informatics · doi:10.1016/j.aei.2026.104792
Neural decoding of speech using deep neural ensemblesPreprint
Yoon, S., Avansino, D. T., Madugula, S. S., Levin, A. D., Fan, C., Meschede-Krasa, B., Singh, A., Vo, C., Hahn, N. V., Card, N. S., et al. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.06.02.729705
Enabling Edge IoT Intelligence: A Survey on Resource-Constrained Continual Learning
Yang, R., Bie, D., Li, D., Yao, Z., & Xiang, J. (2026) · IEEE Internet of Things Journal · doi:10.1109/jiot.2026.3705393
“While Sparse MoE enhances e ffi ciency, the Densing Law suggests that performance scales with the active expert count [102].”
Emotion recognition from body movement through interpretable motion-aware sequential modeling
Esteban-Romero, S., Martín-Fernández, I., San-Segundo, R., Gil-Martín, M., & Fernández-Martínez, F. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1897208
A Capacitorless Graphene Drain Impact Ionization Memory with Ultralow Write Pulses
Tang, C., Yi, F., Huang, H., Chen, W., Liu, W., Lou, F., Xiong, S., Zhao, Y., & Lü, Y. (2026) · Nano Letters · doi:10.1021/acs.nanolett.6c02543
Reducing Memory Requirements of LLM Inference Through Online rANS Decompression
Granat, V.-W., Žádník, J., Makitalo, M. J., Multanen, J., & Jääskeläinen, P. (2026) · IEEE ... International Conference on Cloud Computing · doi:10.1109/cloud72782.2026.00039
“It has been widely noted that the growth trend in large language model (LLM) parameter counts has enabled improved model capabilities in a variety of tasks [1].”
How nanophotonics can drive optical computing toward practical applications
Chen, Y., Yang, G., Yan, T., Tang, C., Wu, J., & Dai, Q. (2026) · Nature Nanotechnology · doi:10.1038/s41565-026-02264-4
Compact Modeling of DNA-Based High-Density Memories for CMOS Integration
Gorthy, A., Keremane, K. S., Wu, H., Priya, S., Yennawar, N., Poudel, B., & Jha, R. (2026) · IEEE International Midwest Symposium on Circuits and Systems (MWSCAS) · doi:10.1109/mwscas67364.2026.11680874
“However, conventional von Neumann architectures suffer from the memory wall, causing latency, bandwidth, and energy bottlenecks [4].”
Irredundant k-fold cross-validation for reliable, scalable, and energy-efficient model evaluation
Aguilar–Ruiz, J. S. (2026) · Machine Learning with Applications · doi:10.1016/j.mlwa.2026.101001
A Transformer Based Pipeline for Software Requirements Classification
Hoque, M. M., Dipu, A. H., Habib, A., & Abrar, A. (2026) · International Conference on Software and Computer Applications (ICSCA) · doi:10.1145/3801619.3801635
Large language models discover complementary heuristics for combinatorial optimization
Gong, H., Wang, S., Song, D., Sheu, J., & Yan, R. (2026) · Nature Machine Intelligence · doi:10.1038/s42256-026-01307-8
“This perspective sits adjacent to, rather than against, the prevailing scaling-law narrative 19 .”
Amiri, M., Mokari, A., & Bocklitz, T. W. (2026) · Discover Artificial Intelligence · doi:10.1007/s44163-026-02369-8
“This data-intensive approach is increasingly viewed as unsustainable without more data-efficient or novel methods [33, 38, 40, 44, 45, 138].”
From authenticity ecosystems to human–AI hybrid societies
Tchernichovski, O., & Jacoby, N. (2026) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2631482123
CopilotJ: a conversational multi-agent system for intelligent and efficient bioimage analysis
Lu, Z., Yuan, Z., Chen, Y., Chen, Z., Qiu, Z., Ren, H., Sun, L., & Wang, Y. (2026) · Nature Methods · doi:10.1038/s41592-026-03290-9
- Authors
- Xiao, C., Cai, J., Zhao, W., Lin, B., Zeng, G., Zhou, J., Zheng, Z., Han, X., Liu, Z., & Sun, M.
- Affiliations
- Beijing National Research Center for Information Science and Technology (CN)
- Corresponding author
- Han, Xu
- Liu, Zhiyuan
- Sun, Maosong
- Venue
- Nature Machine Intelligence
- Published
- 2025-11-06
- Volume
- 7
- Issue
- 11
- Pages
- 1823-1833
- ISSN
- 2522-5839 (ISSN-L)
- OpenAlex
- W4415954773
- Cited by
- 24
- References
- 12
- Citations by year
- 2026: 23 · 2025: 1
- FWCI
- 9.63
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Nature Machine Intelligence (publishedVersion, cc-by)
- Funders
- Beijing Municipal Science and Technology Commission (Z241100001324025)
- National Natural Science Foundation of China (2022ZD0116312)
- Tsinghua University (Guogiang Institute, Shuimu Tsinghua Scholar Program)
- China Postdoctoral Science Foundation (BX20250388)
- Ministry of Industry and Information Technology of the People's Republic of China (DZBM2024-31-0016)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- large language models, scaling laws, model efficiency, parameter efficiency, inference cost, open-source LLMs, performance-efficiency trade-off
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
