The DOI matches its DataCite record; 1 of the 16 references it cites were found in a registry or an index, and 15 are not checked yet.
DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model
Open accessAbstract
We present DeepSeek-V2, a strong Mixture-of-Experts (MoE) language model characterized by economical training and efficient inference. It comprises 236B total parameters, of which 21B are activated for each token, and supports a context length of 128K tokens. DeepSeek-V2 adopts innovative architectures including Multi-head Latent Attention (MLA) and DeepSeekMoE. MLA guarantees efficient inference through significantly compressing the Key-Value (KV) cache into a latent vector, while DeepSeekMoE enables training strong models at an economical cost through sparse computation. Compared with DeepSeek 67B, DeepSeek-V2 achieves significantly stronger performance, and meanwhile saves 42.5% of training costs, reduces the KV cache by 93.3%, and boosts the maximum generation throughput to 5.76 times. We pretrain DeepSeek-V2 on a high-quality and multi-source corpus consisting of 8.1T tokens, and further perform Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) to fully unlock its potential. Evaluation results show that, even with only 21B activated parameters, DeepSeek-V2 and its chat versions still achieve top-tier performance among open-source models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Such an auxiliary loss can be applied at different granularities (e.g., at mini-batch, device, and sequence levels [40,67]), thereby improving communication efficiency.”
From A Survey of Large Language Models (2026) · cites this paper as [67]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, while the open-source model ecosystem has advanced rapidly on the second axis, it has progressed slowly on the first: many recent models remain within or slightly above the 1T-class parameter regime [145,29,135,120].”
From Kimi K3: Open Frontier Intelligence (2026) · cites this paper as [29]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“To reduce communication when distributing experts across different devices using expert parallelism, device-limited routing has been introduced [67], which first selects a small number of devices and then applies topselection only to the experts on those devices.”
From A Survey of Large Language Models (2026) · cites this paper as [67]
The cited source says what the claim attributes to it.
In the citing paper
“At the model level, KV cache reduction can be achieved through architectural modifications, such as layer-wise KV sharing [298] and multi-head latent attention (MLA) [67,299].”
From A Survey of Large Language Models (2026) · cites this paper as [67]
The cited source says what the claim attributes to it.
In the citing paper
“While this effect aligns with explicit compression methods such as MLA [66] and ASVD [42], the approach described herein achieves this via unified factorization within attention layers, thereby reducing both weight and activation memory simultaneously.”
From LittleBit: Ultra Low-Bit Quantization via Latent Factorization (2025) · cites this paper as [66]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 32
- J. Ainslie, J. Lee-Thorp, M. de Jong, Y. Zemlyanskiy, F. Lebrón, and S. Sanghai. Gqa: Training generalized multi-query transformer models from multi-head checkpoints. arXiv preprint arXiv:2305.13245, 2023. · arXiv
- J. Austin, A. Odena, M. Nye, M. Bosma, H. Michalewski, D. Dohan, E. Jiang, C. Cai, M. Terry, Q. Le, et al. Program synthesis with large language models. arXiv preprint arXiv:2108.07732, 2021. · arXiv
- J. Bai, S. Bai, Y. Chu, Z. Cui, K. Dang, X. Deng, Y. Fan, W. Ge, Y. Han, F. Huang, B. Hui, L. Ji, M. Li, J. Lin, R. Lin, D. Liu, G. Liu, C. Lu, K. Lu, J. Ma, R. Men, X. Ren, X. Ren, C. Tan, S. Tan, J. Tu, P. Wang, S. Wang, W. Wang, S. Wu, B. Xu, J. Xu, A. Yang, H. Yang, J. Yang, S. Yang, Y. Yao, B. Yu, H. Yuan, Z. Yuan, J. Zhang, X. Zhang, Y. Zhang, Z. Zhang, C. Zhou, J. Zhou, X. Zhou, and T. Zhu. Qwen technical report. arXiv preprint arXiv:2309.16609, 2023. · arXiv
- M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. de Oliveira Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, N. Ryder, M. Pavlov, A. Power, L. Kaiser, M. Bavarian, C. Winter, P. Tillet, F. P. Such, D. Cummings, M. Plappert, F. Chantzis, E. Barnes, A. Herbert-Voss, W. H. Guss, A. Nichol, A. Paino, N. Tezak, J. Tang, I. Babuschkin, S. Balaji, S. Jain, W. Saunders, C. Hesse, A. N. Carr, J. Leike, J. Achiam, V. Misra, E. Morikawa, A. Radford, M. Knight, M. Brundage, M. Murati, K. Mayer, P. Welinder, B. McGrew, D. Amodei, S. McCandlish, I. Sutskever, and W. Zaremba. Evaluating large language models trained on code. CoRR, abs/2107.03374, 2021. URL https://arxiv.org/abs/2107.03374. · arXiv
- P. Clark, I. Cowhey, O. Etzioni, T. Khot, A. Sabharwal, C. Schoenick, and O. Tafjord. Think you have solved question answering? try arc, the AI2 reasoning challenge. CoRR, abs/1803.05457, 2018. URL http://arxiv.org/abs/1803.05457. · arXiv
- K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021. · arXiv
- Y. Dubois, B. Galambosi, P. Liang, and T. B. Hashimoto. Length-controlled alpacaeval: A simple way to debias automatic evaluators. arXiv preprint arXiv:2404.04475, 2024. · arXiv
- W. Fedus, B. Zoph, and N. Shazeer. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. CoRR, abs/2101.03961, 2021. URL https://arxiv.org/ abs/2101.03961. · arXiv
- L. Gao, S. Biderman, S. Black, L. Golding, T. Hoppe, C. Foster, J. Phang, H. He, A. Thite, N. Nabeshima, et al. The Pile: An 800GB dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027, 2020. · arXiv
- D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300, 2020. · arXiv
- D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Steinhardt. Mea- suring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874, 2021. · arXiv
- S. Hu, Y. Tu, X. Han, C. He, G. Cui, X. Long, Z. Zheng, Y. Fang, Y. Huang, W. Zhao, et al. Minicpm: Unveiling the potential of small language models with scalable training strategies. arXiv preprint arXiv:2404.06395, 2024. · arXiv
- Y. Huang, Y. Bai, Z. Zhu, J. Zhang, J. Zhang, T. Su, J. Liu, C. Lv, Y. Zhang, J. Lei, et al. C-Eval: A multi-level multi-discipline chinese evaluation suite for foundation models. arXiv preprint arXiv:2305.08322, 2023. · arXiv
- N. Jain, K. Han, A. Gu, W.-D. Li, F. Yan, T. Zhang, S. Wang, A. Solar-Lezama, K. Sen, and I. Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code. arXiv preprint arXiv:2403.07974, 2024. · arXiv
- W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. E. Gonzalez, H. Zhang, and I. Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023.
- D. Lepikhin, H. Lee, Y. Xu, D. Chen, O. Firat, Y. Huang, M. Krikun, N. Shazeer, and Z. Chen. Gshard: Scaling giant models with conditional computation and automatic sharding. In 9th International Conference on Learning Representations, ICLR 2021. OpenReview.net, 2021. URL https://openreview.net/forum?id=qrwe7XHTmYb.
- H. Li, Y. Zhang, F. Koto, Y. Yang, H. Zhao, Y. Gong, N. Duan, and T. Baldwin. CMMLU: Measur- ing massive multitask language understanding in Chinese. arXiv preprint arXiv:2306.09212, 2023. · arXiv
- I. Loshchilov and F. Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. · arXiv
- OpenAI. GPT4 technical report. arXiv preprint arXiv:2303.08774, 2023. · arXiv
- L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730-27744, 2022.
- B. Peng, J. Quesnelle, H. Fan, and E. Shippole. Yarn: Efficient context window extension of large language models. arXiv preprint arXiv:2309.00071, 2023. · arXiv
- P. Qi, X. Wan, G. Huang, and M. Lin. Zero bubble pipeline parallelism. arXiv preprint arXiv:2401.10241, 2023. · arXiv
- S. Rajbhandari, J. Rasley, O. Ruwase, and Y. He. Zero: Memory optimizations toward training tril- lion parameter models. In SC20: International Conference for High Performance Computing, Networking, Storage and Analysis, pages 1-16. IEEE, 2020.
- C. Riquelme, J. Puigcerver, B. Mustafa, M. Neumann, R. Jenatton, A. S. Pinto, D. Keysers, and N. Houlsby. Scaling vision with sparse mixture of experts. In Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, pages 8583-8595, 2021. URL https://proceedings.neurips.cc/paper /2021/hash/48237d9f2dea8c74c2a72126cf63d933-Abstract.html.
- Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, M. Zhang, Y. Li, Y. Wu, and D. Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024. · arXiv
- N. Shazeer. Fast transformer decoding: One write-head is all you need. CoRR, abs/1911.02150, 2019. URL http://arxiv.org/abs/1911.02150. · arXiv
- N. Shazeer, A. Mirhoseini, K. Maziarz, A. Davis, Q. V. Le, G. E. Hinton, and J. Dean. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. In 5th International Conference on Learning Representations, ICLR 2017. OpenReview.net, 2017. URL https: //openreview.net/forum?id=B1ckMDqlg.
- M. Suzgun, N. Scales, N. Schärli, S. Gehrmann, Y. Tay, H. W. Chung, A. Chowdhery, Q. V. Le, E. H. Chi, D. Zhou, et al. Challenging big-bench tasks and whether chain-of-thought can solve them. arXiv preprint arXiv:2210.09261, 2022. · arXiv
- A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polo- sukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
- A. Young, B. Chen, C. Li, C. Huang, G. Zhang, G. Zhang, H. Li, J. Zhu, J. Chen, J. Chang, et al. Yi: Open foundation models by 01. ai. arXiv preprint arXiv:2403.04652, 2024. · arXiv
- C. Zhou, P. Liu, P. Xu, S. Iyer, J. Sun, Y. Mao, X. Ma, A. Efrat, P. Yu, L. Yu, et al. Lima: Less is more for alignment. Advances in Neural Information Processing Systems, 36, 2024.
- J. Zhou, T. Lu, S. Mishra, S. Brahma, S. Basu, Y. Luan, D. Zhou, and L. Hou. Instruction-following evaluation for large language models. arXiv preprint arXiv:2311.07911, 2023. · arXiv
Cited by
Sorted by citations
A Survey of Large Language Models
Zhao, W. X., Zhou, K., Li, J., Tang, T., Dong, Z., Hou, Y., Zhang, B., Min, Y., Zhang, J., Liu, P., et al. (2026) · Frontiers of Computer Science · doi:10.1007/s11704-026-60308-3 · Cited by 2,093
Self-citationDeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning
DeepSeek-AI, Guo, D., Yang, D., Zhang, H., Song, J., Wang, P., Zhu, Q., Xu, R., Zhang, R., Ma, S., et al. (2025) · arXiv · doi:10.48550/arxiv.2501.12948 · Cited by 538
Self-citationA Survey on Mixture of Experts in Large Language Models
Cai, W., Jiang, J., Wang, F., Tang, J., Kim, S., & Huang, J. (2025) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2025.3554028 · Cited by 560
Exploring DeepSeek: A Survey on Advances, Applications, Challenges and Future Directions
Deng, Z., Ma, W., Han, Q., Zhou, W., Zhu, X., Wen, S., & Xiang, Y. (2025) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2025.125498 · Cited by 184
Wang, F., Zhang, Z., Zhang, X., Wu, Z., Mo, T., Lu, Q., Wang, W., Li, R., Xu, J., Tang, X., et al. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3768165 · Cited by 106
Self-citationUAVs meet LLMs: Overviews and perspectives towards agentic low-altitude mobility
Tian, Y., Lin, F., Li, Y., Zhang, T., Zhang, Q., Fu, X., Huang, J., Dai, X., Wang, Y., Tian, C., et al. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103158 · Cited by 139
Self-citationCell-free DNA fragmentomics in cancer
Tsui, W., Jiang, P., & Lo, Y. M. D. (2025) · Cancer Cell · doi:10.1016/j.ccell.2025.09.006 · Cited by 72
Accelerating battery innovation: AI-powered molecular discovery
Gao, Y., Chen, X., Yuan, Y., Chen, Y., Niu, Y.-L., Yao, N., Gao, Y.-B., Li, W. Z., & Zhang, Q. (2025) · Chemical Society Reviews · doi:10.1039/d5cs00053j · Cited by 63
Self-citationDeepSeek vs. ChatGPT: prospects and challenges
Jin, I., Tangsrivimol, J. A., Darzidehkalani, E., Virk, H. U. H., Wang, Z., Egger, J., Hacking, S. M., Glicksberg, B. S., Strauß, M., & Krittanawong, C. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1576992 · Cited by 62
Self-citationAgentic AI: The age of reasoning—A review
Nisa, U., Shirazi, M., Saip, M. A., & Pozi, M. S. M. (2025) · Journal of Automation and Intelligence · doi:10.1016/j.jai.2025.08.003 · Cited by 84
Xu, P., Wu, Y., Jin, K., Chen, X., He, M., & Shi, D. (2025) · Advances in Ophthalmology Practice and Research · doi:10.1016/j.aopr.2025.05.001 · Cited by 53
Self-citationDeepSeek: Paradigm Shifts and Technical Evolution in Large AI Models
Xiong, L., Wang, H., Chen, X., Sheng, L., Xiong, Y., Liu, J., Xiao, Y., Chen, H., Han, Q., & Tang, Y. S. (2025) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2025.125495 · Cited by 50
Self-citationInsights into DeepSeek-V3: Scaling Challenges and Reflections on Hardware for AI Architectures
Zhao, C., Deng, C., Ruan, C., Dai, D., Gao, H., Li, J., Zhang, L., Huang, P., Zhou, S., Ma, S., et al. (2025) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1145/3695053.3731412 · Cited by 92
Self-citationLarge models in medical imaging: Advances and prospects
Fang, M., Wang, Z., Pan, S., Feng, X., Zhao, Y., Hou, D., Wu, L., Xie, X., Zhang, X.-Y., Tian, J., et al. (2025) · Chinese Medical Journal · doi:10.1097/cm9.0000000000003699 · Cited by 43
Self-citationLarge-Scale Hierarchical Causal Discovery via Weak Prior Knowledge
Wang, X., Ban, T., Chen, L., Lyu, D., Zhu, Q., & Chen, H. (2025) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2025.3537832 · Cited by 37
Self-citationMixture of experts (MoE): A big data perspective
Gan, W., Ning, Z., Qi, Z., & Yu, P. S. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103664 · Cited by 52
Aydın, Ö., Karaarslan, E., Erenay, F. S., & Bačanin, N. (2026) · Turkish Journal of Engineering · doi:10.31127/tuje.1838765 · Cited by 69
BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions
Zhuo, T. Y., Vu, M. C., Chim, J., Hu, H., Yu, W., Widyasari, R., Yusuf, I. N. B., Zhan, H., He, J., Paul, I., et al. (2024) · arXiv · doi:10.48550/arxiv.2406.15877 · Cited by 29
Self-citationGou, L., Wang, Y., Gao, L., Liu, S., Wang, M., Chai, Q., Fang, J., Zhan, L., Shen, X., Jiang, T., et al. (2025) · Cell · doi:10.1016/j.cell.2025.06.005 · Cited by 27
Self-citationLi, J., Shen, Y., Zhou, W., & Hui, C. H. E. N. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103665 · Cited by 34
Self-citationPhysical reservoir computing for Edge AI applications
Liu, J., Feng, G., Li, W., Hao, S., Han, S., Zhu, Q., Tian, B., Duan, C., & Chu, J. (2025) · The Innovation Materials · doi:10.59717/j.xinn-mater.2025.100127 · Cited by 23
Self-citationXiao, C., Cai, J., Zhao, W., Lin, B., Zeng, G., Zhou, J., Zheng, Z., Han, X., Liu, Z., & Sun, M. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01137-0 · Cited by 24
“Specifically, we evaluated the density of the following models: Llama series of models 4,11,63 , Falcon 64 , MPT 65 , Phi series of models 66–68 , Mistral series of models 69 , StableLM 70 , TinyLlama 71 , MiniCPM series of models 17 , DeepSeek series of models 72 and Gemma series of models 45,73 .”
Self-citationFire-Flyer AI-HPC: A Cost-Effective Software-Hardware Co-Design for Deep Learning
An, W., Bi, X., Chen, G.-T., Chen, S., Deng, C., Ding, H., Dong, K., Du, Q., Gao, W., Guan, K., et al. (2024) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1109/sc41406.2024.00089 · Cited by 35
Self-citationXie, C., Gao, H., Huang, Y., Xue, Z., Xu, C. B., & Dai, K. (2025) · Earthquake research advances · doi:10.1016/j.eqrea.2025.100378 · Cited by 24
Self-citationDeepSeek-V3.2: Pushing the Frontier of Open Large Language Models
DeepSeek-AI, Liu, A., Mei, A., Lin, B., Xue, B., Wang, B., Xu, B., Wu, B., Zhang, B., Lin, C., et al. (2025) · arXiv · doi:10.48550/arxiv.2512.02556 · Cited by 19
Self-citationCrystal Structure Prediction Meets Artificial Intelligence
Chen, Z., Meng, Z., He, T., Li, H., Cao, J. Z., Xu, L., Xiao, H., Zhang, Y., He, X. D., & Fang, G. (2025) · The Journal of Physical Chemistry Letters · doi:10.1021/acs.jpclett.4c03727 · Cited by 20
Self-citationNdum, Z. N., Tao, J., Ford, J. R., & Liu, Y. (2025) · Energy and AI · doi:10.1016/j.egyai.2025.100555 · Cited by 15
Self-citationMultimodal large language model for wheat breeding: A new exploration of smart breeding
Yang, G., Li, Y., He, Y., Zhou, Z., Ye, L., Fang, H., Luo, Y., & Feng, X. (2025) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2025.03.027 · Cited by 17
Self-citationCOFFE: A Code Efficiency Benchmark for Code Generation
Peng, Y., Wan, J., Li, Y., & Ren, X. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3715727 · Cited by 15
Self-citationThe evolution of language models: From N-Grams to LLMs, and beyond
Raeini, M. (2025) · Natural Language Processing Journal · doi:10.1016/j.nlp.2025.100168 · Cited by 26
Wu, Y., & Tang, F. (2025) · Genome biology · doi:10.1186/s13059-025-03639-x · Cited by 14
Self-citationCKGFuzzer: LLM-Based Fuzz Driver Generation Enhanced By Code Knowledge Graph
Xu, H., Ma, W., Zhou, T., Zhao, Y., Chen, K., Hu, Q., Liu, Y., & Wang, H. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse-companion66252.2025.00079 · Cited by 37
Self-citationLarge Language Models for Combinatorial Optimization: A Systematic Review
Ros, F. D., Soprano, M., Gaspero, L. D., & Roitero, K. (2026) · ACM Computing Surveys · doi:10.1145/3801961 · Cited by 40
FSMoE: A Flexible and Scalable Training System for Sparse Mixture-of-Experts Models
Pan, X., Lin, W., Zhang, L., Shi, S., Tang, Z., Wang, R., Li, B., & Chu, X. (2025) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3669940.3707272 · Cited by 22
Self-citationLLM Knowledge-Driven Target Prototype Learning for Few-Shot Segmentation
Li, P., Liu, F., Jiao, L., Li, S., Liu, X., Chen, P., Li, L., & Hao, Z. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113149 · Cited by 14
Self-citationA Survey of Large Language Models in Mental Health Disorder Detection on Social Media
Ge, Z., Hu, N., Li, D., Wang, Y., Qi, S., Xu, Y., Han, S., & Zhang, J. Y. (2025) · IEEE International Conference on Data Engineering Workshops (ICDEW) · doi:10.1109/icdew67478.2025.00027 · Cited by 15
Self-citationRepoTransBench: A Real-World Multilingual Benchmark for Repository-Level Code Translation
Wang, Y., Wang, Y., Wang, S., Guo, D., Chen, J., Grundy, J., Liu, X., Ma, Y., Mao, M., Zhang, H., et al. (2026) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3645056 · Cited by 12
Self-citationPath to Artificial General Intelligence: Past, present, and future
Sarikaya, R. (2025) · Annual Reviews in Control · doi:10.1016/j.arcontrol.2025.101021 · Cited by 15
From intelligence to autopoiesis: rethinking artificial intelligence through systems theory
Zönnchen, B., Dzhimova, M., & Socher, G. (2025) · Frontiers in Communication · doi:10.3389/fcomm.2025.1585321 · Cited by 17
Towards wireless native big AI model: the mission and approach differ from large language model
Chen, Z., Zhang, Z., Liu, C., & Xing, Z. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4464-8 · Cited by 10
Self-citationKundu, R. K., Khalil, K., Garcia, E., Grassia, E., Calyam, P. P., & Hoque, K. A. (2025) · IEEE Access · doi:10.1109/access.2025.3592030 · Cited by 9
Event segmentation applications in large language model enabled automated recall assessments
Panela, R. A., Barnett, A. J., Barense, M. D., & Herrmann, B. (2025) · Communications Psychology · doi:10.1038/s44271-025-00359-7 · Cited by 10
Are Small Language Models Ready to Compete with Large Language Models for Practical Applications?
Sinha, N., Jain, V., & Chadha, A. (2025) · Proceedings of the 5th Workshop on Trustworthy NLP (TrustNLP 2025) · doi:10.18653/v1/2025.trustnlp-main.25 · Cited by 9
Zhang, C., You, Y., Wang, N., Park, J., & Zhang, L. (2025) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2025.3575272 · Cited by 13
Self-citationGao, C., Chen, X., & Zhang, G. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2025.107803 · Cited by 14
Self-citationLog Parsing Using LLMs with Self-Generated In-Context Learning and Self-Correction
Wu, Y., Yu, S., & Li, Y. (2025) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc66645.2025.00063 · Cited by 11
Self-citationCan foundation language models predict fluid dynamics?
Wang, G., & Cheng, S. L. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.111427 · Cited by 8
Memon, Y., Zeng, F., & Yao, L. (2025) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2025.108270 · Cited by 7
Automatic Generation of Job Safety Reports with Explainable RAG-Based LLMs
Bernardi, M. L., Cimitile, M., Panella, G., Pecori, R., & Simoncelli, G. (2025) · Information Systems Frontiers · doi:10.1007/s10796-025-10634-x · Cited by 10
Zhu, Y., Shu, X., Liu, X., Li, Y., Yi, B., & Ma, D. (2025) · Anesthesiology and Perioperative Science · doi:10.1007/s44254-025-00156-9 · Cited by 9
Self-citationRepairBench: Leaderboard of Frontier Models for Program Repair
Silva, A., & Monperrus, M. (2025) · International Workshop on Large Language Models for Code (LLM4Code) · doi:10.1109/llm4code66737.2025.00006 · Cited by 23
Ji, T., Guo, B., Wu, Y., Guo, Q., Shenlixing, S., Chenzhan, C., Qiu, X., Zhang, Q., & Gui, T. (2025) · Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2025.acl-long.1597 · Cited by 6
Self-citationA Self-Evolving Framework for Multi-Agent Medical Consultation Based on Large Language Models
Chen, K., Qi, J., Huo, J., Tian, P., Meng, F., Yang, X., & Gao, Y. (2025) · ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp49660.2025.10889517 · Cited by 6
Self-citationOpenResearcher: Unleashing AI for Accelerated Scientific Research
Zheng, Y., Sun, S., Qiu, L., Ru, D., Jiayang, C., Li, X., Lin, J., Wang, B., Luo, Y., Pan, R., et al. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2024.emnlp-demo.22 · Cited by 5
Self-citationAn Empirical Study on Commit Message Generation Using LLMs via In-Context Learning
Wu, Y., Wang, Y., Li, Y., Tao, W., Yu, S., Yang, H.-W., Jiang, W., & Li, J. (2025) · International Conference on Software Engineering · doi:10.1109/icse55347.2025.00091 · Cited by 22
Self-citationDeepSeek: implications for data science and management in the AI era
Xu, Z., Liu, S., Huang, W., Shi, J., Xu, F., Tang, X., & Lu, H. (2025) · Data Science and Management · doi:10.1016/j.dsm.2025.09.001 · Cited by 6
Self-citationEvaluating LLMs' divergent thinking capabilities for scientific idea generation with minimal context
Ruan, K., Wang, X., Hong, J., Wang, P., Liu, Y., & Sun, H. (2026) · Nature Communications · doi:10.1038/s41467-026-70245-1 · Cited by 50
Self-citationChurch, K. (2025) · International Journal of Lexicography · doi:10.1093/ijl/ecaf007 · Cited by 5
Wang, Y., Meng, X., Wang, Y., Liang, J., Wei, J., Zhang, H., & Zhao, D. (2024) · Qeios · doi:10.32388/1smziq · Cited by 5
Self-citationExperience-guided multi-agent interpretable framework for radiology report summarization
Li, J., Zhou, T., Zhou, Z., Wei, X., Song, H., Wang, Z., Chen, Y., & Lv, H. (2026) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2025.109078 · Cited by 4
Self-citationFu, Y., Li, B., Li, L., Zhang, W., & Xie, T. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3728947 · Cited by 7
Self-citationKey, Value, Compress: A Systematic Exploration of KV Cache Compression Techniques
Javidnia, N., Rouhani, B. D., & Koushanfar, F. (2025) · Custom Integrated Circuits Conference · doi:10.1109/cicc63670.2025.10983416 · Cited by 7
AI in flow cytometry: Current applications and future directions
Yue, A., Brinkman, R. R., Nash, V., Junker, F., Bogdanoski, G., Divekar, A., Tyznik, A. J., Špidlen, J., Kern, W., Pétriz, J., et al. (2025) · Cytometry Part B Clinical Cytometry · doi:10.1002/cyto.b.22255 · Cited by 7
Toward more economical large-scale foundation models: No longer a game for the few
Wu, Y., Zhang, Z., Wang, F., Xu, Y., & Huang, J. (2025) · The Innovation · doi:10.1016/j.xinn.2025.100832 · Cited by 4
Self-citationDeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence
DeepSeek-AI, Xu, A., Lin, B., Xue, B., Wang, B., Xu, B., Wu, B., Zhang, B., Lin, C., Dong, C., et al. (2026) · arXiv · doi:10.48550/arxiv.2606.19348 · Cited by 4
Self-citationMoEcho: Exploiting Side-Channel Attacks to Compromise User Privacy in Mixture-of-Experts LLMs
Ding, R., Xu, T., Shen, X., Ding, A. A., & Fei, Y. (2025) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3719027.3765174 · Cited by 4
Self-citationXue, J., Wang, S., Yang, J., Guo, X., Shen, J., & Wang, Q. (2026) · JMIR Formative Research · doi:10.2196/90673 · Cited by 3
Self-citationSix advances in research on bilingualism
MacWhinney, B., & Gao, Z. (2025) · Brain and Language · doi:10.1016/j.bandl.2025.105614 · Cited by 3
Qiao, Y., Le, H. Q., Raha, A. D., Tran, P.-N., Adhikary, A., Zhang, M., Nguyen, L. X., Huh, E., Niyato, D. T., & Hong, C. S. (2025) · doi:10.36227/techrxiv.174762883.35612385/v1 · Cited by 3
Self-citationSeed-Coder: Let the Code Model Curate Data for Itself
Seed, B., Zhang, Y., Su, J., Sun, Y., Xi, C., Xiao, X., Zheng, S., Zhang, A., Liu, K., Zan, D., et al. (2025) · arXiv · doi:10.48550/arxiv.2506.03524 · Cited by 3
“Although synthetic data offers an effective and cost-efficient alternative to real-13 world data [Liu et al., 2024a], it often lacks the stylistic diversity observed in actual user prompts.”
Quantifying Variance in Evaluation Benchmarks
Madaan, L., Singh, A. K., Schaeffer, R., Poulton, A., Koyejo, S., Stenetorp, P., Narang, S., & Hupkes, D. (2024) · arXiv · doi:10.48550/arxiv.2406.10229 · Cited by 2
“5 {0.5, 1.8, 4, 7, 14, 32, 72, 110}B (Bai et al., 2023), Pythia {1, 1.4, 2.8, 6.9, 12}B (Biderman et al., 2023), Falcon {7, 40}B (Almazrouei et al., 2023), DeepSeek {7, 67}B (Bi et al., 2024), DeepSeek-MoE 16B (Bi et al., 2024), DeepSeek V2 (DeepSeek-AI, 2024), StableLM {1.6, 3, 7}B (StabilityAI,…”
InfluentialObservational Scaling Laws and the Predictability of Language Model Performance
Ruan, Y., Maddison, C. J., & Hashimoto, T. (2024) · arXiv · doi:10.48550/arxiv.2405.10938 · Cited by 3
Keep the Cost Down: A Review on Methods to Optimize LLM' s KV-Cache Consumption
Shi, L., Zhang, H., Yao, Y., Li, Z., & Zhao, H. (2024) · arXiv · doi:10.48550/arxiv.2407.18003 · Cited by 2
Self-citationSpecialized model initialization and architecture optimization for few-shot code search
Zhang, F., Wu, Q., Peng, M., & Shen, Y. (2024) · Information and Software Technology · doi:10.1016/j.infsof.2024.107571 · Cited by 2
Toward Universal Embodied Planning in Scalable Heterogeneous Field Robots Collaboration and Control
Wan, H., Zhang, Y., Wang, J., Wu, D., Li, M., Chen, X., Deng, Y., Huang, Y., Sun, Z., Li, Z., et al. (2025) · Journal of Field Robotics · doi:10.1002/rob.22522 · Cited by 4
Self-citationXu, P., Wu, H., Wang, J., Lin, R., & Tan, L. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-4298-4_2 · Cited by 2
Large Language Models Imitate Logical Reasoning, but at What Cost?
McGinness, L. P., & Baumgartner, P. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-4969-6_7 · Cited by 4
Pham, M.-K., Mai, T. T., Dao, T. T. P., Tran, M., Crane, M., & Bezbradica, M. (2025) · doi:10.36227/techrxiv.175021755.55836808/v1 · Cited by 3
Online Scheduling for LLM Inference with KV Cache Constraints
Jaillet, P., Jiang, J., Mellou, K., Molinaro, M., Podimata, C., & Zhou, Z. (2025) · arXiv · doi:10.48550/arxiv.2502.07115 · Cited by 1
“Liu et al. [2024] boosts LLM inference efficiency by introducing multi-head latent attention, which reduces the KV cache through low-rank key-value joint compression.”
A Multi-Agent Framework for Accessible Web Code Generation
Aljedaani, W., Shaik, M. A., & Aljohani, A. (2026) · Proceedings of the 23rd International Web for All Conference · doi:10.1145/3800424.3800453 · Cited by 1
Research on Predicting Public Opinion Event Heat Levels Based on Large Language Models
Ren, Y., Zhang, T., Li, W., Zhou, D., Qin, C., & Dong, F. (2024) · doi:10.1109/ccat64370.2024.00013 · Cited by 1
An Aerospace Equipment Operation and Maintenance Scheduling Method Based on LLM and Solver
Xia, T., Wang, H. E., Wang, T., & Zhang, L. (2024) · International Conference on Artificial Intelligence, Robotics, and Communication (ICAIRC) · doi:10.1109/icairc64177.2024.10899990 · Cited by 1
Self-citationA Survey of Mixture Models for Scientific Literature
Deng, Y., Liu, Q., Li, X., Zhang, G., & Zhu, M. Q. (2024) · International Symposium on Artificial Intelligence and Intelligent Manufacturing (AIIM) · doi:10.1109/aiim64537.2024.10934484 · Cited by 1
Self-citationA Survey of Low-Rank Adaptation Techniques
Feng, Z., Xie, Y., Yang, J., Hou, W., & Li, Z. (2025) · International Conference on Advanced Electronic Materials, Computers and Software Engineering (AEMCSE) · doi:10.1109/aemcse65292.2025.11042839 · Cited by 1
Self-citationAutomated Coding Utterances Toward Chinese Course Core Competence with Large Language Models
Yue, M., Qi, T., Wang, M., Ruan, W., He, J. Y. D., Sun, B., & Zheng, G. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-0014-7_19 · Cited by 1
Self-citationM-ReDet: A mamba-based method for remote sensing ship object detection and fine-grained recognition
Liu, X.-H., Feng, C., Zi, S., Qin, Z., & Guan, Q. (2025) · PLoS ONE · doi:10.1371/journal.pone.0330485 · Cited by 1
Self-citationA Lightweight Large Language Model for Personal Sleep Quality Estimation
Liu, S., Chen, Y., & Wang, J. (2025) · IEEE Conference on Industrial Electronics and Applications (ICIEA) · doi:10.1109/iciea65512.2025.11148483 · Cited by 1
Zhang, R., Zhao, X., Han, Y., Yang, Y., Ruan, J., Zhang, J., Chen, D., & Wang, H. (2025) · Transactions on Emerging Telecommunications Technologies · doi:10.1002/ett.70249 · Cited by 3
Self-citationEmpowering college students to select ideal advisors: a text-based recommendation model
Jian, L., Zhou, J., Jian, L., Yin, Y., & Li, L. (2025) · Frontiers in Education · doi:10.3389/feduc.2025.1673956 · Cited by 2
Self-citationRealHD: A High-Quality Dataset for Robust Detection of State-of-the-Art AI-Generated Images
Yu, H., Ye, Y., Rong, J., Xuan, Q., & Ma, C. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3755207 · Cited by 4
Artificial Intelligence at Seventy: From Symbolic Aspirations to Emergent Realities
Ochoa-Barragán, R., Raya-Tapia, A. Y., López-Flores, F. J., Márquez, C. R., & Ponce‐Ortega, J. M. (2026) · Industrial & Engineering Chemistry Research · doi:10.1021/acs.iecr.5c04476 · Cited by 1
HiLFS: FPGA-Orchestrated File System for High-Level Synthesis
Na, Y., Wong, L. Y., DeHon, A. M., & Li, J. (. (2026) · ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (FPGA) · doi:10.1145/3748173.3779207 · Cited by 1
Self-citationParallelsim: an accurate, generic, and efficient simulator for distributed deep learning
Liang, P., Qiao, L., Lai, Z., & Li, D. (2026) · CCF Transactions on High Performance Computing · doi:10.1007/s42514-025-00271-w · Cited by 1
Cross-layer Attention Sharing for Pre-trained Large Language Models
Mu, Y., Wu, Y., Fan, Y., Wang, C. Q., Li, H., Zeng, J., He, Q., Yang, M., Meng, F., Zhou, J., et al. (2026) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl.a.616 · Cited by 25
Self-citationAI for battery-accelerated discovery of high-voltage electrolytes for advanced lithium batteries
Zhan, Y. Z., Chen, N., Li, L., Wu, F., & Chen, R. (2026) · Chemical Society Reviews · doi:10.1039/d4cs01250j · Cited by 1
Self-citationDeepSeek-V3: Architecture and Optimizations-A Practical Review
Zouhdi, Y., & Hdioud, B. (2026) · EPJ Web of Conferences · doi:10.1051/epjconf/202638002002 · Cited by 1
Yang, L. Y. U., Can, H. E., Jian-ming, Z. H. A. O., & Zhang, Z. J. (2025) · Bulletin of Mineralogy Petrology and Geochemistry · doi:10.3724/j.issn.1007-2802.20240161 · Cited by 1
Self-citationUnion of Experts: Adapting Hierarchical Routing to Equivalently Decomposed TransformerPreprint
Yang, Y. Y., Lian, J., & Li, L. (2025) · Qeios · doi:10.32388/9qh8rx · Cited by 1
Kimi K3: Open Frontier Intelligence
Kimi Team, Bai, T., Bai, Y., Bao, Y., C., M., Cai, J., Cai, X., Cao, P., Cao, Y., Chai, Z., et al. (2026) · arXiv · doi:10.48550/arxiv.2607.24653
“Multi-head Latent Attention (MLA), introduced in DeepSeek-V2 [28], compresses the key–value representation of each token into a low-dimensional latent vector c t = W c x t .”
Sun, Y., Wang, H., Li, D., Wang, G., & Zhang, H. (2025) · arXiv · doi:10.48550/arxiv.2503.16402
“This ensures that our framework can handle both binary and continuous evaluation setups, further broadening its applicability.”
Metadata
- Authors
- DeepSeek-AI, Liu, A., Feng, B., Wang, B., Wang, B., Liu, B., Zhao, C., Dengr, C., Ruan, C., Dai, D., Guo, D., Yang, D., Chen, D., Ji, D., Li, E., Lin, F., Luo, F., Hao, G., Chen, G., Li, G., Zhang, H., Xu, H., Yang, H., Zhang, H., Ding, H., Xin, H., Gao, H., Li, H., Qu, H., Cai, J. L., Liang, J., Guo, J., Ni, J., Li, J., Chen, J., Yuan, J., Qiu, J., Song, J., Dong, K., Gao, K., Guan, K., Wang, L., Zhang, L., Xu, L., Xia, L., Zhao, L., Zhang, L., Li, M., Wang, M., Zhang, M., Zhang, M., Tang, M., Li, M., Tian, N., Huang, P., Wang, P., Zhang, P., Zhu, Q., Chen, Q., Du, Q., Chen, R. J., Jin, R. L., Ge, R., Pan, R., Xu, R., Chen, R., Li, S. S., Lu, S., Zhou, S., Chen, S., Wu, S., Ye, S., Ma, S., Wang, S., Zhou, S., Yu, S., Zhou, S., Zheng, S., Wang, T., Pei, T., Yuan, T., Sun, T., Xiao, W. L., Zeng, W., An, W., Liu, W., Liang, W., Gao, W., Zhang, W., Li, X. Q., Jin, X., Wang, X., Bi, X., Liu, X., Wang, X., Shen, X., Chen, X., Chen, X., Nie, X., Sun, X., Wang, X., Liu, X., Xie, X., Yu, X., Song, X., Zhou, X., Yang, X., Lu, X., Su, X., Wu, Y., Li, Y. K., Wei, Y. X., Zhu, Y. X., Xu, Y., Huang, Y., Li, Y., Zhao, Y., Sun, Y., Li, Y., Wang, Y., Zheng, Y., Zhang, Y., Xiong, Y., Zhao, Y., He, Y., Tang, Y., Piao, Y., Dong, Y., Tan, Y., Liu, Y., Wang, Y., Guo, Y., Zhu, Y., Wang, Y., Zou, Y., Zha, Y., Ma, Y., Yan, Y., You, Y., Liu, Y., Ren, Z. Z., Ren, Z., Sha, Z., Fu, Z., Huang, Z., Zhang, Z., Xie, Z., Hao, Z., Shao, Z., Wen, Z., Xu, Z., Zhang, Z., Li, Z., Wang, Z., Gu, Z., Li, Z., & Xie, Z.
- Corresponding author
- DeepSeek-AI
- ORCID
- Song, Junxiao
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2024
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2405.04434
- OpenAlex
- W4396815229
- Cited by
- 109
- References
- 0
- Citations by year
- 2026: 26 · 2025: 76 · 2024: 7
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- mixture of experts, language models, multi-head latent attention, key-value cache compression, sparse training, inference efficiency, large language models, training cost reduction, open-source models, chatbots
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
