The DOI matches its Crossref record; 215 of the 215 references it cites were found in a registry or an index.
A Survey: Collaborative Hardware and Software Design in the Era of Large Language Models
Open accessAbstract
The rapid development of large language models (LLMs) has significantly transformed the field of artificial intelligence, demonstrating remarkable capabilities in natural language processing and moving towards multi-modal functionality. These models are increasingly integrated into diverse applications, impacting both research and industry. However, their development and deployment present substantial challenges, including the need for extensive computational resources, high energy consumption, and complex software optimizations. Unlike traditional deep learning systems, LLMs require unique optimization strategies for training and inference, focusing on system-level efficiency. This paper surveys hardware and software co-design approaches specifically tailored to address the unique characteristics and constraints of large language models. This survey analyzes the challenges and impacts of LLMs on hardware and algorithm research, exploring algorithm optimization, hardware design, and system-level innovations. It aims to provide a comprehensive understanding of the trade-offs and considerations in LLM-centric computing systems, guiding future advancements in AI. Finally, we summarize the existing efforts in this space and outline future directions toward realizing production-grade co-design methodologies for the next generation of large language models and AI systems.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Additionally, the collaborative hardware and software design is also necessary to further break the bottleneck for training and inference acceleration [153].”
From A Survey of Large Language Models (2026) · cites this paper as [153]
In the citing paper
“This architectural evolution marks a critical shift in AI system design: from content creation to autonomous task execution [118], [119].”
From AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges (2025) · cites this paper as [119]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 2×
“Finally, we summarize the existing efforts in this space and outline future directions toward realizing production-grade co-design methodologies for the next generation of large language models and AI systems.”
Self-citationVaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 3×
“The backbone of modern LLMs is the transformer architecture, which has revolutionized NLP by addressing the limitations of previous recurrent and convolutional models.”
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173 · cited 1×
“Encoder-decoder and Encoder-only models are considered BERT-style models [15], while Decoder-only models are termed GPT-style models [1].”
MethodBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 1×
“Encoder-decoder and Encoder-only models are considered BERT-style models [15], while Decoder-only models are termed GPT-style models [1].”
MethodLanguage Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 6×
“The landscape changed significantly after 2021 with the introduction of transformative models like GPT-3 [4], which led to a surge in the adoption of decoder-only architectures.”
InfluentialLearning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293 · cited 1×
“The construction of these multi-modal LLMs follows a well-established recipe.”
LoRA: Low-Rank Adaptation of Large Language Models
Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., & Chen, W. (2021) · arXiv · doi:10.48550/arxiv.2106.09685 · Cited by 13,453 · cited 1×
“LoRA, or Low-Rank Adaptation, is a technique designed to fine-tune large pre-trained models in a pa-rameter-efficient manner [36].”
TensorFlow: A system for large-scale machine learning
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08695 · Cited by 11,822
Scaling Laws for Neural Language Models
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · doi:10.48550/arxiv.2001.08361 · Cited by 9,550 · cited 2×
“In 2020, OpenAI’s introduction of the “scaling law” [17] highlighted that model performance is strongly correlated with size.”
Research works printed but not linked: 23
- Radford, 2018, Improving language understanding by generative pre-training
- 2023, GPT-4 technical report
- Radford, 2019, Language models are unsupervised multitask learners, OpenAI blog, 1, 9
- Ren, Zero-offload: Democratizing billionscale model training, 551
- 2020, NVIDIA A100 Tensor Core GPU Architecture
- 2022, NVIDIA H100 Tensor Core GPU Architecture
- 1982, Why systolic architectures?, 15, 3746
- Dao, 2023, Flashdecoding for Long-Context Inference
- Vaidya, 2023, Optimizing Inference on Large Language Models With NVIDIA TensorRT-LLM, Now Publicly Available
- Lefaudeux, 2022, xformers: A Modular and Hackable Transformer Modelling Library
- Yu, Orca: A distributed serving system for transformer-based generative models, 521
- 2024, Lightllm
- Jin, s3: Increasing GPU utilization during generative inference for higher throughput, 36, 18015
- Jiang, 2024, Mixtral of experts, arXiv:2401.04088 · arXiv
- Du, SiDA-MoE: Sparsity-inspired data-aware serving for efficient and scalable large mixture-of-experts models, 6, 224
- Gupta, 2023, FLuRKA: Fast and accurate unified low-rank & kernel attention, arXiv:2306.15799 · arXiv
- Zhang, 2023, Draft & verify: Lossless large language model acceleration via self-speculative decoding, arXiv:2309.08168 · arXiv
- Liu, 2024, LLaVA-NeXT: Improved Reasoning, OCR, and World Knowledge
- Dettmers, GPT3.int8(): 8-bit matrix multiplication for transformers at scale, 35, 30318
- Tu, A 28 nm $15.59 \mu\text{J}$ /token full-digital bitline-transpose CIM-based sparse transformer accelerator with pipeline/parallel reconfigurable modes, 466
- Tambe, A 12 nm 18.1TFLOPs/W sparse transformer processor with entropy-based early exit, mixed-precision predication and fine-grained power management, 342
- Amirshahi, TiC-SAT: tightly-coupled systolic accelerator for transformers, 657
- Imani, FloatPIM: in-memory acceleration of deep neural network training with high precision, 802
Cited by
A Survey of Large Language Models
Zhao, W. X., Zhou, K., Li, J., Tang, T., Dong, Z., Hou, Y., Zhang, B., Min, Y., Zhang, J., Liu, P., et al. (2026) · Frontiers of Computer Science · doi:10.1007/s11704-026-60308-3 · Cited by 2,093
Self-citationAI Agents vs. Agentic AI: A Conceptual taxonomy, applications and challenges
Sapkota, R., Roumeliotis, K. I., & Karkee, M. (2026) · Information Fusion · doi:10.1016/j.inffus.2025.103599 · Cited by 671
“This architectural evolution marks a critical shift in AI system design: from content creation to autonomous task execution [116,117].”
AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges
Sapkota, R., Roumeliotis, K. I., & Karkee, M. (2025) · arXiv · doi:10.48550/arxiv.2505.10468 · Cited by 295
“This architectural evolution marks a critical shift in AI system design: from content creation to autonomous task execution [116,117].”
AI Agents vs. Agentic AI: A Conceptual Taxonomy, Applications and Challenges
Sapkota, R., Roumeliotis, K. I., & Karkee, M. (2025) · SuperIntelligence - Robotics - Safety & Alignment · doi:10.70777/si.v2i3.15161 · Cited by 186
A Comprehensive Survey on Tiny Machine Learning for Human Behavior Analysis
Lamaakal, I., Essahraui, S., Maleh, Y., Makkaoui, K. E., Ouahbi, I., Bouami, M. F., El‐Latif, A. A. A., Almousa, M., Peng, J., & Niyato, D. T. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3565688 · Cited by 76
“Unlike TinyML, which prioritizes efficiency and low power usage, LLMs require extensive computational resources, high-end GPUs or TPUs, and vast amounts of training data [199].”
Metadata
- Authors
- Guo, C., Cheng, F., Du, Z., Kiessling, J., Ku, J., Li, S., Li, Z., Ma, M., Molom-Ochir, T., Morris, B., Shan, H., Sun, J., Wang, Y., Wei, C., Wu, X., Wu, Y., Yang, H. F., Zhang, J., Zhang, J., Zheng, Q., Zhou, G., Li, H., & Chen, Y.
- Affiliations
- Duke University (US)
- ORCID
- Venue
- IEEE Circuits and Systems Magazine
- Published
- 2025
- Volume
- 25
- Issue
- 1
- Pages
- 35-57
- ISSN
- 0163-6812 (ISSN-L)
- 1531-636X
- 1558-0830
- OpenAlex
- W4407212618
- Cited by
- 32
- References
- 212
- Citations by year
- 2026: 12 · 2025: 20
- FWCI
- 21.86
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Circuits and Systems Magazine (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Hardware and Architecture › Embedded Systems Design Techniques
- Keywords
- large language models, hardware-software co-design, energy efficiency, algorithm optimization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A Survey: Collaborative Hardware and Software Design in the Era of Large Language Models. https://app.cochat.ai/papers/10.1109/mcas.2024.3476008
- The paper
- Guo, C., Cheng, F., Du, Z., Kiessling, J., Ku, J., Li, S., Li, Z., Ma, M., Molom-Ochir, T., Morris, B., Shan, H., Sun, J., Wang, Y., Wei, C., Wu, X., Wu, Y., Yang, H. F., Zhang, J., Zhang, J., . . . Chen, Y. (2025). A Survey: Collaborative Hardware and Software Design in the Era of Large Language Models. IEEE Circuits and Systems Magazine. https://doi.org/10.1109/mcas.2024.3476008
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.