Large Language Models: A Comprehensive Survey of its Applications, Challenges, Limitations, and Future Prospects
Open accessThe DOI matches its Crossref record; 255 of the 255 references it cites were found in a registry or an index.
Abstract
Within the vast expanse of computerized language processing, a revolutionary entity known as Large Language Models (LLMs) has emerged, wielding immense power in its capacity to comprehend intricate linguistic patterns and conjure coherent and contextually fitting responses. Large language models (LLMs) are a type of artificial intelligence (AI) that have emerged as powerful tools for a wide range of tasks, including natural language processing (NLP), machine translation, and question-answering. This survey paper provides a comprehensive overview of LLMs, including their history, architecture, training methods, applications, and challenges. The paper begins by discussing the fundamental concepts of generative AI and the architecture of generative pre- trained transformers (GPT). It then provides an overview of the history of LLMs, their evolution over time, and the different training methods that have been used to train them. The paper then discusses the wide range of applications of LLMs, including medical, education, finance, and engineering. It also discusses how LLMs are shaping the future of AI and how they can be used to solve real-world problems. The paper then discusses the challenges associated with deploying LLMs in real-world scenarios, including ethical considerations, model biases, interpretability, and computational resource requirements. It also highlights techniques for enhancing the robustness and controllability of LLMs, and addressing bias, fairness, and generation quality issues. Finally, the paper concludes by highlighting the future of LLM research and the challenges that need to be addressed in order to make LLMs more reliable and useful. This survey paper is intended to provide researchers, practitioners, and enthusiasts with a comprehensive understanding of LLMs, their evolution, applications, and challenges. By consolidating the state-of-the-art knowledge in the field, this survey serves as a valuable resource for further advancements in the development and utilization of LLMs for a wide range of real-world applications. The GitHub repo for this project is available at https://github.com/anas-zafar/LLM-Survey
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Items 201–255 of 255
Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning
Pan, L., Albalak, A., Wang, X., & Wang, W. Y. (2023) · arXiv · doi:10.48550/arxiv.2305.12295 · Cited by 144
Emotional intelligence of Large Language Models
Wang, X., Li, X., Yin, Z., Wu, Y., & Liu, J. (2023) · Journal of Pacific Rim Psychology · doi:10.1177/18344909231213958 · Cited by 128
Reducing the Carbon Impact of Generative AI Inference (today and in 2035)
Chien, A. A., Lin, L., Nguyen, H., Rao, V., Sharma, T., & Wijayawardana, R. (2023) · Proceedings of the 2nd Workshop on Sustainable Computer Systems · doi:10.1145/3604930.3605705 · Cited by 128
Jungmann, S. M., Klan, T., Kuhn, S., & Jungmann, F. (2019) · JMIR Formative Research · doi:10.2196/13863 · Cited by 127
The Science of Detecting LLM-Generated Text
Tang, R., Chuang, Y.-N., & Hu, X. (2024) · Communications of the ACM · doi:10.1145/3624725 · Cited by 125
Hungry Hungry Hippos: Towards Language Modeling with State Space Models
Fu, D. Y., Dao, T., Saab, K. K., Thomas, A. W., Rudra, A., & Ré, C. (2022) · arXiv · doi:10.48550/arxiv.2212.14052 · Cited by 123
(2023) · doi:10.18653/v1/2023.emnlp-main.632 · Cited by 115
Large Language Models for Software Engineering: A Systematic Literature Review
Hou, X., Zhao, Y., Liu, Y., Yang, Z., Wang, K., Li, L., Luo, X., Lo, D., Grundy, J., & Wang, H. (2023) · arXiv · doi:10.48550/arxiv.2308.10620 · Cited by 114
On the creativity of large language models
Franceschelli, G., & Musolesi, M. (2025) · AI & SOCIETY · doi:10.1007/s00146-024-02127-3 · Cited by 113
The Cost of Training NLP Models: A Concise Overview
Sharir, O., Peleg, B., & Shoham, Y. (2020) · arXiv · doi:10.48550/arxiv.2004.08900 · Cited by 113
Parallel Context Windows for Large Language Models
Ratner, N., Levine, Y., Belinkov, Y., Ram, O., Magar, I., Abend, O., Karpas, E., Shashua, A., Leyton-Brown, K., & Shoham, Y. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.352 · Cited by 111
Ignore Previous Prompt: Attack Techniques For Language Models
Perez, F., & Ribeiro, I. (2022) · arXiv · doi:10.48550/arxiv.2211.09527 · Cited by 110
Larger language models do in-context learning differently
Wei, J., Wei, J., Tay, Y., Tran, D., Webson, A., Lu, Y., Chen, X., Liu, H., Huang, D., Zhou, D., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.03846 · Cited by 109
The impact of AI on carbon emissions: evidence from 66 countries
Zhong, J., Zhong, Y., Han, M., Yang, T., & Zhang, Q. (2024) · Applied Economics · doi:10.1080/00036846.2023.2203461 · Cited by 105
Deng, W. H., Guo, B., Devrio, A., Shen, H., Eslami, M., & Holstein, K. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581026 · Cited by 103
Prompt Injection attack against LLM-integrated Applications
Liu, Y., Deng, G., Li, Y., Wang, K., Wang, Z., Wang, X., Zhang, T., Liu, Y., Wang, H., Zheng, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.05499 · Cited by 100
HONEST: Measuring Hurtful Sentence Completion in Language Models
Nozza, D., Bianchi, F., & Hovy, D. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.191 · Cited by 95
Shortcut Learning of Large Language Models in Natural Language Understanding
Du, M., He, F., Zou, N., Tao, D., & Hu, X. (2024) · Communications of the ACM · doi:10.1145/3596490 · Cited by 94
The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset
Laurençon, H., Saulnier, L., Wang, T., Akiki, C., del Moral, A. V., Scao, T. L., Von Werra, L., Mou, C., Ponferrada, E. G., Nguyen, H., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.03915 · Cited by 93
Dang, H., Mecke, L., Lehmann, F., Goller, S., & Buschek, D. (2022) · arXiv · doi:10.48550/arxiv.2209.01390 · Cited by 92
The Gradient of Generative AI Release: Methods and Considerations
Solaiman, I. (2023) · 2023 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3593013.3593981 · Cited by 90
Chat Generative Pre-trained Transformer: why we should embrace this technology
Chavez, M. R., Butler, T. S., Rekawek, P., Heo, H., & Kinzler, W. L. (2023) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2023.03.010 · Cited by 83
OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization
Iyer, S., Lin, X. V., Pasunuru, R., Mihaylov, T., Simig, D., Yu, P., Shuster, K., Wang, T., Liu, Q., Koura, P. S., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.12017 · Cited by 81
Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervision
Sun, Z., Shen, Y., Zhou, Q., Zhang, H., Chen, Z., Cox, D., Yang, Y., & Gan, C. (2023) · arXiv · doi:10.48550/arxiv.2305.03047 · Cited by 81
(2023) · doi:10.52202/075280-0115 · Cited by 81
Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender System
Gao, Y., Sheng, T., Xiang, Y., Xiong, Y., Wang, H., & Zhang, J. (2023) · arXiv · doi:10.48550/arxiv.2303.14524 · Cited by 79
A Survey on Efficient Training of Transformers
Zhuang, B., Liu, J., Pan, Z., He, H., Weng, Y., & Shen, C. (2023) · Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/764 · Cited by 78
Preference Ranking Optimization for Human Alignment
Song, F., Yu, B., Li, M., Yu, H., Huang, F., Li, Y., & Wang, H. (2023) · arXiv · doi:10.48550/arxiv.2306.17492 · Cited by 77
Preference Ranking Optimization for Human Alignment
Song, F., Yu, B., Li, M., Yu, H., Huang, F., Li, Y., & Wang, H. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29865 · Cited by 77
Citation: A Key to Building Responsible and Accountable Large Language Models
Huang, J., & Chang, K. (2024) · Findings of the Association for Computational Linguistics: NAACL 2024 · doi:10.18653/v1/2024.findings-naacl.31 · Cited by 72
How do we know how smart AI systems are?
Mitchell, M. (2023) · Science · doi:10.1126/science.adj5957 · Cited by 71
Valmeekam, K., Marquez, M., Olmo, A., Sreedharan, S., & Kambhampati, S. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1693 · Cited by 70
Valmeekam, K., Marquez, M., Olmo, A., Sreedharan, S., & Kambhampati, S. (2022) · arXiv · doi:10.48550/arxiv.2206.10498 · Cited by 70
Aligning Large Language Models with Human: A Survey
Wang, Y., Zhong, W., Li, L., Mi, F., Zeng, X., Huang, W., Shang, L., Jiang, X., & Liu, Q. (2023) · arXiv · doi:10.48550/arxiv.2307.12966 · Cited by 69
Segment Anything in Medical Images
Ma, J., He, Y., Li, F., Han, L., You, C., & Wang, B. (2023) · arXiv · doi:10.48550/arxiv.2304.12306 · Cited by 67
Internet-augmented language models through few-shot prompting for open-domain question answering
Lazaridou, A., Gribovskaya, E., Stokowiec, W., & Grigorev, N. (2022) · arXiv · doi:10.48550/arxiv.2203.05115 · Cited by 64
Extending Context Window of Large Language Models via Positional Interpolation
Chen, S., Wong, S., Chen, L., & Tian, Y. (2023) · arXiv · doi:10.48550/arxiv.2306.15595 · Cited by 50
LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models
Chen, Y., Qian, S., Tang, H., Lai, X., Liu, Z., Han, S., & Jia, J. (2023) · arXiv · doi:10.48550/arxiv.2309.12307 · Cited by 50
Applying ChatGPT and AI‐Powered Tools to Accelerate Evidence Reviews
Nguyen-Trung, K., Saeri, A. K., & Kaufman, S. (2024) · Human Behavior and Emerging Technologies · doi:10.1155/2024/8815424 · Cited by 47
Fundamental Limitations of Alignment in Large Language Models
Wolf, Y., Wies, N., Avnery, O., Levine, Y., & Shashua, A. (2023) · arXiv · doi:10.48550/arxiv.2304.11082 · Cited by 47
Academic Writing with GPT-3.5 (ChatGPT): Reflections on Practices, Efficacy and Transparency
Buruk, O. '. (2023) · 26th International Academic Mindtrek Conference · doi:10.1145/3616961.3616992 · Cited by 46
Moral Foundations of Large Language Models
Abdulhai, M., Serapio-García, G., Crepy, C., Valter, D., Canny, J., & Jaques, N. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.982 · Cited by 45
Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them
Suzgun, M., Scales, N., Schärli, N., Gehrmann, S., Tay, Y., Chung, H. W., Chowdhery, A., Le, Q. V., Chi, E. H., Zhou, D., et al. (2022) · arXiv · doi:10.48550/arxiv.2210.09261 · Cited by 44
Poisoning Language Models During Instruction Tuning
Wan, A., Wallace, E., Shen, S., & Klein, D. (2023) · arXiv · doi:10.48550/arxiv.2305.00944 · Cited by 42
Allen, B., Agarwal, S., Kalpathy-Cramer, J., & Dreyer, K. (2019) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2019.04.023 · Cited by 40
GPT-4 vs. GPT-3.5: A Concise Showdown
Koubaa, A. (2023) · doi:10.20944/preprints202303.0422.v1 · Cited by 37
Evaluating model drift in machine learning algorithms
Nelson, K., Corbin, G., Anania, M., Kovacs, M., Tobias, J., & Blowers, M. (2015) · 2015 IEEE Symposium on Computational Intelligence for Security and Defense Applications (CISDA) · doi:10.1109/cisda.2015.7208643 · Cited by 31
Zhuo, T. Y., Li, Z., Huang, Y., Shiri, F., Wang, W., Haffari, G., & Li, Y.-F. (2023) · Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.77 · Cited by 31
Volum, R., Rao, S., Xu, M., DesGarennes, G., Brockett, C., Van Durme, B., Deng, O., Malhotra, A., & Dolan, B. (2022) · Proceedings of the 3rd Wordplay: When Language Meets Games Workshop (Wordplay 2022) · doi:10.18653/v1/2022.wordplay-1.3 · Cited by 30
D4: Improving LLM Pretraining via Document De-Duplication and Diversification
Tirumala, K., Simig, D., Aghajanyan, A., & Morcos, A. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-2348 · Cited by 28
Scaling Laws and Interpretability of Learning from Repeated Data
Hernandez, D., Brown, T., Conerly, T., DasSarma, N., Drain, D., El-Showk, S., Elhage, N., Hatfield-Dodds, Z., Henighan, T., Hume, T., et al. (2022) · arXiv · doi:10.48550/arxiv.2205.10487 · Cited by 24
SkinGPT-4: an interactive dermatology diagnostic system with visual large language model
Zhou, J., He, X., Sun, L., Xu, J., Chen, X., Chu, Y., Zhou, L., Liao, X., Zhang, B., & Gao, X. (2023) · arXiv · doi:10.48550/arxiv.2304.10691 · Cited by 17
Nottingham, K., Ammanabrolu, P., Suhr, A., Choi, Y., Hajishirzi, H., Singh, S., & Fox, R. (2023) · arXiv · doi:10.48550/arxiv.2301.12050 · Cited by 12
Plan, Eliminate, and Track - Language Models are Good Teachers for Embodied Agents
Wu, Y., Min, S. Y., Bisk, Y., Salakhutdinov, R., Azaria, A., Li, Y., Mitchell, T., & Prabhumoye, S. (2023) · arXiv · doi:10.48550/arxiv.2305.02412 · Cited by 7
Academic Writing with GPT-3.5: Reflections on Practices, Efficacy and Transparency
Buruk, O. '. (2023) · doi:10.31222/osf.io/vg5f6 · Cited by 3
Cited by
A Comprehensive Systematic Review of YOLO for Medical Object Detection (2018 to 2023)
Ragab, M. G., Abdulkadir, S. J., Muneer, A., Alqushaibi, A., Sumiea, E. H., Qureshi, R., Al-Selwi, S. M., & Alhussian, H. (2024) · IEEE Access · doi:10.1109/access.2024.3386826 · Cited by 306
Self-citationOpportunities and challenges of using generative AI to personalize educational assessment
Arslan, B., Lehman, B., Tenison, C., Sparks, J. R., López, A. A., Gu, L., & Zapata-Rivera, D. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1460651 · Cited by 52
Garg, A., Nisumba Soodhani, K., & Rajendran, R. (2025) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2025.100380 · Cited by 49
Generative AI and Language Diversity: Implications for Teachers and Learners
Hamad M Albeihi, H., & F. Rice, M. (2025) · Arab World English Journal · doi:10.24093/awej/vol16no1.3 · Cited by 19
The Role of Agentic Artificial Intelligence in Healthcare: A Systematic Review
Collaco, B. G., Haider, S. A., Prabha, S., Gomez-Cabello, C. A., Genovese, A., Wood, N. G., Bagaria, S. P., Gopala, N., Tao, C., & Forte, A. J. (2025) · doi:10.21203/rs.3.rs-7234499/v1 · Cited by 68
Metadata
- Authors
- Hadi, M. U., tashi, Q. A., Qureshi, R., Shah, A., muneer, A., Irfan, M., Zafar, A., Shaikh, M. B., Akhtar, N., Wu, J., & Mirjalili, S.
- Published
- 2023-11-16
- OpenAlex
- W4388725733
- Cited by
- 103
- References
- 255
- Type
- posted content
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.