The DOI matches its Crossref record; 501 of the 501 references it cites were found in a registry or an index.
A survey of GPT-3 family large language models including ChatGPT and GPT-4
Open accessAbstract
Large language models (LLMs) are a special class of pretrained language models (PLMs) obtained by scaling model size, pretraining corpus and computation. LLMs, because of their large size and pretraining on large volumes of text data, exhibit special abilities which allow them to achieve remarkable performances without any task-specific training in many of the natural language processing tasks. The era of LLMs started with OpenAI’s GPT-3 model, and the popularity of LLMs has increased exponentially after the introduction of models like ChatGPT and GPT4. We refer to GPT-3 and its successor OpenAI models, including ChatGPT and GPT4, as GPT-3 family large language models (GLLMs). With the ever-rising popularity of GLLMs, especially in the research community, there is a strong need for a comprehensive survey which summarizes the recent research progress in multiple dimensions and can guide the research community with insightful future research directions. We start the survey paper with foundation concepts like transformers, transfer learning, self-supervised learning, pretrained language models and large language models. We then present a brief overview of GLLMs and discuss the performances of GLLMs in various downstream tasks, specific domains and multiple languages. We also discuss the data labelling and data augmentation abilities of GLLMs, the robustness of GLLMs, the effectiveness of GLLMs as evaluators, and finally, conclude with multiple insightful future research directions. To summarize, this comprehensive survey paper will serve as a good resource for both academic and industry people to stay updated with the latest research related to GLLMs.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Large Language Models (LLMs), such as OpenAI's chatGPT [Kalyan 2024], are examples of generative models capable of producing human-like text.”
From Generative-AI-based game asset creation: developing a system for supporting game production brainstorming (2025) · cites this paper as Kalyan 2024
In the citing paper
“TLMs are significantly smaller than LLMs, typically containing tens to hundreds of millions of parameters compared to billions in LLMs like GPT-3 [4].”
From Tiny Language Models for Automation and Control: Overview, Potential Applications, and Future Research Directions (2025) · cites this paper as [4]
In the citing paper
“For example, the popular GPT-3 model [4], with 175 billion parameters, necessitates vast computing resources and is impractical to deploy directly on mobile devices, edge systems, or IoT platforms [16].”
From Tiny Language Models for Automation and Control: Overview, Potential Applications, and Future Research Directions (2025) · cites this paper as [4]
Citations
Cites
Items 401–501 of 501
Chali, Y., Hasan, S. A., & Joty, S. (2010) · Information Processing & Management · doi:10.1016/j.ipm.2010.10.002 · Cited by 34
Do GPTs Produce Less Literal Translations?
Raunak, V., Menezes, A., Post, M., & Hassan, H. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-short.90 · Cited by 34
Bot or Human? Detecting ChatGPT Imposters with A Single QuestionPreprint
Wang, H., Luo, X., Wang, W., & Yan, X. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.06424 · Cited by 33
Benchmarking Foundation Models with Language-Model-as-an-Examiner
Bai, Y., Ying, J., Cao, Y., Lv, X., He, Y., Wang, X., Yu, J., Zeng, K., Xiao, Y., Lyu, H., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.04181 · Cited by 33
Stammbach, D., Antoniak, M., & Ash, E. (2022) · doi:10.18653/v1/2022.wnu-1.6 · Cited by 32
How to Design Translation Prompts for ChatGPT: An Empirical StudyPreprint
Gao, Y., Wang, R., & Hou, F. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.02182 · Cited by 32
ArguGPT: evaluating, understanding and identifying argumentative essays generated by GPT models
Liu, Y., Zhang, Z., Zhang, W., Yue, S., Zhao, X., Cheng, X., Zhang, Y., & Hu, H. (2023) · arXiv · doi:10.48550/arxiv.2304.07666 · Cited by 32
AI-assisted coding: Experiments with GPT-4Preprint
Poldrack, R. A., Lu, T., & Beguš, G. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.13187 · Cited by 32
Giorgi, J., Toma, A., Xie, R., Chen, S., An, K., Zheng, G., & Wang, B. (2023) · Clinical Natural Language Processing Workshop (ClinicalNLP) · doi:10.18653/v1/2023.clinicalnlp-1.36 · Cited by 31
Zhuo, T. Y., Li, Z., Huang, Y., Shiri, F., Wang, W., Haffari, G., & Li, Y.-F. (2023) · Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.77 · Cited by 31
Using GPT-3 to Build a Lexicon of Drugs of Abuse Synonyms for Social Media Pharmacovigilance
Carpenter, K. A., & Altman, R. B. (2023) · Biomolecules · doi:10.3390/biom13020387 · Cited by 30
Zero is Not Hero Yet: Benchmarking Zero-Shot Performance of LLMs for Financial TasksPreprint
Shah, A., & Chava, S. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4458613 · Cited by 30
LM-CPPF: Paraphrasing-Guided Data Augmentation for Contrastive Prompt-Based Few-Shot Fine-Tuning
Abaskohi, A., Rothe, S., & Yaghoobzadeh, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-short.59 · Cited by 30
FLM-101B: An Open LLM and How to Train It with $100K BudgetPreprint
Li, X., Yao, Y., Jiang, X., et al. (2023) · arXiv.org · doi:10.48550/arxiv.2309.03852 · Cited by 30
MGTBench: Benchmarking Machine-Generated Text DetectionPreprint
He, X., Shen, X., Chen, Z., Michael Backes, A., & Zhang, Y. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.14822 · Cited by 29
KoLA: Carefully Benchmarking World Knowledge of Large Language Models
Yu, J., Wang, X., Tu, S., Cao, S., Zhang-Li, D., Lv, X., Peng, H., Yao, Z., Zhang, X., Li, H., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.09296 · Cited by 29
Time Travel in LLMs: Tracing Data Contamination in Large Language Models
Golchin, S., & Surdeanu, M. (2023) · arXiv · doi:10.48550/arxiv.2308.08493 · Cited by 29
Retrieval Augmented Chest X-Ray Report Generation using OpenAI GPT modelsPreprint
Ranjit, M., Ganapathy, G., Manuel, R. F., & Ganu, T. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.03660 · Cited by 28
Do LLMs Understand User Preferences? Evaluating LLMs On User Rating Prediction
Kang, W.-C., Ni, J., Mehta, N., Sathiamoorthy, M., Hong, L., Chi, E., & Cheng, D. Z. (2023) · arXiv · doi:10.48550/arxiv.2305.06474 · Cited by 28
Prompting Frameworks for Large Language Models: A Survey
Liu, X., Wang, J., Sun, J., Yuan, X., Dong, G., Di, P., Wang, W., & Wang, D. (2023) · arXiv · doi:10.48550/arxiv.2311.12785 · Cited by 28
A Video Is Worth 4096 Tokens: Verbalize Story Videos To Understand Them In Zero Shot
Bhattacharyya, A., Singla, Y., Krishnamurthy, B., Shah, R. R., & Chen, C. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.608 · Cited by 28
Chinese fine-grained financial sentiment analysis with large language models
Lan, Y., Wu, Y., Wang, X., Feng, W., & Zhang, Y. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-10603-6 · Cited by 28
GPT3-to-plan: Extracting plans from text using GPT-3
Olmo, A., Sreedharan, S., & Kambhampati, S. (2021) · arXiv · doi:10.48550/arxiv.2106.07131 · Cited by 27
Zero-Shot Next-Item Recommendation using Large Pretrained Language Models
Wang, L., & Lim, E.-P. (2023) · arXiv · doi:10.48550/arxiv.2304.03153 · Cited by 27
G3Detector: General GPT-Generated Text DetectorPreprint
Zhan, H., He, X., Xu, Q., Wu, Y., & Stenetorp, P. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.12680 · Cited by 27
Detection of Fake Generated Scientific Abstracts
Theocharopoulos, P. C., Anagnostou, P., Tsoukala, A., Georgakopoulos, S. V., Tasoulis, S. K., & Plagianakos, V. P. (2023) · IEEE International Conference on Big Data Computing Service and Applications (BigDataService) · doi:10.1109/bigdataservice58306.2023.00011 · Cited by 27
Through the Lens of Core Competency: Survey on Evaluation of Large Language ModelsPreprint
Zhuang, Z., Chen, Q., Ma, L., Li, M.-D., Han, Y., Qian, Y., Bai, H., Feng, Z.-X., Zhang, W.-N., & Liu, T. (2023) · China National Conference on Chinese Computational Linguistics · doi:10.48550/arxiv.2308.07902 · Cited by 26
Evaluation of AI Chatbots for Patient-Specific EHR QuestionsPreprint
Hamidi, A., & Roberts, K. E. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.02549 · Cited by 25
TaskBench: Benchmarking Large Language Models for Task Automation
Shen, Y., Song, K., Tan, X., Zhang, W., Ren, K., Yuan, S., Lu, W., Li, D., & Zhuang, Y. (2023) · arXiv · doi:10.48550/arxiv.2311.18760 · Cited by 24
TRAM: Benchmarking Temporal Reasoning for Large Language Models
Wang, Y., & Zhao, Y. (2024) · Findings of the Association for Computational Linguistics ACL 2024 · doi:10.18653/v1/2024.findings-acl.382 · Cited by 24
DNA-GPT: Divergent N-Gram Analysis for Training-Free Detection of GPT-Generated Text
Yang, X., Cheng, W., Wu, Y., Petzold, L., Wang, W. Y., & Chen, H. (2023) · arXiv · doi:10.48550/arxiv.2305.17359 · Cited by 23
Kwan, W.-C., Zeng, X., Wang, Y., Sun, Y., Li, L., Jiang, Y., Shang, L., Liu, Q., & Wong, K. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.832 · Cited by 23
Large Language Models as Instructors: A Study on Multilingual Clinical Entity Extraction
Méoni, S., de la Clergerie, É. V., & Ryffel, T. (2023) · Workshop on Biomedical Language Processing (BioNLP) · doi:10.18653/v1/2023.bionlp-1.15 · Cited by 22
MedEval: A Multi-Level, Multi-Task, and Multi-Domain Medical Benchmark for Language Model Evaluation
He, Z., Wang, Y., Yan, A., Liu, Y., Chang, E. Y., Gentili, A., McAuley, J., & Hsu, C. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.540 · Cited by 22
Question classification using limited labelled data
Mallikarjuna, C., & Sangeetha, S. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.103094 · Cited by 21
GPT-Sentinel: Distinguishing Human and ChatGPT Generated Content
Chen, Y., Kang, H., Zhai, V., Li, L., Singh, R., & Raj, B. (2023) · arXiv · doi:10.48550/arxiv.2305.07969 · Cited by 21
Understanding the Effectiveness of Very Large Language Models on Dialog EvaluationPreprint
Huynh, J., Jiao, C., Gupta, P., Mehri, S., Bajaj, P., Chaudhary, V., & Eskénazi, M. (2023) · arXiv.org · doi:10.48550/arxiv.2301.12004 · Cited by 20
Toward Human-Like Evaluation for Natural Language Generation with Error Analysis
Lu, Q., Ding, L., Xie, L., Zhang, K., Wong, D. F., & Tao, D. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.324 · Cited by 20
TimeBench: A Comprehensive Evaluation of Temporal Reasoning Abilities in Large Language Models
Chu, Z., Chen, J., Chen, Q., Yu, W., Wang, H., Liu, M., & Qin, B. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.66 · Cited by 20
Open-Source LLMs for Text Annotation: A Practical Guide for Model Setting and Fine-Tuning
Alizadeh, M., Kubli, M., Samei, Z., Dehghani, S., Zahedivafa, M., Bermeo, J. D., Korobeynikova, M., & Gilardi, F. (2023) · arXiv · doi:10.48550/arxiv.2307.02179 · Cited by 18
Gu, Y., Zhang, S., Usuyama, N., Woldesenbet, Y., Wong, C., Sanapathi, P., Wei, M., Valluri, N., Strandberg, E., Naumann, T., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.06439 · Cited by 18
Michail, A., Konstantinou, S., & Clematide, S. (2023) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/2023.semeval-1.140 · Cited by 18
Breaking the Bank with ChatGPT: Few-Shot Text Classification for FinancePreprint
Loukas, L., Stogiannidis, I., Malakasiotis, P., et al. (2023) · RePEc: Research Papers in Economics · doi:10.48550/arxiv.2308.14634 · Cited by 18
Is ChatGPT A Good Keyphrase Generator? A Preliminary StudyPreprint
Song, M., Jiang, H., Shi, S., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.13001 · Cited by 17
Tanaka, Y., Nakata, T., Aiga, K., et al. (2023) · medRxiv · doi:10.1101/2023.04.17.23288603 · Cited by 16
Destefanis, G., Bartolucci, S., & Ortu, M. (2023) · arXiv · doi:10.48550/arxiv.2305.09402 · Cited by 16
Is ChatGPT Involved in Texts? Measure the Polish Ratio to Detect ChatGPT-Generated Text
Yang, L., Jiang, F., & Li, H. (2024) · APSIPA Transactions on Signal and Information Processing · doi:10.1561/116.00000250 · Cited by 16
FinanceBench: A New Benchmark for Financial Question Answering
Islam, P., Kannappan, A., Kiela, D., Qian, R., Scherrer, N., & Vidgen, B. (2023) · arXiv · doi:10.48550/arxiv.2311.11944 · Cited by 16
Hulmán, Á., Dollerup, O. L., Mortensen, J. F., et al. (2023) · medRxiv · doi:10.1101/2023.02.13.23285745 · Cited by 15
Wu, Z., Zhang, L., Cao, C., Yu, X., Dai, H., Ma, C., Liu, Z., Zhao, L., Li, G., Liu, W., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.09138 · Cited by 15
Can GPT-4 Perform Neural Architecture Search?
Zheng, M., Su, X., You, S., Wang, F., Qian, C., Xu, C., & Albanie, S. (2023) · arXiv · doi:10.48550/arxiv.2304.10970 · Cited by 15
Nair, V., Schumacher, E., & Kannan, A. (2023) · Clinical Natural Language Processing Workshop (ClinicalNLP) · doi:10.18653/v1/2023.clinicalnlp-1.26 · Cited by 15
Multi-Dimensional Evaluation of Text Summarization with In-Context Learning
Jain, S., Keshava, V., Mysore Sathyendra, S., Fernandes, P., Liu, P., Neubig, G., & Zhou, C. (2023) · Findings of the Association for Computational Linguistics: ACL 2023 · doi:10.18653/v1/2023.findings-acl.537 · Cited by 15
Wang, J., Yao, Z., Mitra, A., Osebe, S., Yang, Z., & Yu, H. Q. (2023) · Clinical Natural Language Processing Workshop (ClinicalNLP) · doi:10.18653/v1/2023.clinicalnlp-1.49 · Cited by 15
Uncovering the Potential of ChatGPT for Discourse Analysis in Dialogue: An Empirical StudyPreprint
Fan, Y., Jiang, F. X., Li, P., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.08391 · Cited by 14
Tang, X., Tran, A., Tan, J. T. C., & Gerstein, M. (2023) · Clinical Natural Language Processing Workshop (ClinicalNLP) · doi:10.18653/v1/2023.clinicalnlp-1.58 · Cited by 14
Explainable Automated Debugging via Large Language Model-driven Scientific DebuggingPreprint
Kang, S., Chen, B., Yoo, S., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.02195 · Cited by 13
BizBench: A Quantitative Reasoning Benchmark for Business and Finance
Krumdick, M., Koncel-Kedziorski, R., Lai, V. D., Reddy, V., Lovering, C., & Tanner, C. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.452 · Cited by 13
Evaluating Large Language Models at Evaluating Instruction Following
Zeng, Z., Yu, J., Gao, T., Meng, Y., Goyal, T., & Chen, D. (2023) · arXiv · doi:10.48550/arxiv.2310.07641 · Cited by 13
Collins, K. M., Wong, C., Feng, J., Wei, M., & Tenenbaum, J. B. (2022) · arXiv · doi:10.48550/arxiv.2205.05718 · Cited by 13
DPIC: Decoupling Prompt and Intrinsic Characteristics for LLM Generated Text DetectionPreprint
Yu, X., Qi, Y., Chen, K., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.12519 · Cited by 12
InheritSumm: A General, Versatile and Compact Summarizer by Distilling from GPT
Yi‐chong, X., Xu, R., Iter, D., Liu, Y., Wang, S., Zhu, C., & Zeng, M. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.927 · Cited by 12
Gu, W. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.15587 · Cited by 11
Recurrent Attention Networks for Long-text Modeling
Li, X., Li, Z., Luo, X., Xie, H., Lee, X. Z., Zhao, Y., Wang, F. L., & Li, Q. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.188 · Cited by 11
When and how to paraphrase for named entity recognition?
Sharma, S., Joshi, A., Zhao, Y., Mukhija, N., Bhathena, H., Singh, P., & Santhanam, S. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.390 · Cited by 11
Zhang, Z., Yao, Y., Zhang, A., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.11797 · Cited by 11
Learning from Sibling Mentions with Scalable Graph Inference in Fine-Grained Entity Typing
Chen, Y., Cheng, J., Jiang, H., et al. (2022) · Annual Meeting of the Association for Computational Linguistics (ACL) · doi:10.18653/v1/2022.acl-long.147 · Cited by 10
Ji, Y., & Gao, S. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.giscience.2023.43 · Cited by 10
SentBS: Sentence-level Beam Search for Controllable Summarization
Shen, C., Cheng, L., Bing, L., You, Y., & Si, L. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.699 · Cited by 10
Target Concept Guided Medical Concept Normalization in Noisy User-Generated Texts
Kalyan, K. S., & Sangeetha, S. (2020) · Workshop on Knowledge Extraction and Integration for Deep Learning Architectures: Deep Learning Inside Out (DeeLIO) · doi:10.18653/v1/2020.deelio-1.8 · Cited by 9
Self-citationA Survey of Intent Classification and Slot-Filling Datasets for Task-Oriented DialogPreprint
Larson, S., & Leach, K. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2207.13211 · Cited by 9
TwitSenti: A Real-Time Twitter Sentiment Analysis and Visualization Framework
Murthy, J. S., Siddesh, G. M., & Srinivasa, K. G. (2019) · Journal of Information & Knowledge Management · doi:10.1142/s0219649219500138 · Cited by 8
LLMScore: Unveiling the Power of Large Language Models in Text-to-Image Synthesis EvaluationPreprint
Lu, Y., Yang, X., Li, X., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.11116 · Cited by 8
FLIRT: Feedback Loop In-context Red Teaming
Mehrabi, N., Goyal, P., Dupuy, C., et al. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.41 · Cited by 8
Collaborative Generative AI: Integrating GPT-k for Efficient Editing in Text-to-Image Generation
Zhu, W., Wang, X., Lu, Y., Fu, T.-J., Wang, X. E., Eckstein, M. P., & Wang, W. Y. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.685 · Cited by 8
Medical Concept Normalization in User-Generated Texts by Learning Target Concept Embeddings
Kalyan, K. S., & Sangeetha, S. (2020) · International Workshop on Health Text Mining and Information Analysis (LOUHI) · doi:10.18653/v1/2020.louhi-1.3 · Cited by 7
Self-citationLarge Language Models Need Holistically Thought in Medical Conversational QAPreprint
Weng, Y.-X., Li, B., Xia, F., Zhu, M., Sun, B., He, S.-Z., Liu, K., & Zhao, J. (2023) · arXiv.org · doi:10.48550/arxiv.2305.05410 · Cited by 7
Think Outside the Code: Brainstorming Boosts Large Language Models in Code GenerationPreprint
Li, X.-Y., Xue, J.-T., Xie, Z. Y., & Li, M. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.10679 · Cited by 7
Evaluating ChatGPT's Performance for Multilingual and Emoji-based Hate Speech DetectionPreprint
Das, M., Pandey, S. K., & Mukherjee, A. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.13276 · Cited by 7
DirecT2V: Large Language Models are Frame-Level Directors for Zero-Shot Text-to-Video GenerationPreprint
Hong, S., Seo, J., Shin, H., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.14330 · Cited by 7
Sarker, S., Qian, L., & Dong, X. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.07297 · Cited by 7
On the Multilingual Capabilities of Very Large-Scale English Language Models
Armengol-Estapé, J., Bonet, O. D. G., & Melero, M. (2022) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/4swrjz4r87aw · Cited by 6
Multidimensional Evaluation for Text Style Transfer Using ChatGPTPreprint
Lai, H., Toral, A., & Nissim, M. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.13462 · Cited by 6
Chain-of-Symbol Prompting Elicits Planning in Large Langauge ModelsPreprint
Hu, H., Lu, H., Zhang, H., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.10276 · Cited by 6
Malkiel, I., Alon, U., Yehuda, Y., Keren, S., Barkan, O., Ronen, R., & Koenigstein, N. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.07941 · Cited by 6
ChatGPT as Data Augmentation for Compositional Generalization: A Case Study in Open Intent DetectionPreprint
Fang, Y., Li, X., Thomas, S. W., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.13517 · Cited by 6
Exploring the Robustness of Large Language Models for Solving Programming ProblemsPreprint
Shirafuji, A., Watanobe, Y., Ito, T., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.14583 · Cited by 5
Jing, Y., Jin, R., Hu, J., Qiu, H., Wang, X., Wang, P., & Xiong, D. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.09829 · Cited by 5
Towards Zero-Shot and Few-Shot Table Question Answering using GPT-3Preprint
Srivastava, P., Ganu, T., & Guha, S. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.17284 · Cited by 4
Not All Metrics Are Guilty: Improving NLG Evaluation by Diversifying References
Tang, T., Lu, H., Jiang, Y. E., Huang, H., Zhang, D., Zhao, W. X., Kocmi, T., & Wei, F. (2023) · arXiv · doi:10.48550/arxiv.2305.15067 · Cited by 3
Data Augmentation for Neural Machine Translation using Generative Language ModelPreprint
Oh, S. J., Lee, S. A., & Jung, W. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2307.16833 · Cited by 3
PromptShots at the FinNLP-2022 ERAI Task: Pairwise Comparison and Unsupervised Ranking
Wiriyathammabhum, P. (2022) · Workshop on Financial Technology and Natural Language Processing (FinNLP) · doi:10.18653/v1/2022.finnlp-1.12 · Cited by 3
L2CEval: Evaluating Language-to-Code Generation Capabilities of Large Language ModelsPreprint
Ni, A., Yin, P., Zhao, Y., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.17446 · Cited by 3
Prompt scoring system for dialogue summarization using GPT-3Preprint
Prodan, G., & Pelican, E. (2022) · doi:10.36227/techrxiv.16652392.v2 · Cited by 2
MedGPTEval: A Dataset and Benchmark to Evaluate Responses of Large Language Models in MedicinePreprint
Xu, J., Lu, L., Yang, S., Liang, B., Peng, X., Pang, J., Ding, J., Shi, X., Yang, L., Song, H., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.07340 · Cited by 2
Neural Machine Translation Data Generation and Augmentation using ChatGPTPreprint
Yang, W., & Nicolai, G. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2307.05779 · Cited by 2
OPT: Open Pre-trained Transformer Language Models
Zhang, S., Roller, S., Goyal, N., Artetxe, M., Chen, M., Chen, S., Dewan, C., Diab, M., Li, X., Lin, X. V., et al. (2022) · arXiv · doi:10.48550/arxiv.2205.01068
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
Workshop, B., :, Scao, T. L., Fan, A., Akiki, C., Pavlick, E., Ilić, S., Hesslow, D., Castagné, R., Luccioni, A. S., et al. (2022) · arXiv · doi:10.48550/arxiv.2211.05100
Is ChatGPT a Good Sentiment Analyzer? A Preliminary Study
Wang, Z., Xie, Q., Feng, Y., Ding, Z., Yang, Z., & Xia, R. (2023) · arXiv · doi:10.48550/arxiv.2304.04339
A Paradigm Shift: The Future of Machine Translation Lies with Large Language Models
Lyu, C., Du, Z., Xu, J., Duan, Y., Wu, M., Lynn, T., Aji, A. F., Wong, D. F., Liu, S., & Wang, L. (2023) · arXiv · doi:10.48550/arxiv.2305.01181
Flagged first, then by citations
Research works printed but not linked: 18
- Abacha, A.B., Yim, W.-w., Adams, G., Snider, N., Yetisgen-Yildiz, M., 2023. Overview of the MEDIQA-Chat 2023 Shared Tasks on the Summarization & Generation of Doctor-Patient Conversations. In: Proceedings of the 5th Clinical Natural Language Processing Workshop. pp. 503–513.
- Choromanski, K.M., Likhosherstov, V., Dohan, D., Song, X., Gane, A., Sarlos, T., Hawkins, P., Davis, J.Q., Mohiuddin, A., Kaiser, L., et al., 2020. Rethinking Attention with Performers. In: International Conference on Learning Representations.
- Dai, 2015, Semi-supervised sequence learning, Adv. Neural Inf. Process. Syst., 28
- Gilson, 2023, How does ChatGPT perform on the United States medical licensing examination? The implications of large language models for medical education and knowledge assessment, JMIR Med. Educ., 9
- Guo, D., Ren, S., Lu, S., Feng, Z., Tang, D., Shujie, L., Zhou, L., Duan, N., Svyatkovskiy, A., Fu, S., et al., 2020. GraphCodeBERT: Pre-training Code Representations with Data Flow. In: International Conference on Learning Representations.
- Lin, C.-Y., 2004. Rouge: A package for automatic evaluation of summaries. In: Text Summarization Branches Out. pp. 74–81.
- Liu, 2021, Self-supervised learning: Generative or contrastive, IEEE Trans. Knowl. Data Eng., 35, 857
- Lu, S., Guo, D., Ren, S., Huang, J., Svyatkovskiy, A., Blanco, A., Clement, C., Drain, D., Jiang, D., Tang, D., et al., 2021. CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation. In: Thirty-Fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1).
- Radford, 2019, Language models are unsupervised multitask learners, OpenAI Blog, 1, 9
- Sennrich, R., Haddow, B., Birch, A., 2016. Improving Neural Machine Translation Models with Monolingual Data. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 86–96. · DOI · Paper page
- Sharma, S., Joshi, A., Mukhija, N., Zhao, Y., Bhathena, H., Singh, P., Santhanam, S., Biswas, P., 2022. Systematic review of effect of data augmentation using paraphrasing on Named entity recognition. In: NeurIPS 2022 Workshop on Synthetic Data for Empowering ML Research.
- Thakur, N., Reimers, N., Rücklé, A., Srivastava, A., Gurevych, I., 2021. BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models. In: Thirty-Fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2).
- Thapa, S., Naseem, U., Nasim, M., 2023. From humans to machines: can ChatGPT-like LLMs effectively replace human annotators in NLP tasks. In: Workshop Proceedings of the 17th International AAAI Conference on Web and Social Media.
- Valmeekam, K., Olmo, A., Sreedharan, S., Kambhampati, S., 2022. Large Language Models Still Can’t Plan (A Benchmark for LLMs on Planning and Reasoning about Change). In: NeurIPS 2022 Foundation Models for Decision Making Workshop.
- Vaswani, 2017, Attention is all you need, Adv. Neural Inf. Process. Syst., 30
- Xu, 2023, Multimodal learning with transformers: A survey, IEEE Trans. Pattern Anal. Mach. Intell., 10.1109/TPAMI.2023.3275156/mm1 · DOI
- Yang, 2019, Xlnet: Generalized autoregressive pretraining for language understanding, Adv. Neural Inf. Process. Syst., 32
- Zeng, A., Liu, X., Du, Z., Wang, Z., Lai, H., Ding, M., Yang, Z., Xu, Y., Zheng, W., Xia, X., et al., 2022. GLM-130B: An Open Bilingual Pre-trained Model. In: The Eleventh International Conference on Learning Representations.
Cited by
Sorted by citations
Large language models for code completion: A systematic literature review
Husein, R. A., Aburajouh, H., & Catal, C. (2025) · Computer Standards & Interfaces · doi:10.1016/j.csi.2024.103917 · Cited by 72
Pudasaini, S., Miralles-Pechuán, L., Lillis, D., & Llorens Salvador, M. (2025) · Journal of Academic Ethics · doi:10.1007/s10805-024-09576-x · Cited by 70
Lamaakal, I., Maleh, Y., El Makkaoui, K., Ouahbi, I., Pławiak, P., Alfarraj, O., Almousa, M., & Abd El-Latif, A. A. (2025) · Sensors · doi:10.3390/s25051318 · Cited by 60
Galli, C., Gavrilova, A. V., & Calciolari, E. (2025) · Information · doi:10.3390/info16050378 · Cited by 36
Perceptions of Artificial Intelligence and ChatGPT by Speech-Language Pathologists and Students
Austin, J., Benas, K., Caicedo, S., Imiolek, E., Piekutowski, A., & Ghanim, I. (2025) · American Journal of Speech-Language Pathology · doi:10.1044/2024_ajslp-24-00218 · Cited by 31
Metadata
- Authors
- Kalyan, K. S.
- Affiliations
- Department of Public (IN)
- Corresponding author
- Kalyan, Katikapalli Subramanyam
- Venue
- Natural Language Processing Journal
- Publisher
- Elsevier BV
- Published
- 2024-03
- Volume
- 6
- Pages
- 100048
- ISSN
- 2949-7191 (ISSN-L)
- OpenAlex
- W4389977189
- Cited by
- 335
- References
- 501
- Citations by year
- 2026: 65 · 2025: 165 · 2024: 105
- FWCI
- 31.54
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Natural Language Processing Journal (publishedVersion, cc-by-nc-nd)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- large language models, ChatGPT, GPT-4, pre-trained language models, transformer models, transfer learning, self-supervised learning, downstream tasks, multilingual performance, data annotation, data augmentation, robustness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
