How is ChatGPT's behavior changing over time?
Open accessAbstract
GPT-3.5 and GPT-4 are the two most widely used large language model (LLM) services. However, when and how these models are updated over time is opaque. Here, we evaluate the March 2023 and June 2023 versions of GPT-3.5 and GPT-4 on several diverse tasks: 1) math problems, 2) sensitive/dangerous questions, 3) opinion surveys, 4) multi-hop knowledge-intensive questions, 5) generating code, 6) US Medical License tests, and 7) visual reasoning. We find that the performance and behavior of both GPT-3.5 and GPT-4 can vary greatly over time. For example, GPT-4 (March 2023) was reasonable at identifying prime vs. composite numbers (84% accuracy) but GPT-4 (June 2023) was poor on these same questions (51% accuracy). This is partly explained by a drop in GPT-4's amenity to follow chain-of-thought prompting. Interestingly, GPT-3.5 was much better in June than in March in this task. GPT-4 became less willing to answer sensitive questions and opinion survey questions in June than in March. GPT-4 performed better at multi-hop questions in June than in March, while GPT-3.5's performance dropped on this task. Both GPT-4 and GPT-3.5 had more formatting mistakes in code generation in June than in March. We provide evidence that GPT-4's ability to follow user instructions has decreased over time, which is one common factor behind the many behavior drifts. Overall, our findings show that the behavior of the "same" LLM service can change substantially in a relatively short amount of time, highlighting the need for continuous monitoring of LLMs.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Unexpected deterioration of model performance over time occurs for both open-source and closed-source generative AI models.
“For both open-source and closed-source models, unexpected deterioration of model performance over time highlights the need for continuous model monitoring (Chen et al. 2023).”
From Generative AI (2024) · cites this paper as Chen et al. 2023
The Conclusions and Future Work section reports that GPT-3.5 and GPT-4 worsened on some tasks and recommends continuous monitoring. The paper studies these LLM services, but does not establish deterioration over time for open-source models. Page numbers are unavailable in our copy. (LLM, conf=0.89)
Experimental — accuracy not yet measuredThe dependence of criteria on a specific prompt and LLM's outputs means that any change to the LLM pipeline can cause criteria drift; for example, LLM APIs can be replaced with new models, resulting in prompt drift.
“In particular, the dependence of criteria on a specific prompt and LLM's outputs means that any change to the LLM pipeline can cause criteria drift: for example, LLM APIs can be replaced with new models, resulting in prompt drift [4].”
From Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences (2024) · cites this paper as [4]
The Introduction (page unavailable) says that changes in an LLM’s response to a prompt can break a downstream pipeline. Section 3.6 and the accompanying discussion (pages unavailable) describe prompt-engineered LangChain modules as brittle when LLM behavior drifts. The paper does not establish that any pipeline change causes criteria drift, or describe replacing an API with a new model as causing prompt drift. (LLM, conf=0.84)
Experimental — accuracy not yet measuredThe referenced LLMs may be updated and yield different outputs for the configuration and prompts used; temperature was set to 0.0 to mitigate this, but studies report significant changes in GPT-3.5 and GPT-4 over time.
“In our task-solving evaluation, the referenced LLMs may be updated and yield different outputs for the configuration and prompts used in this work. The temperature parameter of the models was set to 0.0 to mitigate this as much as possible. However, some studies have reported significant changes in the behavior of GPT-3.5 and GPT-4 over time [Chen et al. 2023], though it is not clear whether these”
From How Do Programming Students Use Generative AI? (2025) · cites this paper as Chen et al. 2023
The reproducibility of LLMs can sometimes be affected by changes in the underlying model that are not transparently documented and available to users, and which can change the outputs generated in response to the same prompt over time.
“However, the reproducibility of LLMs can sometimes be affected by changes in the underlying model that are not transparently documented and available to users, and which can change the outputs generated in response to the same prompt over time (Chen et al., 2023; Gartlehner et al., 2024; O’Connor et al., 2024; Syriani et al., 2023).”
From Analysis of article screening and data extraction performance by an AI systematic literature review platform (2025) · cites this paper as Chen et al., 2023
Our findings on accuracy, precision, and recall reinforce empirical studies that highlight substantial differences in performance and behavior between consecutive model generations like GPT-3.5 and GPT-4.
“Our findings on accuracy, precision, and recall reinforce empirical studies that highlight substantial differences in performance and behavior between consecutive model generations like GPT-3.5 and GPT-4 [12].”
From Sentiment Analysis in the Age of Generative AI (2024) · cites this paper as [12]
Differences in model outputs under the same external parameter settings are particularly pronounced in closed-source models, which often undergo model updates that can significantly impact their performance.
“Differentiating model outputs under the same external parameter settings are particularly pronounced in closed-source models, which often undergo model updates that can significantly impact their performance [12].”
From Sentiment Analysis in the Age of Generative AI (2024) · cites this paper as [12]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2201.11903 · Cited by 4,263 · see citing works
ReAct: Synergizing Reasoning and Acting in Language Models
Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., & Cao, Y. (2022) · arXiv · doi:10.48550/arxiv.2210.03629 · Cited by 608 · see citing works
Capabilities of GPT-4 on Medical Challenge Problems
Nori, H., King, N., McKinney, S. M., Carignan, D., & Horvitz, E. (2023) · arXiv · doi:10.48550/arxiv.2303.13375 · Cited by 509 · see citing works
Program Synthesis with Large Language Models
Austin, J., Odena, A., Nye, M., Bosma, M., Michalewski, H., Dohan, D., Jiang, E., Cai, C., Terry, M., Le, Q., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07732 · Cited by 388 · see citing works
Bang, Y., Cahyawijaya, S., Lee, N., Dai, W., Su, D., Wilie, B., Lovenia, H., Ji, Z., Yu, T., Chung, W., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.04023 · Cited by 337 · see citing works
Holistic Evaluation of Language Models
Liang, P., Bommasani, R., Lee, T., Tsipras, D., Soylu, D., Yasunaga, M., Zhang, Y., Narayanan, D., Wu, Y., Kumar, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2211.09110 · Cited by 255 · see citing works
On the Measure of Intelligence
Chollet, F. (2019) · arXiv · doi:10.48550/arxiv.1911.01547 · Cited by 218 · see citing works
Yu, T., Zhang, R., Yang, K., Yasunaga, M., Wang, D., Li, Z., Ma, J., Li, I., Yao, Q., Roman, S., et al. (2018) · arXiv · doi:10.48550/arxiv.1809.08887 · Cited by 193 · see citing works
News Summarization and Evaluation in the Era of GPT-3
Goyal, T., Li, J. J., & Durrett, G. (2022) · arXiv · doi:10.48550/arxiv.2209.12356 · Cited by 175 · see citing works
Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned
Ganguli, D., Lovitt, L., Kernion, J., Askell, A., Bai, Y., Kadavath, S., Mann, B., Perez, E., Schiefer, N., Ndousse, K., et al. (2022) · arXiv · doi:10.48550/arxiv.2209.07858 · Cited by 130 · see citing works
Whose Opinions Do Language Models Reflect?
Santurkar, S., Durmus, E., Ladhak, F., Lee, C., Liang, P., & Hashimoto, T. (2023) · arXiv · doi:10.48550/arxiv.2303.17548 · Cited by 114 · see citing works
Evaluating the logical reasoning ability of chatgpt and GPT-4
Liu, H., Ning, R., Teng, Z., Liu, J., Zhou, Q., & Zhang, Y. (2023) · arXiv · doi:10.48550/arxiv.2304.03439 · Cited by 100 · see citing works
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W. W., Salakhutdinov, R., & Manning, C. D. (2018) · arXiv · doi:10.48550/arxiv.1809.09600 · Cited by 97 · see citing works
How Language Model Hallucinations Can Snowball
Zhang, M., Press, O., Merrill, W., Liu, A., & Smith, N. A. (2023) · arXiv · doi:10.48550/arxiv.2305.13534 · Cited by 77 · see citing works
Did the Model Change? Efficiently Assessing Machine Learning API ShiftsPreprint
Chen, L., Cai, T., Zaharia, M., & Zou, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.14203 · Cited by 8 · see citing works
Self-citationRealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models
Gehman, S., Gururangan, S., Sap, M., Choi, Y., & Smith, N. A. (2020) · arXiv · doi:10.48550/arxiv.2009.11462
Flagged first, then by citations
Research works printed but not linked: 8
- Lingjiao Chen, Zhihua Jin, Evan Sabri Eyuboglu, Christopher Ré, Matei Zaharia, and James Y Zou. Hapi: A large-scale longitudinal dataset of commercial ml api predictions. Advances in Neural Information Processing Systems, 35:24571-24585, 2022.
- Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 21), pages 2633-2650, 2021.
- Joost CF de Winter. Can chatgpt pass high school exams on english language comprehension. Researchgate. Preprint, 2023.
- Richard Guy. Unsolved problems in number theory, volume 1. Springer Science & Business Media, 2004.
- Wenxiang Jiao, Wenxuan Wang, Jen-tse Huang, Xing Wang, and Zhaopeng Tu. Is chatgpt a good translator? a preliminary study. arXiv preprint arXiv:2301.08745, 2023. · arXiv
- Daniel Martin Katz, Michael James Bommarito, Shang Gao, and Pablo Arredondo. Gpt-4 passes the bar exam. Available at SSRN 4389233, 2023.
- Tiffany H Kung, Morgan Cheatham, Arielle Medenilla, Czarina Sillos, Lorie De Leon, Camille Elepaño, Maria Madriaga, Rimel Aggabao, Giezel Diaz-Candido, James Maningo, et al. Performance of chatgpt on usmle: Potential for ai-assisted medical education using large language models. PLoS digital health, 2(2):e0000198, 2023.
- Shangqing Tu, Chunyang Li, Jifan Yu, Xiaozhi Wang, Lei Hou, and Juanzi Li. Chatlog: Recording and analyzing chatgpt across time. arXiv preprint arXiv:2304.14106, 2023. · arXiv
Cited by
Sorted by citations
Feuerriegel, S., Hartmann, J., Janiesch, C., & Zschech, P. (2024) · Business & Information Systems Engineering · doi:10.1007/s12599-023-00834-7 · Cited by 1,409 · see citing works
“For both open-source and closed-source models, unexpected deterioration of model performance over time highlights the need for continuous model monitoring (Chen et al. 2023).”
Generative AI in Medicine and Healthcare: Promises, Opportunities and Challenges
Zhang, P., & Kamel Boulos, M. N. (2023) · Future Internet · doi:10.3390/fi15090286 · Cited by 411 · see citing works
“The more recent a model’s version is, the better it tends to perform, e.g., [60], but this is not always guaranteed [61].”
BackgroundPolak, M. P., & Morgan, D. (2024) · Nature Communications · doi:10.1038/s41467-024-45914-8 · Cited by 365 · see citing works
Can Generative AI improve social science?
Bail, C. A. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2314021121 · Cited by 369 · see citing works
Testing theory of mind in large language models and humans
Strachan, J. W. A., Albergo, D., Borghini, G., Pansardi, O., Scaliti, E., Gupta, S., Saxena, K., Rufo, A., Panzeri, S., Manzi, G., et al. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-01882-z · Cited by 299 · see citing works
Sentiment Analysis in the Age of Generative AI
Krugmann, J. O., & Hartmann, J. (2024) · Customer Needs and Solutions · doi:10.1007/s40547-024-00143-4 · Cited by 189 · see citing works
“Differentiating model outputs under the same external parameter settings are particularly pronounced in closed-source models, which often undergo model updates that can significantly impact their performance [12].”
Hallucination‐Free? Assessing the Reliability of Leading <scp>AI</scp> Legal Research Tools
Magesh, V., Surani, F., Dahl, M., Suzgun, M., Manning, C. D., & Ho, D. E. (2025) · Journal of Empirical Legal Studies · doi:10.1111/jels.12413 · Cited by 182 · see citing works
Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferences
Shankar, S., Zamfirescu-Pereira, J. D., Hartmann, B., Parameswaran, A., & Arawjo, I. (2024) · Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3654777.3676450 · Cited by 151 · see citing works
Biomedical knowledge graph-optimized prompt generation for large language models
Soman, K., Rose, P. W., Morris, J. H., Akbas, R. E., Smith, B., Peetoom, B., Villouta-Reyes, C., Cerono, G., Shi, Y., Rizk-Jackson, A., et al. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btae560 · Cited by 135 · see citing works
Art or Artifice? Large Language Models and the False Promise of Creativity
Chakrabarty, T., Laban, P., Agarwal, D., Muresan, S., & Wu, C.-S. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642731 · Cited by 135 · see citing works
Fraser, H., Crossland, D., Bacher, I., Ranney, M., Madsen, T., & Hilliard, R. (2023) · JMIR mHealth and uHealth · doi:10.2196/49995 · Cited by 128 · see citing works
Generalization bias in large language model summarization of scientific research
Peters, U., & Chin-Yee, B. (2025) · Royal Society Open Science · doi:10.1098/rsos.241776 · Cited by 126 · see citing works
Performance of large language models on a neurology board-style examination
Schubert, M. C., Wick, W., & Venkataramani, V. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.46721 · Cited by 115 · see citing works
Evaluating the persuasive influence of political microtargeting with large language models
Hackenburg, K., & Margetts, H. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2403116121 · Cited by 114 · see citing works
Asai, A., Wu, Z., Wang, Y., Sil, A., & Hajishirzi, H. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.11511 · Cited by 101 · see citing works
“However, depending on such proprietary LMs can raise API costs and diminish reproducibility (Chen et al., 2023).”
BackgroundDeception abilities emerged in large language models
Hagendorff, T. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2317967121 · Cited by 100 · see citing works
Performance of Large Language Models on Medical Oncology Examination Questions
Longwell, J. B., Hirsch, I., Binder, F., Gonzalez Conchas, G. A., Mau, D., Jang, R., Krishnan, R. G., & Grant, R. C. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.17641 · Cited by 81 · see citing works
Yan, C., Grabowska, M. E., Dickson, A. L., Li, B., Wen, Z., Roden, D. M., Michael Stein, C., Embí, P. J., Peterson, J. F., Feng, Q., et al. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01038-3 · Cited by 73 · see citing works
Tran, V.-T., Gartlehner, G., Yaacoub, S., Boutron, I., Schwingshackl, L., Stadelmaier, J., Sommer, I., Alebouyeh, F., Afach, S., Meerpohl, J., et al. (2024) · Annals of Internal Medicine · doi:10.7326/m23-3389 · Cited by 66 · see citing works
Evaluating the Utilities of Foundation Models in Single-cell Data Analysis
Liu, T., Li, K., Wang, Y., Li, H., & Zhao, H. (2023) · doi:10.1101/2023.09.08.555192 · Cited by 62 · see citing works
Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMs
Balloccu, S., Schmidtová, P., Lango, M., & Dusek, O. (2024) · Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.eacl-long.5 · Cited by 70 · see citing works
Dotan, R., Parker, L. S., & Radzilowicz, J. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3659023 · Cited by 71 · see citing works
“For example, inaccuracy and bias in the output of GenAI are well-documented [e.g., 13, 26, 31, 51, 71], and the methods and algorithms by which the output is generated are opaque, which undermines the replicability of results [e.g., 7].”
MethodHow Do Programming Students Use Generative AI?
Rahe, C., & Maalej, W. (2025) · Proceedings of the ACM on Software Engineering · doi:10.1145/3715762 · Cited by 45 · see citing works
“5 and GPT-4 over time [Chen et al. 2023], though it is not clear whether these results are based on architectural changes, continuous training, or simply the non-deterministic nature of the LLMs.”
Human innovation and the creative agency of the world in the age of generative AI
Peschl, M. F. (2024) · Possibility Studies & Society · doi:10.1177/27538699241238049 · Cited by 23 · see citing works
van Duijn, M., van Dijk, B., Kouwenhoven, T., de Valk, W., Spruit, M., & vanderPutten, P. (2023) · Proceedings of the 27th Conference on Computational Natural Language Learning (CoNLL) · doi:10.18653/v1/2023.conll-1.25 · Cited by 24 · see citing works
Keeping Humans in the Loop: Human-Centered Automated Annotation with Generative AI
Pangakis, N., & Wolken, S. (2025) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v19i1.35883 · Cited by 27 · see citing works
OctoPack: Instruction Tuning Code Large Language Models
Muennighoff, N., Liu, Q., Zebaze, A., Zheng, Q., Hui, B., Zhuo, T. Y., Singh, S., Tang, X., von Werra, L., & Longpre, S. (2023) · arXiv · doi:10.48550/arxiv.2308.07124 · Cited by 17 · see citing works
“This approach is problematic as (1) closed-source APIs keep changing and have unpredictable availability (Pozzobon et al., 2023; Chen et al., 2023a), (2) it relies on the assumption that a more capable model exists (3) it can reinforce model hallucination (Gudibande et al., 2023) and (4) ,…”
BackgroundBeyond Accuracy: Investigating Error Types in GPT-4 Responses to USMLE Questions
Roy, S., Khatua, A., Ghoochani, F., Hadler, U., Nejdl, W., & Ganguly, N. (2024) · Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3626772.3657882 · Cited by 17 · see citing works
Nykvist, B., Macura, B., Xylia, M., & Olsson, E. (2025) · Environmental Evidence · doi:10.1186/s13750-025-00360-x · Cited by 17 · see citing works
Language-Models-as-a-Service: Overview of a New Paradigm and its Challenges
La Malfa, E., Petrov, A., Frieder, S., Weinhuber, C., Burnell, R., Nazar, R., Cohn, A., Shadbolt, N., & Wooldridge, M. (2024) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.15865 · Cited by 9 · see citing works
Cassell, K., Ologunowa, A., Rastegar-Mojarad, M., Chun, B., Huang, Y.-L., Wang, D., & Cossrow, N. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1662202 · Cited by 6 · see citing works
Fanfiction in the Age of AI: Community Perspectives on Creativity, Authenticity and Adoption
Alfassi, R., Cooper, A., Mitchell, Z., Calabro, M., Shaer, O., & Mokryn, O. (2026) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2025.2531272 · Cited by 4 · see citing works
- Authors
- Chen, L., Zaharia, M., & Zou, J.
- Affiliations
- Stanford University (US)
- University of California, Berkeley (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- Volume
- 6
- Issue
- 2
- ISSN
- 2644-2353 (ISSN-L)
- OpenAlex
- W4392681182
- Cited by
- 507
- References
- 16
- Citations by year
- 2026: 128 · 2025: 139 · 2024: 163 · 2023: 77
- FWCI
- 63.46
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Harvard Data Science Review (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Keywords
- large language models, GPT-3.5, GPT-4, model drift, model updating, chain-of-thought prompting, opinion surveys, multi-hop question answering, code generation, visual reasoning, medical licensing examinations, instruction following
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
