How Is ChatGPT’s Behavior Changing Over Time?
Open accessThe DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
GPT-3.5 and GPT-4 are the two most widely used large language model (LLM) services. However, when and how these models are updated over time is opaque. Here, we evaluate the March 2023 and June 2023 versions of GPT-3.5 and GPT-4 on several diverse tasks: 1) math problems, 2) sensitive/dangerous questions, 3) opinion surveys, 4) multi-hop knowledge-intensive questions, 5) generating code, 6) US Medical License tests, and 7) visual reasoning. We find that the performance and behavior of both GPT-3.5 and GPT-4 can vary greatly over time. For example, GPT-4 (March 2023) was reasonable at identifying prime vs. composite numbers (84% accuracy) but GPT-4 (June 2023) was poor on these same questions (51% accuracy). This is partly explained by a drop in GPT-4’s amenity to follow chain-of-thought prompting. Interestingly, GPT-3.5 was much better in June than in March in this task. GPT-4 became less willing to answer sensitive questions and opinion survey questions in June than in March. GPT-4 performed better at multi-hop questions in June than in March, while GPT-3.5’s performance dropped on this task. Both GPT-4 and GPT-3.5 had more formatting mistakes in code generation in June than in March. We provide evidence that GPT-4’s ability to follow user instructions has decreased over time, which is one common factor behind the many behavior drifts. Overall, our findings show that the behavior of the “same” LLM service can change substantially in a relatively short amount of time, highlighting the need for continuous monitoring of LLMs.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Our finding that most LLMs consistently produced algorithmic overgeneralizations across multiple occasions adds nuance to prior research that found that, in some domains, LLMs produced unstable or inconsistent responses over time [48–50].”
From Generalization bias in large language model summarization of scientific research (2025) · cites this paper as [48]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Our findings thus offer novel contributions to research on overgeneralization in science communication [17,18,54] and the shortcomings of LLMs in text summarization [48,52,55], revealing a subtle form of ‘hallucination’ that has not yet been documented in the literature.”
From Generalization bias in large language model summarization of scientific research (2025) · cites this paper as [48]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Part of this paper’s reference list could not be split into single entries.
Cited by
Items 301–400 of 517
Bang, S. H., Han, S., Turchioe, M. R., Ellison, M., Dai, S., Happ, M. B., Russell, D., & Creber, R. M. (2026) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocag123 · Cited by 4
Cho, A.-Y., Eo, S., Jeon, M., Kim, J., Ihm, J., & Seo, D. (2026) · JMIR Medical Education · doi:10.2196/91809 · Cited by 2
Take a Chance on Me: Aleatory Poetry, Generative AI, and the External Demarcation Problem
Chen, M. (2023) · Journal of Aesthetics and Art Criticism · doi:10.1093/jaac/kpad042 · Cited by 2
Evolution of ChatGPT evaluations in healthcare: Still at the beginning?
Kocbek, P., Fijačko, N., & Štiglic, G. (2023) · Resuscitation · doi:10.1016/j.resuscitation.2023.110042 · Cited by 5
España-Bonet, C. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.787 · Cited by 14
Exploring Failure Cases in Multimodal Reasoning About Physical Dynamics
Ghaffari, S., & Krishnaswamy, N. (2024) · Proceedings of the AAAI Symposium Series · doi:10.1609/aaaiss.v3i1.31189 · Cited by 9
Ethical Judgment using Large Language Model
Hashimoto, T. (2024) · International Conference on Computer and Automation Engineering · doi:10.1109/iccae59995.2024.10569797 · Cited by 2
Wang, A., Prihar, E., Haim, A., & Heffernan, N. T. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-64312-5_29 · Cited by 6
Lu, W. (2024) · Communication Teacher · doi:10.1080/17404622.2024.2385343 · Cited by 5
Programming Chatbots Using Natural Language: Generating Cervical Spine MRI Impressions
Javan, R., Kim, T., Abdelmonem, A. M., Ismail, A. S., Jaamour, F., Melnyk, O., & Heekin, M. (2024) · Cureus · doi:10.7759/cureus.69410 · Cited by 4
CoURAGE: A Framework to Evaluate RAG Systems
Galla, D., Hoda, S., Zhang, M., Quan, W., Yang, T. D., & Voyles, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-70242-6_37 · Cited by 4
Multi-Lingual Semantic-Based Sentiment Analysis Using Generative AI
Biswas, P., Arockiam, D., Neogi, S. G., Sengupta, S., Saraswat, S., & Chanda, R. (2024) · International Conference on Computer, Electronics & Electrical Engineering & their Applications (IC2E3) · doi:10.1109/ic2e362166.2024.10827103 · Cited by 2
Woolsey, C., Gondy Leroy, A., & Maltman, N. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.128151 · Cited by 2
Reducing echo chamber effects: an allostatic regulator for recommendation algorithms
Henry, N., Pedersen, M., Williams, M. N., Martin, J. L. B., & Donkin, L. (2025) · Journal of Psychology and AI · doi:10.1080/29974100.2025.2517191 · Cited by 3
Evaluating ChatGPT’s Concordance with Clinical Guidelines of Ménière’s Disease in Chinese
Lin, M.-J., Hsieh, L.-C., & Chen, C. (2025) · Diagnostics · doi:10.3390/diagnostics15162006 · Cited by 3
Comparative Analysis of Styles in LLM-Generated Code for LeetCode Problems: A Preliminary Study
Zhang, Y., Chen, T. Y., Huang, R., Pike, M., Towey, D., Ying, Z., & Zhou, Z. Q. (2025) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac65507.2025.00219 · Cited by 2
Ghaharian, K., Soligo, M., Young, R., Golab, L., Kraus, S. W., & Wells, S. A. (2025) · Journal of Gambling Studies · doi:10.1007/s10899-025-10430-x · Cited by 3
Generation of Robot Manipulation Plans Using Generative Large Language Models
Töberg, J.-P., & Cimiano, P. (2023) · IEEE International Conference on Robotic Computing (IRC) · doi:10.1109/irc59093.2023.00039 · Cited by 4
The Illusion of Boundless AI: Analyzing Limitations and Ethical ConcernsPreprint
Bhuiyan, M. M. (2024) · doi:10.36227/techrxiv.171742375.53309794/v1 · Cited by 2
ChatGPT: A Reliable Laboratory Medicine Consult?
El-Khoury, J. M. (2024) · Clinical Chemistry · doi:10.1093/clinchem/hvae100 · Cited by 2
A Scoping Review of LLM-as-a-Judge in Healthcare and the MedJUDGE Framework
Li, C., Akhtar, Z., Kwak, M., Ji, Y., Zhang, H., Obi, T., Ren, Y., Wu, X., Sivarajkumar, S., Lehmann, H. P., et al. (2026) · arXiv · doi:10.48550/arxiv.2604.25933 · Cited by 1
SWEET - Weakly Supervised Person Name Extraction for Fighting Human Trafficking
Liu, J., Yu, H., Sujaya, V., Nair, P., Pelrine, K., & Rabbany, R. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.219 · Cited by 9
OpenJustice.ai: A Global Open-Source Legal Language Model
Dahan, S., Bhambhoria, R. V., Liang, D., & Zhu, X. (2023) · Frontiers in artificial intelligence and applications · doi:10.3233/faia230995 · Cited by 7
Mejía-Trejo, J. (2024) · doi:10.55965/abib.2024.9786076984505 · Cited by 11
Comparing ChatGPT And LSTM In Predicting Changes In Quarterly Financial Metrics
Chaiko, V. (2024) · Business & Management Compass · doi:10.56065/crmc1136 · Cited by 2
GENIUS- A Revolutionary SaaS Platform Empowering Users With AI Capabilities
Sahasra, K., Kumar, A. Y. V., Blessy, C., Nikhil, G. S. S., Vanathi, A., & Kishore, V. R. (2024) · Advances in computer science research · doi:10.2991/978-94-6463-471-6_86 · Cited by 3
Prompt Optimization with Human Annotations Using Pairwise Comparison
Lu, Y.-E., & Lee, S.-J. (2024) · IEEE International Conference on e-Business Engineering (ICEBE) · doi:10.1109/icebe62490.2024.00012 · Cited by 1
Enhancing API Labelling with BERT and GPT: An Exploratory Study
Morais, G., Lemelin, E., Adda, M., & Bork, D. (2025) · Lecture notes in business information processing · doi:10.1007/978-3-031-79059-1_11 · Cited by 1
Xue, C., Gao, J., & Gu, Z. (2025) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2025.3574299 · Cited by 2
Contextual Knowledge Pursuit for Faithful Visual Synthesis
Luo, J., Chan, K. H. R., Dimos, D., & Vidal, R. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-91907-7_12 · Cited by 5
Yi, V. N., Scialdone, A., Flusché, A. M., Reitz, K., Lewis, H. C., Tian, W. M., Fisher, E., Rezak, K. M., & Patel, A. (2025) · Journal of Health Communication · doi:10.1080/10810730.2025.2547321 · Cited by 3
The Future of Artificial Intelligence and Artificial Intelligence in Primary Care
Cheema, A. M. (2025) · Primary Care Clinics in Office Practice · doi:10.1016/j.pop.2025.07.011 · Cited by 1
Ignjatović, A., Apostolovic, M. A., Stevanović, L., Radovanović, P., Topalović, M., Filipović, T., & Otašević, S. (2025) · Health Informatics Journal · doi:10.1177/14604582251381260 · Cited by 4
Dilbaz, O. F., Ozates, M. N., Bolat, B., Gündüz-Demir, Ç., & Kulaç, İ. (2025) · American Journal of Clinical Pathology · doi:10.1093/ajcp/aqaf091 · Cited by 2
Malmqvist, L. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-01823-6_4 · Cited by 3
Rattananan, W. (2025) · Medicina · doi:10.3390/medicina61111931 · Cited by 1
Liao, Z., Huang, J., Liu, Y., Li, F., Tang, L., Cong, L., Wang, H., & Luo, S. (2025) · Aesthetic Plastic Surgery · doi:10.1007/s00266-025-05314-9 · Cited by 2
Parenti, S. I., Maglioni, A., Evangelisti, E., Gracco, A. L. T., Badiali, G., Bonetti, G. A., & Bartolucci, M. L. (2025) · Applied Sciences · doi:10.3390/app152312441 · Cited by 1
Ayravainen, L. E. M., Hinds, J. M., & Davidson, B. I. (2025) · PLoS ONE · doi:10.1371/journal.pone.0336269 · Cited by 4
Nordmann, K., & Fischer, F. (2025) · BMC Health Services Research · doi:10.1186/s12913-025-13901-4 · Cited by 2
Towards More Realistic Extraction Attacks: An Adversarial Perspective
More, Y., Ganesh, P., & Farnadi, G. (2025) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl.a.62 · Cited by 19
Nguyen, T. A., Le, T. H. M., & Babar, M. A. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3744916.3787796 · Cited by 3
From Requirements to Models: A Case Study of Large Language Models on Epidemiological Modeling
Karamchandani, A., Seyeddabbaghi, F., & Fokaefs, M. (2025) · IEEE International Conference on Collaborative Advances in Software and Computing (CASCON) · doi:10.1109/cascon66301.2025.00082 · Cited by 2
Position Bias in LLMs for Critical Decision Support: A Case Study on Multiple Casualty Triage
Bolin, U. W., Cohen, K., Björnesjö, H., & Tegen, A. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-14107-1_42 · Cited by 2
Language-based assessments can predict psychological and subjective well-being
Mesquiti, S., Cosme, D., Nook, E. C., Falk, E. B., & Burns, S. M. (2026) · Communications Psychology · doi:10.1038/s44271-026-00400-3 · Cited by 5
An MCP Tool for Bug Localization by Leveraging Modification Frequencies of Similar Bug Reports
Ahn, J., Lee, S., & Kang, S. (2026) · IEEE Access · doi:10.1109/access.2026.3665337 · Cited by 2
Health or taste? Investigating food decision-making patterns in humans and large language models
Lyu, M., & Huang, J. (2026) · Appetite · doi:10.1016/j.appet.2026.108512 · Cited by 1
PromptFuzz: Harnessing Fuzzing Techniques for Robust Testing of Prompt Injection in LLMs
Shao, Y., Yu, J., Miao, H., Gou, G., Li, Z., & Shi, J. (2026) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2026.3666893 · Cited by 32
Adaptive diagnostic reasoning framework for pathology with multimodal large language models
Hong, Y., Kao, K.-C., Edwards, L., Liu, N. T., Huang, C., Oliveira-Kowaleski, A., Hsieh, C.-J., & Lin, N. Y. C. (2026) · Communications Medicine · doi:10.1038/s43856-026-01491-z · Cited by 3
When ChatGPT joins the team: a mixed-methods study of AI-mediated collaborative lesson design
Agostini, D., Serbati, A., Picasso, F., & Lipnevich, A. A. (2026) · Frontiers in Education · doi:10.3389/feduc.2026.1751618 · Cited by 4
Ouyang, J., & Narechania, A. A. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798415 · Cited by 2
Buldur, M., Gizem, A. Y. A. N., & Misilli, T. (2026) · Dental Traumatology · doi:10.1111/edt.70071 · Cited by 2
Evaluating large language models for pharmacotherapy simulations: a mixed-methods study
Farrag, A. N., El-Zeiny, A., & Ali, A. M. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02626-1 · Cited by 4
Chow, J., & Patlas, M. N. (2026) · Japanese Journal of Radiology · doi:10.1007/s11604-026-02009-2 · Cited by 1
ChatGPT as a digital pharmacist: A systematic review of medication counselling accuracy
Azmakan, H., Nabipour, A., Tehrani, N. G., Najari, N., hafshjani, P. F., Marvast, A. F., Mani, S., Sichani, N. A., Pakdaman, S. F., Shieh, M., et al. (2026) · Intelligence-Based Medicine · doi:10.1016/j.ibmed.2026.100417 · Cited by 4
Baskaran, R., Sirikonda, S., Singh, A., Srinivasan, S., Leveridge, B., Casals-Farre, O., Kaur, H., Manivannan, S., Elangovan, K., Quek, C. W. N., et al. (2026) · Frontiers in Medicine · doi:10.3389/fmed.2026.1888765 · Cited by 1
Altan, B., & ÇINAR, Ş. (2026) · BMC Oral Health · doi:10.1186/s12903-026-09438-5 · Cited by 1
How Reflection Enhances Task Factuality in the Use of Large Language Models
Tarafdar, M., Adam, M., & Nguyen, L. T. (2026) · Journal of Management Information Systems · doi:10.1080/07421222.2026.2692283 · Cited by 2
Yu, H., Iqbal, S., Neto, E. C. P., Buffett, S., & Taylor, A. (2026) · International Journal of Information Security · doi:10.1007/s10207-026-01321-2 · Cited by 2
Cai, H. (2026) · JMIR AI · doi:10.2196/111307 · Cited by 1
Künstliche-Intelligenz-Quellen
Besenbäck, R., & Prager, L. (2024) · De Gruyter eBooks · doi:10.1515/9783111351490-032 · Cited by 1
الشريف, د. م. س., & الله, د. م. ن. ع. ع. (2024) · Maǧallaẗ al-'ulūm al-Qānūniyyaẗ wa-al-Iqtiṣādiyyaẗ/Maǧallaẗ al-'ulūm al-Qānūniyyaẗ wa-al-Iqtiṣādiyyaẗ · doi:10.21608/jelc.2024.342115 · Cited by 1
Scientific Paradigm of Informatics as a Third Culture
Zatsman, I. M. (2023) · Scientific and Technical Information Processing · doi:10.3103/s0147688223040111 · Cited by 1
Improving the Diproche CNL through Autoformalization via Large Language Models
Carl, M. (2024) · Electronic Proceedings in Theoretical Computer Science · doi:10.4204/eptcs.400.4 · Cited by 1
Evaluating ChatGPT's effectiveness and tendencies in Japanese internal medicine
Kaneda, Y., Tayuinosho, A., Tomoyose, R., Takita, M., Hamaki, T., Tanimoto, T. E., & Ozaki, A. (2024) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.14011 · Cited by 3
How Can Generative AI Enhance Software Management? Is It Better Done than Perfect?
Cabrero‐Daniel, B., Fazelidehkordi, Y., & Nouri, A. (2024) · Springer eBooks · doi:10.1007/978-3-031-55642-5_11 · Cited by 1
Alcances y límites educativos de la IA: control e ideología en el uso de ChatGPT
Jarquín-Ramírez, M.-R., Martínez, H. A., & Gutiérrez, E. J. D. (2024) · DIDAC · doi:10.48102/didac.2024..84_jul-dic.217 · Cited by 4
The norm-definition of artificial intelligence: a new perspective
Kovanov, E. S. (2024) · Current Issues of the State and Law · doi:10.20310/2587-9340-2024-8-2-206-217 · Cited by 1
Aplicaciones Prácticas de la Generativa en la Educación Inclusiva
Balladares, G. E. G., Caiza, N. E. S., Córdova, A. A. A., Pilaguano, S. M. A., Achig, V. H. Z., Chalapu, J. G. O., & Morillo, A. J. R. (2024) · Estudios y Perspectivas Revista Científica y Académica · doi:10.61384/r.c.a..v4i2.344 · Cited by 3
Ficko, A., Sarkki, S., Gültekin, Y. S., Egli, A., & Hiedanpää, J. (2024) · Geography and sustainability · doi:10.1016/j.geosus.2024.100257 · Cited by 7
RealDriftGenerator: A Novel Approach to Generate Concept Drift in Real World Scenario
Lin, B., Huang, C., Zhu, X., & Jin, N. (2024) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc54092.2024.10831569 · Cited by 1
Hanna, D. R., Creswell, M. L., Terry, R. S., Vergamini, L. B., Sardiu, M. E., Du, H. E., McMahon, A. K., Molina, W., & Whiles, B. B. (2025) · World Journal of Urology · doi:10.1007/s00345-025-05533-4 · Cited by 2
Lambert, M., Döring, N., Schuster, T., & Schmitz, N. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-87217-4_8 · Cited by 1
Assessing the Use of ChatGPT as a Pedagogical Tool: A Small Study
Otto, L. (2024) · Africa Education Review · doi:10.1080/18146627.2025.2471272 · Cited by 5
Multimodales Fenster in die Vergangenheit der ehemaligen Vauban-Festung Saarlouis mittels ChatGPT
Chase, A. V., Chikobava, M., Deru, M., Hauck, C., Poller, P., Ndiaye, A., Baus, J., Schmeyer, T. A., Gampfer, R., & Brandherm, B. (2025) · Angewandte Wirtschaftsinformatik · doi:10.1007/978-3-658-47311-2_9 · Cited by 1
LLMs model how humans induce logically structured rules
Loo, A. M., Pavlick, E., & Feiman, R. (2025) · Journal of Memory and Language · doi:10.1016/j.jml.2025.104675 · Cited by 11
Heterogeneous Effects of Generative AI on Knowledge Seeking in Online CommunitiesPreprint
Quinn, M., & Gutt, D. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4522386 · Cited by 1
Large Language Models' Understanding of Math: Source Criticism and ExtrapolationPreprint
Yousefzadeh, R., & Cao, X. (2023) · Research Square · doi:10.21203/rs.3.rs-3598853/v1 · Cited by 7
Gruber, J. B., & van der Velden, M. A. C. G. (2024) · doi:10.31219/osf.io/a3kwf · Cited by 1
Perspective on Harnessing Large Language Models to Uncover Insights in Diabetes Wearable DataPreprint
Alavi, A., Cha, K., Esfarjani, D. P., Patel, B. J., Li‐Pook‐Than, J., Lee, A., Nebeker, C., Snyder, M. P., & Bahmani, A. H. (2024) · medRxiv · doi:10.1101/2024.07.29.24310315 · Cited by 3
Mesquiti, S., Seraj, S., Weyland, A., Ashokkumar, A., Boyd, R. L., Mihalcea, R. F., & Pennebaker, J. W. (2024) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/sj4fd · Cited by 1
2 Best Practices for Leveraging Generative AI in Experimental ResearchPreprint
Chang, S., Kennedy, A., Leonard, A., & List, J. A. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4979191 · Cited by 1
Man vs. Machine: The Influence of AI Forecasts on Investor BeliefsPreprint
Stradi, F., & Verdickt, G. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4952791 · Cited by 2
Personas shift clinical action thresholds in large language models
Klang, E., Gorenshtein, A., Omar, M., Barash, Y., & Nadkarni, G. N. (2026) · npj Health Systems · doi:10.64898/2026.01.01.26343302 · Cited by 5
Naser, M. Z. (2026) · Language Resources and Evaluation · doi:10.1007/s10579-026-09938-4 · Cited by 3
DAMPAK APLIKASI CHATGPT TERHADAP DUNIA USAHA
Sugangga, R., Sari, I. P., & WIJAYANTO, B. T. (2023) · Akademika · doi:10.51881/jak.v21i2.45
Ciberseguridad: los datos, las semillas del caos
Quintero Fernández, L. F., Rivera Carrillo, J. A., Angulo Salvador, J. F., García Trujillo, D., Sarasti Taguado, M. A., Arce Rendón, A., Samacá Burbano, A., Torres Primero, D. A., Urcuqui, C. C., & Navarro Cadavid, A. (2023) · Editorial Universidad Icesi eBooks · doi:10.18046/eui/ee.10.2023 · Cited by 1
More about the semantic approach to NLP problem solving
Гордей, А. Н. (2023) · Иностранные языки в высшей школе · doi:10.37724/rsu.2023.66.3.001
Evaluating Large Language Models for Assisting in Meta-AnalysisPreprint
Ji, F., Han, J., Zhang, Y., Chen, S., & He, J. (2023) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/dc6tz
Proceedings 12th International Workshop on Theorem proving components for Educational software
Narboux, J., Neuper, W., & Quaresma, P. (2024) · Electronic Proceedings in Theoretical Computer Science · doi:10.4204/eptcs.400
Data Science Experiences and Emotions: From Lonesome to Awesome
Meng, X. (2024) · Harvard Data Science Review · doi:10.1162/99608f92.16558941
Do AI Chatbots Provide an Outside View?Preprint
Shu, S., Das, S., Hernandez, D. P., Fayaz, O., Lei, J., Manavalan, P. K. M., Peng, X., Sakaguchi, N., & Suresh, S. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4874756 · Cited by 1
Advancements in Machine Learning and Deep Learning
Darwish, D. A. M. (2024) · Advances in geospatial technologies book series · doi:10.4018/979-8-3693-2913-9.ch006 · Cited by 2
Liu, Z., Li, G., Shi, B., Zhang, X., Ge, J., Wu, Y., & Lyu, H. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650172 · Cited by 3
Blackburn, K. W., & Chatterjee, S. (2024) · Journal of Thoracic and Cardiovascular Surgery · doi:10.1016/j.jtcvs.2024.10.006
Wuli-Shili-Renli-Jili System Approach and Its Application in Large Model Value Alignment
Qi, J., Jia, K., Zhang, Y., Feng, X., & Yan, H. (2024) · IEEE International Conference on Software Quality, Reliability and Security Companion (QRS-C) · doi:10.1109/qrs-c63300.2024.00069
Bypassing the Nuances of Portuguese Covert Hate Speech Through Contextual Analysis
Ramos, G., Batista, F., Ribeiro, R., Fialho, P., Moro, S., Fonseca, A. J. F., Guerra, R., Carvalho, P. C. Q. D. F., Marques, C. M. V. A., & Silva, C. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73503-5_20
Language Conditioned Robotics: Current Trends and Future Directions – a Survey PaperPreprint
Ninov, N. N., & Asif, W. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.5073634
Generative AI-based Approach to Concept Drift Generation in Streaming Text Data
Belov, B., & Panfilov, P. (2024) · WSEAS Transactions on Information Science and Applications archive · doi:10.37394/23209.2025.22.2
Metadata
- Authors
- Chen, L., Zaharia, M., & Zou, J.
- Affiliations
- Stanford University (US)
- University of California, Berkeley (US)
- Venue
- Harvard Data Science Review
- Publisher
- MIT Press
- Published
- 2024-03-12
- Volume
- 6
- Issue
- 2
- ISSN
- 2644-2353 (ISSN-L)
- OpenAlex
- W4392681182
- Cited by
- 507
- References
- 0
- Citations by year
- 2026: 128 · 2025: 139 · 2024: 163
- FWCI
- 63.33
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- Harvard Data Science Review (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Keywords
- large language models, GPT-3.5, GPT-4, model drift, model updating, chain-of-thought prompting, opinion surveys, multi-hop question answering, code generation, visual reasoning, medical licensing examinations, instruction following
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.