The DOI matches its Crossref record; 58 of the 58 references it cites were found in a registry or an index.
ChatGPT as Research Scientist: Probing GPT’s capabilities as a Research Librarian, Research Ethicist, Data Generator, and Data Predictor
Open accessAbstract
How good a research scientist is ChatGPT? We systematically probed the capabilities of GPT-3.5 and GPT-4 across four central components of the scientific process: as a Research Librarian, Research Ethicist, Data Generator, and Novel Data Predictor, using psychological science as a testing field. In Study 1 (Research Librarian), unlike human researchers, GPT-3.5 and GPT-4 hallucinated, authoritatively generating fictional references 36.0% and 5.4% of the time, respectively, although GPT-4 exhibited an evolving capacity to acknowledge its fictions. In Study 2 (Research Ethicist), GPT-4 (though not GPT-3.5) proved capable of detecting violations like p-hacking in fictional research protocols, correcting 88.6% of blatantly presented issues, and 72.6% of subtly presented issues. In Study 3 (Data Generator), both models consistently replicated patterns of cultural bias previously discovered in large language corpora, indicating that ChatGPT can simulate known results, an antecedent to usefulness for both data generation and skills like hypothesis generation. Contrastingly, in Study 4 (Novel Data Predictor), neither model was successful at predicting new results absent in their training data, and neither appeared to leverage substantially new information when predicting more vs. less novel outcomes. Together, these results suggest that GPT is a flawed but rapidly improving librarian, a decent research ethicist already, capable of data generation in simple domains with known characteristics but poor at predicting novel patterns of empirical data to aid future experimentation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The findings of this study align with previous research across various disciplines showing that citation fabrication and bibliographic inaccuracies are common in LLM-generated outputs [7-9,14].”
From Influence of Topic Familiarity and Prompt Specificity on Citation Fabrication in Mental Health Research Using Large Language Models: Experimental Study (2025) · cites this paper as [14]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“That most models produced overgeneralizations reliably across time suggests that they may have incorporated an ‘algorithmic bias’ from the training data similar to the way models trained on linguistic data can acquire gender, racial, or political biases, by picking up human tendencies implicit in training text corpora [51–53].”
From Generalization bias in large language model summarization of scientific research (2025) · cites this paper as [52]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Our findings thus offer novel contributions to research on overgeneralization in science communication [17,18,54] and the shortcomings of LLMs in text summarization [48,52,55], revealing a subtle form of ‘hallucination’ that has not yet been documented in the literature.”
From Generalization bias in large language model summarization of scientific research (2025) · cites this paper as [52]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 1×
“The rise of new deep learning architectures like the Transformer ( 1 ) has yielded models capable of an array of impressive tasks, from seamless human - like conversation to composing sym-phonies.”
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764 · cited 1×
“As just a few examples, machine learning has been used to discover new protein structures ( 3 ), to render complicated problems more tractable in quantum mechanics ( 4 ) and fluid dynamics ( 5 ), and to accelerate the retrosynthesis of organic molecules ( 6 ).”
Advances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,887
Proceedings of the 24th international conference on Machine learning
(2007) · doi:10.1145/1273496 · Cited by 11,630
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887 · cited 3×
“For analysis, GPT’s estimates were treated as analogous to cosine similarity measures from word embedding research (38).”
Self-citationJost, J. T., Banaji, M. R., & Nosek, B. A. (2004) · Political Psychology · doi:10.1111/j.1467-9221.2004.00402.x · Cited by 2,883
Self-citationMeasuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling
John, L. K., Loewenstein, G., & Prelec, D. (2012) · Psychological Science · doi:10.1177/0956797611430953 · Cited by 2,239 · cited 1×
“One source of these issues is unquestionably poor statistical practices by generally well - intentioned scientists: Studies show that poor statistical practices are widely prevalent across scientific research (31, 32).”
Solving the quantum many-body problem with artificial neural networks
Carleo, G., & Troyer, M. (2017) · Science · doi:10.1126/science.aag2302 · Cited by 2,215
Research works printed but not linked: 2
- X. Zhang Artificial intelligence for science in quantum atomistic and continuum systems. arXiv [Preprint] (2023). https://arxiv.org/pdf/2307.08423.pdf (Accessed 19 January 2024). · arXiv
- OpenAI GPT-4 technical report. arXiv [Preprint] (2023). https://arxiv.org/pdf/2303.08774.pdf (Accessed 19 January 2024). · arXiv
Cited by
Using LLMs for Market Research
Brand, J., Israeli, A., & Ngwe, D. (2023) · doi:10.2139/ssrn.4395751 · Cited by 190
Risks of AI scientists: prioritizing safeguarding over autonomy
Tang, X., Jin, Q., Zhu, K., Yuan, T., Zhang, Y., Zhou, W., Qu, M., Zhao, Y., Tang, J., Zhang, Z., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-63913-1 · Cited by 178
“For instance, in chemistry and biology, these AI scientists can design experiments, control laboratory equipment, and make research decisions [13, 14, 2, 15, 16, 17].”
AgentRxiv: Towards Collaborative Autonomous ResearchPreprint
Schmidgall, S., & Moor, M. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.18102 · Cited by 140
Generative AI in Marketing and Principles for Ethical Design and Deployment
Hermann, E., & Puntoni, S. (2024) · Journal of Public Policy & Marketing · doi:10.1177/07439156241309874 · Cited by 133
“…(Qiu, Singh, and Srinivasan 2023) and human preferences of intertemporal rewards (Goli and Singh 2024) and appears to be limited in predicting new outcomes absent in its training data and in leveraging substantially new information when predicting more versus less novel outcomes (Lehr et al. 2024).”
Generalization bias in large language model summarization of scientific research
Peters, U., & Chin-Yee, B. (2025) · Royal Society Open Science · doi:10.1098/rsos.241776 · Cited by 126
Metadata
- Authors
- Lehr, S. A., Caliskan, A., Liyanage, S., & Banaji, M. R.
- Affiliations
- Cangrade, Inc., Watertown, MA 02472
- University of Washington (US)
- Harvard University (US)
- Corresponding author
- Lehr, Steven A.
- Banaji, Mahzarin R.
- Venue
- Proceedings of the National Academy of Sciences
- Publisher
- National Academy of Sciences
- Published
- 2024-08-20
- Volume
- 121
- Issue
- 35
- Pages
- e2404328121-e2404328121
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- PMC
- PMC11363351
- OpenAlex
- W4401729468
- Cited by
- 66
- References
- 44
- Citations by year
- 2026: 21 · 2025: 39 · 2024: 6
- FWCI
- 27.35
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the National Academy of Sciences (publishedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- US National Institute of Standards and Technology (60NANB23D194)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- ChatGPT, large language models, GPT-3.5, GPT-4, research librarianship, research ethics, hallucinations, data generation, hypothesis generation, artificial intelligence in science
- MeSH terms
- Humans, Research Personnel, Librarians, Ethicists, Ethics, Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: ChatGPT as Research Scientist: Probing GPT’s capabilities as a Research Librarian, Research Ethicist, Data Generator, and Data Predictor. https://app.cochat.ai/papers/10.1073/pnas.2404328121
- The paper
- Lehr, S. A., Caliskan, A., Liyanage, S., & Banaji, M. R. (2024). ChatGPT as Research Scientist: Probing GPT’s capabilities as a Research Librarian, Research Ethicist, Data Generator, and Data Predictor. Proceedings of the National Academy of Sciences. https://doi.org/10.1073/pnas.2404328121
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.