The DOI matches its Crossref record; 165 of the 165 references it cites were found in a registry or an index.
Artificial intelligence and illusions of understanding in scientific research
Abstract
Scientists are enthusiastically imagining ways in which artificial intelligence (AI) tools might improve research. Why are AI tools so attractive and what are the risks of implementing them across the research pipeline? Here we develop a taxonomy of scientists' visions for AI, observing that their appeal comes from promises to improve productivity and objectivity by overcoming human shortcomings. But proposed AI solutions can also exploit our cognitive limitations, making us vulnerable to illusions of understanding in which we believe we understand more about the world than we actually do. Such illusions obscure the scientific community's ability to see the formation of scientific monocultures, in which some types of methods, questions and viewpoints come to dominate alternative approaches, making science less innovative and more vulnerable to errors. The proliferation of AI tools in science risks introducing a phase of scientific enquiry in which we produce more but understand less. By analysing the appeal of these tools, we provide a framework for advancing discussions of responsible knowledge production in the age of AI.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This can result in widespread misinterpretations of findings, illusions of understanding, research lacunas, and risky practices [13].”
From Generalization bias in large language model summarization of scientific research (2025) · cites this paper as [13]
In the citing paper
“And open-washing is bad for the public understanding of AI, because it creates artefacts designed to impress without providing people with the resources to reach a deeper understanding of the technology [39].”
From Rethinking open source generative AI: open washing and the EU AI Act (2024) · cites this paper as [39]
In the citing paper
“In science, Messeri and Crockett (2024) warn that AI tools can produce illusions of understanding and scientific monocultures, where apparent productivity coexists with reduced epistemic diversity.”
From Hermeneutic Sovereignty under Generative AI: A Humanities Framework for Protecting Human Meaning-Making from Interpretive Foreclosure (2026) · cites this paper as Messeri and Crockett (2024)
In the citing paper
“The same logic responds to the risk of monoculture identified by Messeri and Crockett (2024).”
From Hermeneutic Sovereignty under Generative AI: A Humanities Framework for Protecting Human Meaning-Making from Interpretive Foreclosure (2026) · cites this paper as Messeri and Crockett, 2024
In the citing paper
“Messeri and Crockett (2024) show how AI-supported science can encourage illusions of understanding, where researchers feel that they comprehend more than they actually do, while the scientific community becomes vulnerable to monocultures.”
From Hermeneutic Sovereignty under Generative AI: A Humanities Framework for Protecting Human Meaning-Making from Interpretive Foreclosure (2026) · cites this paper as Messeri and Crockett (2024)
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764
Situated Learning: Legitimate Peripheral Participation
Lave, J., & Wenger, E. (1991) · Cambridge University Press eBooks · doi:10.1017/cbo9780511815355 · Cited by 35,324
Situated Knowledges: The Science Question in Feminism and the Privilege of Partial Perspective
Haraway, D. (1988) · Feminist Studies · doi:10.2307/3178066 · Cited by 18,364
Hutchins, E. (1995) · doi:10.7551/mitpress/1881.001.0001 · Cited by 10,801
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
boyd, D., & Crawford, K. (2012) · Information, Communication & Society · doi:10.1080/1369118x.2012.678878 · Cited by 5,479
Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author)
Breiman, L. (2001) · Statistical Science · doi:10.1214/ss/1009213726 · Cited by 4,340
Science as Social Knowledge: Values and Objectivity in Scientific Inquiry
Longino, H. E. (1990) · doi:10.1515/9780691209753 · Cited by 3,092
Algorithms of Oppression: How Search Engines Reinforce Racism
Noble, S. U. (2020) · doi:10.2307/j.ctt1pwt9w5 · Cited by 2,709
Research works printed but not linked: 24
- Gil, Y. Will AI write scientific papers in the future? AI Mag. 42, 3–15 (2022).
- Benjamin, R. Race After Technology: Abolitionist Tools for the New Jim Code (Oxford Univ. Press, 2020). This book examines how social norms about race become embedded in technologies, even those that are focused on providing good societal outcomes.
- Collingridge, D. The Social Control of Technology (St Martin’s Press, 1980).
- Hastings, J. AI for Scientific Discovery (CRC Press, 2023). This book reviews current and future incorporation of AI into the scientific research pipeline.
- Gray, K., Yam, K. C., Zhen’An, A. E., Wilbanks, D. & Waytz, A. The psychology of robots and artificial intelligence. In The Handbook of Social Psychology (eds Gilbert, D. et al.) (in the press).
- Aher, G., Arriaga, R. I. & Kalai, A. T. Using large language models to simulate multiple humans and replicate human subject studies. In Proc. 40th International Conference on Machine Learning (eds Krause, A. et al.) 337–371 (JMLR.org, 2023).
- Karjus, A. Machine-assisted mixed methods: augmenting humanities and social sciences with artificial intelligence. Preprint at https://arxiv.org/abs/2309.14379 (2023). · arXiv
- Dhar, P. Peer review of scholarly research gets an AI boost. IEEE Spectrum spectrum.ieee.org/peer-review-of-scholarly-research-gets-an-ai-boost (2020).
- Johnson, S. G. B., Johnston, A. M., Toig, A. E. & Keil, F. C. Explanatory scope informs causal strength inferences. In Proc. 36th Annual Meeting of the Cognitive Science Society 2453–2458 (Cognitive Science Society, 2014).
- Ylikoski, P. in Scientific Understanding: Philosophical Perspectives (eds De Regt, H. et al.) 100–119 (Univ. Pittsburgh Press, 2009).
- Clarke, A. E. & Fujimura, J. H. (eds) The Right Tools for the Job: At Work in Twentieth-Century Life Sciences (Princeton Univ. Press, 2014).
- Latour, B. in The Social After Gabriel Tarde: Debates and Assessments (ed. Candea, M.) 145–162 (Routledge, 2010).
- Gillespie, T. in Media Technologies: Essays on Communication, Materiality, and Society (eds Gillespie, T. et al.) 167–194 (MIT Press, 2014). · doi:10.7551/mitpress/9780262525374.003.0009 · Paper page
- Latour, B. Science In Action: How to Follow Scientists and Engineers Through Society (Harvard Univ. Press, 1987). This book provides strategies and approaches for thinking about science as a social endeavour.
- Daston, L. & Galison, P. Objectivity (Princeton Univ. Press, 2007). This book is a historical analysis of the shifting modes of ‘objectivity’ that scientists have pursued, arguing that objectivity is not a universal concept but that it shifts alongside scientific techniques and ambitions.
- Schiebinger, L. The Mind Has No Sex? Women in the Origins of Modern Science (Harvard Univ. Press, 1991).
- Harding, S. Rethinking standpoint epistemology: What is “strong objectivity”? Centen. Rev. 36, 437–470 (1992). In this article, Harding outlines her position on ‘strong objectivity’, by which clearly articulating one’s standpoint can lead to more robust knowledge claims.
- Oreskes, N. Why Trust Science? (Princeton Univ. Press, 2019). This book introduces the reader to 20 years of scholarship in science and technology studies, arguing that the tools the discipline has for understanding science can help to reinstate public trust in the institution.
- National Academies of Sciences, Engineering, and Medicine et al. Advancing Antiracism, Diversity, Equity, and Inclusion in STEMM Organizations: Beyond Broadening Participation (National Academies Press, 2023).
- Winner, L. Do artifacts have politics? Daedalus 109, 121–136 (1980).
- Eubanks, V. Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor (St. Martin’s Press, 2018).
- Raghu, M. & Schmidt, E. A survey of deep learning for scientific discovery. Preprint at arxiv.org/abs/2003.11755 (2020). · arXiv
- Stengers, I. Another Science is Possible: A Manifesto for Slow Science (Wiley, 2018).
- Feinman, R. & Lake, B. M. Learning task-general representations with generative neuro-symbolic modeling. Preprint at arxiv.org/abs/2006.14448 (2021). · arXiv
Cited by
Self-Driving Laboratories for Chemistry and Materials Science
Tom, G., Schmid, S. P., Baird, S. G., Cao, Y., Darvish, K., Hao, H., Lo, S., Pablo-García, S., Rajaonson, E. M., Skreta, M., et al. (2024) · Chemical Reviews · doi:10.1021/acs.chemrev.4c00055 · Cited by 707
Larger and more instructable language models become less reliable
Zhou, L., Schellaert, W., Martínez-Plumed, F., Moros-Daval, Y., Ferri, C., & Hernández-Orallo, J. (2024) · Nature · doi:10.1038/s41586-024-07930-y · Cited by 203
“Millions of people are using general-purpose artificial intelligence (AI) systems based on large language models (LLMs), which have become commonplace in areas such as education 6 , medicine 7 , science 8,9 and administration 10,11 .”
Resnik, D. B., & Hosseini, M. (2025) · AI and Ethics · doi:10.1007/s43681-024-00493-8 · Cited by 228
“This is important because given the accessibility of AI systems and the fact that they can exploit our cognitive shortcomings, they are creating an illusion of understanding [148].”
How Interpretable Machine Learning Can Benefit Process Understanding in the Geosciences
Jiang, S., Sweet, L., Blougouras, G., Brenning, A., Li, W., Reichstein, M., Denzler, J., Shangguan, W., Yu, G., Huang, F., et al. (2024) · Earth's Future · doi:10.1029/2024ef004540 · Cited by 182
“In many cases, applying different interpretation methods to a single model (Mamalakis, Barnes, & Ebert‐Uphoff, 2022), or even applying the same interpretation method but with varying settings or hyperparameters (S. Müller et al., 2023), can lead to different results.”
REFORMS: Consensus-based Recommendations for Machine-learning-based Science
Kapoor, S., Cantrell, E. M., Peng, K., Pham, T. H., Bail, C. A., Gundersen, O. E., Hofman, J. M., Hullman, J. R., Lones, M. A., Malik, M. M., et al. (2024) · Science Advances · doi:10.1126/sciadv.adk3452 · Cited by 303
“However, the rapid uptake of ML methods has been accompanied by concerns of validity, reproducibility, and generalizability ( 13 – 20 ).”
Metadata
- Authors
- Messeri, L., & Crockett, M. J.
- Affiliations
- Yale University (US)
- Princeton University (US)
- Corresponding author
- Messeri, Lisa
- Crockett, M. J.
- Venue
- Nature
- Published
- 2024-03-06
- Volume
- 627
- Issue
- 8002
- Pages
- 49-58
- ISSN
- 0028-0836 (ISSN-L)
- 1476-4687
- OpenAlex
- W4392504765
- Cited by
- 707
- References
- 161
- Citations by year
- 2026: 309 · 2025: 288 · 2024: 110
- FWCI
- 293.02
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springernature.com/gp/researchers/text-and-data-mining (tdm)
- www.springernature.com/gp/researchers/text-and-data-mining (vor)
- Locations
- Nature (publishedVersion)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Social Sciences › Business, Management and Accounting › Management of Technology and Innovation › Research, Science, and Academia
- Keywords
- artificial intelligence in science, cognitive limitations, research productivity, scientific objectivity, scientific innovation, research errors
- MeSH terms
- Artificial Intelligence, Cognition, Diffusion of Innovation, Efficiency, Humans, Illusions, Physicians, Research Design, Research Personnel, Risk, Reproducibility of Results, Knowledge
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Artificial intelligence and illusions of understanding in scientific research. https://app.cochat.ai/papers/10.1038/s41586-024-07146-0
- The paper
- Messeri, L., & Crockett, M. J. (2024). Artificial intelligence and illusions of understanding in scientific research. Nature. https://doi.org/10.1038/s41586-024-07146-0
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.