Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 90 of the 90 references it cites were found in a registry or an index; 12 of the 43 sources its claims cite were checked for support.

How should the advancement of large language models affect the practice of science?

Open access

Abstract

Large language models (LLMs) are being increasingly incorporated into scientific workflows. However, we have yet to fully grasp the implications of this integration. How should the advancement of large language models affect the practice of science? For this opinion piece, we have invited four diverse groups of scientists to reflect on this query, sharing their perspectives and engaging in debate. Schulz et al. make the argument that working with LLMs is not fundamentally different from working with human collaborators, while Bender et al. argue that LLMs are often misused and overhyped, and that their limitations warrant a focus on more specialized, easily interpretable tools. Marelli et al. emphasize the importance of transparent attribution and responsible use of LLMs. Finally, Botvinick and Gershman advocate that humans should retain responsibility for determining the scientific roadmap. To facilitate the discussion, the four perspectives are complemented with a response from each group. By putting these different perspectives in conversation, we aim to bring attention to important considerations within the academic community regarding the adoption of LLMs and their impact on both current and future scientific practices.

Overview

CoChat Summary

TL;DR

This PNAS opinion piece convenes four groups of scientists to debate how LLM advancement should affect scientific practice, with each perspective followed by responses from the other groups. The contributors disagree sharply on whether LLMs are collaborator-like aids, overhyped and often inappropriate tools, or technologies requiring principles of transparency, accountability, and fairness while reserving core scientific judgment for humans.

Key findings

  • Schulz et al. argue that working with LLMs is “not fundamentally different from working with other collaborators” and that existing rules for scientific practice suffice; they advocate open-source models for reproducibility.
  • Bender et al. argue LLMs are “often mischaracterized, misused, and overhyped,” that they are models of word form distributions rather than meaning, and that bespoke, interpretable tools are preferable for most scientific tasks.
  • Marelli et al. call for principles rather than strict regulation, emphasizing transparency, accountability, and fairness; they cite Liang et al.’s finding that “more than half (57.4%) of the users found GPT-4 generated feedback helpful or very helpful and 82.4% found it more beneficial than feedback from at least some human reviewers.”
  • Botvinick and Gershman argue that two core aspects of science should remain human: the normative aspect of choosing problems and the epistemic aspect of human understanding.

Methods

  • Opinion piece format: four invited groups of scientists each provide a perspective on how LLM advancement should affect science, followed by a response from each group and a concluding general discussion.
  • No new empirical data are collected; the article synthesizes arguments and cites prior studies, including examples such as Meta’s Galactica being taken offline after three days and Yax et al.’s failed replication of proprietary LLM reasoning results after three months.
  • The discussion is organized around applications in navigating, doing, and communicating science, with cross-responses among the four contributor groups.

Limitations

  • The paper states that it is an opinion piece and that the invited perspectives were selected to cover a broad spectrum, not to provide a systematic review or empirical test.
  • The authors note that it remains unclear whether benchmarks can be established for scientific workflows and whether benchmark success would translate to meaningful scientific output.
  • Bender et al. state that there are “no published user studies that quantify just how much effort this checking process is, nor how accurately researchers can carry it out, especially while working under pressure”; more generally, the text makes evident that the perspectives rely on argument and cited examples rather than new controlled evidence.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1073/pnas.2401227121

Cites

90 of 90 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

8 Valid · 7 Not valid · 20 Not checked · 28 Own results and interpretations

22 more claims

Create a free account to see all 63 claims and to verify the 31 references still open.

See all claims — free account
  • Probably not a substantial claim. 30
  • It may state two separate claims as one. 14

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

2 of 2

Citations

Cites

Show all 68 references

Research works printed but not linked: 3

  1. S. Narayan S. B. Cohen M. Lapata Ranking sentences for extractive summarization with reinforcement learning. arXiv [Preprint] (2018). https://arxiv.org/abs/1802.08636 (Accessed 19 February 2024). · arXiv
  2. 10.1073/pnas.2305016120 · DOI · Paper page
  3. C. Flaherty The peer review crisis (2023). https://www.insidehighered.com/news/2022/06/13/peer-review-crisis-creates-problems-journals-and-scholars. Accessed 30 October 2023.

Cited by

See all 74

Metadata

Authors
Binz, M., Alaniz, S., Roskies, A., Aczel, B., Bergstrom, C. T., Allen, C., Schad, D., Wulff, D., West, J. D., Zhang, Q., Shiffrin, R. M., Gershman, S. J., Popov, V., Bender, E. M., Marelli, M., Botvinick, M. M., Akata, Z., & Schulz, E.
Affiliations
  • Max Planck Institute for Biological Cybernetics (DE)
  • Munich Center for Machine Learning (DE)
  • University of California, Santa Barbara (US)
  • Institute of Psychology, Eötvös Loránd University
  • University of Washington (US)
  • HMU Health and Medical University (DE)
  • University of Basel (CH)
  • Rutgers, The State University of New Jersey (US)
  • Indiana University Bloomington (US)
  • Harvard University (US)
  • University of Zurich (CH)
  • University of Milano-Bicocca (IT)
  • Google DeepMind (United Kingdom) (GB)
Corresponding author
  • Binz, Marcel
  • Alaniz, Stephan
  • Bender, Emily M.
  • Marelli, Marco
  • Botvinick, Matthew M.
  • Akata, Zeynep
  • Schulz, Eric
Venue
Proceedings of the National Academy of Sciences
Published
2025-01-27
Volume
122
Issue
5
Pages
e2401227121-e2401227121
ISSN
  • 0027-8424 (ISSN-L)
  • 1091-6490
OpenAlex
W4406855515
Cited by
74
References
68
Citations by year
2026: 42 · 2025: 31
FWCI
29.28
Type
journal article
Language
en
Open access
Locations
  • Proceedings of the National Academy of Sciences (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • eScholarship (California Digital Library) (submittedVersion, cc-by)
  • BOA (University of Milano-Bicocca) (submittedVersion, other-oa)
  • Zurich Open Repository and Archive (University of Zurich) (submittedVersion, cc-by)
Funders
  • Deutsche Forschungsgemeinschaft (206/4-1, 390727645)
  • Bundesministerium für Bildung und Forschung (01IS18039A, FKZ: 01IS18039A)
  • H2020 European Research Council (853489)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
large language models, scientific practice, scientific workflows, LLM misuse, human-AI collaboration, responsible AI use
MeSH terms
Large Language Models, Diffusion of Innovation, Humans, Language, Models, Theoretical, Science

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: How should the advancement of large language models affect the practice of science? https://app.cochat.ai/papers/10.1073/pnas.2401227121
The paper
Binz, M., Alaniz, S., Roskies, A., Aczel, B., Bergstrom, C. T., Allen, C., Schad, D., Wulff, D., West, J. D., Zhang, Q., Shiffrin, R. M., Gershman, S. J., Popov, V., Bender, E. M., Marelli, M., Botvinick, M. M., Akata, Z., & Schulz, E. (2025). How should the advancement of large language models affect the practice of science? Proceedings of the National Academy of Sciences. https://doi.org/10.1073/pnas.2401227121

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.