The DOI matches its Crossref record.
ChatGPT is fun, but not an author
Open accessAbstract
In less than 2 months, the artificial intelligence (AI) program ChatGPT has become a cultural sensation. It is freely accessible through a web portal created by the tool’s developer, OpenAI. The program—which automatically creates text based on written prompts—is so popular that it’s likely to be “at capacity right now” if you attempt to use it. When you do get through, ChatGPT provides endless entertainment. I asked it to rewrite the first scene of the classic American play Death of a Salesman , but to feature Princess Elsa from the animated movie Frozen as the main character instead of Willy Loman. The output was an amusing conversation in which Elsa—who has come home from a tough day of selling—is told by her son Happy, “Come on, Mom. You’re Elsa from Frozen . You have ice powers and you’re a queen. You’re unstoppable.” Mash-ups like this are certainly fun, but there are serious implications for generative AI programs like ChatGPT in science and academia.
Overview
CoChat Summary
TL;DR
Thorp argues that ChatGPT is entertaining but unsuitable as a scientific author, citing its tendency to produce plausible-sounding errors and a study in which reviewers caught only 68% of AI-generated abstracts. Science journals are updating policies to bar ChatGPT/AI-generated text, figures, images, and graphics from papers and to prohibit AI authorship.
Key findings
- ChatGPT can be fun and conversational, but it “sometimes writes plausible-sounding but incorrect or nonsensical answers,” including referencing a scientific study that does not exist.
- In a cited recent study, abstracts created by ChatGPT were submitted to academic reviewers, who only caught 68% of these fakes.
- The Science family of journals is updating its license and Editorial Policies to specify that text generated by ChatGPT or any other AI tools cannot be used in the work, nor can figures, images, or graphics be products of such tools; an AI program cannot be an author.
- Violations will constitute scientific misconduct no different from altered images or plagiarism, though legitimate AI-generated data sets in research papers are not covered by this change.
Methods
- Editorial/commentary by the Editor-in-Chief of Science journals; no empirical study or systematic analysis is reported.
- Uses anecdotal demonstrations, including asking ChatGPT to rewrite the first scene of Death of a Salesman with Princess Elsa as the main character, and giving it an exam and final project from a science denial class taught at George Washington University.
- Cites a recent study in which ChatGPT-created abstracts were submitted to academic reviewers and only 68% were caught.
Limitations
- The paper states no formal limitations.
- Evident from the text: the argument relies on anecdotal examples and a cited study without methodological detail, and it is an editorial policy statement rather than an empirical evaluation.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 Valid · 1 Not valid · 1 Not checked · 6 Own results and interpretations
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It certainly can write essays about a range of topics.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Nevertheless, as the website states, “ChatGPT sometimes writes plausible-sounding but incorrect or nonsensical answers.””
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In a recent study, abstracts created by ChatGPT were submitted to academic reviewers, who only caught 68% of these fakes.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
- Probably not a substantial claim. 5
- It may state two separate claims as one. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“However, there is cause for concern as one study has shown that peer reviewers can only identify 63% of abstracts written using Chat GPT, indicating that such instances of academic fraud may precipitate a reputation crisis within academia (Thorp, 2023).”
From Reflection on whether Chat GPT should be banned by academia from the perspective of education and teaching (2023) · cites this paper as Thorp, 2023
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Third, the generation of non-original, over-detailed, or excessive content can be an additional burden for researchers who should carefully supervise the ChatGPT-generated content [14,24,25,26,65,71].”
From ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid Concerns (2023) · cites this paper as [24]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The disapproval of including ChatGPT or any other LLM in the list of authors was clearly explained in Science, Nature, and the Lancet editorials, which referred to such practice as scientific misconduct, and this view was echoed by many scientists [24,27,35,40,45].”
From ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid Concerns (2023) · cites this paper as [24]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The need to rethink and revise the current assessment tools in health care education comes in light of ChatGPT’s ability to pass reputable exams (e.g., USMLE) and possibility of ChatGPT misuse, which would result in academic dishonesty [24,34,58,59,62,76,85,86,87].”
From ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid Concerns (2023) · cites this paper as [24]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Some journals, such as Science, have banned the use of LLMs because of the ethical problems they raise concerning responsible authorship.”
From The ethics of disclosing the use of artificial intelligence tools in writing scholarly manuscripts (2023) · cites this paper as Thorp, 2023
Experimental — accuracy not yet measured
5 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Still being checked.
Cited by
Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., Baabdullah, A. M., Koohang, A., Raghavan, V., Ahuja, M., et al. (2023) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2023.102642 · Cited by 4,584
Large language models in medicine
Thirunavukarasu, A. J., Ting, D. S. J., Elangovan, K., Gutierrez, L., Tan, T. F., & Ting, D. S. W. (2023) · Nature Medicine · doi:10.1038/s41591-023-02448-8 · Cited by 3,944
Sallam, M. (2023) · Healthcare · doi:10.3390/healthcare11060887 · Cited by 3,126
“The disapproval of including ChatGPT or any other LLM in the list of authors was clearly explained in Science, Nature, and the Lancet editorials, which referred to such practice as scientific misconduct, and this view was echoed by many scientists [24,27,35,40,45].”
BackgroundMethodRay, P. P. (2023) · Internet of Things and Cyber-Physical Systems · doi:10.1016/j.iotcps.2023.04.003 · Cited by 2,463
What Is the Impact of ChatGPT on Education? A Rapid Review of the Literature
Lo, C. K. (2023) · Education Sciences · doi:10.3390/educsci13040410 · Cited by 2,024
Metadata
- Authors
- Thorp, H. H.
- Affiliations
- H. Holden Thorp Editor-in-Chief, Science journals.
- Corresponding author
- Thorp, H. Holden
- ORCID
- Thorp, H. Holden
- Venue
- Science
- Published
- 2023-01-27
- Volume
- 379
- Issue
- 6630
- Pages
- 313-313
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W4318263917
- Semantic Scholar
- eddfb9be78cfe94193766e3722eb0e56c3d24cef
- Cited by
- 1,437
- References
- 0
- Citations by year
- 2026: 176 · 2025: 280 · 2024: 456 · 2023: 522
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- ChatGPT, generative AI, authorship, AI-generated text, academic integrity, scientific writing, artificial intelligence in education, OpenAI, Death of a Salesman
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-03, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-03 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: ChatGPT is fun, but not an author. https://app.cochat.ai/papers/10.1126/science.adg7879
- The paper
- Thorp, H. H. (2023). ChatGPT is fun, but not an author. Science. https://doi.org/10.1126/science.adg7879
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.