The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index; none of the 18 sources its claims cite is checked for support yet.
ChatGPT and large language models in academia: opportunities and challenges
Open accessAbstract
The introduction of large language models (LLMs) that allow iterative “chat” in late 2022 is a paradigm shift that enables generation of text often indistinguishable from that written by humans. LLM-based chatbots have immense potential to improve academic work efficiency, but the ethical implications of their fair use and inherent bias must be considered. In this editorial, we discuss this technology from the academic’s perspective with regard to its limitations and utility for academic writing, education, and programming. We end with our stance with regard to using LLMs and chatbots in academia, which is summarized as (1) we must find ways to effectively use them, (2) their use does not constitute plagiarism (although they may produce plagiarized text), (3) we must quantify their bias, (4) users must be cautious of their poor accuracy, and (5) the future is bright for their application to research and as an academic tool.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This editorial argues that LLM-based chatbots such as ChatGPT should be embraced in academia for writing, education, and programming, while cautioning about bias, hallucination, and poor accuracy. The authors conclude that LLM use does not constitute plagiarism, that bias metrics are needed, and that users remain responsible for validating outputs.
Key findings
The authors’ stance is that (1) LLMs must be embraced, (2) their use does not constitute plagiarism although they may produce plagiarized text, (3) their bias must be quantified, (4) users must be cautious of poor accuracy, and (5) the future is bright for their application to research and as an academic tool.
- ChatGPT can present false information as truth: when asked to help find relevant publications to cite, it “proceeded to make up a list of five entirely fictional publications”; on repeat trials it sometimes listed one or two real papers, but other times fabricated a paper or returned an irrelevant one.
- On scalability and accessibility, the paper reports ChatGPT gained 100 million active users in 2 months, with an estimated 13 million daily queries, compared with Google search’s estimated 8.5 billion daily queries; ChatGPT offers a free tier and a $20/month premium plan. It also notes BLOOM as an open access 176B parameter multilingual LLM, ChatGPT as a 175 billion parameter LLM, speculation that OpenAI had access to 10,000 Nvidia V100s costing close to $5 million, and an estimate that training BERT (described as a 6 billion parameter LLM) would produce 21 to 78 metric tons of CO2.
- For academic writing, LLMs may particularly help non-native English speakers; the paper cites that 49% of postdoctoral fellows trained in the United States were born overseas, and in engineering, math, and computer science, 60% of PhDs were awarded to international students. For programming, ChatGPT can help write, interpret, debug, simplify, and translate code, but it may present incorrect code as correct, miss edge cases, and is more reliable for smaller blocks than larger complex programs.
Methods
Design: editorial/commentary, not an empirical study or systematic review.
- Data or sample: no primary data; the paper draws on published literature, illustrative examples, and the authors’ experiences.
- Main analysis: narrative discussion of opportunities and challenges across academic writing, education, programming, bias, and scalability/accessibility.
Limitations
The paper does not report a formal limitations section for its own approach. Evident from the text: it is an editorial/commentary, so its examples are illustrative and include personal experiences rather than systematic evidence.
- The paper states LLM limitations: hallucination/false information, bias from training data, poor accuracy, misrepresentation of capabilities, and inability to anticipate edge cases in code.
- The paper notes unresolved ethical/legal uncertainty around LLM use in academic writing and grants, and that detection of cheating remains an unsolved challenge for educators.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
17 Own interpretation · 9 Not checked yet · 7 No basis given · 4 Own result
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“A fundamental challenge with LLM-based chatbots is that they can present false information as truth, also known as “hallucination”.”
- Also stated in: § Background · plain
“For example, when asked if it could help find relevant publications to cite in a review paper, ChatGPT confirmed that it could, but then proceeded to make up a list of five entirely fictional publications.”
§ ChatGPT and large language models in academia: opportunities and challenges > Background
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Background · plain
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In fact, ChatGPT and Bard can misrepresent their capabilities.”
§ ChatGPT and large language models in academia: opportunities and challenges > Background
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“For example, LLMs may adopt bias, perpetuate stereotypes in the training dataset, and present false information as truth.”
§ ChatGPT and large language models in academia: opportunities and challenges > Introduction
What the sources say
- Not checked yetcites [3] Sallam M. ChatGPT utility in healthcare education, research, and practice: systematic review on the promising perspectives and valid concerns. Healthcare. 2023;11(6):887. https://doi.org/10.3390/healthcare11060887. source
- Not checked yetcites [4] Kung TH, Cheatham M, Medenilla A, Sillos C, De Leon L, Elepaño C, et al. Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digit Health. 2023;2:e0000198. source
- Not checked yetcites [5] Liebrenz M, Schleifer R, Buadze A, Bhugra D, Smith A. Generating scholarly content with ChatGPT: ethical challenges for medical publishing. Lancet Digit Health. 2023;5:e105–6. source
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These issues limit the utility of LLM chatbots in some domains, such as the medical field, where facts are essential to ensure the best health outcomes.”
§ ChatGPT and large language models in academia: opportunities and challenges > Background
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Thus, an answer generated by a large language model can be formatted correctly but not necessarily be factual.”
§ ChatGPT and large language models in academia: opportunities and challenges > Background
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Over reliance may hinder students ability to develop critical skills, such as writing [21].”
§ ChatGPT and large language models in academia: opportunities and challenges > Education
What the sources say
- Not checked yetcites [21] Shidiq M. The use of artificial intelligence-based chat-gpt and its challenges for the world of education; from the viewpoint of the development of creative writing skills. Proc Int Conf Educ Soc Humanity. 2023;1(1):353–7. ISSN 2986-5832.
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Consequently, employing LMMs with pre-written text is unlikely to raise ethical concerns such as plagiarism, which may arise when using prompt-based text generation by LMMs.”
§ ChatGPT and large language models in academia: opportunities and challenges > ChatGPT for academic writing > ChatGPT as a writing and editing tool
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Unethically and irresponsibly designed AI/ML solutions deployed in the healthcare setting can exacerbate and perpetuate systematic biases and disparities for those from marginalized groups [27–29].”
§ ChatGPT and large language models in academia: opportunities and challenges > Bias
What the sources say
- Not checked yetcites [27] Vyas DA, Eisenstein LG, Jones DS. Hidden in Plain Sight - Reconsidering the Use of Race Correction in Clinical Algorithms. N Engl J Med. 2020;383:874–82. source
- Not checked yetcites [28] Gijsberts CM, Groenewegen KA, Hoefer IE, Eijkemans MJC, Asselbergs FW, Anderson TJ, et al. Race/Ethnic Differences in the Associations of the Framingham Risk Factors with Carotid IMT and Cardiovascular Events. PLoS ONE. 2015;10:e0132321. source
- Not checked yetcites [29] Obermeyer Z, Powers B, Vogeli C, Mullainathan S. Dissecting Racial Bias in an Algorithm used to Manage the Health of Populations. Science. 2019;366:447–53. source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“A recent study estimated that the training of BERT (a 6 billion parameter LLM) would produce between 21 to 78 metric tons of CO2 to train to completion [14].”
§ ChatGPT and large language models in academia: opportunities and challenges > Scalability and accessibility of LLMs
What the sources say
- Not checked yetcites [14] Dodge J, Prewitt T, Des Combes RT, Odmark E, Schwartz R, Strubell E, Luccioni AS, Smith NA, DeCario N, Buchanan W. Measuring the Carbon Intensity of AI in Cloud Instances. arXiv. 2022;2206.05229. source
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Until the root-causes of the encoded biases of LLMs are addressed, LLMs for clinical applications will suffer the same fate, propagating biases [30, 31].”
§ ChatGPT and large language models in academia: opportunities and challenges > Bias
What the sources say
- Not checked yetcites [30] Neeley T, Ruper S. Timnit Gebru: 'SILENCED No More' on AI Bias and The Harms of Large Language Models. Harvard Business School Case. 2022;422–085.
- Not checked yetcites [31] Bender EM, Gebru T, McMillan-Major A, Shmitchell S. On the dangers of stochastic parrots: can language models be too big? In: Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. New York, NY, USA: ACM; 2021. p. 610–23. source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Probably not a substantial claim. 23
- It may state two separate claims as one. 11
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In one author's experience experimenting with how ChatGPT might help write grants, ChatGPT can help provide some ideas for interesting directions, even suggesting sound logical reasons that we need to do research that could find their way into the significance section of NIH grants.”
§ ChatGPT and large language models in academia: opportunities and challenges > ChatGPT for academic writing > ChatGPT and LLMs for grant proposal writing
plaindirectSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“However, grants must convey many complex ideas in a short amount of space, and as such, since much of the text from ChatGPT is so general and often repetitive, in our experience none of the text produced by ChatGPT made it into the final grant proposal.”
§ ChatGPT and large language models in academia: opportunities and challenges > ChatGPT for academic writing > ChatGPT and LLMs for grant proposal writing
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“ChatGPT can even provide general text that might be logically used in parts of the grant, such as the aims page.”
§ ChatGPT and large language models in academia: opportunities and challenges > ChatGPT for academic writing > ChatGPT and LLMs for grant proposal writing
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“A more important concern regarding students learning from a LLM is that it typically responds in a confident, and seemingly knowing manner, whether the provided information is accurate or not.”
§ ChatGPT and large language models in academia: opportunities and challenges > Education
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It can suggest specific experiments that might test a hypothesis of interest.”
§ ChatGPT and large language models in academia: opportunities and challenges > ChatGPT for academic writing > ChatGPT and LLMs for grant proposal writing
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A surprising effect for one author of this editorial was that it helped overcome writer's block in that simply having some text on the page was sufficient to move forward with editing.”
§ ChatGPT and large language models in academia: opportunities and challenges > ChatGPT for academic writing > ChatGPT and LLMs for grant proposal writing
plaindirectSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The model can generate correctly formatted information such as references, that do not exist.”
§ ChatGPT and large language models in academia: opportunities and challenges > Conclusion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“ChatGPT will misrepresent its knowledge; instead of responding “I don’t know”, it will readily provide fabricated information with apparent confidence.”
§ ChatGPT and large language models in academia: opportunities and challenges > Conclusion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“For example, LLMs may adopt bias, perpetuate stereotypes in the training dataset, and present false information as truth.”
§ ChatGPT and large language models in academia: opportunities and challenges > Introduction
What the sources say
- Not checked yetcites [3] Sallam M. ChatGPT utility in healthcare education, research, and practice: systematic review on the promising perspectives and valid concerns. Healthcare. 2023;11(6):887. https://doi.org/10.3390/healthcare11060887. source
- Not checked yetcites [4] Kung TH, Cheatham M, Medenilla A, Sillos C, De Leon L, Elepaño C, et al. Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digit Health. 2023;2:e0000198. source
- Not checked yetcites [5] Liebrenz M, Schleifer R, Buadze A, Bhugra D, Smith A. Generating scholarly content with ChatGPT: ethical challenges for medical publishing. Lancet Digit Health. 2023;5:e105–6. source
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For example, it would not be wise to base treatment plans on outputs from ChatGPT, at least in its current state, because this would lead to a large proportion of misdiagnoses and ineffective treatments that could cause patient harm.”
§ ChatGPT and large language models in academia: opportunities and challenges > Background
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Deploying AI models at scale is a challenge that future AI technologies will have to face even for tech giants such as Google (Bard power by LaMDA) and Microsoft (AI powered Bing) that have announced AI integration within their respective search engines.”
§ ChatGPT and large language models in academia: opportunities and challenges > Scalability and accessibility of LLMs
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In line with warnings for general use, ChatGPT may somewhat unpredictably present incorrect code as being correct.”
§ ChatGPT and large language models in academia: opportunities and challenges > Assistance with programming
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
In the paper
“Concerns have arisen regarding how students might easily use ChatGPT dishonestly, either to cheat in the completion of homework or exams, or to write reports/essays without citation in a manner that could be construed as plagiarism [22].”
§ ChatGPT and large language models in academia: opportunities and challenges > Education
What the sources say
- Not checked yetcites [22] Neumann M, Rauschenberger M, Schön E-M. “We Need To Talk About ChatGPT”: The Future of AI and Higher Education. 2023:4. https://doi.org/10.25968/opus-2467. source
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Currently, ChatGPT is more likely to be successful in accurately writing smaller blocks of code, whereas its reliability in writing larger/more complex programs (e.g. a software package) is questionable.”
§ ChatGPT and large language models in academia: opportunities and challenges > Assistance with programming
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Ultimately, code written by ChatGPT may offer a useful starting point, however both the code and comments that it generates should be checked and validated by both the programmer and users to confirm that it fully satisfies the intended purpose.”
§ ChatGPT and large language models in academia: opportunities and challenges > Assistance with programming
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“ChatGPT also promises to facilitate computer programming in terms of (1) learning to code or use specific packages, libraries, or frameworks, (2) writing new code, (3) interpreting existing code, (4) debugging existing code, (5) increasing the compactness or efficiency of code, or (6) translating code from one programming language to another.”
§ ChatGPT and large language models in academia: opportunities and challenges > Assistance with programming
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“LLMs provide benefits and opportunities in education for students by assisting with research and academic writing [15] or using them as an interactive study guide which may include generating practice exams and being provided immediate feedback [16, 17].”
§ ChatGPT and large language models in academia: opportunities and challenges > Education
What the sources say
- Not checked yetcites [15] Kasneci E, Sessler K, Küchemann S, Bannert M, Dementieva D, Fischer F, et al. ChatGPT for good? On opportunities and challenges of large language models for education. Learn Individ Differ. 2023;103:102274. source
- Not checked yetcites [16] MacNeil S, Tran A, Mogil D, Bernstein S, Ross E, Huang Z. Generating Diverse Code Explanations using the GPT-3 Large Language Model. In: Vahrenhold J, Fisler K, Hauswirth M, Franklin D, editors. Proceedings of the 2022 ACM Conference on International Computing Education Research - Volume 2. New York, NY, USA: ACM; 2022. p. 37–9. source
- Not checked yetcites [17] Tate TP, Doroudi S, Ritchie D, Xu Y, Mark Warschauer UCI. Educational Research and AI-Generated Writing: Confronting the Coming Tsunami. 2023.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“While AI has the potential to revolutionize education and academic research, paywalled access could further increase the divide between the wealthy and the poor.”
§ ChatGPT and large language models in academia: opportunities and challenges > Scalability and accessibility of LLMs
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
- Rests on an assumption: AI has the potential to revolutionize education and academic research.
In the paper
“Data produced from ChatGPT may be inaccurate or biased [23], which users need to be aware of to ensure these deficiencies are not propagated in their works.”
§ ChatGPT and large language models in academia: opportunities and challenges > Education
What the sources say
- Not checked yetcites [23] Baidoo-Anu D, Owusu Ansah L. Education in the Era of Generative Artificial Intelligence (AI): Understanding the Potential Benefits of ChatGPT in Promoting Teaching and Learning. 2023. Available at SSRN: https://ssrn.com/abstract=4337484 or https://doi.org/10.2139/ssrn.4337484. source
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This editorial provides several examples and perspectives on how LLMs increase the efficiency of academic writing, education, and programming.”
§ ChatGPT and large language models in academia: opportunities and challenges > Conclusion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In many contexts, LLMs have low accuracy.”
§ ChatGPT and large language models in academia: opportunities and challenges > Conclusion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Output from ChatGPT will match existing text on the internet, particularly if the sample is small enough.”
§ ChatGPT and large language models in academia: opportunities and challenges > Conclusion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
- Rests on an assumption: The output of ChatGPT is from most publicly available text on the internet until 2021.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“However, ChatGPT does not do anything without a prompt.”
§ ChatGPT and large language models in academia: opportunities and challenges > Conclusion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“When LLMs for clinical applications are socially, consciously, and transparently designed with such considerations, they can become an additional tool for promoting equity and improving access to care.”
§ ChatGPT and large language models in academia: opportunities and challenges > Bias
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The performance of ChatGPT and other LLMs therefore mimics the available text, and they are as biased as their training data.”
§ ChatGPT and large language models in academia: opportunities and challenges > Conclusion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
- Rests on an assumption: The output of ChatGPT is from most publicly available text on the internet until 2021.
- Rests on an assumption: ChatGPT is known to perpetuate stereotypes such as nurses being female and doctors being male, and this bias comes from the training data.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Thus, in a strict sense, LLMs like ChatGPT do produce plagiarism.”
§ ChatGPT and large language models in academia: opportunities and challenges > Conclusion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
- Rests on an assumption: The output of ChatGPT is from most publicly available text on the internet until 2021.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As companies race to innovate with LLMs like ChatGPT and others, encoded biases will be amplified, and harm will manifest.”
§ ChatGPT and large language models in academia: opportunities and challenges > Bias
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds. It may state two separate claims as one.
The paper relies on this without support.
Effective academic writing is essential to the success and quality of academic publications and is a major challenge facing international postdocs and students.
“Effective academic writing plays an essential role in the success and quality of academic publications, nevertheless, it is one of the major challenges facing international postdocs and students.”
The paper relies on this without support.
The current grant review system is subjective; some proposals scored very poorly in one study section while nearly the same proposal at another study section is funded.
“Despite this massive investment, the current review system is subjective; for example, some proposals sent to one study section are scored very poorly, while nearly the same proposal at another study section is funded.”
The paper relies on this without support.
ChatGPT has already been applied to help write grant proposals.
“ChatGPT has undoubtedly already been applied to help write grant proposals.”
The paper relies on this without support.
ChatGPT is a 175 billion parameter LLM.
“As a reminder, ChatGPT is a 175 billion parameter LLM and this only considers the cost of training and not any cost related to generating text in response to potentially billions of daily queries.”
The paper relies on this without support.
The output of ChatGPT is from most publicly available text on the internet until 2021.
“The output of ChatGPT is from most publicly available text on the internet until 2021.”
The paper relies on this without support.
ChatGPT is known to perpetuate stereotypes such as nurses being female and doctors being male, and this bias comes from the training data.
“For example, ChatGPT is known to perpetuate stereotypes such as nurses being female and doctors being male, and this bias comes from the training data.”
The paper relies on this without support.
The use of ChatGPT and other LLMs by students is inevitable.
“As the use of ChatGPT and other LLMs by students is inevitable, educators will need to change the pedagogy and assessments in education, such as integrating the use of ChatGPT in lessons [26], and a focus more on the process rather than the final product for assessment.”
The paper relies on this without support.
Detection of cheaters remains an unsolved challenge for educators.
“While the detection of cheaters remains an unsolved challenge for educators, if we start from the assumption that students are allowed, if not encouraged, to utilize ChatGPT, strategies could be adopted to account for this.”
The paper relies on this without support.
It is difficult to define metrics that assess the level of bias in training data and model outputs.
“It is difficult to define metrics that assess the level of bias in the training data and in the model outputs.”
The paper relies on this without support.
AI has the potential to revolutionize education and academic research.
“While AI has the potential to revolutionize education and academic research, paywalled access could further increase the divide between the wealthy and the poor.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“using GenAI to automate routine administrative tasks in higher education, such as consolidation of grades, creating performance summaries of students, crafting out evaluation reports, and maintaining student enrolment and attendance records (Meyer et al., 2023).”
From Higher Education's Generative Artificial Intelligence Paradox: The Meaning of Chatbot Mania (2024) · cites this paper as Meyer et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Third, LLMs can act as a thought partner for students and enhance understanding by providing detailed explanations and step-by-step solutions (Mollick and Mollick 2022, Kasneci et al. 2023, Meyer et al. 2023, Extance 2023).”
From AI Meets the Classroom: When Do Large Language Models Harm Learning? (2024) · cites this paper as Meyer et al. 2023
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“there has been a notable upswing in discussions among scholars and academicians regarding the transformative potential of generative AI, particularly ChatGPT, in higher education and the risks associated with it (Dalalah & Dalalah, 2023; Meyer et al., 2023; Peters et al., 2023; Yilmaz & Yilmaz, 2023a).”
From Is it harmful or helpful? Examining the causes and consequences of generative AI usage among university students (2024) · cites this paper as Meyer et al., 2023
In the citing paper
“The consequence of a self-referential loop in LLMs can lead to several problematic outcomes, including the propagation of biases [30], increased homogeneity in generated data, and ultimately, hindered performance.”
From Large Language Models and User Trust: Consequence of Self-Referential Learning Loop and the Deskilling of Health Care Professionals (2024) · cites this paper as [30]
In the citing paper
“ChatGPT is considered the fastest-growing technology in history: in just three months following its public launch, it amassed an estimated 120 million monthly active users [16] with an estimated 13 million daily queries [49], surpassing all other applications [64].”
From A systematic literature review of empirical research on ChatGPT in education (2024) · cites this paper as [49]
In the citing paper
“a fundamental challenge with LLM-based chatbots, including ChatGPT, is the accuracy and quality of the provided information and responses, as they provide false information as truth—a phenomenon often referred to as “hallucination” [3, 49]”
From A systematic literature review of empirical research on ChatGPT in education (2024) · cites this paper as [49]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ ChatGPT and large language models in academia: opportunities and challenges > ChatGPT for academic writing > Ethics and rules of using text from ChatGPT
“In one case, ChatGPT was listed as an author in the preprint [9] but missing as an author in the final publication [4].”
§ ChatGPT and large language models in academia: opportunities and challenges > References
“9. Kung TH, Cheatham M, Medenilla A, Sillos C, De Leon L, et al. Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digital Health. 2023;2(2):e0000198. 10.1371/journal.pdig.0000198. PMC9931230 36812645”
Citations
Cites
ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education
Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · doi:10.35542/osf.io/5er8f · Cited by 7,552
ChatGPT for good? On opportunities and challenges of large language models for education
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · Learning and Individual Differences · doi:10.1016/j.lindif.2023.102274 · Cited by 7,326 · cited 1×
“LLMs provide benefits and opportunities in education for students by assisting with research and academic writing [15] or using them as an interactive study guide which may include generating practice exams and being provided immediate feedback [16, 17].”
BackgroundDissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137 · cited 1×
“Unethically and irresponsibly designed AI/ML solutions deployed in the healthcare setting can exacerbate and perpetuate systematic biases and disparities for those from marginalized groups [27–29].”
BackgroundOn the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000198 · Cited by 4,115 · cited 1×
“In one case, ChatGPT was listed as an author in the preprint [9] but missing as an author in the final publication [4].”
BackgroundBAİDOO-ANU, D., & OWUSU ANSAH, L. (2023) · Journal of AI · doi:10.61969/jai.1337500 · Cited by 3,939
Sallam, M. (2023) · Healthcare · doi:10.3390/healthcare11060887 · Cited by 3,126 · cited 1×
“On the other hand, there are also unsettling ethical issues to be considered [3–5].”
BackgroundChatting and cheating: Ensuring academic integrity in the era of ChatGPT
Cotton, D. R. E., Cotton, P. A., & Shipway, J. R. (2024) · Innovations in Education and Teaching International · doi:10.1080/14703297.2023.2190148 · Cited by 3,087
Chatting and Cheating. Ensuring academic integrity in the era of ChatGPT
Cotton, D., Cotton, P., & Shipway, J. R. (2023) · doi:10.35542/osf.io/mrz8h · Cited by 3,079
Baidoo-Anu, D., & Owusu Ansah, L. (2023) · doi:10.2139/ssrn.4337484 · Cited by 3,004 · cited 1×
“Data produced from ChatGPT may be inaccurate or biased [23], which users need to be aware of to ensure these deficiencies are not propagated in their works.”
Background
Research works printed but not linked: 4
- Kung TH, Cheatham M, Medenilla A, Sillos C, De Leon L, et al. Performance of ChatGPT on USMLE: Potential for AI-assisted medical education using large language models. PLOS Digital Health. 2023;2(2):e0000198. https://doi.org/10.1371/journal.pdig.0000198. · DOI · Paper page
- Shidiq M. The use of artificial intelligence-based chat-gpt and its challenges for the world of education; from the viewpoint of the development of creative writing skills. Proc Int Conf Educ Soc Humanity. 2023;1(1):353–7. ISSN 2986-5832.
- Trust T, Whalen J, Mouza C. Editorial: ChatGPT: Challenges, opportunities, and implications for teacher education. Contemp Issues Technol Teacher Educ. 2023;23(1):1–23.
- Dijkstra R, Genç Z, Kayal S, Kamps J. Reading Comprehension Quiz Generation using Generative Pre-trained Transformers. 2022.
Cited by
Sorted by citations
Items 101–200 of 574
Wang, J., Hu, H., Wang, Z., Yan, S., Sheng, Y., & He, D. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641917 · Cited by 21
ALGAN: Time Series Anomaly Detection with Adjusted-LSTM GAN
Bashar, M. A., & Nayak, R. (2025) · International Journal of Data Science and Analytics · doi:10.1007/s41060-025-00810-2 · Cited by 33
ChatGPT as a Virtual Laboratory Teaching Assistant in Undergraduate Biology
Doğru, M. S., & Faulconer, E. K. (2025) · Research in Science Education · doi:10.1007/s11165-025-10271-z · Cited by 28
“(Meyer et al., 2023; Omar et al., 2023).”
Urzúa, C. A. C., Ranjan, R., Saavedra, E. E. M., Quintana, M. G. B., Lepe-Martínez, N. F., & Philominraj, A. S. (2025) · Education Sciences · doi:10.3390/educsci15101396 · Cited by 45
“The inaccuracy and low reliability of results are attributed to factors such as insufficient training, poor-quality or inadequate data sets, and the way users formulate their prompts (Fuchs, 2023; Meyer et al., 2023; Wardat et al., 2023).”
How ChatGPT shapes the future labour market situation of software engineers: A Finnish Delphi study
Komp, K. (2024) · Futures · doi:10.1016/j.futures.2024.103382 · Cited by 34
A comparative study of AI-human-made and human-made test forms for a university TESOL theory course
Kyung-Mi, O. (2024) · Language Testing in Asia · doi:10.1186/s40468-024-00291-3 · Cited by 27
Rasheed, Z., Waseem, M., Ahmad, A., Kemell, K., Wang, X., NguyenDuc, A., & Abrahamsson, P. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5149683 · Cited by 23
Beck, S. M., Kuhner, M., Haar, M., Daubmann, A., Semmann, M., & Kluge, S. (2024) · Scandinavian Journal of Trauma Resuscitation and Emergency Medicine · doi:10.1186/s13049-024-01266-2 · Cited by 24
“…the literature, we anticipated the following error types: “superficial” (the statement lacks sufficient detail for full application), “inaccurate” (certain elements of the statement are not applicable), and “hallucination” (the statement appears credible but is unsupported by evidence) [18, 24].”
BackgroundTudino, G., & Qin, Y. (2024) · Lingua · doi:10.1016/j.lingua.2024.103838 · Cited by 32
Lang, G., & Gürpinar, T. (2025) · Information Systems Education Journal · doi:10.62273/zklk5988 · Cited by 20
“These chatbots use deep learning models, particularly transformer architectures, to understand and generate human-like text (Meyer et al., 2023).”
Large language modeling and classical AI methods for the future of healthcare
Banerjee, S. K., Dunn, P. J., Conard, S. E., & Ng, R. (2023) · Journal of Medicine Surgery and Public Health · doi:10.1016/j.glmedi.2023.100026 · Cited by 22
ChatGPT Interventions in Higher Education: A Systematic Review of Experimental Studies
Jin, Y., & Sercu, L. (2025) · Journal of Computer Assisted Learning · doi:10.1111/jcal.70072 · Cited by 21
The ethical security of large language models: A systematic review
Liu, F., Jiang, J., Lu, Y., Huang, Z., & Jiang, J. (2025) · Frontiers of Engineering Management · doi:10.1007/s42524-025-4082-6 · Cited by 23
“These issues include hallucination of outputs (Cascella et al., 2023; Ji et al., 2023), potential misinformation and bias (Meyer et al., 2023; Zhou et al., 2023), risks of data privacy breaches (Su, 2024), and impacts on human autonomy (Ellis et al., 2024).”
Coal Mine Accident Risk Analysis with Large Language Models and Bayesian Networks
Du, G., & Chen, A. (2025) · Sustainability · doi:10.3390/su17051896 · Cited by 25
Grilo, A. M., Marques, C. P., Corte-Real, M., Carolino, E., & Caetano, M. (2025) · JMIR Cancer · doi:10.2196/63677 · Cited by 21
“Although AI can revolutionize education and research, GPT-4 may widen the gap between the wealthy and poor [58].”
The impact of large language models on computer science student writing
Zdravkova, K., & Ilijoski, B. (2025) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-025-00525-1 · Cited by 23
“LLMs need mechanisms to help them validate outputs prone to inaccuracy or hallucinations (Meyer, 2023).”
Systematic exploration and in-depth analysis of ChatGPT architectures progression
Banik, D., Pati, N., & Sharma, A. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10832-0 · Cited by 24
“The primary objective of this research study is to provide a comprehensive comparative analysis of different architectural iterations of ChatGPT (Meyer et al. 2023), emphasizing their unique features, progressions, and limitations.”
Generative Artificial Intelligence in Instructional System Design
Chai, D. S., Kim, H. S., Kim, K. N., Ha, Y., Shin, S. S. H., & Yoon, S. W. (2025) · Human Resource Development Review · doi:10.1177/15344843251320256 · Cited by 20
“These GAI applications excel in assisting with basic-level writing, exploring information, summarizing complex information, and encouraging learner re fl ection (Khan et al., 2023; Meyer et al., 2023; P´erez et al., 2020; Ross, 2023).”
InfluentialLi, S., Gao, H. O., & You, F. (2024) · Nexus · doi:10.1016/j.ynexs.2024.100026 · Cited by 21
Derner, E., Kučera, D., Oliver, N., & Zahálka, J. (2024) · Computers in Human Behavior Artificial Humans · doi:10.1016/j.chbah.2024.100088 · Cited by 25
Siddiqui, M. N., Pea, R. D., & Subramonyam, H. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714119 · Cited by 19
Troy, C., Sturley, S., Calero, J. M. A., & Wang, Q. (2023) · IEEE Access · doi:10.1109/access.2023.3329071 · Cited by 18
“There are several ChatGPT-based [23] research focused on the usage of generative AI for modeling tasks [24] and for modeling software use cases based on system requirements [25], to name a few.”
MethodExploring Public Response to ChatGPT With Sentiment Analysis and Knowledge Mapping
Zhou, J., Liang, Z., Fang, Y., & Zhou, Z. (2024) · IEEE Access · doi:10.1109/access.2024.3386362 · Cited by 18
“Meyer pointed out that ChatGPT can generate bias from training data, such as nurses being female and doctors being male [65].”
BackgroundUsing GPT-4 to write a scientific review article: a pilot evaluation study
Wang, Z. P., Bhandary, P., Wang, Y., & Moore, J. H. (2024) · BioData Mining · doi:10.1186/s13040-024-00371-3 · Cited by 25
Self-citationCrawford, L. M., Hendzlik, P., Lam, J., Cannon, L. M., Qi, Y., DeCaporale-Ryan, L. N., & Wilson, N. A. (2024) · Journal of Surgical Research · doi:10.1016/j.jss.2024.06.020 · Cited by 18
Exploring the role of generative AI in academia: Opportunities and challenges
Panda, S., & Kaur, N. (2024) · IP Indian Journal of Library Science and Information Technology · doi:10.18231/j.ijlsit.2024.003 · Cited by 30
Zhu, Y., Ye, Z., Yuan, Y., Tang, W., Yu, C., & Shi, Y. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714261 · Cited by 23
Evaluating ChatGPT-4 Vision on Brazil's National Undergraduate Computer Science Exam
Mendonça, N. C. (2024) · ACM Transactions on Computing Education · doi:10.1145/3674149 · Cited by 22
ResumeFlow: An LLM-facilitated Pipeline for Personalized Resume Generation and Refinement
Zinjad, S. B., Bhattacharjee, A., Bhilegaonkar, A., & Liu, H. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657680 · Cited by 20
Alnaimat, F., Al-Halaseh, S., & AlSamhori, A. R. F. (2024) · Journal of Korean Medical Science · doi:10.3346/jkms.2024.39.e231 · Cited by 20
“102 103 To ensure reproducibility, the use of LLMs to create content is expected in the current era, so tools that can detect AI-generated or human-generated texts could help ensure that a study is reproducible.”
ChatGPT Evaluation: Can It Replace Grammarly and Quillbot Tools?
A. younis, H., Mohammed, O., Muthmainnah, Sahib, T. M., Akhtom, D., Hayder, I. M., Salisu, S., & Shahid, M. (2023) · British Journal of Applied Linguistics · doi:10.32996/bjal.2023.3.2.4 · Cited by 17
Awareness, perception, and adoption of ChatGPT in African HEIs: A multi-dimensional analysis
Ojubanire, O. A., Olaleye, S. A., Marhraoui, M. A., Kamihanda, M. W., Oke, O. I., & Ojubanire, O. A. (2025) · The Internet and Higher Education · doi:10.1016/j.iheduc.2025.100999 · Cited by 22
Elrawy, S., & Wagdy, B. (2025) · AI & Society · doi:10.1007/s00146-025-02193-1 · Cited by 16
“(Kashyap 2023; Meyer et al. 2023).”
Legal regulation of AI-assisted academic writing: challenges, frameworks, and pathways
Gao, R., Yu, D., Gao, B., Hua, H., Hui, Z., Gao, J., & Yin, C. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1546064 · Cited by 27
López‐Pernas, S., Misiejuk, K., Kaliisa, R., & Saqr, M. (2025) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2025.3568599 · Cited by 18
“…in overcoming most of the existing limitations of curating educational datasets: 1) they are easily accessiblethroughwebormobileapplications[30];2)theirlarge trainingcorpusenablesthecreationofadatasetaboutanytopic withnoadditionalinput;3)thedatageneratedare—allegedly—…”
LLM-Generated multiple choice practice quizzes for preclinical medical students
Camarata, T., McCoy, L., Rosenberg, R. L., Temprine, K., Brettschnieder, K., & Berman, J. M. (2025) · AJP Advances in Physiology Education · doi:10.1152/advan.00106.2024 · Cited by 16
Chatbot Performance in Defining and Differentiating Palliative Care, Supportive Care, Hospice Care
Kim, M. J., Admane, S., Chang, Y. K., Shih, K. K., Reddy, A. S., Tang, M. J., de la Cruz, M., Taylor, T. P., Bruera, E. D., & Hui, D. (2024) · Journal of Pain and Symptom Management · doi:10.1016/j.jpainsymman.2024.01.008 · Cited by 16
Levin, G., Pareja, R., Viveros-Carreño, D. A., Díaz, E. S., Yates, E. M., Zand, B., & Ramirez, P. T. (2024) · International Journal of Gynecological Cancer · doi:10.1136/ijgc-2023-005162 · Cited by 16
LLMEffiChecker : Understanding and Testing Efficiency Degradation of Large Language Models
Feng, X., Han, X., Chen, S. H., & Yang, W. M. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3664812 · Cited by 20
“LLMEffiChecker : Understanding and Testing Efficiency Degradation of LLMs 186:3 possible to generate test inputs that could sufficiently delay the generation of EOS such that LLMs would have to go through max iterations to satisfy the pessimistic pre-configured threshold.”
Abernethy, N. J. (2024) · AI and Ethics · doi:10.1007/s43681-024-00575-7 · Cited by 17
“Furthermore, because companies are rapidly developing LLMs that prioritize factual accuracy and citational reliability [95], the time and energy necessary to double-check such LLMs’ work will probably continue decreasing going forward.”
AI-powered language learning: Developing the chatGPT usage scale for foreign language learners
Çobanoğulları, F., & Özbek, Ö. (2025) · Education and Information Technologies · doi:10.1007/s10639-025-13342-w · Cited by 34
“There are worries about the accuracy of texts generated by ChatGPT in different languages, and some authors even suggest that ChatGPT is prone to ’hallucinations’ (Meyer et al., 2023).”
Evaluation of ChatGPT for Patient Counseling in Kidney Stone Clinic: A Prospective Study
Javid, M., Bhandari, M., Parameshwari, P., Reddiboina, M., & Prasad, S. (2024) · Journal of Endourology · doi:10.1089/end.2023.0571 · Cited by 18
“Since ChatGPT’s responses are derived from extensive training data, they are expected to be more detailed and are likely to provide redundant information, which could confuse patients, 15–17 whereas the urologists’ responses were pointed, specific, and brief.”
Zarei, M., Zarei, M., Hamzehzadeh, S., Oliyaei, S. S. B., & Hosseini, M. (2024) · Shiraz E-Medical Journal · doi:10.5812/semj-145840 · Cited by 21
ContextGPT: Infusing LLMs Knowledge into Neuro-Symbolic Activity Recognition Models
Arrotta, L., Bettini, C., Civitarese, G., & Fiori, M. (2024) · IEEE International Conference on Smart Computing · doi:10.1109/smartcomp61445.2024.00029 · Cited by 15
The Evolution of Learning: Assessing the Transformative Impact of Generative AI on Higher Education
Krause, S., Panchal, B. H., & Ubhe, N. (2024) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-981-97-9255-9_24 · Cited by 35
Sarkar, T., Bhattacherjee, A., & Mukhia, M. (2024) · International Conference on Technological Advancements in Computational Sciences (ICTACS) · doi:10.1109/ictacs62700.2024.10840404 · Cited by 15
Silva, N., & Wickramaarachchi, D. (2025) · International Conference on Advanced Research in Computing (ICARC) · doi:10.1109/icarc64760.2025.10963273 · Cited by 15
“Questions about the reliability of AI-generated outputs, the variability in result quality, and the necessity of human oversight remain critical [7] [8].”
Investigating the Accuracy of ChatGPT as a Writing Error Correction Tool
Alsaweed, W., & Aljebreen, S. (2024) · International Journal of Computer-Assisted Language Learning and Teaching · doi:10.4018/ijcallt.364847 · Cited by 15
Textoshop: Interactions Inspired by Drawing Software to Facilitate Text Editing
Masson, D., Kim, Y., & Chevalier, F. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713862 · Cited by 39
Lin, C., Wang, W., Lee, H., Li, P., Huang, Y., & Wu, T. (2025) · Journal of Research on Technology in Education · doi:10.1080/15391523.2025.2504358 · Cited by 14
A Data-Driven Approach for the Identification of Features for Automated Feedback on Academic Essays
Abbas, M., van Rosmalen, P., & Kalz, M. (2023) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2023.3320877 · Cited by 16
Uysal, B. Ç. B., & Yüksel, İ. (2024) · Advances in educational technologies and instructional design book series · doi:10.4018/979-8-3693-0872-1.ch006 · Cited by 16
Plagiarism, Academic Ethics, and the Utilization of Generative AI in Academic Writing
Koplin, J., & Philosophy Documentation Center (2023) · International Journal of Applied Philosophy · doi:10.5840/ijap2023372202 · Cited by 16
Can generative AI Be an effective Co-Teacher? An experiment
Niloy, A. C., Akter, S., Sultana, J., Rahman, M. A., Sultana, N., Islam, T., Isha, N. J., Afroz, M., Zabeen, M., Tabassum, M., et al. (2025) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2025.100418 · Cited by 18
Dellafiore, F., Saba, A., Collaro, C., & Artioli, G. (2025) · Qualitative Health Research · doi:10.1177/10497323251389800 · Cited by 20
Carius, A. C., & Teixeira, A. J. (2024) · AI & Society · doi:10.1007/s00146-024-01988-y · Cited by 13
“…for the first time, any human being on the planet has access to tools of this type, just needing a device with Internet access (notebook, smartphone, tablet) (Gupta 2023; Henrickson and Meroño-Peñuela 2023; Hill-Yardin et al., 2023; Meyer et al., 2023; Rane et al., 2023; Short and Short 2023).”
Leveraging Open Source LLMs for Software Engineering Education and Training
Pereira, J., López, J. M., Garmendia, X., & Azanza, M. (2024) · IEEE International Conference on Software Engineering Education and Training (CSEE&T) · doi:10.1109/cseet62301.2024.10663055 · Cited by 13
“These models use advanced machine learning algorithms to understand and generate human-like text, providing a unique tool for educators to create dynamic and responsive teaching materials [20].”
Beyond borders or building new walls?
Tran, H., & Stell, A. (2024) · Australian Review of Applied Linguistics · doi:10.1075/aral.24135.tra · Cited by 13
Academic Advising Chatbot Powered with AI Agent
Tamascelli, M., Bunch, O., Fowler, B., Taeb, M., & Cohen, A. (2025) · ACM Southeast Regional Conference (ACMSE) · doi:10.1145/3696673.3723065 · Cited by 20
[The stellate ganglion blockade].
Hempel, V. (1993) · PubMed · Cited by 13
GPT-Neo-CRV: Elevating Information Accuracy in GPT-Neo with Cross-Referential ValidationPreprint
Xiong, X., & Zheng, M. (2024) · doi:10.36227/techrxiv.170473976.61241065/v1 · Cited by 13
Freire, Y., Santamaría-Laorden, A., Pérez, J. O., Ortiz-Collado, I., Sánchez, M. G., Vasallo, I. J. T., García, V. D., & Suárez, A. (2025) · PLoS ONE · doi:10.1371/journal.pone.0323086 · Cited by 12
“Among the most significant biases is the possibility of producing mean-ingless content [23] by presenting incorrect information as if it were accurate [24].”
Preliminary Systematic Review of Open-Source Large Language Models in Education
Lin, M. P.-C., Chang, D., Hall, S., & Jhajj, G. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-63028-6_6 · Cited by 12
A Large Language Model Approach to Code and Privacy Policy Alignment
Morales, G. A., Pragyan, K. C., Jahan, S., Hosseini, M. B., & Slavin, R. (2024) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner60148.2024.00016 · Cited by 13
Tanay, B. A., Arinze, L., Joshi, S. S., Davis, K., & Davis, J. C. (2024) · ASEE Annual Conference & Exposition · doi:10.18260/1-2--46557 · Cited by 12
The Impact of LLM Hallucinations on Motor Skill Learning: A Case Study in Badminton
Qiu, Y. (2024) · IEEE Access · doi:10.1109/access.2024.3444783 · Cited by 17
“For example, ChatGPT and Bard can misrepresent their capabilities, providing entirely fabricated publication lists when asked to cite relevant publications in a review paper [37], [88].”
BackgroundXBRL Agent: Leveraging Large Language Models for Financial Report Analysis
Han, S., Kang, H., Jin, B., Liu, X.-Y., & Yang, S. Y. (2024) · ACM International Conference on AI in Finance (ICAIF) · doi:10.1145/3677052.3698614 · Cited by 21
Unmoderated Usability Studies Evolved: Can GPT Ask Useful Follow-up Questions?
Kuric, E., Demcak, P., & Krajcovic, M. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2427978 · Cited by 19
“…(Antaki et al., 2023; Carlbring et al., 2023; Farhat, 2024; Johnson et al., 2023; Nori et al., 2023), law (Bommarito & Katz, 2022; Perlman, 2022), finance (Dowling & Lucey, 2023), education (Kung et al., 2023; Lund & Wang, 2023; Meyer et al., 2023; Susnjak, 2022) and communication (Dynel, 2023).”
The use of generative artificial intelligence in surgical education: a narrative review
Rao, L., Yang, E., Dissanayake, S., Cuomo, R., Seth, I., & Rozen, W. M. (2024) · Plastic and Aesthetic Research · doi:10.20517/2347-9264.2024.102 · Cited by 13
Generative AI as a Coach to Help Students Enhance Proficiency in Question Formulation
Kim, P., Wang, W., & Bonk, C. J. (2025) · Journal of Educational Computing Research · doi:10.1177/07356331251314222 · Cited by 14
“Generative AI presents educators with many new opportunities as well as challenges in transforming teaching and learning as we know it (Adeshola & Adepoju, 2023; Baidoo-Anu & Ansah, 2023; Chan & Hu, 2023; Lo, 2023; Meyer et al., 2023; Rahman & Watanobe, 2023; Yu & Guo, 2023).”
Bell, S. (2023) · BMC Medicine · doi:10.1186/s12916-023-03039-7 · Cited by 12
“The US National Institutes of Health has issued similar guidance, barring reviewers from using AI tech for assessments [9].”
BackgroundAI Meets the Classroom: When Do Large Language Models Harm Learning?
Lehmann, M., Cornelius, P. B., & Sting, F. J. (2024) · arXiv · doi:10.48550/arxiv.2409.09047 · Cited by 11
Liang, H., & Mao, X. (2025) · International Journal of Educational Development · doi:10.1016/j.ijedudev.2025.103434 · Cited by 12
Pirzado, F. A., Ahmed, A., Alipour, M. A., & Terashima‐Marín, H. (2025) · Journal of Computers in Education · doi:10.1007/s40692-025-00375-w · Cited by 19
“The focus should shift from whether students use these tools to how they utilize them, similar to parental assistance with homework (Meyer et al., 2023).”
Practical Assessment of Large Language Models in Cloud Computing Using Real-World Data Applications
Singh, V. K., Moharana, B., Sarkar, T., Maan, A., Bhattacherjee, A., & Rakhra, M. (2024) · International Conference on Cybernation and Computation (CYBERCOM) · doi:10.1109/cybercom63683.2024.10803266 · Cited by 11
Abdelnabi, A. A. B., Soykan, B., Bhatti, D., & Rabadi, G. (2025) · ACM Transactions on Computing for Healthcare · doi:10.1145/3712298 · Cited by 12
“Indeed, ChatGPT and Bard may make false claims about their skills [13].”
Güvel, M. C., Kıyak, Y. S., Varan, H. D., Sezenöz, B., Çoşkun, Ö., & Uluoğlu, C. (2025) · European Journal of Clinical Pharmacology · doi:10.1007/s00228-025-03838-2 · Cited by 17
Hu, L., Chen, M., & Ding, N. (2025) · Scientific Reports · doi:10.1038/s41598-025-99924-7 · Cited by 15
Yu, J., Yan, W., Gong, J., Wang, S., Nah, K., & Cheng, W. (2025) · Applied Sciences · doi:10.3390/app15148088 · Cited by 11
Trending Applications of Large Language Models: A User Perspective Survey
Zhang, Y., Zhao, M., Zhang, Y., & Cheung, Y. (2025) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2025.3620272 · Cited by 13
Ganguly, A., Mehjabin, N., Malik, A. N., & Johri, A. (2025) · AI and Ethics · doi:10.1007/s43681-025-00916-0 · Cited by 19
“…are popular for their ability to generate human-like text and undertake contextual reasoning [21] and therefore they are being integrated into academics and classrooms to support various tasks such as question-answering, summarizing content, providing feedback and scoring and assessments [22].”
Lim, B., Seth, I., Dooreemeah, D., & Lee, C. H. A. (2023) · Langenbeck s Archives of Surgery · doi:10.1007/s00423-023-03173-z · Cited by 10
The Challenges of ChatGPT in Healthcare Scientific Writing
SeyedAlinaghi, S., Abbaspour, F., & Mehraeen, E. (2023) · Shiraz E-Medical Journal · doi:10.5812/semj-141861 · Cited by 10
Large language models reshaping molecular biology and drug development
Tripathi, S., Gabriel, K., Tripathi, P. K., & Kim, E. (2024) · Chemical Biology & Drug Design · doi:10.1111/cbdd.14568 · Cited by 13
“The number of researchers with equitable access to specialized expertise and resources in molecular biology and drug discovery is not satisfactory (Makridakis et al., 2023; Meyer et al., 2023).”
BackgroundEng, P. H., Manohar, S. K., & Liu, R. L. (2024) · Studies in systems, decision and control · doi:10.1007/978-3-031-65207-3_25 · Cited by 10
LLM-Enhanced Learning Environments for CS: Exploring Data Structures and Algorithms with Gurukul
Rachha, A., & Seyam, M. (2024) · IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie61694.2024.10893211 · Cited by 13
“Meyer et al. (2023) discuss ethical concerns surrounding LLM deployment, such as data privacy and bias [12].”
Violakis, P. (2025) · Policing A Journal of Policy and Practice · doi:10.1093/police/paaf012 · Cited by 11
ChatGPT vs. Human Journalists: Analyzing News Summaries Through BERTScore and Moderation Standards
Kim, H.-S., Kang, J.-W., & Choi, S. (2025) · Electronics · doi:10.3390/electronics14112115 · Cited by 12
Passing with ChatGPT? Ethical Evaluations of Generative AI Use in Higher Education
Pérez-Portabella, A., Arias-Oliva, M., Castillo, G. P., & Sánchez, J. D. A. (2025) · Digital · doi:10.3390/digital5030033 · Cited by 10
“Continuous use of AI may deteriorate fundamental skills such as written expression [71], reducing intellectual autonomy and real preparation for the professional world.”
Bonacaro, A., Rubbi, I., Artioli, G., Monaco, F., Sarli, L., & Guasconi, M. (2024) · Studies in health technology and informatics · doi:10.3233/shti240134 · Cited by 15
Comas‐Forgas, R., Koulouris, A., & Kouis, D. (2025) · Learned Publishing · doi:10.1002/leap.1647 · Cited by 9
“…to substitute for most types of formal writing, including proposals to recognize them as authors in academic publications (Dwivedi et al., 2023; Stokel-Walker, 2023), has given way to a more re fi ned comprehension of their strengths and signi fi cant weaknesses (Lingard, 2023; Meyer et al., 2023).”
Petrič, G. (2024) · International Journal of Technology and Human Interaction · doi:10.4018/ijthi.349225 · Cited by 10
Samadi, M. A., Jaquay, S., Lin, Y., Tajik, E., Park, S., & Nixon, N. (2024) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3636555.3636922 · Cited by 11
ChatGPT in Urogynecology Research: Novel or Not?
Choueka, D., Tabakin, A. L., & Shalom, D. F. (2024) · Urogynecology · doi:10.1097/spv.0000000000001505 · Cited by 9
Gupta, A., & Goyal, R. (2024) · IEEE Access · doi:10.1109/access.2024.3401770 · Cited by 9
“After identifying methods in the given software that are not documented as required by the proposed technique, this AI tool can generate a documented version for these methods [6], [7].”
BackgroundMethodKamali, J., Paknejad, A., & Poorghorban, A. (2024) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2024.7.2.8 · Cited by 12
Chen, A., Qilleri, A., Foster, T., Rao, A. S., Gopalakrishnan, S., Niezgoda, J., & Oropallo, A. (2024) · Advances in Skin & Wound Care · doi:10.1097/asw.0000000000000226 · Cited by 9
Can people with epilepsy trust AI chatbots for information on physical exercise?
Silva, R. R., de Lima, B. E., Costa, T. G., Morais, N. S., José, G., Cordeiro, D. F., de Almeida, A. A., Lopim, G. M., Viana, R. B., Sousa, B. S., et al. (2024) · Epilepsy & Behavior · doi:10.1016/j.yebeh.2024.110193 · Cited by 14
“They are trained on vast amounts of publicly available text data, including books, articles, and webpages [20].”
Responsible Use of Large Language Models: An Analogy with the Oxford Tutorial System
Lissack, M. R., & Meagher, B. (2024) · She ji · doi:10.1016/j.sheji.2024.11.001 · Cited by 13
Nordmann, K., Sauter, S., Stein, M., Aigner, J., Redlich, M.-C., Schaller, M., & Fischer, F. (2025) · BMC Medical Research Methodology · doi:10.1186/s12874-025-02532-2 · Cited by 10
Metadata
- Authors
- Meyer, J. G., Urbanowicz, R. J., Martin, P. C. N., O’Connor, K., Li, R., Peng, P.-C., Bright, T. J., Tatonetti, N., Won, K. J., Gonzalez-Hernandez, G., & Moore, J. H.
- Affiliations
- Cedars-Sinai Medical Center (US)
- University of Pennsylvania (US)
- Corresponding author
- Meyer, Jesse G.
- Moore, Jason H.
- ORCID
- Venue
- BioData Mining
- Published
- 2023-07-13
- Volume
- 16
- Issue
- 1
- Pages
- 20-20
- ISSN
- 1756-0381 (ISSN-L)
- PMC
- PMC10339472
- OpenAlex
- W4384200891
- Semantic Scholar
- e95b10b9a8415874f24275b665628fc340eee5dc
- Cited by
- 569
- References
- 23
- Citations by year
- 2026: 104 · 2025: 215 · 2024: 220 · 2023: 29
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- BioData Mining (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Institute on Aging (AG066833)
- National Institute of General Medical Sciences (R35GM142502)
- National Institute of Allergy and Infectious Diseases (R01AI164481)
- U.S. National Library of Medicine (R01LM011176)
- Topics
- Social Sciences › Education › Artificial Intelligence in Education
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- large language models, ChatGPT, academic writing, artificial intelligence in education, AI-assisted programming, plagiarism, algorithmic bias, AI accuracy, research tools, ethics of AI use
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
