Hallucination Rates and Reference Accuracy of ChatGPT and Bard for Systematic Reviews: Comparative Analysis
Open accessPublished version of a preprint
The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index; 2 of the 2 sources its claims cite were checked for support.
Abstract
Background Large language models (LLMs) have raised both interest and concern in the academic community. They offer the potential for automating literature search and synthesis for systematic reviews but raise concerns regarding their reliability, as the tendency to generate unsupported (hallucinated) content persist. Objective The aim of the study is to assess the performance of LLMs such as ChatGPT and Bard (subsequently rebranded Gemini) to produce references in the context of scientific writing. Methods The performance of ChatGPT and Bard in replicating the results of human-conducted systematic reviews was assessed. Using systematic reviews pertaining to shoulder rotator cuff pathology, these LLMs were tested by providing the same inclusion criteria and comparing the results with original systematic review references, serving as gold standards. The study used 3 key performance metrics: recall, precision, and F1-score, alongside the hallucination rate. Papers were considered “hallucinated” if any 2 of the following information were wrong: title, first author, or year of publication. Results In total, 11 systematic reviews across 4 fields yielded 33 prompts to LLMs (3 LLMs×11 reviews), with 471 references analyzed. Precision rates for GPT-3.5, GPT-4, and Bard were 9.4% (13/139), 13.4% (16/119), and 0% (0/104) respectively (P<.001). Recall rates were 11.9% (13/109) for GPT-3.5 and 13.7% (15/109) for GPT-4, with Bard failing to retrieve any relevant papers (P<.001). Hallucination rates stood at 39.6% (55/139) for GPT-3.5, 28.6% (34/119) for GPT-4, and 91.4% (95/104) for Bard (P<.001). Further analysis of nonhallucinated papers retrieved by GPT models revealed significant differences in identifying various criteria, such as randomized studies, participant criteria, and intervention criteria. The study also noted the geographical and open-access biases in the papers retrieved by the LLMs. Conclusions Given their current performance, it is not recommended for LLMs to be deployed as the primary or exclusive tool for conducting systematic reviews. Any references generated by such models warrant thorough validation by researchers. The high occurrence of hallucinations in LLMs highlights the necessity for refining their training and functionality before confidently using them for rigorous academic purposes.
Overview
CoChat Summary
## TL;DR
This comparative analysis tested ChatGPT and Bard on 11 systematic reviews of shoulder rotator cuff pathology and found low retrieval accuracy plus frequent hallucinated references. GPT-4 performed best of the models tested, but the authors conclude that LLM-generated citations still require rigorous human validation and should not be used alone for systematic reviews.
## Key findings
- Across 11 systematic reviews in 4 fields, the study analyzed 471 references from 33 LLM prompts (3 models × 11 reviews).
- Precision was 9.4% (13/139) for GPT-3.5, 13.4% (16/119) for GPT-4, and 0% (0/104) for Bard (P<.001); recall was 11.9% (13/109) for GPT-3.5, 13.7% (15/109) for GPT-4, and 0 for Bard (P<.001).
- Hallucination rates were 39.6% (55/139) for GPT-3.5, 28.6% (34/119) for GPT-4, and 91.4% (95/104) for Bard (P<.001).
- Among nonhallucinated GPT-retrieved papers, title accuracy was very high (83/84, 99% for GPT-3.5 and 84/85, 99% for GPT-4), but DOI accuracy was low (13/82, 16% and 17/84, 20%); American authors were overrepresented in ChatGPT results (44% vs 16.5% in the original reviews for GPT-3.5, and 33% vs 16.5% for GPT-4; P<.001).
## Methods
- Sequential comparative design: PubMed search on July 27, 2023 identified English-language 2020 systematic reviews/meta-analyses on randomized clinical trials in shoulder rotator cuff pathology; 11 systematic reviews were selected after title/abstract/full-text screening and validation with PRISMA/PROSPERO.
- The LLMs tested were GPT-3.5, GPT-4, and Bard (PaLM 2.0); each fresh chat session was prompted with the same inclusion criteria, and two prompt versions were tried, keeping the one that produced more results.
- Outputs were compared with the original systematic review references using recall, precision, F1-score, and hallucination rate; papers were labeled hallucinated if any 2 of title, first author, or year were wrong.
## Limitations
- The authors state the study was restricted to shoulder rotator cuff pathology and to 3 LLMs, so the findings may not generalize to other specialties or models.
- The paper also notes that no established guidelines exist for using LLMs to optimize accuracy, and different prompts might yield different results.
- Evident from the text: the study focused on a narrow, single-topic set of reviews from one year (2020) and English-language publications only, which limits breadth and may constrain generalizability.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures






Verification
Claims the paper makes
1 Valid · 0 Not valid · 0 Not checked · 40 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Papers from American authors were selected in 16.5% (18/109) of original systematic reviews, 44% (37/84) of GPT-3.5 papers, and 33% (28/85) of GPT-4 papers (P<.001).”
§ Hallucination Rates and Reference Accuracy of ChatGPT and Bard for Systematic Reviews: Comparative Analysis > Results
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In most cases, the number of references returned by LLMs was greater or equal to that of the original papers (Table 2).”
§ Hallucination Rates and Reference Accuracy of ChatGPT and Bard for Systematic Reviews: Comparative Analysis > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Papers identified by LLMs were present in the original systematic reviews (precision) in 9.4% (13/139), 13.4% (16/119), and 0% (0/104) of cases for GPT-3.5, GPT-4, and Bard (P<.001), respectively.”
§ Hallucination Rates and Reference Accuracy of ChatGPT and Bard for Systematic Reviews: Comparative Analysis > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“authors list (73/84, 87% vs 74/85, 87%; P>.99)”
§ Hallucination Rates and Reference Accuracy of ChatGPT and Bard for Systematic Reviews: Comparative Analysis > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“randomized studies (33/84, 39% vs 42/85, 49%; P=.24)”
- Also stated in: § Principal Findings · plain
“For instance, the criterion of “randomized study” was adhered to in only 39% (33/84) to 49% (42/85) of nonhallucinated papers generated by ChatGPT, even when the term “randomized” appeared in the title or abstract of the papers from the original systematic reviews.”
§ Hallucination Rates and Reference Accuracy of ChatGPT and Bard for Systematic Reviews: Comparative Analysis > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
- Also stated in: § Principal Findings · plain
18 more claims
Create a free account to see all 41 claims and every source verdict.
- It may state two separate claims as one. 12
- Probably not a substantial claim. 9
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“Hallucination rates have been documented to vary between 3% – 28% in current literature [30, 31].”
From Artificial Intelligence (AI) – Powered Documentation Systems in Healthcare: A Systematic Review (2025) · cites this paper as [30]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Studies in radiology, psychiatry, and clinical rehabilitation have reported fabrication rates ranging from 20 to 70%”
From How LLMs Cite and Why It Matters: A Cross-Model Audit of Reference Fabrication in AI-Assisted Academic Writing and Methods to Detect Phantom Citations (2026) · cites this paper as [12]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Experimental studies from 2023 to 2025 have found that the rate of hallucination in LLMgenerated citations ranged from 19.9% to 91.4%, varying widely across models and time [16], [17], [18], [19], [24].”
From LLM hallucinations in the wild: Large-scale evidence from non-existent citations (2026) · cites this paper as [18]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“When citations are missing, inaccurate, or fabricated, the resulting break in the evidence chain can obscure logical dependencies, weaken argumentation, and jeopardize academic integrity at the level of both individual papers and the broader literature [10, 22, 26].”
From CiteAudit: You Cited It, But Did You Read It? A Benchmark for Verifying Scientific References in the LLM Era (2026) · cites this paper as [10]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Chelli et al. found that Bard produced incorrect references in 91.4% of systematic review prompts, whereas GPT-4 had a lower but still notable error rate of 28.6%18.”
From Multi-model assurance analysis showing large language models are highly vulnerable to adversarial hallucination attacks during clinical decision support (2025) · cites this paper as [18]
Experimental — accuracy not yet measured
16 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gotzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · BMJ · doi:10.1136/bmj.b2700 · Cited by 17,648
Artificial hallucinations in ChatGPT: implications in scientific writing
Alkaissi, H., & McFarlane, S. I. (2023) · Cureus · doi:10.7759/cureus.35179 · Cited by 1,540 · cited 1×
“This concern has been broached in previous literature [26-29], but our study provides an experimental design to probe the matter more deeply.”
BackgroundThe nuts and bolts of PROSPERO: an international prospective register of systematic reviews
Booth, A., Clarke, M., Dooley, G., Ghersi, D., Moher, D., Petticrew, M., & Stewart, L. (2012) · Systematic Reviews · doi:10.1186/2046-4053-1-2 · Cited by 1,433 · cited 1×
“Additionally, the registration status of these reviews was verified in the PROSPERO database [14].”
BackgroundChatting about ChatGPT: how may AI and GPT impact academia and libraries?
Lund, B. D., & Wang, T. (2023) · Library Hi Tech News · doi:10.1108/lhtn-01-2023-0009 · Cited by 1,355
ChatGPT listed as author on research papers: many scientists disapprove
Stokel-Walker, C. (2023) · Nature · doi:10.1038/d41586-023-00107-z · Cited by 868 · cited 1×
“While it is known that ChatGPT can help researchers write papers [3-5], controversy exists about whether it should be used at all, whether its use should be disclosed, and whether it should be listed as an author or not [6].”
BackgroundCan artificial intelligence help for scientific writing?
Salvagno, M., Taccone, F. S., & Gerli, A. G. (2023) · Critical Care · doi:10.1186/s13054-023-04380-2 · Cited by 827
Abstracts written by ChatGPT fool scientists
Else, H. (2023) · Nature · doi:10.1038/d41586-023-00056-7 · Cited by 611
Boileau, P., Baqué, F., Valerio, L., Ahrens, P., Chuinard, C., & Trojani, C. (2007) · The Journal of Bone and Joint Surgery-American Volume · doi:10.2106/00004623-200704000-00008 · Cited by 603
Self-citationChatGPT and the future of medical writing
Biswas, S. (2023) · Radiology · doi:10.1148/radiol.223312 · Cited by 571
Learning to summarize from human feedback
Stiennon, N., Ouyang, L., Wu, J., Ziegler, D. M., Lowe, R., Voss, C., Radford, A., Amodei, D., & Christiano, P. (2020) · arXiv · doi:10.48550/arxiv.2009.01325 · Cited by 550
Cited by
Items 301–400 of 423
El Kayali, M. K., Neumann, E. M., Fahy, S., Bartek, B., Oehme, S., Getgood, A., Jung, T., & Milinkovic, D. D. (2026) · Journal of ISAKOS · doi:10.1016/j.jisako.2026.101160
Sarin, A., Seffren, C. M., Shwaib, H., Matos, M. C., Eisenstein, S., Ferrara, M., Patel, N. M., & Langenfeld, S. J. (2026) · Diseases of the Colon & Rectum · doi:10.1097/dcr.0000000000004178
Artificial Intelligence in Ayurveda: A Contemporary Reflection on Prajnaparadha
Rathi, B. (2026) · Journal of Indian System of Medicine · doi:10.5005/jism-11051-0056
Suhardi, N. M., Oktarina, A., Retzky, J. S., Dhillon, D., Ninan, D., Bostrom, M. P., Xu, Y., & Suhardi, V. J. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.06.11.659165
Cline, D. B., Zgonc, D., Vass, W., Butkus, M., Baideme, M. P., & Krueger, B. (2025) · ASEE Annual Conference & Exposition · doi:10.18260/1-2--55372 · Cited by 1
TreeReader: A Hierarchical Academic Paper Reader Powered by Language Models
Zhang, Z., Chen, P., Du, F., Ye, R., Huang, O., Liut, M., & Aspuru‐Guzik, A. (2025) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl-hcc65237.2025.00039 · Cited by 2
Wilhelm-Weidner, A., Deimann, M., Schubert, M., & Granas, B. (2026) · Springer eBooks · doi:10.1007/978-3-032-26816-7_18
Implicit SDG Reasoning in LLMs: A Classroom-Oriented Benchmark
Hermann, J., Jansen, N., Doğangün, A., & van Ledden, S. (2025) · Australian Conference on Human-Computer Interaction (OzCHI) · doi:10.1145/3764687.3769916
Explainability of decoder-only clinical large language models: A scoping review
Mishra, N., Abu-Hanna, A., & Calixto, I. (2026) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2026.103501
The Validity and Reliability of Artificial Intelligence Chatbots in the Self-management of Diabetes
Dereli, F., & Yıldırım, J. G. (2026) · CIN Computers Informatics Nursing · doi:10.1097/cin.0000000000001573
Zakhari, R. (2026) · CIN Computers Informatics Nursing · doi:10.1097/cin.0000000000001603
Badilla, C. A. C. (2026) · Biblios Journal of Librarianship and Information Science · doi:10.47456/biblios.2026.1425
Knauer, M., Greß, J., Kather, J. N., & May, P. (2026) · doi:10.2196/preprints.93239
Zhang, X., Dai, Y., Zhao, X., Wu, L., Shao, B., Shan, X., Ji, F., Deng, R., & Zhao, B. (2026) · Digital Health · doi:10.1177/20552076261464749 · Cited by 3
Guideline Concordance and Safety of AI Chatbots for Circumcision Anesthesia: A Comparative Study
Şahin, A. T., & Yorulmaz, E. M. (2026) · Journal of Contemporary Medicine · doi:10.16899/jcm.1878107
Pareja, C. G. G., Pera, L. Y. P., Lizares, A. A., Rojas, L. M. H., Gomez, H., Tavera, Z. R. D., Andrade, C. P. A., Wong, F. M. R., Santana, W. M. R., Salcedo, C. M. D., et al. (2026) · Encyclopedia · doi:10.3390/encyclopedia6100209
Correia, L., & Feng, Y. (2026) · Journal of Food Protection · doi:10.1016/j.jfp.2026.100852
Tanas, Y., Gasper, G., Rashidi, K., & Swed, S. (2025) · International Journal of Surgery Protocols · doi:10.1097/sp9.0000000000000052
From scientific events to shared knowledge: An AI-based methodology with clinical supervision
Molteni, A., Girina, M., Cannone, P., Alberga, M., Bandera, V., Pisano, L., Vismara, M. E., Cuoghi, M., Attinà, G. M., Ricciardelli, L., et al. (2026) · Il Giornale di Chirurgia - Journal of the Italian Association of Hospital Surgeons · doi:10.1097/ia9.0000000000000087 · Cited by 5
AnatomyGPT 5.2 Versus ChatGPT 5.2 in Solving Anatomical Questions: A Comparative Study
Chaba-Karnaś, W., Tatarczyk, E., Kozioł, N., Kalita, N., Jaśkiewicz, K., Karnaś, M., Fibiger, G., Możdżeń, K., Walocha, J. A., & Kozioł, T. (2026) · Clinical Anatomy · doi:10.1002/ca.70206
Santhosh, V. N., Vas, R., Roychowdhury, B., Sakthi, K., & Rahaman, M. (2026) · Research Evaluation · doi:10.1093/reseval/rvag024
Stephenson, C. R., Varayil, J. E., Aakre, C. A., Croghan, I. T., & Hurt, R. T. (2026) · Journal of Primary Care & Community Health · doi:10.1177/21501319261455393 · Cited by 1
Automated Data Analytics with GPTs under ChatGPT: Lessons Learned from Two Case Studies
AlMahri, K. R. S., Alzahmi, S. G. M., Ezzaza, M., Alyammahi, J. S. A., & Ertek, G. (2025) · International Conference on Business Intelligence for Technology Innovation (ICBITI) · doi:10.1109/icbiti65527.2025.11501163
Benefits and challenges of integrating generative AI in health technology education—a case study
Pikkarainen, M. A., Lukic, M., Johansen, S., Admiraal, W., Berg, A., & Røe, Y. (2026) · Frontiers in Education · doi:10.3389/feduc.2026.1763138
Современные методы повышения качества цитатно-осведомленной суммаризации
Кузнецов, И. (2026) · Cifra LLC Journals · doi:10.60797/irj.2026.168.94.2
Artificial Intelligence and Medical Distance Learning and Management
Madani, R., Richter, D. M., & Gomes, T. J. (2026) · Springer eBooks · doi:10.1007/978-3-032-19116-8_5
Navigating the future of teaching, learning, and creativity with generative AI
Acar, S., & Ellis, A. (2025) · Elsevier eBooks · doi:10.1016/b978-0-443-34073-4.00003-4
Kamaleddin, M. A., Mirjalili, M., Barzegar, R., Le, N. T., Cote, Z., Winkler, O., Burback, L., Zhang, Y., Zai͏̈ane, O. R., Monson, C. M., et al. (2026) · JMIR Medical Education · doi:10.2196/92964 · Cited by 1
Applications of Artificial Intelligence in Cancer Diagnosis and Treatment
Xie, Y., Wang, Z., Zeng, Z., Xin, T., Yuan, S., Qin, F., & Chen, Z. (2026) · Cancer Medicine · doi:10.1002/cam4.72143 · Cited by 3
From User Needs to Smart Ecosystems Through LLMs: The Smartifier Case Study
Caslini, G., Gianotti, M., & Garzotto, F. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-95452-8_5
Evaluation of six large language models for study identification in an obstetric systematic review
Somer, S., Vardy, M., Eisenberger, J., Shapira, M., Mehler, S., & Safrai, M. (2026) · Minerva Obstetrics and Gynecology · doi:10.23736/s2724-606x.25.05849-x
Natural language querying of biological databases with large language models
Makarov, V. A., Stroganov, O. V., Furlong, L. I., Evarts, B., van den Biggelaar, L., Goulas, A., Stolte, E., Marren, D., & Greiffenberg, L. (2026) · Drug Discovery Today · doi:10.1016/j.drudis.2026.104654 · Cited by 1
Loneliness and social isolation at the emergency department: a scoping review protocol
Chan, W. L., Gao, Z., Liu, A. C., Rainer, T. H., Tang, W. W. C., Wing, Y. S. S., & Lederman, Z. (2026) · BMJ Open · doi:10.1136/bmjopen-2025-108972
Huy, N. T., Minh, P. Q., Bich, L. T., & Nam, T. V. (2026) · Archives of Pharmacy Practice · doi:10.51847/mdbmowwqrb
Madeira, C., Dopke, L., Losch, G., Magnaguagno, M. C., & Griebler, D. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-39898-7_18
Domingo, J. L. (2026) · Qeios · doi:10.32388/9aob4v
Harada, Y. (2026) · Cureus Journal of AI-Augmented Research · doi:10.7759/s44568-026-00185-3
Uçar, M. T., Bostan, E., Ortabağ, T., & Dönmez, E. (2026) · Diagnostics · doi:10.3390/diagnostics16162620 · Cited by 1
Wilson, E., Jong, F. D., & Peters, L. (2026) · Annals of Pharmacy Practice and Pharmacotherapy · doi:10.51847/xt3dqdt8kp
Freudenberg, J., Knitza, J., Gremke, N., Amann, N., Deutsch, T. M., Tauber, N., Muras, K., Oftring, Z. S., Engler, T., Englisch, A., et al. (2026) · Journal of Medical Systems · doi:10.1007/s10916-026-02434-w · Cited by 2
Verdaguer-Tremolosa, M., Kotoreni, G., Hoggart, S., & Cano, M. L. (2026) · Journal of Abdominal Wall Surgery · doi:10.3389/jaws.2026.17401
Rao, A., Cholankeril, G. T., Flores, A. G., Sood, G., White, T., Kanwal, F., & El‐Serag, H. B. (2026) · Hepatology Communications · doi:10.1097/hc9.0000000000000895 · Cited by 1
Sallai, T., Nitto, M. D., Catania, G., Zanini, M. P., Sasso, L., & Bagnasco, A. (2026) · Nurse Education in Practice · doi:10.1016/j.nepr.2026.104801
Naffi, N., Bahlouli, Y. E., Fortier, N., Doumbouya, M. B., Shakeraneh, S., Faraj, D. A., Grégoire, J., Beaulieu, N., & Whelan, K. (2026) · Médiations et médiatisations · doi:10.52358/mm.vi25.507
Vollmer, A., Schorn, L., Schrader, F., Kübler, N., Sproll, C. K., Vollmer, M., Singh, D. D., & Saravi, B. (2026) · Diagnostics · doi:10.3390/diagnostics16152456 · Cited by 2
Основные этапы цитатно-осведомленной суммаризации
Кузнецов, И. (2026) · Cifra LLC Journals · doi:10.60797/irj.2026.168.94.1
Guerschberg, L., & Gutierrez, Y. E. (2026) · Revista Multidisciplinar Epistemología de las Ciencias · doi:10.71112/gpqkd594
Mourelatou, E., & Katakis, I. (2025) · IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT) · doi:10.1109/wi-iat67162.2025.00079
Omar, M., Awan, K., Ramaiya, N., & Mohamed, I. (2026) · Current Problems in Diagnostic Radiology · doi:10.1067/j.cpradiol.2026.03.009 · Cited by 1
Natural Language Querying of Biological Databases with Large Language ModelsPreprint
Makarov, V. A., Stroganov, O. V., Furlong, L. I., et al. (2025) · doi:10.31219/osf.io/94mex_v1
Domingo, J. L. (2026) · Qeios · doi:10.32388/9aob4v.2
ChatGPT and Large Language Models in Contemporary Nursing
Uchmanowicz, I., Aldossary, H. M., Lee, C. S., Figura, M., Jędrzejczyk, M., Maria, M. D., Pucciarelli, G., Molka, E., Straiton, N., & Kołtuniuk, A. (2026) · Journal of Clinical Nursing · doi:10.1111/jocn.70490 · Cited by 4
Generative AI in Surgical Care: Evaluating Large Language Model Performance in Patient Education
Hall, A. E., Patel, A. K., Shariati, K., Perrotta, A. T., Argame, A. A., Tseng, C. Y., Tseng, C., Hidalgo, M. A., & Lee, J. C. (2025) · Journal of Artificial Intelligence for Medical Sciences · doi:10.55578/joaims.251202.001
Evidence-Bound Generation for Medicines Information with Verifiable Claims and Visible Uncertainty
Ferraro, L., Ricci, M., (1560388), G. M., & Greco, P. (2026) · Archives of Pharmacy Practice · doi:10.51847/tzcg1ksqsz
Combining Formal Reasoning and LLMs for Scenario-Based Educational STEM Exercises
Vollmer, L., Loubet, R., Neider, D., Hasse, H., & Leitte, H. (2026) · Human-Centric Intelligent Systems · doi:10.1007/s44230-026-00174-1
Shin, R., Lee, J.-M., Park, J., Kwun, J.-S., Cho, H.-W., Kang, S., & Jeon, K. (2026) · medRxiv · doi:10.64898/2026.08.31.26361806
Anggraini, C. N., Rachmadania, N., Nugroho, C., Wulandari, A., & Rina, N. (2026) · Frontiers in Political Science · doi:10.3389/fpos.2026.1784691 · Cited by 6
Fu, H., Du, Y., Zhao, G., Xiao, Y., Song, S., Zang, X., Zhou, T., Zhuang, Y., Mo, R., Lai, R., et al. (2026) · Liver International · doi:10.1111/liv.70879 · Cited by 1
Suhardi, N. M., Oktarina, A., Bostrom, M. P., Xu, Y., & Suhardi, V. J. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.01.28.635344 · Cited by 2
Zhang, C. (2026) · Open Computer Science · doi:10.1515/comp-2025-0073
BasketVision: Benchmarking MLLMs' Grasp of Complex Dynamic Systems
Jiang, X., Sun, Q., & Lv, X. (2025) · International Conference on Communication and Information Processing (ICCIP) · doi:10.1145/3784833.3784850
Tayo-Ladega, O. T., Dada, O. S., Anih, D. C., & Okorocha, U. C. (2026) · Trends in Social Sciences · doi:10.21124/tss.2026.57.68
Marjanović, M., & Latinović, L. (2026) · Serbian Journal of Engineering Management · doi:10.5937/sjem2600076m
Umpar, N. R., Umpar, S., Magsino, T. A., Nolasco, D. E., & Abuton, D. (2025) · International Journal on Culture History and Religion · doi:10.63931/ijchr.v7i2.598
Кузнецов, И. (2026) · Международный научно-исследовательский журнал · doi:10.60797/irj.2026.168.94
Toward autonomous chemical discovery: agentic AI in organic synthesis and nanochemistry
Babu, B. H., Vutla, P. C., Mahimaluru, J., Chong, B. K. J., & Kambam, S. (2026) · Results in Engineering · doi:10.1016/j.rineng.2026.113180
Avrumova, F., Cesar, M. D., Loggia, G., Strebel, K., Altorfer, F. C., Castro, R. I., Zhu, J., Abjornson, C., Bono, C. M., & Lebl, D. R. (2026) · The Spine Journal · doi:10.1016/j.spinee.2026.08.002
Funa, A. A., Ricafort, J. D., & Gabay, R. A. E. (2026) · Medical Science Educator · doi:10.1007/s40670-026-02736-1 · Cited by 1
Mai, H., Li, W., Chen, J., Huang, B., Li, B., & Long, S. (2026) · International Conference on Computer and Multimedia Technology (ICCMT) · doi:10.1109/iccmt69755.2026.11608235
Blease, C., Hagström, J., Gaab, J., Carey, A., Cipriani, F., Gorman, C., Nascimento, A. F., Fitzgerald, A., Holtz, L., Schwarz, J., et al. (2026) · JMIR Mental Health · doi:10.2196/105052 · Cited by 1
LitPilot: A Human-Centered Platform for Transparent and Interactive Systematic Literature Reviews
Pei, B., Sun, X., Guo, R., & Zhang, Z. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2646419
Venne, D. M., Hartley, D. M., Dorris, C. S., Torres, L. M., Malchione, M. D., Mitgang, E. A., & Goodman, J. L. (2025) · Discover Public Health · doi:10.1186/s12982-025-01245-7
Jafar, A. B., Motair, M., & Hajeri, S. A. (2026) · World Journal of Otorhinolaryngology - Head and Neck Surgery · doi:10.1002/wjo2.70124
Artificial intelligence transforming healthcare research: opportunities, risks, and responsible use
Juan C Calderon1, 2., Karla Robles-Velasco2, 3., & Ferreira, J. C. (2025) · Jornal Brasileiro de Pneumologia · doi:10.36416/1806-3756/e20250433
Reimagining biomedical science workflows in the age of large language models
Roberts, K. F., Koutarapu, S., Melendez, J., Landsness, E. C., Reese, J. T., Elbert, D. L., & Bateman, R. J. (2026) · npj Dementia · doi:10.1038/s44400-026-00126-3
“A frequently encountered hallucination (i.e. con-fabulation)inthiscontextinvolvesfabricatedcitations 14 – 18 .”
Georgiou, G. P. (2026) · AI in Education · doi:10.3390/aieduc2010002 · Cited by 5
A hybrid validation framework for LLM-enhanced open data service: the mindat API as a use case
Zhang, J., Chen, W., Li, C., & Ma, X. (2026) · Earth Science Informatics · doi:10.1007/s12145-026-02208-5
İlleez, Ö. G., Giray, E., Ateş, U., Keles, A., Turkyilmaz, S. G., & Yüksel, M. (2026) · Work · doi:10.1177/10519815261488805
The TRACE Model for Graduate Research Ethics, Publication Integrity, and Global Academic Contexts
KALECİ, F. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-2600-1232-1.ch004 · Cited by 1
Основные этапы комплексной оценки соответствия научных утверждений информации из первоисточника
Кузнецов, И. (2026) · Cifra LLC Journals · doi:10.60797/irj.2026.168.94.3
Simon, B. F., Dénes-Fazakas, L., Eigner, G., Kovács, L., & Szilágyi, L. (2026) · IEEE International Conference on Intelligent Engineering Systems (INES) · doi:10.1109/ines69513.2026.11661038
Bui, D. K. (2026) · International Artificial Intelligence Technology Conference (AITC) · doi:10.1109/aitc70732.2026.11666499
Rezaeian, M. (2025) · مجله علمی دانشگاه علوم پزشکی رفسنجان · doi:10.66224/jrums.24.9.773
Sun, M., & Zhu, H. (2026) · IEEE Access · doi:10.1109/access.2026.3735019
Kutluay, G. E., & KONDİL, Y. (2026) · Cureus · doi:10.7759/cureus.116893
Vassileva, J. (2026) · International Conference on Revolutionary Artificial Intelligence and Future Applications (Rev-AI) · doi:10.1109/rev-ai70456.2026.11622089
Hunt, D., & Miceli, M. D. (2026) · American Journal of STEM Education · doi:10.32674/tfptev94 · Cited by 1
Domingo, J. L. (2026) · Qeios · doi:10.32388/9aob4v.3
A 2026 horizon scan for biodiversity conservation in South Africa
Seymour, C. L., Tolley, K. A., Zengeya, T., Spear, D., Cloete, J. A., Dayaram, A., da Silva, J. M., Alexander, G. J., Handley, K., Joseph, G. S., et al. (2026) · Ambio · doi:10.1007/s13280-026-02365-3
Civelekler, M., & Çıtırık, M. (2026) · Indian Journal of Ophthalmology · doi:10.4103/ijo.ijo_3155_25
HAICO-SLR Guide: Conducting and Reporting Human-AI Collaboration in Systematic Literature ReviewsPreprint
Fernandes, B. S., Schleder, M. O., de Oliveira, O. C., Jaeger, R. D. B., Nascimento, L. D. S., & de Oliveira, F. D. S. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7048543
de Beer, J., Mohammed, K. B., Barrios, J. M., Elnajjar, S., Aldabahy, M. F., Moodley, V., Almuntashiri, A., Basha, M., Eissa, A. O., Barayan, W., et al. (2025) · Journal of Nursing Science and Professional Practice · doi:10.4103/jnspp.jnspp_40_25 · Cited by 1
Artificial Intelligence in Scholarly Publishing: New Tools, Same Standards
Watson, J. L. (2025) · Clinical journal of oncology nursing · doi:10.1188/25.cjon.268-269 · Cited by 2
Critical thinking in human-AI collaborationsPreprint
Bilen, E., & Hervé, J. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6304755 · Cited by 1
Liu, S.-W., Chu, S., Wang, H.-C., & Lee, M.-S. (2026) · JMIR Formative Research · doi:10.2196/98343
AISA: A Web-Based Platform for Evaluating the Literature Search Capabilities of LLMs
Rahimi, A., Wimmer, M., & Gusenbauer, M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-29372-5_30
Metadata
- Authors
- Chelli, M., Descamps, J., Lavoué, V., Trojani, C., Azar, M., Deckert, M., Raynier, J.-L., Clowez, G., Boileau, P., & Ruetsch-Chelli, C.
- Affiliations
- Assistance Publique – Hôpitaux de Paris (FR)
- Inserm (FR)
- Corresponding author
- Chelli, Mikaël
- ORCID
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2024-05-22
- Volume
- 26
- Pages
- e53164
- ISSN
- 1438-8871 (ISSN-L)
- PMC
- PMC11153973
- OpenAlex
- W4398203672
- Semantic Scholar
- 9a0e8c101b72e9094ea554194698daa3aeb3bc6f
- Cited by
- 602
- References
- 30
- Citations by year
- 2026: 378 · 2025: 202 · 2024: 22
- FWCI
- 11.51
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Journal of Medical Internet Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- HAL (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- hallucinations, large language models, ChatGPT, Google Bard, Gemini, systematic reviews, reference retrieval, literature search automation, precision and recall, citation bias, GPT-3.5, GPT-4, AI in scientific writing
- MeSH terms
- Systematic Reviews as Topic, Artificial Intelligence
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-09-28, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-28 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.