The TRIPOD-LLM reporting guideline for studies using large language models
Open accessAbstract
Large language models (LLMs) are rapidly being adopted in healthcare, necessitating standardized reporting guidelines. We present transparent reporting of a multivariable model for individual prognosis or diagnosis (TRIPOD)-LLM, an extension of the TRIPOD + artificial intelligence statement, addressing the unique challenges of LLMs in biomedical applications. TRIPOD-LLM provides a comprehensive checklist of 19 main items and 50 subitems, covering key aspects from title to discussion. The guidelines introduce a modular format accommodating various LLM research designs and tasks, with 14 main items and 32 subitems applicable across all categories. Developed through an expedited Delphi process and expert consensus, TRIPOD-LLM emphasizes transparency, human oversight and task-specific performance reporting. We also introduce an interactive website (https://tripod-llm.vercel.app/) facilitating easy guideline completion and PDF generation for submission. As a living document, TRIPOD-LLM will evolve with the field, aiming to enhance the quality, reproducibility and clinical applicability of LLM research in healthcare through comprehensive reporting. TRIPOD-LLM (transparent reporting of a multivariable model for individual prognosis or diagnosis–large language model) is a checklist of items considered essential for good reporting of studies that are developing or evaluating an LLM for use in healthcare settings. It is a ‘living guideline’ that emphasizes transparency, human oversight and task-specific performance reporting.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
LLM version, prompting approach, session handling, and parameter settings can influence model outputs.
“These details are important because LLM version, prompting approach, session handling, and parameter settings can influence model outputs.106,111”
From Large Language Models in Pharmacist-Led Direct Patient Care: A Rapid Review of Published Evaluations (2026) · cites this paper as [111]
The Discussion says prompting approaches can considerably influence LLM performance and calls for reporting model names and versions. The Rationale for TRIPOD-LLM says choices of fine-tuning hyperparameters and prompting affect model reliability. These passages support much of the claim, but our copy does not establish that session handling influences outputs. (LLM, conf=0.84)
Experimental — accuracy not yet measuredOmitting details on LLM use limits experimental transparency, replicability, and interpretation of evaluation findings.
“Omitting details on LLM use also limits experimental transparency, replicability, and interpretation of evaluation findings.111”
From Large Language Models in Pharmacist-Led Direct Patient Care: A Rapid Review of Published Evaluations (2026) · cites this paper as [111]
We adhere to the TRIPOD-LLM guidelines and completed the accompanying checklist, meeting all reporting requirements.
“We adhere to the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis - Large Language Models (TRIPOD-LLM) guidelines75 and completed the accompanying checklist, meeting all reporting requirements, as detailed in Supplementary Note 4.”
From MoMA: a mixture-of-multimodal-agents architecture for enhancing clinical prediction modelling (2025) · cites this paper as [75]
This study follows established guidelines for the transparent reporting of a multivariable model for individual prognosis or diagnosis (TRIPOD)-LLM.
“This study follows established guidelines for the transparent reporting of a multivariable model for individual prognosis or diagnosis (TRIPOD)-LLM (Table S2).32”
From irAE-GPT: leveraging large language models to identify immune-related adverse events in electronic health records and clinical trial datasets (2026) · cites this paper as [32]
Reporting follows the TRIPOD-LLM guideline for studies using large language models.
“Reporting follows the TRIPOD-LLM guideline for studies using large language models [11], and a completed checklist is provided as a supplementary file.”
From Do Calibrated-Probability Claims Hold Up in Clinical Diagnosis? A Pre-registered Evaluation of a Decision Model Against Frontier Language Models (2026) · cites this paper as [11]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · see citing works
Radiomics: the bridge between medical imaging and personalized medicine
Lambin, P., Leijenaar, R. T. H., Deist, T. M., Peerlings, J., de Jong, E. E. C., van Timmeren, J., Sanduleanu, S., Larue, R. T. H. M., Even, A. J. G., Jochems, A., et al. (2017) · Nature Reviews Clinical Oncology · doi:10.1038/nrclinonc.2017.141 · Cited by 5,688 · see citing works
Collins, G. S., Moons, K. G. M., Dhiman, P., Riley, R. D., Beam, A. L., Van Calster, B., Ghassemi, M., Liu, X., Reitsma, J. B., van Smeden, M., et al. (2024) · BMJ · doi:10.1136/bmj-2023-078378 · Cited by 4,718 · see citing works
Self-citationCollins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · BMJ · doi:10.1136/bmj.g7594 · Cited by 4,388 · see citing works
Self-citationGLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding
Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. (2018) · Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/w18-5446 · Cited by 4,222 · see citing works
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083 · see citing works
Self-citationCollins, P. G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · British Journal of Cancer · doi:10.1038/bjc.2014.639 · Cited by 3,312 · see citing works
Self-citationWhat is ‘quality of evidence’ and why is it important to clinicians?
Guyatt, G. H., Oxman, A. D., Kunz, R., Vist, G. E., Falck-Ytter, Y., & Schünemann, H. J. (2008) · BMJ · doi:10.1136/bmj.39490.551019.be · Cited by 3,250 · see citing works
Liu, X., Cruz Rivera, S., Moher, D., Calvert, M. J., Denniston, A. K., The SPIRIT-AI and CONSORT-AI Working Group, SPIRIT-AI and CONSORT-AI Steering Group, Chan, A.-W., Darzi, A., Holmes, C., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-1034-x · Cited by 1,421 · see citing works
The imperative for regulatory oversight of large language models (or generative AI) in healthcare
Meskó, B., & Topol, E. J. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00873-0 · Cited by 1,047 · see citing works
Vasey, B., Nagendran, M., Campbell, B., Clifton, D. A., Collins, G. S., Denaxas, S., Denniston, A. K., Faes, L., Geerts, B., Ibrahim, M., et al. (2022) · Nature Medicine · doi:10.1038/s41591-022-01772-9 · Cited by 1,039 · see citing works
Self-citationA visual-language foundation model for computational pathology
Lu, M. Y., Chen, B., Williamson, D. F. K., Chen, R. J., Liang, I., Ding, T., Jaume, G., Odintsov, I., Le, L. P., Gerber, G., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-02856-4 · Cited by 966 · cited 1× · see citing works
“Of note, this guideline was developed with text-only LLMs in mind; however, advances in multimodal models incorporating LLMs, such as vision-language models 47 , are now rapidly emerging — illustrating the need for rapid, nimble approaches for reporting guidelines.”
Large Language Model Influence on Diagnostic Reasoning: A Randomized Clinical Trial
Goh, E., Gallo, R., Hom, J., Strong, E., Weng, Y., Kerman, H., Cool, J. A., Kanjee, Z., Parsons, A. S., Ahuja, N., et al. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.40969 · Cited by 724 · see citing works
Minimum information about clinical artificial intelligence modeling: the MI-CLAIM checklist.
Norgeot, B., Quer, G., Beaulieu-Jones, B. K., Torkamani, A., Dias, R., Gianfrancesco, M., Arnaout, R., Kohane, I. S., Saria, S., Topol, E., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-1041-y · Cited by 632 · cited 1× · see citing works
“This is in addition to other guidelines that offer complementary guidance on AI development throughout the model life cycle 12–14 .”
Zack, T., Lehman, E., Suzgun, M., Rodriguez, J. A., Celi, L. A., Gichoya, J., Jurafsky, D., Szolovits, P., Bates, D. W., Abdulnour, R.-E. E., et al. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(23)00225-x · Cited by 510 · see citing works
Self-citationHealth system-scale language models are all-purpose prediction engines
Jiang, L. Y., Liu, X. C., Nejatian, N. P., Nasir-Moin, M., Wang, D., Abidin, A., Eaton, K., Riina, H. A., Laufer, I., Punjabi, P., et al. (2023) · Nature · doi:10.1038/s41586-023-06160-y · Cited by 460 · see citing works
Self-citationAlmanac — Retrieval-Augmented Language Models for Clinical Medicine
Zakka, C., Shad, R., Chaurasia, A., Dalal, A. R., Kim, J. L., Moor, M., Fong, R., Phillips, C., Alexander, K., Ashley, E., et al. (2024) · NEJM AI · doi:10.1056/aioa2300068 · Cited by 424 · see citing works
Salazar, J., Liang, D., Nguyen, T. Q., & Kirchhoff, K. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.240 · Cited by 414 · see citing works
A study of generative large language model for medical research and healthcare
Peng, C., Yang, X., Chen, A., Smith, K. E., PourNejatian, N., Costa, A. B., Martin, C., Flores, M. G., Zhang, Y., Magoc, T., et al. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00958-w · Cited by 408 · see citing works
Ambient artificial intelligence scribes to alleviate the burden of clinical documentation.
Tierney, A. A., Gayre, G., Hoberman, B., Mattern, B., Ballesca, M., Kipnis, P., Liu, V., & Lee, K. (2024) · NEJM Catalyst · doi:10.1056/cat.23.0404 · Cited by 400 · cited 1× · see citing works
“…in several domains across administrative and healthcare delivery use cases, including generating drafts for patient communication, medical document summarization, question answering, information retrieval, medical diagnosis, treatment recommendations, patient education and medical education 1–5 .”
Large Language Models in Medicine: The Potentials and Pitfalls
Omiye, J. A., Gui, H., Rezaei, S. J., Zou, J., & Daneshjou, R. (2024) · Annals of Internal Medicine · doi:10.7326/m23-2772 · Cited by 394 · see citing works
Self-citationLarge language models are few-shot clinical information extractors
Agrawal, M., Hegselmann, S., Lang, H., Kim, Y., & Sontag, D. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.130 · Cited by 358 · see citing works
Living systematic reviews: 4. Living guideline recommendations.
Akl, E. A., Meerpohl, J. J., Elliott, J., Kahale, L. A., Schünemann, H. J., Agoritsas, T., Hilton, J., Perron, C., Akl, E., Hodder, R., et al. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.08.009 · Cited by 328 · see citing works
The shaky foundations of large language models and foundation models for electronic health records
Wornow, M., Xu, Y., Thapa, R., Patel, B., Steinberg, E., Fleming, S., Pfeffer, M. A., Fries, J., & Shah, N. H. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00879-8 · Cited by 324 · see citing works
Koçak, B., D’Antonoli, T. A., Mercaldo, N. D., Alberich‐Bayarri, Á., Baeßler, B., Ambrosini, I., Андрейченко, А. Е., Bakas, S., Beets‐Tan, R. G. H., Bressem, K. K., et al. (2024) · Insights into Imaging · doi:10.1186/s13244-023-01572-w · Cited by 309 · see citing works
Zaretsky, J., Kim, J. M., Baskharoun, S., Zhao, Y., Austrian, J., Aphinyanaphongs, Y., Gupta, R., Blecker, S. B., & Feldman, J. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.0357 · Cited by 303 · see citing works
Self-citationLarge language models to identify social determinants of health in electronic health records
Guevara, M., Chen, S., Thomas, S., Chaunzwa, T. L., Franco, I., Kann, B. H., Moningi, S., Qian, J. M., Goldstein, M., Harper, S., et al. (2024) · npj Digital Medicine · doi:10.1038/s41746-023-00970-0 · Cited by 278 · see citing works
Self-citationUse of Artificial Intelligence Chatbots for Cancer Treatment Information
Chen, S., Kann, B. H., Foote, M. B., Aerts, H. J. W. L., Savova, G. K., Mak, R. H., & Bitterman, D. S. (2023) · JAMA Oncology · doi:10.1001/jamaoncol.2023.2954 · Cited by 251 · see citing works
Self-citationJin, Q., Yang, Y., Chen, Q., & Lu, Z. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btae075 · Cited by 176 · see citing works
Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling
Biderman, S., Schoelkopf, H., Anthony, Q., Bradley, H., O'Brien, K., Hallahan, E., Khan, M. A., Purohit, S., Prashanth, U. S., Raff, E., et al. (2023) · arXiv · doi:10.48550/arxiv.2304.01373 · Cited by 160 · see citing works
AI-Generated Draft Replies Integrated Into Health Records and Physicians’ Electronic Communication
Tai-Seale, M., Baxter, S. L., Vaida, F., Walker, A., Sitapati, A. M., Osborne, C., Diaz, J., Desai, N., Webb, S., Polston, G., et al. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.6565 · Cited by 158 · see citing works
The effect of using a large language model to respond to patient messages
Chen, S., Guevara, M., Moningi, S., Hoebers, F., Elhalawani, H., Kann, B. H., Chipidza, F. E., Leeman, J., Aerts, H. J. W. L., Miller, T., et al. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(24)00060-8 · Cited by 147 · see citing works
Self-citationMEDITRON-70B: Scaling Medical Pretraining for Large Language Models
Chen, Z., Cano, A. H., Romanou, A., Bonnet, A., Matoba, K., Salvi, F., Pagliardini, M., Fan, S., Köpf, A., Mohtashami, A., et al. (2023) · arXiv · doi:10.48550/arxiv.2311.16079 · Cited by 122 · see citing works
A framework for the oversight and local deployment of safe and high-quality prediction models
Bedoya, A. D., Economou-Zavlanos, N. J., Goldstein, B. A., Young, A., Jelovsek, J. E., O’Brien, C., Parrish, A. B., Elengold, S., Lytle, K., Balu, S., et al. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac078 · Cited by 119 · see citing works
Peer review of GPT-4 technical report and systems card
Gallifant, J., Fiske, A., Strekalova, Y. A., Osorio-Valencia, J. S., Parke, R. L., Mwavu, R., Martinez‐Martin, N., Gichoya, J. W., Ghassemi, M., Demner‐Fushman, D., et al. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000417 · Cited by 115 · see citing works
Self-citationA Nationwide Network of Health AI Assurance Laboratories
Shah, N. H., Halamka, J. D., Saria, S., Pencina, M., Tazbaz, T., Tripathi, M., Callahan, A., Hildahl, H., & Anderson, B. (2024) · JAMA · doi:10.1001/jama.2023.26930 · Cited by 114 · see citing works
AI-Generated Clinical Summaries Require More Than Accuracy
Goodman, K. E., Yi, P. H., & Morgan, D. J. (2024) · JAMA · doi:10.1001/jama.2024.0555 · Cited by 107 · see citing works
Advancing entity recognition in biomedicine via instruction tuning of large language models
Keloth, V. K., Hu, Y., Xie, Q., Peng, X., Wang, Y., Zheng, A., Selek, M., Raja, K., Wei, C. H., Jin, Q., et al. (2024) · Bioinformatics · doi:10.1093/bioinformatics/btae163 · Cited by 104 · see citing works
A Framework for the Development of Living Practice Guidelines in Health Care
Mikati, I. K. E., Khabsa, J., Harb, T., Khamis, M. M., Agarwal, A., Hernandez, H. P., Farran, S. H., Khamis, A. M., Zein, O. E., El-Khoury, R., et al. (2022) · Annals of Internal Medicine · doi:10.7326/m22-0514 · Cited by 99 · see citing works
Evaluating the ChatGPT family of models for biomedical reasoning and classification
Chen, S., Li, Y., Lu, S., Van, H., Aerts, H. J. W. L., Savova, G. K., & Bitterman, D. S. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad256 · Cited by 91 · see citing works
Self-citationChatGPT: standard reporting guidelines for responsible use
Cacciamani, G. E., Collins, G. S., & Gill, I. S. (2023) · Nature · doi:10.1038/d41586-023-01853-w · Cited by 56 · see citing works
Self-citationReporting standards for the use of large language model-linked chatbots for health advice.
Huo, B., Cacciamani, G. E., Collins, G. S., McKechnie, T., Lee, Y., & Guyatt, G. (2023) · Nature Medicine · doi:10.1038/s41591-023-02656-2 · Cited by 53 · see citing works
Self-citationRegulating advanced artificial agents
Cohen, M. K., Kolt, N., Bengio, Y., Hadfield, G. K., & Russell, S. (2024) · Science · doi:10.1126/science.adl0625 · Cited by 45 · see citing works
Large Language Models are Few-Shot Health LearnersPreprint
Liu, X., McDuff, D., Kovacs, G., Galatzer‐Levy, I. R., Sunshine, J. E., Zhan, J., Poh, M., Liao, S., Achille, P. D., & Patel, S. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.15525 · Cited by 41 · see citing works
Navarro, D. F., Cheyne, S., Hill, K. M., McFarlane, E., Morgan, R. L., Murad, M. H., Mustafa, R. A., Sultan, S., Tunnicliffe, D. J., Vogel, J. P., et al. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2022.12.022 · Cited by 38 · see citing works
Measuring Progress on Scalable Oversight for Large Language Models
Bowman, S. R., Hyun, J., Perez, E., Chen, E., Pettit, C., Heiner, S., Lukošiūtė, K., Askell, A., Jones, A., Chen, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2211.03540 · Cited by 34 · see citing works
Weak-to-Strong Generalization: Eliciting Strong Capabilities With Weak Supervision
Burns, C., Izmailov, P., Kirchner, J. H., Baker, B., Gao, L., Aschenbrenner, L., Chen, Y., Ecoffet, A., Joglekar, M., Leike, J., et al. (2023) · arXiv · doi:10.48550/arxiv.2312.09390 · Cited by 34 · see citing works
Toward Clinical-Grade Evaluation of Large Language Models
Moreno, A. C., & Bitterman, D. S. (2024) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/j.ijrobp.2023.11.012 · Cited by 28 · cited 2× · see citing works
“To safeguard LLM use and increase transparency, standardization in the development and reporting of LLMs is essential — to ensure consistency, reliability and verifiability, akin to established clinical-grade evaluation in other scientific domains 28–30 .”
Self-citationLanguage Models are Surprisingly Fragile to Drug Names in Biomedical Benchmarks
Gallifant, J., Chen, S., Moreira, P. J. F., Munch, N., Gao, M., Pond, J., Celi, L. A., Aerts, H., Hartvigsen, T., & Bitterman, D. (2024) · Findings of the Association for Computational Linguistics: EMNLP 2024 · doi:10.18653/v1/2024.findings-emnlp.726 · Cited by 19 · see citing works
Self-citationLanguage Models are Surprisingly Fragile to Drug Names in Biomedical Benchmarks
Gallifant, J., Chen, S., Moreira, P., Munch, N., Gao, M., Pond, J., Celi, L. A., Aerts, H., Hartvigsen, T., & Bitterman, D. (2024) · arXiv · doi:10.48550/arxiv.2406.12066 · Cited by 19 · see citing works
Self-citationCross-Care: Assessing the Healthcare Implications of Pre-training Data on Language Model Bias
Chen, S., Gallifant, J., Gao, M., et al. (2024) · neural information processing systems · doi:10.52202/079017-0749 · Cited by 18 · see citing works
Self-citationLCD benchmark: long clinical document benchmark on mortality prediction for language models
Yoon, W., Chen, S., Gao, Y., Zhao, Z., Dligach, D., Bitterman, D. S., Afshar, M., & Miller, T. A. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae287 · Cited by 13 · see citing works
Self-citationLLM Critics Help Catch LLM Bugs
McAleese, N., Pokorny, R. M., Uribe, J. F. C., Nitishinskaya, E., Trebacz, M., & Leike, J. (2024) · arXiv · doi:10.48550/arxiv.2407.00215 · Cited by 10 · see citing works
Red Teaming Large Language Models in Medicine: Real-World Insights on Model BehaviorPreprint
Chang, C. T., Farah, H., Gui, H., Rezaei, S. J., Bou-Khalil, C., Park, Y., Swaminathan, A., Omiye, J. A., Kolluri, A., Chaurasia, A. R., et al. (2024) · medRxiv · doi:10.1101/2024.04.05.24305411 · Cited by 9 · see citing works
Self-citationExamining Large Pre-Trained Language Models for Machine Translation: What You Don't Know About ItPreprint
Han, L., Erofeev, G., Sorokina, I., et al. (2022) · Research Explorer (The University of Manchester) · doi:10.48550/arxiv.2209.07417 · Cited by 3 · see citing works
Hansen, L., Andersen, N., Gallifant, J., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2405.05049 · Cited by 2 · see citing works
Self-citationOpenAI, Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.08774
Self-citation
Flagged first, then by citations
Research works printed but not linked: 2
- [34]Living systematic reviews | Cochrane Community. https://community.cochrane.org/review-development/resources/living-systematic-reviews.
- [64]Yoon W LCD Benchmark: Long Clinical Document Benchmark on Mortality Prediction. 2024.03.26.24304920 Preprint at 10.1101/2024.03.26.24304920 (2024). PMC11756648 39602813 · DOI
Cited by
Sorted by citations
The STARD-AI reporting guideline for diagnostic accuracy studies using artificial intelligence
Sounderajah, V., Guni, A., Liu, X., Collins, G. S., Karthikesalingam, A., Markar, S. R., Golub, R. M., Denniston, A. K., Shetty, S., Moher, D., et al. (2025) · Nature Medicine · doi:10.1038/s41591-025-03953-8 · Cited by 352 · see citing works
Self-citationScientific evidence for clinical text summarization using large language models: scoping review
Bednarczyk, L., Reichenpfader, D., Gaudet-Blavignac, C., Ette, A. K., Zaghir, J., Zheng, Y., Bensahla, A., Bjelogrlic, M., & Lovis, C. (2025) · Journal of Medical Internet Research · doi:10.2196/68998 · Cited by 83 · see citing works
Berkhout, W. E. M., van Wijngaarden, J. J., Workum, J. D., van de Sande, D., Hilling, D. E., Jung, C., Meyfroidt, G., Gommers, D., Buijsman, S. N. R., & van Genderen, M. E. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.22866 · Cited by 75 · see citing works
Advancing healthcare AI governance through a comprehensive maturity model based on systematic review
Hussein, R., Zink, A., Ramadan, B., Howard, F. M., Hightower, M., Shah, S., & Beaulieu-Jones, B. K. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02418-7 · Cited by 42 · see citing works
ESMO guidance on the use of Large Language Models in Clinical Practice (ELCAP)
Wong, E. Y. T., Verlingue, L., Aldea, M., Franzoi, M. A., Umeton, R., Halabi, S., Harbeck, N., Indini, A., Prelaj, A., Romano, E., et al. (2025) · Annals of Oncology · doi:10.1016/j.annonc.2025.09.001 · Cited by 44 · see citing works
Self-citationWeisman, D., Sugarman, A., Huang, Y. M., Gelberg, L., Ganz, P. A., & Comulada, W. S. (2025) · JMIR Formative Research · doi:10.2196/65670 · Cited by 37 · see citing works
Cross-sectional studies: strengths, limitations, and methodological considerations
Baumeister, S., Kocher, T., Papapanou, P. N., Holtfreter, B., & Demmer, R. T. (2026) · Journal of Periodontal Research · doi:10.1111/jre.70063 · Cited by 38 · see citing works
Artificial Intelligence–Assisted Data Extraction With a Large Language Model: A Study Within Reviews
Gartlehner, G., Kugley, S., Crotty, K., Viswanathan, M., Dobrescu, A., Nussbaumer-Streit, B., Booth, G., Treadwell, J. R., Han, J. M., Wagner, J., et al. (2025) · Annals of Internal Medicine · doi:10.7326/annals-25-00739 · Cited by 36 · see citing works
Tripathi, S., Alkhulaifat, D., Doo, F. X., Rajpurkar, P., McBeth, R., Daye, D., & Cook, T. S. (2025) · NEJM AI · doi:10.1056/aip2401106 · Cited by 31 · see citing works
Bejan, C. A., Wang, M., Venkateswaran, S., Bergmann, E. A., Hiles, L., Xu, Y., Chandler, G. S., Brondfield, S., Silverstein, J., Wright, F., et al. (2026) · eBioMedicine · doi:10.1016/j.ebiom.2026.106227 · Cited by 16 · see citing works
Large language models in peer review: challenges and opportunities
Sun, Z. (2025) · Scientometrics · doi:10.1007/s11192-025-05440-w · Cited by 15 · see citing works
Harnessing AI in critical care: opportunities, challenges and key steps for success
Kalimouttou, A., Stevens, R. D., & Pirracchio, R. (2026) · Thorax · doi:10.1136/thorax-2024-222125 · Cited by 12 · see citing works
Loaiza-Bonilla, A., Yost, C., Kurnaz, S., Tuysuz, E., Thaker, N. G., Giritlioglu, D., & Noel Meza, J. P. (2026) · ESMO Real World Data and Digital Oncology · doi:10.1016/j.esmorw.2026.100706 · Cited by 8 · see citing works
“14”
Innovating global regulatory frameworks for generative AI in medical devices is an urgent priority
Ong, J. C. L., Ning, Y., Liu, M., Ma, Y., Zhao, L., Singh, K., Chang, R. T., Vogel, S., Lim, J. C. W., Tan, I. S. K., et al. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02552-2 · Cited by 7 · see citing works
Self-citationOne shot at trust: building credible evidence for medical artificial intelligence
Kouzy, R., Hong, J. C., & Bitterman, D. S. (2025) · The Lancet Digital Health · doi:10.1016/j.landig.2025.100883 · Cited by 4 · see citing works
Self-citationHabib, D. R. S., Mahajan, I., Evancha, B., Micheel, C., & Fabbri, D. (2025) · Journal of Medical Systems · doi:10.1007/s10916-025-02303-y · Cited by 3 · see citing works
Xie, E., Cheng, L., Shireman, J., Cai, Y., Liu, J., Mohanty, C., Dey, M., & Kendziorski, C. (2024) · doi:10.1101/2024.12.04.626476 · Cited by 3 · see citing works
MoMA: a mixture-of-multimodal-agents architecture for enhancing clinical prediction modelling
Gao, J., Rahman, M., Caskey, J., Oguss, M., O’Rourke, A., Brown, R., Stey, A., Mayampurath, A., Churpek, M. M., Chen, G., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-02219-4 · Cited by 2 · see citing works
“We adhere to the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis - Large Language Models (TRIPOD-LLM) guidelines 75 and completed the accompanying checklist, meeting all reporting requirements, as detailed in Supplementary Note 4.”
Self-citationMultimodal Artificial Intelligence for Precision Critical Care: A Scoping Review
Yang, M., Shi, N., Chen, H., Po, E. Y. P., Ng, P. Y., Lai, P. C. K., Wu, Y.-C., & Fong, D. Y. T. (2026) · Health Data Science · doi:10.34133/hds.0356 · Cited by 3 · see citing works
“For example, the TRIPOD + AI statement [ 125 ] provides reporting guidance for machine-learning-based diagnosis or prognosis prediction models, the TRIPOD-LLM [ 126 ] reporting guideline for using LLMs, and the CONSORT-AI [ 127 ] for AI-based clinical trials.”
Large Language Models in Pharmacist-Led Direct Patient Care: A Rapid Review of Published Evaluations
Yakob, R. A., Bamgboje-Ayodele, A., Collins, J. C., & Aslani, P. (2026) · Research in Social and Administrative Pharmacy · doi:10.1016/j.sapharm.2026.09.007
Rahgozar, A., & Mortezaagha, P. (2026) · arXiv · doi:10.48550/arxiv.2608.14551
Deobhakta, A., Chang, J., Chen, R. W. S., Jaipalli, S., Pinhas, A., Tania Tai, T. Y., & Rosen, R. B. (2026) · doi:10.64898/2026.10.09.26365153
- Authors
- Gallifant, J., Afshar, M., Ameen, S., Aphinyanaphongs, Y., Chen, S., Cacciamani, G., Demner-Fushman, D., Dligach, D., Daneshjou, R., Fernandes, C., Hansen, L. H., Landman, A., Lehmann, L., McCoy, L. G., Miller, T., Moreno, A., Munch, N., Restrepo, D., Savova, G., Umeton, R., Gichoya, J. W., Collins, G. S., Moons, K. G. M., Celi, L. A., & Bitterman, D. S.
- Affiliations
- St Thomas' Hospital (GB)
- University of Wisconsin–Madison (US)
- University of Tasmania (AU)
- NYU Langone Health (US)
- Brigham and Women's Hospital (US)
- University of Southern California (US)
- Loyola University Chicago (US)
- Stanford Medicine (US)
- Massachusetts Institute of Technology (US)
- Aarhus University (DK)
- Mass General Brigham (US)
- University of Alberta (CA)
- Boston Children's Hospital (US)
- The University of Texas MD Anderson Cancer Center (US)
- Universidad del Cauca (CO)
- Dana-Farber Cancer Institute (US)
- Emory University (US)
- Utrecht University (NL)
- Beth Israel Deaconess Medical Center (US)
- Corresponding author
- Bitterman, Danielle S.
- ORCID
- Gallifant, Jack
- Ameen, Saleem
- Aphinyanaphongs, Yindalon
- Chen, Shan
- Cacciamani, Giovanni
- Dligach, Dmitriy
- Daneshjou, Roxana
- Fernandes, Chrystinne
- Hansen, Lasse Hyldig
- Landman, Adam
- Lehmann, Lisa
- McCoy, Liam G.
- Miller, Timothy
- Moreno, Amy
- Restrepo, David
- Savova, Guergana
- Umeton, Renato
- Gichoya, Judy Wawira
- Collins, Gary S.
- Moons, Karel G. M.
- Celi, Leo A.
- Bitterman, Danielle S.
- Venue
- Nature Medicine
- Published
- 2025-01
- Volume
- 31
- Issue
- 1
- Pages
- 60-69
- ISSN
- 1078-8956 (ISSN-L)
- 1546-170X
- PMC
- PMC12104976
- OpenAlex
- W4406152263
- Semantic Scholar
- fe3a18a7f96842bb7b79d785f1e252f3c04f87b4
- Cited by
- 615
- References
- 57
- Citations by year
- 2026: 458 · 2025: 155 · 2024: 2
- FWCI
- 36.27
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Nature Medicine (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- Digital Commons-TMC (Texas Medical Center) (submittedVersion)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion)
- Funders
- National Science Foundation (1928614, 2129076)
- Gordon and Betty Moore Foundation
- Radiological Society of North America (#EIHD2204)
- Cancer Research UK (C49297/A27294)
- National Institute for Health and Care Research (C49297/A27294)
- Department of Health and Social Care
- National Institutes of Health (OT2OD032701, 75N92020C00021, 5r01eb017205-07, 75N92020C00008, R01-EB017205, UL1TR001445)
- Engineering and Physical Sciences Research Council (EP/Y018516/1)
- National Heart, Lung, and Blood Institute (75N92020C00008, 75N92020C00021, R01HL167811)
- National Cancer Institute (U54CA274516, R01CA294033)
- Fogarty International Center (U54TW012043)
- National Institute of Biomedical Imaging and Bioengineering (R01-EB017205)
- U.S. National Library of Medicine (R01LM013486)
- National Center for Advancing Translational Sciences
- National Institute on Minority Health and Health Disparities
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- large language models, reporting guidelines, diagnostic model, artificial intelligence in healthcare, transparency, human oversight, reproducibility, living guidelines, Delphi consensus
- MeSH terms
- Large Language Models, Artificial Intelligence, Delphi Technique, Humans, Prognosis, Research Design, Guidelines as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
