Toward Clinical-Grade Evaluation of Large Language Models
Open accessThe claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Automated metrics fail to capture semantic nuances, contextual dependencies, and domain-specific knowledge crucial for accurate medical discourse.
“These metrics, primarily focused on surface-level textual similarity, fail to capture the semantic nuances, contextual dependencies, and domain-specific knowledge crucial for accurate medical discourse30.”
From A framework to assess clinical safety and hallucination rates of LLMs for medical text summarisation (2025) · cites this paper as [30]
The passage on automated evaluation (section labeled Abstract; page unavailable) says similarity metrics may not accurately capture content differences or reflect human preferences and should not be used alone for clinical applications. It does not establish that these metrics categorically fail, nor does it specifically identify contextual dependencies or domain-specific knowledge as what they miss. (LLM, conf=0.84)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
(2004) · Cited by 9,718 · see citing works
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505 · see citing works
(2005) · Cited by 4,152 · see citing works
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083 · see citing works
Bleu: a method for automatic evaluation of machine translation
Papineni, K. (2024) · doi:10.57702/ljteqmrb · Cited by 3,682 · see citing works
Kincaid, J. P., Fishburne, Jr., Robert P., R., Richard L., C., & Brad, S. (1975) · doi:10.21236/ada006655 · Cited by 3,497 · see citing works
Collins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0697 · Cited by 3,267 · see citing works
BERTScore: Evaluating Text Generation with BERT
Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q., & Artzi, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.09675 · Cited by 2,865 · see citing works
Ayers, J. W., Poliak, A., Dredze, M., Leas, E. C., Zhu, Z., Kelley, J. B., Faix, D. J., Goodman, A. M., Longhurst, C. A., Hogarth, M., et al. (2023) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2023.1838 · Cited by 2,613 · see citing works
Large language models are zero-shot reasoners
Kojima, T., Gu, S. (., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1613 · Cited by 2,013 · see citing works
Toward expert-level medical question answering with large language models
Singhal, K., Tu, T., Gottweis, J., Sayres, R., Wulczyn, E., Amin, M., Hou, L., Clark, K., Pfohl, S. R., Cole-Lewis, H., et al. (2025) · Nature Medicine · doi:10.1038/s41591-024-03423-7 · Cited by 1,734 · see citing works
Proceedings of the 40th Annual Meeting on Association for Computational Linguistics
Isabelle, P. (2002) · Cited by 1,633 · see citing works
Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints
Zhao, J., Wang, T., Yatskar, M., Ordonez, V., & Chang, K.-W. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1323 · Cited by 829 · see citing works
Murray, G., Renals, S. J., Carletta, J., & Moore, J. D. (2005) · Meeting of the Association for Computational Linguistics · Cited by 716 · see citing works
Capabilities of GPT-4 on Medical Challenge Problems
Nori, H., King, N., McKinney, S. M., Carignan, D., & Horvitz, E. (2023) · arXiv · doi:10.48550/arxiv.2303.13375 · Cited by 509 · see citing works
Accuracy and Reliability of Chatbot Responses to Physician Questions
Goodman, R. S., Patrinely, J. R., Stone, C. A., Zimmerman, E., Donald, R. R., Chang, S. S., Berkowitz, S. T., Finn, A. P., Jahangir, E., Scoville, E. A., et al. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.36483 · Cited by 484 · see citing works
Rao, A. S., Kim, J., Kamineni, M., Pang, M., Lie, W., Dreyer, K. J., & Succi, M. D. (2023) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2023.05.003 · Cited by 481 · see citing works
Rao, A., Pang, M., Kim, J., Kamineni, M., Lie, W., Prasad, A. K., Landman, A., Dreyer, K., & Succi, M. D. (2023) · Journal of Medical Internet Research · doi:10.2196/48659 · Cited by 412 · see citing works
Towards Expert-Level Medical Question Answering with Large Language Models
Singhal, K., Tu, T., Gottweis, J., Sayres, R., Wulczyn, E., Hou, L., Clark, K., Pfohl, S., Cole-Lewis, H., Neal, D., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.09617 · Cited by 338 · see citing works
Can Large Language Models Be an Alternative to Human Evaluations?
Chiang, C.-H., & Lee, H.-Y. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.870 · Cited by 331 · see citing works
Johnson, S. B., King, A. J., Warner, E. L., Aneja, S., Kann, B. H., & Bylund, C. L. (2023) · JNCI Cancer Spectrum · doi:10.1093/jncics/pkad015 · Cited by 289 · see citing works
Temporal Annotation in the Clinical Domain
Styler, W., Bethard, S., Finan, S., Palmer, M., Pradhan, S., de Groen, P. D., Erickson, B., Miller, T., Lin, C., Savova, G., et al. (2014) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00172 · Cited by 285 · see citing works
Assessment of Artificial Intelligence Chatbot Responses to Top Searched Queries About Cancer
Pan, A., Musheyev, D., Bockelman, D., Loeb, S., & Kabarriti, A. E. (2023) · JAMA Oncology · doi:10.1001/jamaoncol.2023.2947 · Cited by 278 · see citing works
Large language models to identify social determinants of health in electronic health records
Guevara, M., Chen, S., Thomas, S., Chaunzwa, T. L., Franco, I., Kann, B. H., Moningi, S., Qian, J. M., Goldstein, M., Harper, S., et al. (2024) · npj Digital Medicine · doi:10.1038/s41746-023-00970-0 · Cited by 278 · see citing works
Self-citationEvaluating ChatGPT as an Adjunct for Radiologic Decision-Making
Rao, A., Kim, J., Kamineni, M., Pang, M., Lie, W., & Succi, M. D. (2023) · doi:10.1101/2023.02.02.23285399 · Cited by 257 · see citing works
Use of Artificial Intelligence Chatbots for Cancer Treatment Information
Chen, S., Kann, B. H., Foote, M. B., Aerts, H. J. W. L., Savova, G. K., Mak, R. H., & Bitterman, D. S. (2023) · JAMA Oncology · doi:10.1001/jamaoncol.2023.2954 · Cited by 251 · see citing works
Self-citationHandbook of Linguistic Annotation
Biemann, C., Bontcheva, K., de Castilho, R. E., Gurevych, I., & Yimam, S. M. (2017) · Springer eBooks · doi:10.1007/978-94-024-0881-2 · Cited by 192 · see citing works
Human evaluation of automatically generated text: Current trends and best practice guidelines
van der Lee, C., Albert Gatt, A., van Miltenburg, E., & Krahmer, E. (2020) · Computer Speech & Language · doi:10.1016/j.csl.2020.101151 · Cited by 178 · see citing works
A Critical Evaluation of Evaluations for Long-form Question AnsweringPreprint
Xu, F., Song, Y., Iyyer, M., & Choi, E. (2023) · Annual Meeting of the Association for Computational Linguistics · doi:10.48550/arxiv.2305.18201 · Cited by 160 · see citing works
Feng, S., Park, C. Y., Liu, Y., & Tsvetkov, Y. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.656 · Cited by 149 · see citing works
Readability formulas have even more limitations than Klare discusses
Redish, J. C. (2000) · ACM Journal of Computer Documentation · doi:10.1145/344599.344637 · Cited by 142 · see citing works
Sharma, M., Tong, M., Korbak, T., Duperrouzel, D., Was, H., Korbak, J., Cheng, M., Askell, A., Bai, Y., Brain, C., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.13548 · Cited by 132 · see citing works
Huang, Y., Gomaa, A., Semrau, S., Haderlein, M., Lettmaier, S., Weißmann, T., Grigo, J., Tkhayat, H. B., Frey, B., Gaipl, U. S., et al. (2023) · Frontiers in Oncology · doi:10.3389/fonc.2023.1265024 · Cited by 116 · see citing works
Hurdles to Progress in Long-form Question Answering
Krishna, K., Roy, A., & Iyyer, M. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.393 · Cited by 114 · see citing works
It’s All in the Name: Mitigating Gender Bias with Name-Based Counterfactual Data Substitution
Maudslay, R. H., Gonen, H., Cotterell, R., & Teufel, S. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1530 · Cited by 107 · see citing works
Large Language Models Help Humans Verify Truthfulness – Except When They Are Convincingly WrongPreprint
Si, C., Goyal, N., Wu, T.-S., Zhao, C., Feng, S., Daumé, H., & Boyd-Graber, J. L. (2023) · North American Chapter of the Association for Computational Linguistics · doi:10.48550/arxiv.2310.12558 · Cited by 96 · see citing works
Evaluation of ChatGPT Family of Models for Biomedical Reasoning and Classification
Chen, S., Li, Y., Lu, S., Van, H., Aerts, H. J., Savova, G. K., & Bitterman, D. S. (2023) · arXiv · doi:10.48550/arxiv.2304.02496 · Cited by 91 · see citing works
Self-citationEvaluating the ChatGPT family of models for biomedical reasoning and classification
Chen, S., Li, Y., Lu, S., Van, H., Aerts, H. J. W. L., Savova, G. K., & Bitterman, D. S. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad256 · Cited by 91 · see citing works
Self-citationBuilding gold standard corpora for medical natural language processing tasks.
Deléger, L., Li, Q., Lingren, T., Kaiser, M., Molnár, K., Stoutenborough, L., Kouril, M., Marsolo, K. A., & Solti, I. (2012) · PubMed Central · Cited by 79 · see citing works
Kersloot, M. G., van Putten, F. J. P., Abu–Hanna, A., Ronald Cornet, A., & Arts, D. L. (2020) · Journal of Biomedical Semantics · doi:10.1186/s13326-020-00231-z · Cited by 77 · see citing works
Czarnowska, P., Vyas, Y., & Shah, K. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00425 · Cited by 72 · see citing works
Clinical Natural Language Processing for Radiation Oncology: A Review and Practical Primer
Bitterman, D. S., Miller, T. A., Mak, R. H., & Savova, G. (2021) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/j.ijrobp.2021.01.044 · Cited by 68 · see citing works
Self-citationA Trip Towards Fairness: Bias and De-Biasing in Large Language Models
Ranaldi, L., Ruzzetti, E. S., Venditti, D., Onorati, D., & Zanzotto, F. M. (2023) · STARSEM · doi:10.18653/v1/2024.starsem-1.30 · Cited by 47 · see citing works
A Critical Evaluation of Evaluations for Long-form Question Answering
Xu, F., Song, Y., Iyyer, M., & Choi, E. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.181 · Cited by 33 · see citing works
Chapman, W. W., Dowling, J., & Hripcsak, G. M. (2007) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2007.01.002 · Cited by 28 · see citing works
An AI-Enhanced Electronic Health Record Could Boost Primary Care Productivity
Harris, J. E. (2023) · JAMA · doi:10.1001/jama.2023.14525 · Cited by 19 · see citing works
“Fifty Shades of Bias”: Normative Ratings of Gender Bias in GPT Generated English Text
Hada, R., Seth, A., Diddee, H., & Bali, K. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.115 · Cited by 12 · see citing works
Large Language Models to Identify Social Determinants of Health in Electronic Health RecordsPreprint
Guevara, M., Chen, S., Thomas, S., Chaunzwa, T. L., Franco, I. “., Kann, B. H., Moningi, S., Qian, J. M., Goldstein, M. H., Harper, S., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2308.06354 · Cited by 7 · see citing works
Self-citationModeling Exemplification in Long-form Question Answering via Retrieval
Wang, S., Xu, F., Thompson, L., Choi, E., & Iyyer, M. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.151 · Cited by 7 · see citing works
Annotating the Clinical Text – MiPACQ, ShARe, SHARPn and THYME Corpora
Savova, G., Pradhan, S., Palmer, M. S., Styler, W., Chapman, W. W., & Elhadad, N. (2017) · Springer eBooks · doi:10.1007/978-94-024-0881-2_52 · Cited by 4 · see citing works
''Fifty Shades of Bias'': Normative Ratings of Gender Bias in GPT Generated English TextPreprint
Hada, R., Seth, A., Diddee, H., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.17428
Flagged first, then by citations
Research works printed but not linked: 2
- [34]34. Novikova J, Dušek O, Rieser V. RankME: Reliable human ratings for natural language generation. Available at: https://arxiv.org/abs/1803.05928. Accessed October 25, 2023. · arXiv
- [38]38. Nadeem M, Bethke A, Reddy S. StereoSet: Measuring stereotypical bias in pretrained language models. Available at: https://arxiv.org/abs/2004.09456. Accessed October 25, 2023. · arXiv
Cited by
Sorted by citations
The TRIPOD-LLM reporting guideline for studies using large language models
Gallifant, J., Afshar, M., Ameen, S., Aphinyanaphongs, Y., Chen, S., Cacciamani, G., Demner-Fushman, D., Dligach, D., Daneshjou, R., Fernandes, C., et al. (2025) · Nature Medicine · doi:10.1038/s41591-024-03425-5 · Cited by 615 · see citing works
“To safeguard LLM use and increase transparency, standardization in the development and reporting of LLMs is essential — to ensure consistency, reliability and verifiability, akin to established clinical-grade evaluation in other scientific domains 28–30 .”
Self-citationA framework to assess clinical safety and hallucination rates of LLMs for medical text summarisation
Asgari, E., Montaña-Brown, N., Dubois, M., Khalil, S., Balloch, J., Yeung, J. A., & Pimenta, D. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01670-7 · Cited by 423 · see citing works
“These metrics, primarily focused on surface-level textual similarity, fail to capture the semantic nuances, contextual dependencies, and domain-speci fi c knowledge crucial for accurate medical discourse 30 .”
Fleurence, R., Dawoud, D., Bian, J., Higashi, M. K., Wang, X., Xu, H., Chhatwal, J., & Ayer, T. (2025) · Value in Health · doi:10.1016/j.jval.2025.06.018 · Cited by 27 · see citing works
The TRIPOD-LLM Statement: A Targeted Guideline For Reporting Large Language Models UsePreprint
Gallifant, J., Afshar, M., Ameen, S., Aphinyanaphongs, Y., Chen, S., Cacciamani, G. E., Demner‐Fushman, D., Dligach, D., Daneshjou, R., Fernandes, C. O., et al. (2024) · medRxiv · doi:10.1101/2024.07.24.24310930 · Cited by 21 · see citing works
Self-citationComparative performance of large language models for patient-initiated ophthalmology consultations
Huang, M., Wang, X., Zhou, S., Cui, X., Zhang, Z., Xu, Y., Yang, W., & Chi, W. (2025) · Frontiers in Public Health · doi:10.3389/fpubh.2025.1673045 · Cited by 13 · see citing works
Large language models for clinical artificial intelligence in healthcare a systematic review
Ghnemat, R., & Saleh, A. (2026) · Discover Artificial Intelligence · doi:10.1007/s44163-025-00784-x · Cited by 12 · see citing works
“Furthermore, data privacy is of the essence and since there is sensitive patient information, compliance with regulations is a must if AI technologies are to be used for better health outcomes [124].”
Sarvari, P., & Al-Fagih, Z. (2025) · JMIRx Med · doi:10.2196/67661 · Cited by 7 · see citing works
“Moreno and Bitterman [26] also hinted at nonhuman evaluation as a method to allow for a larger-scale beta test, and Zack et al [27] actively employed this method to match generated diagnoses to ground truth diagnoses and shared the prompt as supplementary material.”
Herbert, D. N., Westendorf, J., Farmer, M., & Reeder, B. (2025) · Journal of Studies on Alcohol and Drugs · doi:10.15288/jsad.24-00319 · Cited by 6 · see citing works
Prompt design for medical question answering with Large Language Models
Kuligin, L., Lammert, J., Ostapenko, A., Bressem, K. K., Boeker, M., & Tschochohei, M. (2025) · Machine Learning with Applications · doi:10.1016/j.mlwa.2025.100758 · Cited by 7 · see citing works
Review of Large Language Models for Patient and Caregiver Support in Cancer Care Delivery
Kouzy, R., Cha, E. E., Rosen, A., & Bitterman, D. S. (2025) · JCO Clinical Cancer Informatics · doi:10.1200/cci-25-00044 · Cited by 8 · see citing works
Self-citationSarvari, P., Al-Fagih, Z., Abou-Chedid, A., Jewell, P., Taylor, R., & Imtiaz, A. (2025) · JMIR Biomedical Engineering · doi:10.2196/66691 · Cited by 5 · see citing works
“We must also be mindful of other harmful biases LLMs may learn during training [84], as well as the risks that over-reliance on AI systems may bring to medicine (eg, automation bias) [90].”
From benchmarks to deployment: a comprehensive review of agentic AI evaluation
Kehkashan, T., Abdullah, M. I., Al‐Shamayleh, A. S., Ivković, N., Ismail, N. A., Ahmad, S. S. S., Rehman, A., & Akhunzada, A. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-026-11571-0 · Cited by 10 · see citing works
Wei, S., Hu, A., Wang, Z., Meng, X., Yu, L., Yang, B., & Qiu, J. (2025) · Journal of Applied Clinical Medical Physics · doi:10.1002/acm2.70263 · Cited by 4 · see citing works
Current Perspectives on the Artificial Intelligence in Critical Care Medicine
Lee, J., & Yoon, J. H. (2025) · Anesthesiology Clinics · doi:10.1016/j.anclin.2025.05.005 · Cited by 4 · see citing works
Improving Patient Engagement: Is There a Role for Large Language Models?
Kouzy, R., & Bitterman, D. S. (2024) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/j.ijrobp.2024.06.001 · Cited by 3 · see citing works
Self-citationServicing open-source large language models for oncology
Ray, P. P. (2024) · The Oncologist · doi:10.1093/oncolo/oyae264 · Cited by 2 · see citing works
Beyond the Algorithm: A Field Guide to Deploying AI Agents in Clinical PracticePreprint
Gallifant, J., Kellogg, K. C., Butler, M., Centi, A., Chen, S., Doyle, P., Dutta, S., Guo, J., Hadfield, M., Kim, E., et al. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5732262 · Cited by 2 · see citing works
Self-citationMedFrenchmark, a Small Set for Benchmarking Generative LLMs in Medical French
Quercia, A., Zaghir, J., Lovis, C., & Gaudet-Blavignac, C. (2024) · Studies in health technology and informatics · doi:10.3233/shti240486 · Cited by 1 · see citing works
Alfonzetti, T., & Xia, J. (2025) · Practical Radiation Oncology · doi:10.1016/j.prro.2025.02.006 · Cited by 1 · see citing works
Zhang, S., Yılmaz, A., Gençoğlan, G., & Temelkuran, B. (2026) · Diagnostics · doi:10.3390/diagnostics16040603 · Cited by 1 · see citing works
“Such a multifaceted evaluation is crucial for LLMs in the medical domain, where outputs must not only be lexically accurate and complete, but also semantically aligned with clinical standards [32,33].”
Assessing Llm Hallucinations And The Reliability Of Using LLms For Automated Hallucination DetectionPreprint
Cheong, J., Chen, D. S., Leung, W., Chou, C., Corcoran, C. M., Kelly, S., Bearden, C. E., Cecchi, G., Baker, J. T., Kane, J. M., et al. (2026) · Research Square · doi:10.21203/rs.3.rs-9394250/v1
Kaftan, A. N. (2026) · Accreditation and Quality Assurance · doi:10.1007/s00769-026-01707-8
Kaftan, A. N. (2026) · Scandinavian Journal of Clinical and Laboratory Investigation · doi:10.1080/00365513.2026.2684025
“This framework enables laboratories to move from anecdotal impressions toward auditable, evidence-based assessment of AI outputs [17].”
Development of a benchmarking dataset for symptom detection using large language models
Davis, J., Durieux, B. N., Dongen, C. V., Sciacca, K. R., Gutierrez, J. M., & Lindvall, C. J. (2026) · JAMIA Open · doi:10.1093/jamiaopen/ooag134
Cangül, S., Adıgüzel, Ö., Tunç, T., Taşyürek, M., & Ortaç, H. (2026) · Acta Medica Nicomedia · doi:10.53446/actamednicomedia.1949463
Performance of DeepSeek V3 and ChatGPT-4o in answering esophageal cancer-related questions
Yang, Q., Yi, J., Gong, A., Li, Y., Feng, Q., Fu, A., Li, J., & Zhan, Y. (2026) · Medicine · doi:10.1097/md.0000000000049896
“[25,26]”
Zhong, Y., El-Diraby, T. E., & Goodfellow, S. D. (2026) · Journal of Computing in Civil Engineering · doi:10.1061/jccee5.cpeng-7192
Cornelisse, E., Kousa, D., Menz, B. D., Murphy-Reuter, B., Shahnam, A., Chin-Yee, B. H., Calabro, R., Li, L. X., Ramsey, I., Abuhelwa, A. Y., et al. (2026) · Journal of Medical Internet Research · doi:10.2196/100531 · Cited by 1 · see citing works
- Authors
- Moreno, A. C., & Bitterman, D. S.
- Affiliations
- The University of Texas MD Anderson Cancer Center (US)
- Brigham and Women's Hospital (US)
- Corresponding author
- Bitterman, Danielle S.
- Venue
- International Journal of Radiation Oncology*Biology*Physics
- Publisher
- Elsevier BV
- Published
- 2024-03
- Volume
- 118
- Issue
- 4
- Pages
- 916-920
- ISSN
- 0360-3016 (ISSN-L)
- PMC
- PMC11221761
- OpenAlex
- W4392077089
- Cited by
- 28
- References
- 48
- Citations by year
- 2026: 11 · 2025: 13 · 2024: 4
- FWCI
- 2.62
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- International Journal of Radiation Oncology*Biology*Physics (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Digital Commons-TMC (Texas Medical Center) (submittedVersion)
- Funders
- National Cancer Institute (U54CA274516)
- National Institute of Dental and Craniofacial Research (R21DE031082, K01DE030524)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- large language models, clinical evaluation, clinical decision support, ChatGPT, health care applications, biomedical knowledge, burnout, documentation burden
- MeSH terms
- Brain Neoplasms, Humans, Language
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
