CoChat
Sign Up
PreprintCited by 4,083Artificial Intelligence in Healthcare and EducationTopic ModelingMachine Learning in Healthcare

Large Language Models Encode Clinical Knowledge

Open access
Singhal, Karan, Azizi, Shekoofeh, Tu, Tao, Mahdavi, S. Sara, Wei, Jason, Chung, Hyung Won, Scales, Nathan, Tanwani, Ajay +22 more·2022·arXiv
DOI 10.48550/arxiv.2212.13138arXiv 2212.13138PMC PMC10396962
Open this paper in CoChat — free accountOpen PDF

Abstract

Large language models (LLMs) have demonstrated impressive capabilities in natural language understanding and generation, but the quality bar for medical and clinical applications is high. Today, attempts to assess models' clinical knowledge typically rely on automated evaluations on limited benchmarks. There is no standard to evaluate model predictions and reasoning across a breadth of tasks. To address this, we present MultiMedQA, a benchmark combining six existing open question answering datasets spanning professional medical exams, research, and consumer queries; and HealthSearchQA, a new free-response dataset of medical questions searched online. We propose a framework for human evaluation of model answers along multiple axes including factuality, precision, possible harm, and bias. In addition, we evaluate PaLM (a 540-billion parameter LLM) and its instruction-tuned variant, Flan-PaLM, on MultiMedQA. Using a combination of prompting strategies, Flan-PaLM achieves state-of-the-art accuracy on every MultiMedQA multiple-choice dataset (MedQA, MedMCQA, PubMedQA, MMLU clinical topics), including 67.6% accuracy on MedQA (US Medical License Exam questions), surpassing prior state-of-the-art by over 17%. However, human evaluation reveals key gaps in Flan-PaLM responses. To resolve this we introduce instruction prompt tuning, a parameter-efficient approach for aligning LLMs to new domains using a few exemplars. The resulting model, Med-PaLM, performs encouragingly, but remains inferior to clinicians. We show that comprehension, recall of knowledge, and medical reasoning improve with model scale and instruction prompt tuning, suggesting the potential utility of LLMs in medicine. Our human evaluations reveal important limitations of today's models, reinforcing the importance of both evaluation frameworks and method development in creating safe, helpful LLM models for clinical applications.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2212.13138

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • In the medical domain, specialized terminology requires precise handling, as minor lexical deviations can lead to substantially divergent interpretations.

    “This gap becomes particularly consequential in the medical domains, where specialized terminology requires precise handling, as minor lexical deviations can lead to substantially divergent interpretations (Singhal et al., 2022).”

    From MedHallu: A Comprehensive Benchmark for Detecting Medical Hallucinations in Large Language Models (2025) · cites this paper as Singhal et al., 2022

    The Abstract and Scientific consensus section support the need for accurate medical language: the paper describes medical AI as safety-critical and evaluates incorrect or missing content for its clinical importance. It does not establish that minor lexical deviations in specialized terminology lead to substantially divergent interpretations; searches for terminology and changes in wording found no such passage. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

  • General-purpose models such as Flan-PaLM or ChatGPT can accurately answer medical questions, achieving passing scores on the United States Medical Licensing Exam.

    “For example, by drawing on medical texts present within their training datasets, general-purpose models such as Flan-PaLM or ChatGPT can accurately answer medical questions, achieving passing scores on the United States Medical Licensing Exam 10,52,53 .”

    From Foundation models for generalist medical artificial intelligence (2023) · cites this paper as [10]

    The ‘State of the art on MedQA’ section reports that Flan-PaLM answered 67.6% of questions correctly on a USMLE-style benchmark. It does not establish a passing score on the actual USMLE or report a ChatGPT result. The Discussion treats medical content in the training corpus as a possible explanation, not a demonstrated cause. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • AI using EHR data has become much more general and accurate in the last few years, allowing prediction of perioperative events and learning effective treatment strategies.

    “AI using EHR data has become much more general and accurate in the last few years,35,36 allowing prediction of perioperative events37–41 and learning effective treatment strategies.42”

    From Potential uses of AI for perioperative nursing handoffs: a qualitative study (2023) · cites this paper as [36]

    The Abstract (page unavailable) describes recent advances that make medical language models usable across diverse tasks and reports improved accuracy on medical question-answering benchmarks. It does not establish that AI using EHR data has become more general or accurate. The later claims about perioperative prediction and treatment strategies are attached to other citations, not this source. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • In the medical domain, generation of incorrect information can exacerbate health disparities.

    “This issue is particularly problematic in high-stakes fields such as the medical domains, where the generation of incorrect information can exacerbate health disparities (Singhal et al., 2022).”

    From MedHallu: A Comprehensive Benchmark for Detecting Medical Hallucinations in Large Language Models (2025) · cites this paper as Singhal et al., 2022

  • MultiMedQA is a medical QA benchmark focusing on medical examinations, medical research, and consumer healthcare questions.

    “MultiMedQA [177] is a medical QA benchmark that focuses on medical examinations, medical research, and consumer healthcare questions.”

    From A Survey on Evaluation of Large Language Models (2024) · cites this paper as [177]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

No citation lists for this paper

No citation lists for this paper yet — checked again daily.

References 85 · Cited by 4,083

Research works printed but not linked: 74

  1. Esteva, A. et al. Deep learning-enabled medical computer vision. NPJ Digit. Med. 4, 5 (2021).
  2. Lakkaraju, H., Slack, D., Chen, Y., Tan, C. & Singh, S. Rethinking explainability as a dialogue: a practitioner's perspective. Preprint at https://doi.org/10.48550/arXiv.2202.01875 (2022). · DOI · Paper page
  3. Bommasani, R. et al. On the opportunities and risks of foundation models. Preprint at https://doi.org/10.48550/arXiv.2108.07258 (2021). · DOI · Paper page
  4. Jin, D. et al. What disease does this patient have? A large-scale open domain question answering dataset from medical exams. Appl. Sci. 11, 6421 (2021).
  5. Papineni, K., Roukos, S., Ward, T. & Zhu, W.-J. BLEU: a method for automatic evaluation of machine translation. In Proc. 40th Annual Meeting of the Association for Computational Linguistics 311-318 (Association of Computational Machinery, 2002).
  6. Pal, A., Umapathi, L. K. & Sankarasubbu, M. MedMCQA: a large-scale multi-subject multi- choice dataset for medical domain question answering. In Conference on Health, Inference, and Learning 248-260 (Proceedings of Machine Learning Research, 2022).
  7. Jin, Q., Dhingra, B., Liu, Z., Cohen, W. W. & Lu, X. PubMedQA: a dataset for biomedical research question answering. Preprint at https://doi.org/10.48550/arXiv.1909.06146 (2019). · DOI
  8. Abacha, A. B. et al. in Studies in Health Technology and Informatics (eds Ohno-Machado, L. & Séroussi, B.) 25-29 (IOS Press, 2019).
  9. Hendrycks, D. et al. Measuring massive multitask language understanding. Preprint at https://doi.org/10.48550/arXiv.2009.03300 (2020). · DOI · Paper page
  10. Chowdhery, A. et al. PaLM: scaling language modeling with pathways. Preprint at https:// doi.org/10.48550/arXiv.2204.02311 (2022). · DOI · Paper page
  11. Chung, H. W. et al. Scaling instruction-finetuned language models. Preprint at https://doi. org/10.48550/arXiv.2210.11416 (2022). · DOI · Paper page
  12. Brown, T. et al. Language models are few-shot learners. Adv. Neural Inf. Process. Syst. 33, 1877-1901 (2020).
  13. Wei, J. et al. Chain of thought prompting elicits reasoning in large language models. Preprint at https://doi.org/10.48550/arXiv.2201.11903 (2022). · DOI · Paper page
  14. Wang, X. et al. Self-consistency improves chain of thought reasoning in language models. Preprint at https://doi.org/10.48550/arXiv.2203.11171 (2022). · DOI · Paper page
  15. Yasunaga, M. et al. Deep bidirectional language-knowledge graph pretraining. Preprint at https://doi.org/10.48550/arXiv.2210.09338 (2022). · DOI
  16. Taylor, R. et al. Galactica: a large language model for science. Preprint at https://doi.org/ 10.48550/arXiv.2211.09085 (2022). · DOI · Paper page
  17. Lin, S., Hilton, J. & Evans, O. Teaching models to express their uncertainty in words. Preprint at https://doi.org/10.48550/arXiv.2205.14334 (2022). · DOI · Paper page
  18. Kadavath, S. et al. Language models (mostly) know what they know. Preprint at https:// doi.org/10.48550/arXiv.2207.05221 (2022). · DOI · Paper page
  19. Tran, D. et al. Plex: towards reliability using pretrained large model extensions. Preprint at https://doi.org/10.48550/arXiv.2207.07411 (2022). · DOI
  20. Feng, S. Y., Khetan, V., Sacaleanu, B., Gershman, A. & Hovy, E. CHARD: clinical health-aware reasoning across dimensions for text generation models. Preprint at https://doi.org/ 10.48550/arXiv.2210.04191 (2022). · DOI
  21. Walsh, K. E. et al. Measuring harm in healthcare: optimizing adverse event review. Med. Care 55, 436 (2017).
  22. Wei, J. et al. Emergent abilities of large language models. Preprint at https://doi.org/ 10.48550/arXiv.2206.07682 (2022). · DOI · Paper page
  23. Kington, R. S. et al. Identifying credible sources of health information in social media: principles and attributes. NAM Perspectives https://doi.org/10.31478%2F202107a (2021). · DOI
  24. Boateng, G. O., Neilands, T. B., Frongillo, E. A., Melgar-Quiñonez, H. R. & Young, S. L. Best practices for developing and validating scales for health, social, and behavioral research: a primer. Front. Public Health 6, 149 (2018).
  25. Hooker, S. Moving beyond "algorithmic bias is a data problem". Patterns 2, 100241 (2021).
  26. Weidinger, L. et al. Ethical and social risks of harm from language models. Preprint at https://doi.org/10.48550/arXiv.2112.04359 (2021). · DOI · Paper page
  27. Liang, P. et al. Holistic evaluation of language models. Preprint at https://doi.org/10.48550/ arXiv.2211.09110 (2022). · DOI · Paper page
  28. Raji, I. D. et al. Closing the AI accountability gap: defining an end-to-end framework for internal algorithmic auditing. In Proc. 2020 Conference on Fairness, Accountability, and Transparency 33-44 (Association for Computing Machinery, 2020).
  29. Rostamzadeh, N. et al. Healthsheet: development of a transparency artifact for health datasets. Preprint at https://doi.org/10.48550/arXiv.2202.13028 (2022). · DOI
  30. Gebru, T. et al. Datasheets for datasets. Commun. ACM 64, 86-92 (2021).
  31. Mitchell, M. et al. Model cards for model reporting. In Proc. conference on Fairness, Accountability, and Transparency 220-229 (Association for Computing Machinery, 2019). Nature | www.nature.com | 9
  32. Garg, S. et al. Counterfactual fairness in text classification through robustness. In Proc. 2019 AAAI/ACM Conference on AI, Ethics, and Society 219-226 (Association for Computing Machinery, 2019).
  33. Prabhakaran, V., Hutchinson, B. & Mitchell, M. Perturbation sensitivity analysis to detect unintended model biases. Preprint at https://doi.org/10.48550/arXiv.1910.04210 (2019). · DOI
  34. Zhang, H., Lu, A. X., Abdalla, M., McDermott, M. & Ghassemi, M. Hurtful words: quantifying biases in clinical contextual word embeddings. In Proc. ACM Conference on Health, Inference, and Learning 110-120 (Association for Computing Machinery, 2020).
  35. Pampari, A., Raghavan, P., Liang, J. & Peng, J. emrQA: a large corpus for question answering on electronic medical records. Preprint at https://doi.org/10.48550/arXiv.1809.00732 (2018). · DOI
  36. Tsatsaronis, G. et al. An overview of the bioasq large-scale biomedical semantic indexing and question answering competition. BMC Bioinformatics 16, 138 (2015).
  37. Morgado, F. F., Meireles, J. F., Neves, C., Amaral, A. & Ferreira, M. E. Scale development: ten main limitations and recommendations to improve future research practices. Psic. Reflex. Crit. 30, 5 (2017).
  38. Barham, P. et al. Pathways: asynchronous distributed dataflow for ML. Proc. Mach. Learn. Syst. 4, 430-449 (2022).
  39. Thoppilan, R. et al. Lamda: language models for dialog applications. Preprint at https:// doi.org/10.48550/arXiv.2201.08239 (2022). · DOI · Paper page
  40. Du, N. et al. Glam: efficient scaling of language models with mixture-of-experts. In International Conference on Machine Learning 5547-5569 (PMLR, 2022).
  41. Srivastava, A. et al. Beyond the imitation game: quantifying and extrapolating the capabilities of language models. Preprint at https://doi.org/10.48550/arXiv.2206.04615 (2022). · DOI · Paper page
  42. Clark, J. H. et al. Tydi qa: A benchmark for information-seeking question answering in typologically diverse languages. Trans. Assoc. Comput. Linguist. 8, 454-470 (2020).
  43. Lester, B., Al-Rfou, R. & Constant, N. The power of scale for parameter-efficient prompt tuning. Preprint at https://doi.org/10.48550/arXiv.2104.08691 (2021). · DOI · Paper page
  44. Nye, M. et al. Show your work: scratchpads for intermediate computation with language models. Preprint at https://doi.org/10.48550/arXiv.2112.00114 (2021). · DOI · Paper page
  45. Zhou, D. et al. Least-to-most prompting enables complex reasoning in large language models. Preprint at https://doi.org/10.48550/arXiv.2205.10625 (2022). · DOI · Paper page
  46. Cobbe, K. et al. Training verifiers to solve math word problems. Preprint at https://doi.org/ 10.48550/arXiv.2110.14168 (2021). · DOI · Paper page
  47. Lewkowycz, A. et al. Solving quantitative reasoning problems with language models. Preprint at https://doi.org/10.48550/arXiv.2206.14858 (2022). · DOI · Paper page
  48. Ficler, J. & Goldberg, Y. Controlling linguistic style aspects in neural language generation. Preprint at https://doi.org/10.48550/arXiv.1707.02633 (2017). · DOI
  49. Li, X. L. & Liang, P. Prefix-tuning: optimizing continuous prompts for generation. Preprint at https://doi.org/10.48550/arXiv.2101.00190 (2021). · DOI · Paper page
  50. Wei, J. et al. Finetuned language models are zero-shot learners. Preprint at https://doi.org/ 10.48550/arXiv.2109.01652 (2021). · DOI · Paper page
  51. Liu, P. et al. Pre-train, prompt, and predict: a systematic survey of prompting methods in natural language processing. Preprint at https://doi.org/10.48550/arXiv.2107.13586 (2021). · DOI · Paper page
  52. Liu, X. et al. GPT understands, too. Preprint at https://doi.org/10.48550/arXiv.2103.10385 (2021). · DOI · Paper page
  53. Gu, Y., Han, X., Liu, Z. & Huang, M. PPT: Pre-trained prompt tuning for few-shot learning. Preprint at https://doi.org/10.48550/arXiv.2109.04332 (2021). · DOI
  54. Ye, S., Jang, J., Kim, D., Jo, Y. & Seo, M. Retrieval of soft prompt enhances zero-shot task generalization. Preprint at https://doi.org/10.48550/arXiv.2210.03029 (2022). · DOI
  55. Hoffmann, J. et al. Training compute-optimal large language models. Preprint at https:// doi.org/10.48550/arXiv.2203.15556 (2022). · DOI · Paper page
  56. Scao, T. L. et al. BLOOM: a 176B-parameter open-access multilingual language model. Preprint at https://doi.org/10.48550/arXiv.2211.05100 (2022). · DOI · Paper page
  57. Rae, J. W. et al. Scaling language models: methods, analysis & insights from training Gopher. Preprint at https://doi.org/10.48550/arXiv.2112.11446 (2021). · DOI · Paper page
  58. Zhang, S. et al. OPT: open pre-trained transformer language models. Preprint at https:// doi.org/10.48550/arXiv.2205.01068 (2022). · DOI · Paper page
  59. Vaswani, A. et al. Attention is all you need. In 31st Conference on Neural Information Processing Systems (Association of Computational Machinery, 2017).
  60. Kaplan, J. et al. Scaling laws for neural language models. Preprint at https://doi.org/ 10.48550/arXiv.2001.08361 (2020). · DOI · Paper page
  61. Lampinen, A. K. et al. Can language models learn from explanations in context? Preprint at https://doi.org/10.48550/arXiv.2204.02329 (2022). · DOI
  62. Kojima, T., Gu, S. S., Reid, M., Matsuo, Y. & Iwasawa, Y. Large language models are zero- shot reasoners. Preprint at https://doi.org/10.48550/arXiv.2205.11916 (2022). · DOI · Paper page
  63. Joshi, M., Choi, E., Weld, D. S. & Zettlemoyer, L. TriviaQA: a large scale distantly supervised challenge dataset for reading comprehension. Preprint at https://doi.org/10.48550/ arXiv.1705.03551 (2017). · DOI · Paper page
  64. Beltagy, I., Lo, K. & Cohan, A. SciBERT: a pretrained language model for scientific text. Preprint at https://doi.org/10.48550/arXiv.1903.10676 (2019). · DOI · Paper page
  65. Lewis, P., Ott, M., Du, J. & Stoyanov, V. Pretrained language models for biomedical and clinical tasks: Understanding and extending the state-of-the-art. In Proc. 3rd Clinical Natural Language Processing Workshop (eds Roberts, K., Bethard, S. & Naumann, T.) 146-157 (Association for Computational Linguistics, 2020).
  66. Shin, H.-C. et al. BioMegatron: larger biomedical domain language model. Preprint at https://doi.org/10.48550/arXiv.2010.06060 (2020). · DOI
  67. Gu, Y. et al. Domain-specific language model pretraining for biomedical natural language processing. ACM Trans. Comput. Healthc. 3, 2 (2021).
  68. Papanikolaou, Y. & Pierleoni, A. DARE: data augmented relation extraction with GPT-2. Preprint at https://doi.org/10.48550/arXiv.2004.13845 (2020). · DOI
  69. Hong, Z. et al. The diminishing returns of masked language models to science. Preprint at https://doi.org/10.48550/arXiv.2205.11342 (2023). · DOI
  70. Korngiebel, D. M. & Mooney, S. D. Considering the possibilities and pitfalls of generative pre-trained transformer 3 (GPT-3) in healthcare delivery. NPJ Digit. Med. 4, 93 (2021).
  71. Sezgin, E., Sirrianni, J. & Linwood, S. L. et al. Operationalizing and implementing pretrained, large artificial intelligence linguistic models in the us health care system: outlook of generative pretrained transformer 3 (GPT-3) as a service model. JMIR Med. Informatics 10, e32875 (2022).
  72. Agrawal, M., Hegselmann, S., Lang, H., Kim, Y. & Sontag, D. Large language models are zero-shot clinical information extractors. Preprint at https://doi.org/10.48550/ arXiv.2205.12689 (2022). · DOI
  73. Liévin, V., Hother, C. E. & Winther, O. Can large language models reason about medical questions? Preprint at https://doi.org/10.48550/arXiv.2207.08143 (2022). · DOI · Paper page
  74. Ouyang, L. et al. Training language models to follow instructions with human feedback. Preprint at https://doi.org/10.48550/arXiv.2203.02155 (2022). · DOI · Paper page

Authors
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., Payne, P., Seneviratne, M., Gamble, P., Kelly, C., Scharli, N., Chowdhery, A., Mansfield, P., Arcas, B. A. Y., Webster, D., Corrado, G. S., Matias, Y., Chou, K., Gottweis, J., Tomasev, N., Liu, Y., Rajkomar, A., Barral, J., Semturs, C., Karthikesalingam, A., & Natarajan, V.
Affiliations
Google (United States) (US)
Corresponding author
  • Singhal, Karan
  • Azizi, Shekoofeh
  • Karthikesalingam, Alan
  • Natarajan, Vivek
ORCID
Venue
arXiv
Publisher
arXiv
Published
2022
Volume
620
Issue
7972
Pages
172-180
ISSN
0028-0836 (ISSN-L)
DOI
10.48550/arxiv.2212.13138
arXiv
2212.13138
PMC
PMC10396962
OpenAlex
W4384071683
Cited by
4,083
References
85
Citations by year
2026: 1627 · 2025: 1275 · 2024: 933 · 2023: 247 · 2022: 1
FWCI
162.53
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Funders
  • DeepMind
  • U.S. National Library of Medicine
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
Keywords
large language models, clinical knowledge, medical question answering, human evaluation, Arecaceae, MedQA, MedMCQA, PubMedQA, MMLU, clinical reasoning
MeSH terms
Clinical Competence, Computer Simulation, Humans, Licensure, Medicine, Natural Language Processing, Physicians, Bias, Knowledge, Benchmarking, Comprehension, Patient Safety, Datasets as Topic

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review