CoChat
Sign Up
Cited by 76Artificial Intelligence in Healthcare and EducationMachine Learning in HealthcareTopic Modeling

Prompt Engineering in Clinical Practice: Tutorial for Clinicians

Open access
Liu, Jialin, Liu, Fang, Wang, Changyu, Liu, Siru·2025·Journal of Medical Internet Research
DOI 10.2196/72644PMC PMC12439060
Open this paper in CoChat — free account

Abstract

Large language models (LLMs), such as OpenAI’s GPT series and Google’s PaLM, are transforming health care by improving clinical decision-making, enhancing patient communication, and simplifying administrative tasks. However, their performance relies heavily on prompt design, as small changes in wording or structure can greatly impact output quality. This presents challenges for clinicians who are not experts in natural language processing (NLP). This tutorial combines prompt engineering techniques tailored for clinical use, covering methods like zero-shot prompting, one-shot prompting, few-shot prompting, chain-of-thought prompting, self-consistency prompting, generated knowledge prompting, and meta-prompting. We provide actionable guidance on defining objectives, applying core principles, iterative prompt refinement, and integration into interoperable electronic health record (EHR) systems. This framework helps clinicians leverage LLMs to improve decision-making, streamline documentation, and enhance patient communication while maintaining ethical standards and ensuring patient safety.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.2196/72644

Cites

13 of 13 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Prompt framing and persona assignment can meaningfully shift LLM outputs and measured performance, including hallucination rate and factual accuracy.

    “In particular, prompt framing and persona assignment (e.g., instructing the model to respond as if to a clinician vs a patient) can meaningfully shift LLM outputs and measured performance, including hallucination rate and factual accuracy.”

    From Large Language Model Hallucinations in Spine Surgery: A Comparative Analysis of Clinician vs Patient-Level Prompts (2026) · cites this paper as [15]

    The Introduction (page unavailable) says small changes in prompt wording, structure, or tone can markedly change output relevance and accuracy, supporting the prompt-framing and factual-accuracy portion. The ‘Evaluate and Refine the Output’ section recommends checking for hallucinations, but our copy does not establish that assigning a clinician-versus-patient persona changes hallucination rates or other measured performance. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • How an LLM is prompted (prompt engineering) significantly affects the accuracy and ethical behavior of its output.

    “It is well-established that how an LLM is prompted (“prompt engineering”) significantly affects the accuracy and ethical behavior of its output.2,15,16”

    From Large Language Model Hallucinations in Spine Surgery: A Comparative Analysis of Clinician vs Patient-Level Prompts (2026) · cites this paper as [15]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 61

  1. [1]Singhal K Azizi S Tu T et al Large language models encode clinical knowledge Nature Aug 2023 620 7972 172 180 doi 10.1038/s41586-023-06291-2 Medline 37438534 PMC10396962 · DOI · Paper page
  2. [2]Liu J Wang C Liu S Utility of ChatGPT in clinical practice J Med Internet Res Jun 28 2023 25 e48568 doi 10.2196/48568 Medline 37379067 PMC10365580 · DOI · Paper page
  3. [3]Thirunavukarasu AJ Mahmood S Malem A et al Large language models approach expert-level clinical knowledge and reasoning in ophthalmology: A head-to-head cross-sectional study PLOS Digit Health Apr 2024 3 4 e0000341 doi 10.1371/journal.pdig.0000341 Medline 38630683 PMC11023493 · DOI
  4. [4]Webster P Six ways large language models are changing healthcare Nat Med Dec 2023 29 12 2969 2971 doi 10.1038/s41591-023-02700-1 Medline 38036704 · DOI
  5. [5]Liu S McCoy AB Wright AP et al Why do users override alerts? Utilizing large language model to summarize comments and optimize clinical decision support J Am Med Inform Assoc 05 20 2024 31 6 1388 1396 doi 10.1093/jamia/ocae041 38452289 PMC11105133 · DOI
  6. [6]Liu S McCoy AB Peterson JF et al Leveraging explainable artificial intelligence to optimize clinical decision support J Am Med Inform Assoc Apr 3 2024 31 4 968 974 doi 10.1093/jamia/ocae019 Medline 38383050 PMC10990514 · DOI
  7. [7]Liu S Wright AP Patterson BL et al Using AI-generated suggestions from ChatGPT to optimize clinical decision support J Am Med Inform Assoc Jun 20 2023 30 7 1237 1245 doi 10.1093/jamia/ocad072 Medline 37087108 PMC10280357 · DOI · Paper page
  8. [8]Bernstein IA Zhang YV Govil D et al Comparison of ophthalmologist and large language model chatbot responses to online patient eye care questions JAMA Netw Open Aug 1 2023 6 8 e2330320 doi 10.1001/jamanetworkopen.2023.30320 Medline 37606922 PMC10445188 · DOI
  9. [9]Liu S Wright AP Mccoy AB et al Using large language model to guide patients to create efficient and comprehensive clinical care message J Am Med Inform Assoc Aug 1 2024 31 8 1665 1670 doi 10.1093/jamia/ocae142 Medline 38917441 PMC11258400 · DOI
  10. [10]Liu S McCoy AB Wright AP et al Leveraging large language models for generating responses to patient messages-a subjective analysis J Am Med Inform Assoc 05 20 2024 31 6 1367 1379 doi 10.1093/jamia/ocae052 Medline 38497958 PMC11105129 · DOI
  11. [11]Patil R Heston TF Bhuse V Prompt engineering in healthcare Electronics (Basel) 2024 13 15 2961 doi 10.3390/electronics13152961 · DOI
  12. [14]White J Fu Q Hays S Sandborn M Olea C Gilbert H et al A prompt pattern catalog to enhance prompt engineering with chatgpt Presented at Proceedings of the 30th Conference on Pattern Languages of Programs USA: The Hillside Group Oct 22-25, 2023 1 31 Allerton Park in Monticello, Illinois, USA
  13. [16]Zhou S Xu Z Zhang M et al Large language models for disease diagnosis: a scoping review NPJ Artif Intell 2025 1 1 9 doi 10.1038/s44387-025-00011-z Medline 40607112 PMC12216946 · DOI
  14. [17]Wang C Liu S Li A Liu J Text dialogue analysis for primary screening of mild cognitive impairment: development and validation study J Med Internet Res Dec 29 2023 25 e51501 doi 10.2196/51501 Medline 38157230 PMC10787336 · DOI
  15. [18]Velásquez-Henao JD Franco-Cardona CJ Cadavid-Higuita L Prompt engineering: a methodology for optimizing interactions with AI-language models in the field of engineering DYNA 2023 90 230 9 17 doi 10.15446/dyna.v90n230.111700 · DOI
  16. [19]Jha S Jha SK Lincoln P Bastian ND Velasquez A Neema S Dehallucinating large language models using formal methods guided iterative prompting Presented at 2023 IEEE International Conference on Assured Autonomy (ICAA) Jun 6-8, 2023 Laurel, MD, USA doi 10.1109/ICAA58325.2023.00029 · DOI
  17. [20]Busch K Rochlitzer A Sola D Leopold H Just tell me: prompt engineering in business process management van der Aa H Bork D Proper HA Schmidt R Enterprise, Business-Process and Information Systems Modeling Springer Nature 2023 3 11 doi 10.1007/978-3-031-34241-7_1 · DOI
  18. [21]Kumar K Geotechnical parrot tales (GPT): harnessing large language models in geotechnical engineering J Geotech Geoenviron Eng 01 2024 150 1 02523001 doi 10.1061/JGGEFK.GTENG-11828 · DOI
  19. [22]Hill P Narine L Miller A Prompt engineering principles for generative AI use in extension JOE 2024 62 3 1 7 URL https://open.clemson.edu/joe/vol62/iss3 doi 10.34068/joe.62.03.20 · DOI
  20. [24]Kumar P Large language models (LLMs): survey, technical frameworks, and future challenges Artif Intell Rev 2024 57 10 doi 10.1007/s10462-024-10888-y · DOI · Paper page
  21. [25]Sivarajkumar S Kelley M Samolyk-Mazzanti A Visweswaran S Wang Y An empirical evaluation of prompting strategies for large language models in zero-shot clinical natural language processing: algorithm development and validation study JMIR Med Inform Apr 8 2024 12 e55318 doi 10.2196/55318 Medline 38587879 PMC11036183 · DOI
  22. [26]Brown T Mann B Ryder N Subbiah M Kaplan JD Dhariwal P et al Language models are few-shot learners Presented at NIPS’20: Proceedings of the 34th International Conference on Neural Information Processing Systems Dec 6-12, 2020 1877 1901 Vancouver, British Columbia, Canada
  23. [27]Betz G Richardson K Voigt C Thinking aloud: dynamic context generation improves zero-shot reasoning performance of GPT-2 ArXiv Preprint posted online on 2021 doi 10.48550/arXiv.2103.13033 · DOI
  24. [28]Sivarajkumar S Wang Y HealthPrompt: a zero-shot learning paradigm for clinical natural language processing AMIA Annu Symp Proc 2022 2022 972-981 972 981 Medline 37128372 PMC10148337
  25. [29]Kojima T Gu SS Reid M Matsuo Y Iwasawa Y Large language models are zero-shot reasoners Presented at Proceedings of the 36th International Conference on Neural Information Processing Systems Red Hook Nov 2022 to Dec 2022 22199 22213 New Orleans, Louisiana, USA
  26. [30]Leiser F Guse R Sunyaev A Large language model architectures in health care: scoping review of research perspectives J Med Internet Res Jun 19 2025 27 1 e70315 doi 10.2196/70315 Medline 40536801 PMC12226782 · DOI
  27. [31]Agrawal M Hegselmann S Lang H Kim Y Sontag D Large language models are few-shot clinical information extractors Presented at Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing Dec 2022 Association for Computational Linguistics 1998 2022 Abu Dhabi, United Arab Emirates doi 10.18653/v1/2022.emnlp-main.130 · DOI · Paper page
  28. [32]Huotala A Kuutila M Ralph P Mäntylä M The promise and challenges of using llms to accelerate the screening process of systematic reviews Presented at Proceedings of the 28th International Conference on Evaluation and Assessment in Software Engineering Jun 18, 2024 262 271 Salerno Italy doi 10.1145/3661167.3661172 · DOI
  29. [33]Shyr C Hu Y Bastarache L et al Identifying and extracting rare diseases and their phenotypes with large language models J Healthc Inform Res Jun 2024 8 2 438 461 doi 10.1007/s41666-023-00155-0 Medline 38681753 PMC11052982 · DOI
  30. [34]Zaghir J Naguib M Bjelogrlic M Névéol A Tannier X Lovis C Prompt engineering paradigms for medical applications: scoping review J Med Internet Res Sep 10 2024 26 e60501 doi 10.2196/60501 Medline 39255030 PMC11422740 · DOI · Paper page
  31. [37]Wei J Tay Y Bommasani R Raffel C Zoph B Borgeaud S et al Emergent abilities of large language models Transactions on Machine Learning Research 2022 URL https://openreview.net/forum?id=yzkSU5zdwD&utm_source=substack&utm_medium=email Accessed 08-09-2025
  32. [39]Wei J Wang X Schuurmans D Bosma M Ichter B et al Chain-of-thought prompting elicits reasoning in large language models Presented at Proceedings of the 36th International Conference on Neural Information Processing Systems Red 2024 24824 24837 NY, USA
  33. [40]Yu Z He L Wu Z Dai X Chen J Towards better chain-of-thought prompting strategies: a survey arXiv Preprint posted online on 2023 doi 10.48550/arXiv.2310.04959 · DOI
  34. [41]Wang X Wei J Schuurmans D Le Q Chi E Narang S et al Self-consistency improves chain of thought reasoning in language models arXiv Preprint posted online on 2023 doi 10.48550/arXiv.2203.11171 · DOI · Paper page
  35. [42]Kao JP Kao HT Large Language Models in radiology: A technical and clinical perspective Eur J Radiol Artif Intell Jun 2025 2 100021 doi 10.1016/j.ejrai.2025.100021 · DOI
  36. [43]B T B Chen JM Performance assessment of ChatGPT versus Bard in detecting Alzheimer’s dementia Diagnostics (Basel) Apr 15 2024 14 8 817 doi 10.3390/diagnostics14080817 Medline 38667463 PMC11048951 · DOI
  37. [44]Lee JH Choi E McDougal R Lytton WW GPT-4 performance for neurologic localization Neur Clin Pract Jun 2024 14 3 doi 10.1212/CPJ.0000000000200293 PMC11003355 38596779 · DOI
  38. [45]Jang D Yun TR Lee CY Kwon YK Kim CE GPT-4 can pass the Korean National Licensing Examination for Korean Medicine Doctors PLOS Digit Health Dec 2023 2 12 e0000416 doi 10.1371/journal.pdig.0000416 Medline 38100393 PMC10723673 · DOI
  39. [46]Chang CH Lucas MM Lee Y Yang CC Lu-Yao G Beyond self-consistency: ensemble reasoning boosts consistency and accuracy of llms in cancer staging Finkelstein J Moskovitch R Parimbelli E Artif Intell Med Springer Nature Switzerland 2024 224 228 doi 10.1007/978-3-031-66538-7_23 · DOI
  40. [47]Wang L Bi W Zhao S et al Investigating the impact of prompt engineering on the performance of large language models for standardizing obstetric diagnosis text: comparative study JMIR Form Res Feb 8 2024 8 e53216 doi 10.2196/53216 Medline 38329787 PMC10884897 · DOI
  41. [48]Liu J Liu A Lu X Welleck S West P Bras RL et al Generated knowledge prompting for commonsense reasoning arXiv Preprint posted online on 2022 doi 10.48550/arXiv.2110.08387 · DOI
  42. [50]Liu Z Patwary M Prenger R et al Multi-stage prompting for knowledgeable dialogue generation In Muresan S Nakov P Villavicencio A Presented at Findings of the Association for Computational Linguistics May 22-27, 2022 1317 1337 Dublin, Ireland doi 10.18653/v1/2022.findings-acl.104 · DOI
  43. [51]Bai J Yan Z Zhang S Yang J Guo H Li Z Infusing internalized knowledge of language models into hybrid prompts for knowledgeable dialogue generation Knowl Based Syst 07 2024 296 111874 doi 10.1016/j.knosys.2024.111874 · DOI
  44. [52]Lu Y Liu X Du Z Gao Y Wang G MedKPL: A heterogeneous knowledge enhanced prompt learning framework for transferable diagnosis J Biomed Inform 07 2023 143 104417 doi 10.1016/j.jbi.2023.104417 Medline 37315832 · DOI
  45. [53]Long C Subburam D Lowe K et al ChatENT: augmented large language model for expert knowledge retrieval in otolaryngology-head and neck surgery Otolaryngol Head Neck Surg Oct 2024 171 4 1042 1051 doi 10.1002/ohn.864 Medline 38895862 · DOI
  46. [54]Afshar M Gao Y Gupta D Croxford E Demner-Fushman D On the role of the UMLS in supporting diagnosis generation proposed by Large Language Models J Biomed Inform Sep 2024 157 104707 104707 doi 10.1016/j.jbi.2024.104707 Medline 39142598 PMC11402555 · DOI
  47. [55]Zhang Y Yuan Y Yao AC Meta prompting for AI systems arXiv Preprint posted online on 2024 doi 10.48550/arXiv.2311.11482 · DOI
  48. [56]Tripathi S Sukumaran R Cook TS Efficient healthcare with large language models: optimizing clinical workflow and enhancing patient care J Am Med Inform Assoc 05 20 2024 31 6 1436 1440 doi 10.1093/jamia/ocad258 Medline 38273739 PMC11105142 · DOI
  49. [57]Hou Y Dong H Wang X Li B Che W MetaPrompting: learning to learn better prompts Presented at Proceedings of the 29th International Conference on Computational Linguistics Oct 12-17, 2022 3251 3262 Gyeongju, Republic of Korea
  50. [59]Kataoka Y So R Banno M et al Development of meta-prompts for large language models to screen titles and abstracts for diagnostic test accuracy reviews medRxiv Preprint posted online on 2023 doi 10.1101/2023.10.31.23297818 · DOI
  51. [60]Wu C Restrepo D Shuai Z Liu Z Shen L Efficient in-context medical segmentation with meta-driven visual prompt selection Medical Image Computing and Computer Assisted Intervention – MICCAI 2024 Springer Nature 2024 255 265 doi 10.1007/978-3-031-72114-4_25 · DOI
  52. [62]Li H Fu JF Python A Implementing large language models in health care: clinician-focused review with interactive guideline J Med Internet Res 07 11 2025 27 e71916 doi 10.2196/71916 Medline 40644686 PMC12299950 · DOI
  53. [63]Sweeney L k-Anonymity: a model for protecting privacy Int J Unc Fuzz Knowl Based Syst Oct 2002 10 5 557 570 doi 10.1142/S0218488502001648 · DOI · Paper page
  54. [64]Kohlmayer F Lautenschläger R Prasser F Pseudonymization for research data collection: is the juice worth the squeeze? BMC Med Inform Decis Mak Sep 4 2019 19 1 178 doi 10.1186/s12911-019-0905-x Medline 31484555 PMC6727563 · DOI
  55. [65]Leiser F Guse R Sunyaev A Large language model architectures in health care: scoping review of research perspectives J Med Internet Res Jun 19 2025 27 e70315 doi 10.2196/70315 Medline 40536801 PMC12226782 · DOI
  56. [66]Morgan KL Kukhareva PV Warner PB et al Using CDS Hooks to increase SMART on FHIR app utilization: a cluster-randomized trial J Am Med Inform Assoc Aug 16 2022 29 9 1461 1470 doi 10.1093/jamia/ocac085 Medline 35641136 PMC9382378 · DOI
  57. [67]Wu Y Wu M Wang C Lin J Liu J Liu S Evaluating the prevalence of burnout among health care professionals related to electronic health record use: systematic review and meta-analysis JMIR Med Inform Jun 12 2024 12 e54811 doi 10.2196/54811 Medline 38865188 PMC11208837 · DOI · Paper page
  58. [68]Real-time clinical decision support via middleware-AI pipelines: bridging data silos for actionable healthcare intelligence World J Adv Res Rev 2025 26 2 3532 3544 URL https://journalwjarr.com/ArchiveIssue-2025-Vol26-Issue2 doi 10.30574/wjarr.2025.26.2.2007 · DOI
  59. [69]Zhang J Li B Zhou S Hierarchical modeling for medical visual question answering with cross-attention fusion Appl Sci (Basel) 2025 15 9 4712 doi 10.3390/app15094712 · DOI
  60. [70]Yin H Lu P Li Z Sun B Li K MODE: a multimodal open-domain dialogue dataset with explanation Appl Intell Apr 2024 54 7 5891 5906 doi 10.1007/s10489-024-05479-x · DOI
  61. [71]Bewersdorff A Hartmann C Hornberger M et al Taking the next step with generative artificial intelligence: The transformative role of multimodal large language models in science education Learn Individ Differ Feb 2025 118 102601 doi 10.1016/j.lindif.2024.102601 · DOI

Cited by

Sorted by citations

Authors
Liu, J., Liu, F., Wang, C., & Liu, S.
Affiliations
  • Sichuan University (CN)
  • Nanyang Technological University (SG)
  • Vanderbilt University Medical Center (US)
Corresponding author
Liu, Siru
ORCID
Venue
Journal of Medical Internet Research
Publisher
JMIR Publications Inc.
Published
2025-09-15
Volume
27
Pages
e72644-e72644
ISSN
1438-8871 (ISSN-L)
DOI
10.2196/72644
PMC
PMC12439060
OpenAlex
W4412884948
Cited by
76
References
13
Citations by year
2026: 69 · 2025: 7
FWCI
15.48
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Journal of Medical Internet Research (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DR-NTU (Nanyang Technological University) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords
prompt engineering, large language models, clinical decision support, zero-shot prompting, few-shot prompting, chain-of-thought prompting, meta-prompting, electronic health records, clinical documentation, patient communication, patient safety, ethical use of AI
MeSH terms
Humans, Natural Language Processing, Electronic Health Records

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review