CoChat
Sign Up
PreprintCited by 150Topic ModelingArtificial Intelligence in Healthcare and EducationNatural Language Processing Techniques

Prompt Engineering for Healthcare: Methodologies and Applications

Open access
Wang, Jiaqi, Shi, Enze, Yu, Sigang, Wu, Zihao, Ma, Chong, Dai, Haixing, Yang, Qiushi, Kang, Yanqing +16 more·2023·arXiv
DOI 10.48550/arxiv.2304.14670arXiv 2304.14670
Open this paper in CoChat — free account

Abstract

Prompt engineering is a critical technique in the field of natural language processing that involves designing and optimizing the prompts used to input information into models, aiming to enhance their performance on specific tasks. With the recent advancements in large language models, prompt engineering has shown significant superiority across various domains and has become increasingly important in the healthcare domain. However, there is a lack of comprehensive reviews specifically focusing on prompt engineering in the medical field. This review will introduce the latest advances in prompt engineering in the field of natural language processing for the medical field. First, we will provide the development of prompt engineering and emphasize its significant contributions to healthcare natural language processing applications such as question-answering systems, text summarization, and machine translation. With the continuous improvement of general large language models, the importance of prompt engineering in the healthcare domain is becoming increasingly prominent. The aim of this article is to provide useful resources and bridges for healthcare natural language processing researchers to better explore the application of prompt engineering in this field. We hope that this review can provide new ideas and inspire for research and application in medical natural language processing.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2304.14670

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Prompt phrasing and model temperature can considerably affect LLM output.

    “Prompt phrasing and model temperature can have a considerable effect on LLM output, as demonstrated in the literature45,46 and in Fig. 2.”

    From Adapted large language models can outperform medical experts in clinical text summarization (2024) · cites this paper as [46]

    The Introduction (page unavailable) says prompt design can significantly affect model performance, and Automated Prompts describes improved performance from prompt paraphrasing. The copy does not address model temperature (searched: “temperature”), so it supports the prompt-phrasing part but not the temperature part. (LLM, conf=0.87)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 73 · Cited by 150

Research works printed but not linked: 75

  1. P. Liu, W. Yuan, J. Fu, Z. Jiang, H. Hayashi, and G. Neubig, "Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing," ACM Computing Surveys, vol. 55, no. 9, pp. 1-35, 2023.
  2. C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu, "Exploring the limits of transfer learn- ing with a unified text-to-text transformer," The Journal of Machine Learning Research, vol. 21, no. 1, pp. 5485-5551, 2020.
  3. X. Qiu, T. Sun, Y. Xu, Y. Shao, N. Dai, and X. Huang, "Pre-trained models for natural language processing: A survey," Science China Technological Sciences, vol. 63, no. 10, pp. 1872-1897, 2020.
  4. B. Lester, R. Al-Rfou, and N. Constant, "The power of scale for parameter-efficient prompt tuning," arXiv preprint arXiv:2104.08691, 2021. · arXiv
  5. J. Kopač, M. Bahor, and M. Soković, "Optimal machining parameters for achieving the desired surface roughness in fine turning of cold pre- formed steel workpieces," International Journal of Machine Tools and Manufacture, vol. 42, no. 6, pp. 707-716, 2002.
  6. N. Ding, S. Hu, W. Zhao, Y. Chen, Z. Liu, H. Zheng, and M. Sun, "Openprompt: An open-source framework for prompt-learning," in Proceedings of the 60th Annual Meeting of the Association for Com- putational Linguistics: System Demonstrations, 2022, pp. 105-113.
  7. V. Liu and L. B. Chilton, "Design guidelines for prompt engineering text-to-image generative models," in Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems, 2022, pp. 1- 23.
  8. T. Sorensen, J. Robinson, C. Rytting, A. Shaw, K. Rogers, A. Delorey, M. Khalil, N. Fulda, and D. Wingate, "An information-theoretic ap- proach to prompt engineering without ground truth labels," in Proceed- ings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2022, pp. 819-862.
  9. G. Rong, A. Mendez, E. B. Assi, B. Zhao, and M. Sawan, "Artificial intelligence in healthcare: review and prediction case studies," Engi- neering, vol. 6, no. 3, pp. 291-301, 2020.
  10. L. Zhou and G. Hripcsak, "Temporal reasoning with medical data-a review with emphasis on medical natural language processing," Journal of biomedical informatics, vol. 40, no. 2, pp. 183-202, 2007.
  11. R. Baud, A.-M. Rassinoux, and J.-R. Scherrer, "Natural language processing and semantical representation of medical texts," Methods of information in medicine, vol. 31, no. 02, pp. 117-125, 1992.
  12. F. Petroni, T. Rocktäschel, S. Riedel, P. Lewis, A. Bakhtin, Y. Wu, and A. Miller, "Language models as knowledge bases?" in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2019, pp. 2463-2473.
  13. Q. Lyu, J. Tan, M. E. Zapadka, J. Ponnatapuram, C. Niu, G. Wang, and C. T. Whitlow, "Translating radiology reports into plain language using chatgpt and gpt-4 with prompt learning: Promising results, limitations, and potential," arXiv preprint arXiv:2303.09038, 2023. · arXiv
  14. E. Wallace, S. Feng, N. Kandpal, M. Gardner, and S. Singh, "Universal adversarial triggers for attacking and analyzing nlp," in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2019.
  15. T. Shin, Y. Razeghi, R. L. Logan IV, E. Wallace, and S. Singh, "Autoprompt: Eliciting knowledge from language models with auto- matically generated prompts," in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020, pp. 4222-4235.
  16. X. L. Li and P. Liang, "Prefix-tuning: Optimizing continuous prompts for generation," in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), 2021, pp. 4582-4597.
  17. K. Zhou, J. Yang, C. C. Loy, and Z. Liu, "Learning to prompt for vision-language models," International Journal of Computer Vision, vol. 130, no. 9, pp. 2337-2348, 2022.
  18. J. D. M.-W. C. Kenton and L. K. Toutanova, "Bert: Pre-training of deep bidirectional transformers for language understanding," in Proceedings of NAACL-HLT, 2019, pp. 4171-4186.
  19. K. He, X. Chen, S. Xie, Y. Li, P. Dollár, and R. Girshick, "Masked autoencoders are scalable vision learners," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022, pp. 16 000-16 009.
  20. P. J. Liu, M. Saleh, E. Pot, B. Goodrich, R. Sepassi, L. Kaiser, and N. Shazeer, "Generating wikipedia by summarizing long sequences," in International Conference on Learning Representations, 2018.
  21. A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever et al., "Language models are unsupervised multitask learners," OpenAI blog, vol. 1, no. 8, p. 9, 2019.
  22. T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell et al., "Language mod- els are few-shot learners," Advances in neural information processing systems, vol. 33, pp. 1877-1901, 2020.
  23. A. Najar and M. Chetouani, "Reinforcement learning with human advice: a survey," Frontiers in Robotics and AI, vol. 8, p. 584075, 2021.
  24. W. B. Knox and P. Stone, "Interactively shaping agents via human reinforcement: The tamer framework," in Proceedings of the fifth international conference on Knowledge capture, 2009, pp. 9-16.
  25. L. Cui, Y. Wu, J. Liu, S. Yang, and Y. Zhang, "Template-based named entity recognition using bart," in Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, 2021, pp. 1835- 1845.
  26. M. Abaho, D. Bollegala, P. Williamson, and S. Dodd, "Position-based prompting for health outcome generation," in Proc. of The 21st BioNLP workshop associated with the ACL SIGBIOMED special interest group, 2022.
  27. J. Holmes, Z. Liu, L. Zhang, Y. Ding, T. T. Sio, L. A. McGee, J. B. Ashman, X. Li, T. Liu, J. Shen et al., "Evaluating large language models on a highly-specialized topic, radiation oncology physics," arXiv preprint arXiv:2304.01938, 2023. · arXiv
  28. J. Yuan, R. Tang, X. Jiang, and X. Hu, "Llm for patient-trial match- ing: Privacy-aware data augmentation towards better performance and generalizability," arXiv preprint arXiv:2303.16756, 2023. · arXiv
  29. H. Wang, C. Liu, N. Xi, S. Zhao, M. Ju, S. Zhang, Z. Zhang, Y. Zheng, B. Qin, and T. Liu, "Prompt combines paraphrase: Teaching pre-trained models to understand rare biomedical words," in Proceedings of the 29th International Conference on Computational Linguistics, 2022, pp. 1422-1431.
  30. A. Romanov and C. Shivade, "Lessons from natural language inference in the clinical domain," in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, 2018, pp. 1586- 1596.
  31. T. Gao, A. Fisch, and D. Chen, "Making pre-trained language models better few-shot learners," in Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL- IJCNLP 2021. Association for Computational Linguistics (ACL), 2021, pp. 3816-3830.
  32. Y. Ruan, X. Lan, D. J. Tan, H. R. Abdullah, and M. Feng, "Medical intervention duration estimation using language-enhanced transformer encoder with medical prompts," arXiv preprint arXiv:2303.17408, 2023. · arXiv
  33. B. Li, X. Wang, X. Xu, Y. Hou, Y. Feng, F. Wang, and W. Che, "Semantic-guided image augmentation with pre-trained models," arXiv preprint arXiv:2302.02070, 2023. · arXiv
  34. D. Jang and C.-E. Kim, "Exploring the potential of large language models in traditional korean medicine: A foundation model approach to culturally-adapted healthcare," arXiv preprint arXiv:2303.17807, 2023. · arXiv
  35. Z. Yang, S. Kwon, Z. Yao, and H. Yu, "Multi-label few-shot icd coding as autoregressive generation with prompt," in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 4, 2023, pp. 5366- 5374.
  36. K. Kolluru, G. Stanovsky et al., ""covid vaccine is against covid but oxford vaccine is made at oxford!" semantic interpretation of proper noun compounds," in Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2022, pp. 10 407-10 420.
  37. C. Niu and G. Wang, "Ct multi-task learning with a large image-text (lit) model," bioRxiv, pp. 2023-04, 2023.
  38. Y. Yang, W. Lei, P. Huang, J. Cao, J. Li, and T.-S. Chua, "A dual prompt learning framework for few-shot dialogue state tracking," in Proceedings of the ACM Web Conference 2023, 2023, pp. 1468-1477.
  39. J. DeYoung, E. Lehman, B. Nye, I. Marshall, and B. C. Wallace, "Evidence inference 2.0: More data, better models," in Proceedings of the 19th SIGBioMed Workshop on Biomedical Language Processing, 2020, pp. 123-132.
  40. M. Keicher, K. Mullakaeva, T. Czempiel, K. Mach, A. Khakzar, and N. Navab, "Few-shot structured radiology report generation using natural language prompts," arXiv preprint arXiv:2203.15723, 2022. · arXiv
  41. Y. Rao, W. Zhao, G. Chen, Y. Tang, Z. Zhu, G. Huang, J. Zhou, and J. Lu, "Denseclip: Language-guided dense prediction with context- aware prompting," in 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2022, pp. 18 061-18 070.
  42. Z. Yao, Y. Cao, Z. Yang, and H. Yu, "Context variance evaluation of pretrained language models for prompt-based biomedical knowledge probing," AMIA Summits on Translational Science Proceedings, vol. 2023, p. 592, 2023.
  43. M. Sung, J. Lee, S. Yi, M. Jeon, S. Kim, and J. Kang, "Can language models be biomedical knowledge bases?" in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, 2021, pp. 4723-4734.
  44. A. Hertz, R. Mokady, J. Tenenbaum, K. Aberman, Y. Pritch, and D. Cohen-or, "Prompt-to-prompt image editing with cross-attention control," in The Eleventh International Conference on Learning Rep- resentations, 2022.
  45. X. Liu, H.-Y. Huang, G. Shi, and B. Wang, "Dynamic prefix-tuning for generative template-based event extraction," in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2022, pp. 5216-5228.
  46. Y.-N. Chuang, R. Tang, X. Jiang, and X. Hu, "Spec: A soft prompt- based calibration on mitigating performance variability in clinical notes summarization," arXiv preprint arXiv:2303.13035, 2023. · arXiv
  47. X. Wang, B. Liu, S. Tang, and L. Wu, "Qrelscore: Better evaluating generated questions with deeper understanding of context-aware rele- vance," in Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, 2022, pp. 562-581.
  48. Y. Du, S. Oraby, V. Perera, M. Shen, A. Narayan-Chen, T. Chung, A. Venkatesh, and D. Hakkani-Tur, "Schema-guided natural language generation," in Proceedings of the 13th International Conference on Natural Language Generation, 2020, pp. 283-295.
  49. N. T. Le, F. Bai, and A. Ritter, "Few-shot anaphora resolution in scientific protocols via mixtures of in-context experts," in Findings of the Association for Computational Linguistics: EMNLP 2022, 2022, pp. 2693-2706.
  50. C. Peng, X. Yang, Z. Yu, J. Bian, W. R. Hogan, and Y. Wu, "Clinical concept and relation extraction using prompt-based machine reading comprehension," Journal of the American Medical Informatics Associ- ation: JAMIA, p. ocad107.
  51. J. Zhou, Q. Zhang, Q. Chen, L. He, and X.-J. Huang, "A multi-format transfer learning model for event argument extraction via variational information bottleneck," in Proceedings of the 29th International Conference on Computational Linguistics, 2022, pp. 1990-2000.
  52. Z. Yao, J. Tsai, W. Liu, D. A. Levy, E. Druhl, J. I. Reisman, and H. Yu, "Automated identification of eviction status from electronic health record notes," Journal of the American Medical Informatics Association, p. ocad081, 2023.
  53. Y. Xing, Q. Wu, D. Cheng, S. Zhang, G. Liang, P. Wang, and Y. Zhang, "Dual modality prompt tuning for vision-language pre-trained model," IEEE Transactions on Multimedia, 2023.
  54. Z. Ding, A. Wang, H. Chen, Q. Zhang, P. Liu, Y. Bao, W. Yan, and J. Han, "Exploring structured semantic prior for multi label recognition with incomplete labels," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023, pp. 3398-3407.
  55. Y. Li, Z. Lin, S. Zhang, Q. Fu, B. Chen, J.-G. Lou, and W. Chen, "On the advance of making language models better reasoners," arXiv preprint arXiv:2206.02336, 2022. · arXiv
  56. A. Azaria, A. Ekblaw, T. Vieira, and A. Lippman, "Medrec: Using blockchain for medical data access and permission management," in 2016 2nd international conference on open and big data (OBD). IEEE, 2016, pp. 25-30.
  57. Y. He, S. Zheng, Y. Tay, J. Gupta, Y. Du, V. Aribandi, Z. Zhao, Y. Li, Z. Chen, D. Metzler et al., "Hyperprompt: Prompt-based task- conditioning of transformers," in International Conference on Machine Learning. PMLR, 2022, pp. 8678-8690.
  58. V. Franc ¸ois-Lavet, P. Henderson, R. Islam, M. G. Bellemare, J. Pineau et al., "An introduction to deep reinforcement learning," Foundations and Trends® in Machine Learning, vol. 11, no. 3-4, pp. 219-354, 2018.
  59. P. Wang, H. Gao, X. Guo, C. Xiao, F. Qi, and Z. Yan, "An experimental investigation of text-based captcha attacks and their robustness," ACM Computing Surveys, vol. 55, no. 9, pp. 1-38, 2023.
  60. F. Perez and I. Ribeiro, "Ignore previous prompt: Attack techniques for language models," in NeurIPS ML Safety Workshop, 2022.
  61. P. Rajpurkar, E. Chen, O. Banerjee, and E. J. Topol, "Ai in health and medicine," Nature medicine, vol. 28, no. 1, pp. 31-38, 2022.
  62. X. Liu, J. Zhao, J. Li, B. Cao, and Z. Lv, "Federated neural architecture search for medical data security," IEEE transactions on industrial informatics, vol. 18, no. 8, pp. 5628-5636, 2022.
  63. K. Wang, B. Zhan, C. Zu, X. Wu, J. Zhou, L. Zhou, and Y. Wang, "Semi-supervised medical image segmentation via a tripled-uncertainty guided mean teacher model with contrastive learning," Medical Image Analysis, vol. 79, p. 102447, 2022.
  64. X. Li, H. Xiong, X. Li, X. Wu, X. Zhang, J. Liu, J. Bian, and D. Dou, "Interpretable deep learning: Interpretation, interpretability, trustworthiness, and beyond," Knowledge and Information Systems, vol. 64, no. 12, pp. 3197-3234, 2022.
  65. R. U. Rasool, H. F. Ahmad, W. Rafique, A. Qayyum, and J. Qadir, "Security and privacy of internet of medical things: A contemporary review in the age of surveillance, botnets, and adversarial ml," Journal of Network and Computer Applications, p. 103332, 2022.
  66. B. R. Andrus, Y. Nasiri, S. Cui, B. Cullen, and N. Fulda, "Enhanced story comprehension for large language models through dynamic document-based knowledge graphs," in Proceedings of the AAAI Con- ference on Artificial Intelligence, vol. 36, 2022, pp. 10 436-10 444.
  67. A. Gilson, C. W. Safranek, T. Huang, V. Socrates, L. Chi, R. A. Taylor, D. Chartash et al., "How does chatgpt perform on the united states medical licensing examination? the implications of large language mod- els for medical education and knowledge assessment," JMIR Medical Education, vol. 9, no. 1, p. e45312, 2023.
  68. B. Jacob, A. Kaushik, P. Velavan, and M. Sharma, "Autonomous drones for medical assistance using reinforcement learning," Advances in Augmented Reality and Virtual Reality, pp. 133-156, 2022.
  69. J. S. Chen and S. L. Baxter, "Applications of natural language pro- cessing in ophthalmology: present and future," Frontiers in Medicine, vol. 9, 2022.
  70. M. Thakur, S. Dhanalakshmi, H. Kuresan, R. Senthil, R. Narayanamoorthi, and K. W. Lai, "Automated restricted boltzmann machine classifier for early diagnosis of parkinson's disease using digitized spiral drawings," Journal of Ambient Intelligence and Humanized Computing, vol. 14, no. 1, pp. 175-189, 2023.
  71. E. A. Martin, A. G. D'Souza, S. Lee, C. Doktorchik, C. A. Eastwood, and H. Quan, "Hypertension identification using inpatient clinical notes from electronic medical records: an explainable, data-driven algorithm study," Canadian Medical Association Open Access Journal, vol. 11, no. 1, pp. E131-E139, 2023.
  72. M. A. Azam, K. B. Khan, S. Salahuddin, E. Rehman, S. A. Khan, M. A. Khan, S. Kadry, and A. H. Gandomi, "A review on multimodal medical image fusion: Compendious analysis of medical modalities, multimodal databases, fusion techniques and quality metrics," Computers in biology and medicine, vol. 144, p. 105253, 2022.
  73. R. Chengoden, N. Victor, T. Huynh-The, G. Yenduri, R. H. Jhaveri, M. Alazab, S. Bhattacharya, P. Hegde, P. K. R. Maddikunta, and T. R. Gadekallu, "Metaverse for healthcare: A survey on potential applications, challenges and future directions," IEEE Access, 2023.
  74. Q. D. Buchlak, N. Esmaili, C. Bennett, and F. Farrokhi, "Natural language processing applications in the clinical neurosciences: A machine learning augmented systematic review," Machine Learning in Clinical Neuroscience: Foundations and Applications, pp. 277-289, 2022.
  75. B. Kutela, K. Msechu, S. Das, and E. Kidando, "Chatgpt's scientific writings: A case study on traffic safety," Available at SSRN 4329120, 2023.

Authors
Wang, J., Shi, E., Yu, S., Wu, Z., Ma, C., Dai, H., Yang, Q., Kang, Y., Wu, J., Hu, H., Yue, C., Zhang, H., Liu, Y., Pan, Y., Liu, Z., Sun, L., Li, X., Ge, B., Jiang, X., Zhu, D., Yuan, Y., Shen, D., Liu, T., & Zhang, S.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2023
Volume
4
Issue
1
Pages
100190-100190
ISSN
2950-1628 (ISSN-L)
DOI
10.48550/arxiv.2304.14670
arXiv
2304.14670
OpenAlex
W4416542771
Cited by
150
References
73
Citations by year
2026: 30 · 2025: 36 · 2024: 67 · 2023: 17
FWCI
26.47
Type
journal article
Language
en
Open access
Licence
Locations
  • Meta-Radiology (publishedVersion, cc-by-nc-nd)
  • arXiv (Cornell University) (submittedVersion, other-oa)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • CityU Scholars (submittedVersion, cc-by-nc-nd)
  • arXiv (Cornell University)
Funders
  • National Natural Science Foundation of China (62276050, 61976131, 62001410)
  • Northwestern Polytechnical University
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Keywords
prompt engineering, large language models, question answering systems, text summarization, machine translation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review