CoChat
Sign Up
Cited by 3Artificial Intelligence in Healthcare and EducationMachine Learning in HealthcareElectronic Health Records Systems

Generating subjective, objective, assessment, and plan (SOAP)-structured medication logs using DeepSeek-R1 through prompt engineering and multi-source clinical information

Open access
Zhu, Yuxuan, Zhang, Jizhong, Sun, Yuhao, Wen, Jiayu, Liu, Zhixian, Liu, Xin, Xu, Silu, Wu, Nan +3 more·2026·DIGITAL HEALTH
DOI 10.1177/20552076261428234PMC PMC12966567
Open this paper in CoChat — free account

Abstract

Objectives: This study evaluated the feasibility of using the open-source large language model (LLM) DeepSeek-R1 to generate standardized Subjective, Objective, Assessment, and Plan (SOAP)-format medication logs and its potential to support clinical pharmacists. Materials and methods: Thirty complete oncology medication profiles were collected, from which 80 days of logs were extracted and converted into simulated pharmacist–patient dialogues. The experiment compared single-information-source inputs (dialogue only) with multi-information-source inputs (dialogue plus patient information, records, and test results), using five prompts of increasing complexity. Performance was measured using the Bidirectional Encoder Representations from Transformers (BERT) score and Recall-Oriented Understudy for Gisting Evaluation (ROUGE), alongside a blinded expert evaluation based on the Seven-Dimension Index (7DI) metric. Results: DeepSeek-R1 effectively generated structured SOAP medication logs when integrated with multi-source information and complex prompts (especially Prompts 4 and 5). Both machine scores and manual 7DI evaluation confirmed the superiority of multi-source inputs over single-source dialogues. While Prompt 4 achieved the highest BERT- F 1 and ROUGE scores, the model's output quality remained highly dependent on input data completeness and required pharmacist review to correct errors (e.g. incomplete analyses), after which scores improved significantly. Discussion: This study confirms DeepSeek-R1's utility in generating SOAP medication logs using multi-source data and structured prompts, potentially enhancing pharmacists’ efficiency. Limitations such as oncology-specific scope and artificial intelligence (AI) hallucinations necessitate pharmacist review and future validation across specialties, alongside comparisons with closed-source LLMs and explainable AI integration. Conclusion: This study demonstrates that DeepSeek-R1 can generate structured SOAP-format medication logs when guided by prompt-engineered multi-source clinical information, while highlighting that output quality depends on input completeness and that pharmacist review remains essential for clinical reliability.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1177/20552076261428234

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Some studies in this review used de-identified data to evaluate the LLMs.

    “Some studies in this review also used de-identified data to evaluate the LLMs, 30,61,65”

    From Large Language Models in Pharmacist-Led Direct Patient Care: A Rapid Review of Published Evaluations (2026) · cites this paper as [30]

  • Another study found that GPT-4 provided more accurate medication responses than pharmacists did when the information queried was within its training limits.

    “another found that GPT-4 provided more accurate medication responses than pharmacists did when the information queried was within its training limits. 30”

    From Large Language Models in Pharmacist-Led Direct Patient Care: A Rapid Review of Published Evaluations (2026) · cites this paper as [30]

    The Methods and Results assess DeepSeek-R1-generated medication logs, and the Discussion says the study did not compare DeepSeek-R1 with other LLMs. Searches for GPT-4, pharmacist response accuracy, and training limits found no passage reporting the claimed comparison. Page numbers were not provided in our copy. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

  • Two studies reported improved GPT-4 accuracy when answers did not require direct access to patient data.

    “For factual knowledge questions or inquiries, two studies reported improved GPT-4 accuracy when answers did not require direct access to patient data.30,90”

    From Large Language Models in Pharmacist-Led Direct Patient Care: A Rapid Review of Published Evaluations (2026) · cites this paper as [30]

    The Results (Machine scoring of generated text; page unavailable) compare DeepSeek-R1 outputs from single- and multi-source inputs, not GPT-4 answers to factual questions. The Discussion (page unavailable) mentions other language-model studies but does not report improved GPT-4 accuracy when answers do not require direct access to patient data. Searches for that comparison found no passage addressing it. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

  • One study reported that the proportion of GPT-4 responses rated as completely accurate increased from 64.3% to 69% when restricted to drug-related inquiries that did not require access to patient-specific information or knowledge beyond the model's training cut-off date.

    “One study descriptively reported that the proportion of GPT-4 responses rated as completely accurate increased from 64.3% to 69% when the analysis was restricted to drug-related inquiries that did not require access to patient-specific information or knowledge beyond the model's training cut-off date.30”

    From Large Language Models in Pharmacist-Led Direct Patient Care: A Rapid Review of Published Evaluations (2026) · cites this paper as [30]

    Split into 2 atoms — 0 confirmed, 0 partially supported, 0 disputed, 0 misattributed, 2 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

Cited by 3

Research works printed but not linked: 40

  1. [1]Lisenby KM Andrus MR Jackson CW, et al. Ambulatory care preceptors’ perceptions on SOAP note writing in advanced pharmacy practice experiences (APPEs). Curr Pharm Teach Learn 2018; 10: 1574– 1578. 30527822
  2. [2]Gogineni H Aranda JP Garavalia LS. Designing professional program instruction to align with students’ cognitive processing. Curr Pharm Teach Learn 2019; 11: 160– 165. 30733012
  3. [3]Andrus MR McDonough SLK Kelley KW, et al. Development and validation of a rubric to evaluate diabetes SOAP note writing in APPE. Am J Pharm Educ 2018; 82: 6725. 30559501 PMC6291674
  4. [4]Suhaili S Nasif H Oktavia Sari Y. The effect of education on writing integrated patient progress notes (IPPNs) at several government hospitals in Bukittinggi, Indonesia. J Farm Dan Ilmu Kefarmasian Indones 2024; 11: 101– 111.
  5. [5]American College of Clinical Pharmacy. Standards of practice for clinical pharmacists. Pharmacother J Hum Pharmacol Drug Ther 2014; 34: 794– 797. 25112523
  6. [6]Chan A Saeteaw M Chui WK, et al. Perceptions of pharmacy students and pharmacists on SOAP note education and utility in pharmacy practice. Curr Pharm Teach Learn 2016; 8: 77– 82.
  7. [7]Sherman JJ Johnson CD. Assessment of pharmacy students’ patient care skills using case scenarios with a SOAP note grading rubric and standardized patient feedback. Curr Pharm Teach Learn 2019; 11: 513– 521. 31171254
  8. [8]Minard LV Deal H Harrison ME, et al. Pharmacists’ perceptions of the barriers and facilitators to the implementation of clinical pharmacy key performance indicators. PLoS ONE 2016; 11: e0152903. PMC4820256 27043716
  9. [9]Canning ML Munns A Tai B. Accuracy of best possible medication history documentation by pharmacists at an Australian tertiary referral metropolitan hospital. Eur J Hosp Pharm 2018; 25: e52– e58. PMC6457157 31157067
  10. [10]Hayavi-Haghighi MH Davoodi S Teshnizi SH, et al. Usability evaluation of electronic prescribing systems from physicians’ perspective: a case study from southern Iran. Inform Med Unlocked 2024; 45: 101460.
  11. [11]Singhal K Tu T Gottweis J, et al. Toward expert-level medical question answering with large language models. Nat Med 2025; 31: 943– 950. 39779926 PMC11922739
  12. [12]He K Mao R Lin Q, et al. A survey of large language models for healthcare: from data, technology, and applications to accountability and ethics. Inf Fusion 2025; 118: 102963.
  13. [13]Park Y-J Pillai A Deng J, et al. Assessing the research landscape and clinical utility of large language models: a scoping review. BMC Med Inform Decis Mak 2024; 24: 72. 38475802 PMC10936025
  14. [14]Thirunavukarasu AJ Ting DSJ Elangovan K, et al. Large language models in medicine. Nat Med 2023; 29: 1930– 1940. 37460753
  15. [15]Yalamanchili A Sengupta B Song J, et al. Quality of large language model responses to radiation oncology patient care questions. JAMA Netw Open 2024; 7: e244630. PMC10988356 38564215
  16. [16]Ayers JW Poliak A Dredze M, et al. Comparing physician and artificial intelligence chatbot responses to patient questions posted to a public social media forum. JAMA Intern Med 2023; 183: 589– 596. 37115527 PMC10148230
  17. [17]Temsah A Alhasan K Altamimi I, et al. DeepSeek in healthcare: revealing opportunities and steering challenges of a new open-source artificial intelligence frontier. Cureus 2025; 17: e79221. PMC11836063 39974299
  18. [18]Sandmann S Hegselmann S Fujarski M, et al. Benchmark evaluation of DeepSeek large language models in clinical decision-making. Nat Med. Epub ahead of print 23 April 2025. DOI: 10.1038/s41591-025-03727-2. PMC12353792 40267970 · DOI · Paper page
  19. [19]Liang W Chen P Zou X, et al. DeepSeek: the “Watson” to doctors—from assistance to collaboration. J Thorac Dis 2025; 17: 1103– 1105. 40083524 PMC11898397
  20. [20]Kaygisiz ÖF Teke MT. Can DeepSeek and ChatGPT be used in the diagnosis of oral pathologies? BMC Oral Health 2025; 25: 638. 40281436 PMC12023442
  21. [21]Jin X Zhang J Yang J, et al. Every cloud has a silver lining: DeepSeek’s light through acute respiratory distress syndrome shadows. J Thorac Dis 2025; 17: 1109– 1113. 40083514 PMC11898352
  22. [22]Marcaccini G Seth I Xie Y, et al. Breaking bones, breaking barriers: ChatGPT, DeepSeek, and Gemini in hand fracture management. J Clin Med 2025; 14: 1983. 40142791 PMC11942733
  23. [23]Chen Y-Q Yu T Song Z-Q, et al. Application of large language models in drug-induced osteotoxicity prediction. J Chem Inf Model 2025; 65: 3370– 3379. 40114317
  24. [24]Zhou M Pan Y Zhang Y, et al. Evaluating AI-generated patient education materials for spinal surgeries: comparative analysis of readability and DISCERN quality across ChatGPT and DeepSeek models. Int J Med Inf 2025; 198: 105871. 40107040
  25. [25]Kayaalp ME Prill R Sezgin EA, et al. DeepSeek versus ChatGPT: multimodal artificial intelligence revolutionizing scientific discovery. From language editing to autonomous content generation-redefining innovation in research and practice. Knee Surg Sports Traumatol Arthrosc Off J ESSKA 2025; 33: 1553– 1556. 39936363
  26. [26]Meskó B. Prompt engineering as an important emerging skill for medical professionals: tutorial. J Med Internet Res 2023; 25: e50638. PMC10585440 37792434
  27. [27]Wang L Chen X Deng X, et al. Prompt engineering in consistency and reliability with the evidence-based guideline for LLMs. Npj Digit Med 2024; 7: 41. 38378899 PMC10879172
  28. [28]Kernberg A Gold JA Mohan V. Using ChatGPT-4 to create structured medical notes from audio recordings of physician–patient encounters: comparative study. J Med Internet Res 2024; 26: e54419. PMC11074889 38648636
  29. [29]Ding H Xia W Zhou Y, et al. Evaluation and practical application of prompt-driven ChatGPTs for EMR generation. Npj Digit Med 2025; 8: 77. 39894840 PMC11788423
  30. [30]Zhang T Kishore V Wu F, et al. BERTScore: Evaluating Text Generation with BERT. In: International conference on learning representations, 2020.
  31. [32]Liu Z Zhang L Wu Z, et al. Surviving ChatGPT in healthcare. Front Radiol 2024; 3: 1224682. 38464946 PMC10920216
  32. [33]Wiest IC Ferber D Zhu J, et al. Privacy-preserving large language models for structured medical information retrieval. Npj Digit Med 2024; 7: 257. 39304709 PMC11415382
  33. [34]Cascella M Montomoli J Bellini V, et al. Evaluating the feasibility of ChatGPT in healthcare: an analysis of multiple clinical and research scenarios. J Med Syst 2023; 47: 33. 36869927 PMC9985086
  34. [35]Chintagunta B Katariya N Amatriain X, et al. Medically aware GPT-3 as a data generator for medical dialogue summarization. In: Proceedings of the second workshop on natural language processing for medical conversations, pp. 66– 76: Association for Computational Linguistics.
  35. [36]Patel SB Lam K. ChatGPT: the future of discharge summaries? Lancet Digit Health 2023; 5: e107– e108. 36754724
  36. [37]Liu J Koopman B Brown NJ, et al. Generating synthetic clinical text with local large language models to identify misdiagnosed limb fractures in radiology reports. Artif Intell Med 2025; 159: 103027. 39580897
  37. [38]Tordjman M Liu Z Yuce M, et al. Comparative benchmarking of the DeepSeek large language model on medical tasks and clinical reasoning. Nat Med 2025; 31: 2550– 2555. 40267969
  38. [39]Yildiz Durak H Eğin F Onan A. A comparison of human-written versus AI-generated text in discussions at educational settings: investigating features for ChatGPT, Gemini and BingAI. Eur J Educ 2025; 60: e70014.
  39. [40]Georgiou GP. Differentiating between human-written and AI-generated texts using linguistic features automatically extracted from an online computational tool. arXiv.org, https://arxiv.org/abs/2407.03646v2 (2024, accessed 28 May 2025). · arXiv
  40. [41]Liu X Barreto EF Dong Y, et al. Discrepancy between perceptions and acceptance of clinical decision support systems: implementation of artificial intelligence for vancomycin dosing. BMC Med Inform Decis Mak 2023; 23: 157. 37568134 PMC10416522

Authors
Zhu, Y., Zhang, J., Sun, Y., Wen, J., Liu, Z., Liu, X., Xu, S., Wu, N., Zhang, Y., Zhou, G., & Wei, J.
Affiliations
  • China Pharmaceutical University (CN)
  • Jiangsu Cancer Hospital (CN)
  • Jiangsu Cancer Hospital, The Affiliated Cancer Hospital of Nanjing Medical University, Jiangsu Institute of Cancer Research, Jiangsu Key Laboratory of Innovative Cancer Diagnosis & Therapeutics, Nanjing, Jiangsu Province, China
Corresponding author
  • Zhou, Guoren
  • Wei, Jifu
ORCID
Venue
DIGITAL HEALTH
Publisher
SAGE Publications
Published
2026-02
Volume
12
Pages
20552076261428234-20552076261428234
ISSN
2055-2076 (ISSN-L)
DOI
10.1177/20552076261428234
PMC
PMC12966567
OpenAlex
W7134197276
Cited by
3
References
0
Citations by year
2026: 3
FWCI
5.90
Type
journal article
Language
en
Open access
Licence
Locations
  • Digital Health (publishedVersion, cc-by-nc)
  • PubMed Central (publishedVersion, cc-by-nc)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • the Yishan Project of Jiangsu Cancer Hospital (YSZD202406)
  • Qunfeng Project of Jiangsu Cancer Hospital (No.DFXK202501)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
Keywords
DeepSeek-R1, large language models, prompt engineering, clinical pharmacists, AI hallucination, clinical documentation automation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review