Building Trustworthy Generative Artificial Intelligence for Diabetes Care and Limb Preservation: A Medical Knowledge Extraction Case
Open accessAbstract
Background: Large language models (LLMs) offer significant potential in medical information extraction but carry risks of generating incorrect information. This study aims to develop and validate a retriever-augmented generation (RAG) model that provides accurate medical knowledge about diabetes and diabetic foot care to laypersons with an eighth-grade literacy level. Improving health literacy through patient education is paramount to addressing the problem of limb loss in the diabetic population. In addition to affecting patient well-being through improved outcomes, improved physician well-being is an important outcome of a self-management model for patient health education. Methods: We used an RAG architecture and built a question-and-answer artificial intelligence (AI) model to extract knowledge in response to questions pertaining to diabetes and diabetic foot care. We utilized GPT-4 by OpenAI, with Pinecone as a vector database. The NIH National Standards for Diabetes Self-Management Education served as the basis for our knowledge base. The model’s outputs were validated through expert review against established guidelines and literature. Fifty-eight keywords were used to select 295 articles and the model was tested against 175 questions across topics. Results: The study demonstrated that with appropriate content volume and few-shot learning prompts, the RAG model achieved 98% accuracy, confirming its capability to offer user-friendly and comprehensible medical information. Conclusion: The RAG model represents a promising tool for delivering reliable medical knowledge to the public which can be used for self-education and self-management for diabetes, highlighting the importance of content validation and innovative prompt engineering in AI applications.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Extensive literature consistently reports that domain-specific models such as BioGPT are prone to hallucination or generating content beyond the retrieved evidence unless explicitly fine-tuned for retrieval-grounded generation.
“Moreover, extensive literature consistently reports that while domain-specific models such as BioGPT excel at producing semantically rich and lexically appropriate text, they are prone to hallucination or generating content beyond the retrieved evidence unless explicitly fine-tuned for retrieval-grounded generation [16,24,29,32]”
From Enhancing Medical Question Answering with LLMs via a Hybrid Retrieval-Augmented Generation Framework (2026) · cites this paper as [29]
Cites
International Diabetes Federation 2017
Carracher, A. M., Marathe, P. H., & Close, K. L. (2018) · Journal of Diabetes · doi:10.1111/1753-0407.12644 · Cited by 13,369 · see citing works
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083 · see citing works
Armstrong, D. G., Tan, T.-W., Boulton, A. J. M., & Bus, S. A. (2023) · JAMA · doi:10.1001/jama.2023.10578 · Cited by 1,658 · see citing works
Self-citationPatients' memory for medical information
Kessels, R. P. C. (2003) · Journal of the Royal Society of Medicine · doi:10.1258/jrsm.96.5.219 · Cited by 1,426 · see citing works
National Standards for Diabetes Self-Management Education
Funnell, M. M., Brown, T. L., Childs, B. P., Haas, L. B., Hosey, G. M., Jensen, B., Maryniuk, M. D., Peyrot, M. F., Piette, J. D., Reader, D., et al. (2008) · Diabetes Care · doi:10.2337/dc09-s087 · Cited by 1,348 · see citing works
Etiology, Epidemiology, and Disparities in the Burden of Diabetic Foot Ulcers
McDermott, K. M., Fang, M., Boulton, A. J. M., Selvin, E., & Hicks, C. W. (2022) · Diabetes Care · doi:10.2337/dci22-0043 · Cited by 1,295 · see citing works
Armstrong, D. G., Swerdlow, M. A., Armstrong, A. A., Conte, M. S., Padula, W. V., & Bus, S. A. (2020) · Journal of Foot and Ankle Research · doi:10.1186/s13047-020-00383-2 · Cited by 975 · see citing works
Self-citationImproving Health Outcomes Through Patient Education and Partnerships with Patients
Paterick, T. E., Patel, N. J., Tajik, A. J., & Chandrasekaran, K. (2017) · Baylor University Medical Center Proceedings · doi:10.1080/08998280.2017.11929552 · Cited by 368 · see citing works
Diabetes, Lower-Extremity Amputation, and Death
Hoffstad, O., Mitra, N., Walsh, J., & Margolis, D. J. (2015) · Diabetes Care · doi:10.2337/dc15-0536 · Cited by 240 · see citing works
Charter on Physician Well-being
Thomas, L. R., Ripp, J. A., & West, C. P. (2018) · JAMA · doi:10.1001/jama.2018.1331 · Cited by 190 · see citing works
National Standards for Diabetes Self-Management Education
Funnell, M. M., Brown, T. L., Childs, B. P., Haas, L. B., Hosey, G. M., Jensen, B., Maryniuk, M. D., Peyrot, M. F., Piette, J. D., Reader, D., et al. (2009) · Diabetes Care · doi:10.2337/dc10-s089 · Cited by 190 · see citing works
Timmers, T., Janssen, L., Kool, R. B., & Kremer, J. A. M. (2020) · Journal of Medical Internet Research · doi:10.2196/17342 · Cited by 127 · see citing works
Meloni, M., Izzo, V., Giurato, L., Lázaro‐Martínez, J. L., & Uccioli, L. (2020) · Journal of Clinical Medicine · doi:10.3390/jcm9061780 · Cited by 124 · see citing works
A systematic review and meta-analysis on ChatGPT and its utilization in medical and dental research
Bagde, H. S., Dhopte, A., Alam, M. K., & Basri, R. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e23050 · Cited by 71 · see citing works
ChatGPT and finetuned BERT: A comparative study for developing intelligent design support systems
Qiu, Y., & Jin, Y. (2023) · Intelligent Systems with Applications · doi:10.1016/j.iswa.2023.200308 · Cited by 65 · see citing works
Donnelly, H. R., Clarke, E. D., Collins, C. E., Collins, R. A., Armstrong, D. G., Mills, J. L., & Tehan, P. E. (2023) · International Wound Journal · doi:10.1111/iwj.14483 · Cited by 9 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Large language models in patient education: a scoping review of applications in medicine
Aydin, S., Karabacak, M., Vlachos, V., & Margetis, K. D. (2024) · Frontiers in Medicine · doi:10.3389/fmed.2024.1477898 · Cited by 333 · see citing works
Retrieval augmented generation for large language models in healthcare: A systematic review
Amugongo, L. M., Mascheroni, P., Brooks, S., Doering, S., & Seidel, J. (2025) · PLOS Digital Health · doi:10.1371/journal.pdig.0000877 · Cited by 267 · see citing works
A survey on retrieval-augmentation generation (RAG) models for healthcare applications
El-Enen, M. A. A., Saad, S., & Nazmy, T. M. (2025) · Neural Computing and Applications · doi:10.1007/s00521-025-11666-9 · Cited by 78 · see citing works
Tozuka, R., Johno, H., Amakawa, A., Sato, J., Muto, M., Seki, S., Komaba, A., & Onishi, H. (2024) · Japanese Journal of Radiology · doi:10.1007/s11604-024-01705-1 · Cited by 57 · see citing works
Kelly, A., Noctor, E., Ryan, L., & van de Ven, P. W. J. (2025) · Journal of Medical Internet Research · doi:10.2196/70131 · Cited by 38 · see citing works
AlSammarraie, A., Al-Saifi, A., Kamhia, H., Aboagla, M., & Househ, M. (2025) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2025-101570 · Cited by 10 · see citing works
Qin, S., Wang, Y., Cui, T., Ma, J., Zhou, X., Guo, X., Zhang, C., Zhou, C., Guo, R., & Li, H. (2025) · Pharmacological Research · doi:10.1016/j.phrs.2025.107883 · Cited by 11 · see citing works
Thanthrige, A. N. W., & Wickramasinghe, N. S. (2025) · JMIR Diabetes · doi:10.2196/80582 · Cited by 7 · see citing works
Creating a biomedical knowledge base by addressing GPT inaccurate responses and benchmarking context
Darnell, S. S., Overall, R. W., Guarracino, A., Colonna, V., Villani, F., Garrison, E. P., Isaac, A., Muli, P., Muriuki, M. F., Kabui, A., et al. (2025) · Open Research Africa · doi:10.12688/openresafrica.15854.1 · Cited by 5 · see citing works
An Automotive Fault Diagnosis Framework Based on Knowledge Graphs and Large Language Models
Lin, W., & Miao, K.-H. (2025) · Electronics · doi:10.3390/electronics14214180 · Cited by 9 · see citing works
Wang, Z., Li, X., Tao, X., Li, J., Zhang, L., He, X., & Yang, J. (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2026.1804524 · Cited by 5 · see citing works
Artificial Intelligence in Diabetes Care
Dubský, M., Liegertová, M., Bém, R., Sojáková, D., Fejfarová, V., & Jude, E. B. (2026) · Diabetes Obesity and Metabolism · doi:10.1111/dom.70720 · Cited by 5 · see citing works
Interpretable machine learning model for predicting recurrence in patients with diabetic foot ulcers
Mou, W., Shan, W., Yu, S., Rui, S., Du, C., Huo, Z., Gu, H., Armstrong, D. G., Tang, D., Wang, Y., et al. (2025) · BMJ Open Diabetes Research & Care · doi:10.1136/bmjdrc-2025-005242 · Cited by 4 · see citing works
Self-citationIncreasing Adherence to Societal Recommendations in Radiology Reporting
Mugu, V. K., Carr, B. M., Olson, M. C., Schupbach, J. C., Eguia, F. A., Schmitz, J. J., & Khandelwal, A. (2024) · Ultrasound Quarterly · doi:10.1097/ruq.0000000000000699 · Cited by 4 · see citing works
Enhancing Medical Question Answering with LLMs via a Hybrid Retrieval-Augmented Generation Framework
Aljohani, B., & Alsanoosy, T. (2026) · Information · doi:10.3390/info17020133 · Cited by 3 · see citing works
Avanços na Inteligência Artificial para o cuidado de úlceras em pessoas com Diabetes Mellitus
Apolinário, P. P., Zanchetta, F. C., Pittella, C. Q. P., & Lima, M. H. M. (2024) · Revista Enfermagem Atual In Derme · doi:10.31011/reaid-2024-v.98-n.3-art.2326 · Cited by 2 · see citing works
Makino, M., She, W. J., Siriaraya, P., Matoba, S., & Senoo, K. (2025) · Scientific Reports · doi:10.1038/s41598-025-34079-z · Cited by 1 · see citing works
Generative AI for Web Engineering Models in Physical Therapy
Siddiqui, H. A. F., Khan, N., Miran, S., Hakeem, A., Khan, M., Siraj, M., & Rehman, A. (2024) · Advances in web technologies and engineering book series · doi:10.4018/979-8-3693-3703-5.ch017
Darnell, S. S., Overall, R. W., Guarracino, A., Colonna, V., Garrison, E. P., Isaac, A., Lisso, F., Yousefi, S., Ashbrook, D. G., Huang, P.-S., et al. (2024) · Qeios · doi:10.32388/1amker
Qadir, A. M., Hassan, B. A., & Khalid, H. (2025) · International Journal of Computers and Applications · doi:10.1080/1206212x.2025.2537892
Thanthrige, A. N. W., & Wickramasinghe, N. S. (2025) · doi:10.2196/preprints.80582
Anghelescu, A., Munteanu, C., Anghelescu, L. A.-M., & Onose, G. (2025) · Balneo and PRM Research Journal · doi:10.12680/balneo.2025.847
Introduction: Implementation of Artificial Intelligence for Improving Healthcare
Hübner, U. H., Rubeis, G., Delaney, C. W., & Ball, M. J. (2026) · Health informatics · doi:10.1007/978-3-032-11938-4_1
Generative AI-Driven Continuous Monitoring and Management of Diabetes
Biswas, S., Sagar, A., Kumari, S., & Soni, A. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-15134-6_4
A Domain-specific Large Language Model for Diabetes Care and ManagementPreprint
Silinda, G. Z., Piotie, P. N., & Daramola, O. (2026) · Research Square · doi:10.21203/rs.3.rs-9563605/v1
Retrieval-augmented generation for medication safety: A case study using drug package inserts
Sun, Y., Wang, W., Cheng, W., Li, R., Liu, X., Pei, Q., Tang, J., & Yang, G. (2026) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2026.106599
Shakir, K., Sarlak, H., Rogati, G., Leardini, A., Berti, L., & Caravaggi, P. (2026) · Informatics · doi:10.3390/informatics13070117
Ratchakom, P., Rosjan, S., Piumsomboon, T., Pongsamakthai, M., & Intharah, T. (2026) · IEEE Access · doi:10.1109/access.2026.3730820
Salam, A. A., Chiranth, R., Benakappa, A., Shrivastava, S. K., & Rao, S. K. (2026) · International Journal of Computational and Experimental Science and Engineering · doi:10.22399/ijcesen.5499
- Authors
- Mashatian, S., Armstrong, D. G., Ritter, A., Robbins, J., Aziz, S., Alenabi, I., Huo, M., Anand, T., & Tavakolian, K.
- Affiliations
- University of North Dakota (US)
- University of Southern California (US)
- United States Department of Veterans Affairs (US)
- Silverberry Group, Inc., Grand Forks, ND, USA
- University of California, Los Angeles (US)
- Emory University (US)
- Corresponding author
- Mashatian, Shayan
- Venue
- Journal of Diabetes Science and Technology
- Publisher
- SAGE Publications
- Published
- 2024-05-20
- Volume
- 19
- Issue
- 5
- Pages
- 1264-1270
- ISSN
- 1932-2968 (ISSN-L)
- PMC
- PMC11572003
- OpenAlex
- W4398141327
- Cited by
- 28
- References
- 16
- Citations by year
- 2026: 9 · 2025: 13 · 2024: 6
- FWCI
- 2.62
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Journal of Diabetes Science and Technology (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institute of Diabetes and Digestive and Kidney Diseases (1R01124789-01A1)
- National Science Foundation Center to Stream Healthcare in Place (2052578)
- Silverberry Group, Inc.
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- large language models, diabetic foot care, diabetes self-management education, limb salvage, health literacy, patient education, GPT-4, prompt engineering, few-shot learning, medical knowledge extraction
- MeSH terms
- Generative Artificial Intelligence, Artificial Intelligence, Diabetes Mellitus, Humans, Health Knowledge, Attitudes, Practice, Patient Education as Topic, Diabetic Foot, Health Literacy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
