An AI Dietitian for Type 2 Diabetes Mellitus Management Based on Large Language and Image Recognition Models: Preclinical Concept Validation Study
Open accessThe DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Abstract
Background: Nutritional management for patients with diabetes in China is a significant challenge due to the low supply of registered clinical dietitians. To address this, an artificial intelligence (AI)–based nutritionist program that uses advanced language and image recognition models was created. This program can identify ingredients from images of a patient’s meal and offer nutritional guidance and dietary recommendations. Objective: The primary objective of this study is to evaluate the competence of the models that support this program. Methods: The potential of an AI nutritionist program for patients with type 2 diabetes mellitus (T2DM) was evaluated through a multistep process. First, a survey was conducted among patients with T2DM and endocrinologists to identify knowledge gaps in dietary practices. ChatGPT and GPT 4.0 were then tested through the Chinese Registered Dietitian Examination to assess their proficiency in providing evidence-based dietary advice. ChatGPT’s responses to common questions about medical nutrition therapy were compared with expert responses by professional dietitians to evaluate its proficiency. The model’s food recommendations were scrutinized for consistency with expert advice. A deep learning–based image recognition model was developed for food identification at the ingredient level, and its performance was compared with existing models. Finally, a user-friendly app was developed, integrating the capabilities of language and image recognition models to potentially improve care for patients with T2DM. Results: Most patients (182/206, 88.4%) demanded more immediate and comprehensive nutritional management and education. Both ChatGPT and GPT 4.0 passed the Chinese Registered Dietitian examination. ChatGPT’s food recommendations were mainly in line with best practices, except for certain foods like root vegetables and dry beans. Professional dietitians’ reviews of ChatGPT’s responses to common questions were largely positive, with 162 out of 168 providing favorable reviews. The multilabel image recognition model evaluation showed that the Dino V2 model achieved an average F1 score of 0.825, indicating high accuracy in recognizing ingredients. Conclusions: The model evaluations were promising. The AI-based nutritionist program is now ready for a supervised pilot study.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000198 · Cited by 4,115 · cited 1×
“They have already shown their potential in medical writing and education [10,11].”
Kung, T. H., Cheatham, M., ChatGPT, Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., et al. (2022) · doi:10.1101/2022.12.19.22283643 · Cited by 4,115
Davies, M. J., D’Alessio, D. A., Fradkin, J., Kernan, W. N., Mathieu, C., Mingrone, G., Rossing, P., Tsapas, A., Wexler, D. J., & Buse, J. B. (2018) · Diabetes Care · doi:10.2337/dci18-0033 · Cited by 3,518
Sallam, M. (2023) · Healthcare · doi:10.3390/healthcare11060887 · Cited by 3,126 · cited 1×
“Advanced large language models (LLMs) like ChatGPT, a type of artificial intelligence (AI), can present medical information in a detailed and explicit manner through interactive dialogues, demonstrating great potential for development in the medical field [9].”
Ayers, J. W., Poliak, A., Dredze, M., Leas, E. C., Zhu, Z., Kelley, J. B., Faix, D. J., Goodman, A. M., Longhurst, C. A., Hogarth, M., et al. (2023) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2023.1838 · Cited by 2,613
Gilson, A., Safranek, C. W., Huang, T., Socrates, V., Chi, L., Taylor, R. A., & Chartash, D. (2023) · JMIR Medical Education · doi:10.2196/45312 · Cited by 2,078
Buse, J. B., Wexler, D. J., Tsapas, A., Rossing, P., Mingrone, G., Mathieu, C., D’Alessio, D. A., & Davies, M. J. (2020) · Diabetes Care · doi:10.2337/dci19-0066 · Cited by 1,192 · cited 1×
“Achieving diabetes remission can reduce the risk of these complications, and obtaining diabetes self-management education (DSME) and medical nutrition therapy (MNT) is crucial in attaining this goal [3,4].”
Quinn, C. C., Shardell, M. D., Terrin, M. L., Barr, E. A., Ballew, S. H., & Gruber-Baldini, A. L. (2011) · Diabetes Care · doi:10.2337/dc11-0366 · Cited by 650 · cited 1×
“Clinical research has shown that digital tools aimed to provide DSME and MNT can significantly lower patients’ hemoglobin A 1c levels [5].”
Consultation length in general practice: cross sectional study in six European countries
Deveugele, M., Derese, A., van den Brink-Muinen, A., Bensing, J., & De Maeseneer, J. (2002) · BMJ · doi:10.1136/bmj.325.7362.472 · Cited by 627 · cited 1×
“Moreover, research from Europe has also shown that the duration of patient interaction with nutritional educators is crucial for managing and reducing hemoglobin A 1c levels [25].”
ChatGPT and the future of medical writing
Biswas, S. (2023) · Radiology · doi:10.1148/radiol.223312 · Cited by 571 · cited 1×
“They have already shown their potential in medical writing and education [10,11].”
The evidence for the effectiveness of medical nutrition therapy in diabetes management
Pastors, J. G., Warshaw, H., Daly, A., Franz, M., & Kulkarni, K. (2002) · Diabetes Care · doi:10.2337/diacare.25.3.608 · Cited by 434
Goldenberg, J. Z., Day, A., Brinkworth, G. D., Sato, J., Yamada, S., Jönsson, T., Beardsley, J., Johnson, J. A., Thabane, L., & Johnston, B. C. (2021) · BMJ · doi:10.1136/bmj.m4743 · Cited by 428 · cited 1×
“The successful management of diabetes hinges mainly on the patients’ adherence to the dietary recommendations provided by their health care provider [31].”
Shan, R., Sarkar, S., & Martin, S. S. (2019) · Diabetologia · doi:10.1007/s00125-019-4864-7 · Cited by 339 · cited 2×
“This rapid and efficient process represents a significant improvement over traditional management applications that require manual input, thereby increasing the patient’s commitment to maintaining a consistent dietary log and overall diet [6,32].”
Johnson, S. B., King, A. J., Warner, E. L., Aneja, S., Kann, B. H., & Bylund, C. L. (2023) · JNCI Cancer Spectrum · doi:10.1093/jncics/pkad015 · Cited by 289
Agarwal, P., Mukerji, G., Desveaux, L., Ivers, N. M., Bhattacharyya, O., Hensel, J. M., Shaw, J., Bouck, Z., Jamieson, T., Onabajo, N., et al. (2019) · JMIR mHealth and uHealth · doi:10.2196/10321 · Cited by 208
Interventions for Reversing Prediabetes: A Systematic Review and Meta-Analysis
Galaviz, K. I., Weber, M. B., Suvada, K., Gujral, U. P., Wei, J., Merchant, R., Dharanendra, S., Haw, J. S., Narayan, K. M. V., & Ali, M. K. (2022) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2021.10.020 · Cited by 149 · cited 1×
“Although exercise is more effective than diet control in managing diabetes, dietary control alone still proves highly effective [23,24].”
Cited by
The Role of Artificial Intelligence in Nutrition Research: A Scoping Review
Sosa-Holwerda, A., Park, O.-H., Albracht-Schulte, K., Niraula, S., Thompson, L., & Oldewage-Theron, W. (2024) · Nutrients · doi:10.3390/nu16132066 · Cited by 114
“Figure 3 shows the nutrition areas in which the included articles used AI. by lifestyle interventions [26–28], and dietary assessment specifically for people [25,42].”
BackgroundMethodInfluentialWang, D., Liang, J., Ye, J., Li, J., Li, J., Zhang, Q., Hu, Q., Pan, C., Wang, D., Liu, Z., et al. (2024) · Journal of Medical Internet Research · doi:10.2196/58041 · Cited by 76
Ulug, E., Gunesli, I., Acıkgoz Pinar, A., & Yildiz, B. O. (2025) · Nutrition Research · doi:10.1016/j.nutres.2024.11.005 · Cited by 12
Metadata
- Authors
- Sun, H., Zhang, K., Lan, W., Gu, Q., Jiang, G., Yang, X., Qin, W., & Han, D.
- Venue
- Journal of Medical Internet Research
- Publisher
- JMIR Publications Inc.
- Published
- 2023-11-09
- Volume
- 25
- Pages
- e51300
- ISSN
- 1438-8871 (ISSN-L)
- OpenAlex
- W4387412164
- Cited by
- 101
- References
- 17
- FWCI
- 3.49
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.