Automated Diet Capture Using Voice Alerts and Speech Recognition on Smartphones: Pilot Usability and Acceptability Study
Open accessPublished version of a preprint
Abstract
BACKGROUND: Effective monitoring of dietary habits is critical for promoting healthy lifestyles and preventing or delaying the onset and progression of diet-related diseases, such as type 2 diabetes. Recent advances in speech recognition technologies and natural language processing present new possibilities for automated diet capture; however, further exploration is necessary to assess the usability and acceptability of such technologies for diet logging. OBJECTIVE: This study explores the usability and acceptability of speech recognition technologies and natural language processing for automated diet logging. METHODS: We designed and developed base2Diet-an iOS smartphone application that prompts users to log their food intake using voice or text. To compare the effectiveness of the 2 diet logging modes, we conducted a 28-day pilot study with 2 arms and 2 phases. A total of 18 participants were included in the study, with 9 participants in each arm (text: n=9, voice: n=9). During phase I of the study, all 18 participants received reminders for breakfast, lunch, and dinner at preselected times. At the beginning of phase II, all participants were given the option to choose 3 times during the day to receive 3 times daily reminders to log their food intake for the remainder of the phase, with the ability to modify the selected times at any point before the end of the study. RESULTS: The total number of distinct diet logging events per participant was 1.7 times higher in the voice arm than in the text arm (P=.03, unpaired t test). Similarly, the total number of active days per participant was 1.5 times higher in the voice arm than in the text arm (P=.04, unpaired t test). Furthermore, the text arm had a higher attrition rate than the voice arm, with only 1 participant dropping out of the study in the voice arm, while 5 participants dropped out in the text arm. CONCLUSIONS: The results of this pilot study demonstrate the potential of voice technologies in automated diet capturing using smartphones. Our findings suggest that voice-based diet logging is more effective and better received by users compared to traditional text-based methods, underscoring the need for further research in this area. These insights carry significant implications for the development of more effective and accessible tools for monitoring dietary habits and promoting healthy lifestyle choices.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Nutritionix has been widely used as an NLP nutrition tracking system in prior research.
“Nutritionix has been widely used as an NLP nutrition tracking system in prior research [52,53].”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Global Burden of Cardiovascular Diseases and Risk Factors, 1990–2019
Roth, G. A., Mensah, G. A., Johnson, C. O., Addolorato, G., Ammirati, E., Baddour, L. M., Barengo, N. C., Beaton, A. Z., Benjamin, E. J., Benziger, C. P., et al. (2020) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2020.11.010 · Cited by 11,449 · see citing works
Saeedi, P., Petersohn, I., Salpea, P., Malanda, B., Karuranga, S., Unwin, N., Colagiuri, S., Guariguata, L., Motala, A. A., Ogurtsova, K., et al. (2019) · Diabetes Research and Clinical Practice · doi:10.1016/j.diabres.2019.107843 · Cited by 10,792 · see citing works
The role of diet in the prevention of type 2 diabetes
Salas‐Salvadó, J., Martínez‐González, M. Á., Bulló, M., & Ros, E. (2011) · Nutrition Metabolism and Cardiovascular Diseases · doi:10.1016/j.numecd.2011.03.009 · Cited by 341 · see citing works
Use of Intelligent Voice Assistants by Older Adults with Low Technology Use
Pradhan, A., Lazar, A., & Findlater, L. (2020) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3373759 · Cited by 285 · see citing works
40-year trends in meal and snack eating behaviors of American adults
Kant, A. K., & Graubard, B. I. (2015) · Journal of the Academy of Nutrition and Dietetics · doi:10.1016/j.jand.2014.06.354 · Cited by 271 · see citing works
Reversing Type 2 Diabetes: A Narrative Review of the Evidence
Hallberg, S. J., Gershuni, V. M., Hazbun, T. L., & Athinarayanan, S. J. (2019) · Nutrients · doi:10.3390/nu11040766 · Cited by 170 · see citing works
Rethinking the Mobile Food Journal
Cordeiro, F., Bales, E., Cherry, E., & Fogarty, J. A. (2015) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702154 · Cited by 166 · see citing works
Mobile Apps for Weight Management: A Review of the Latest Evidence to Inform Practice
Ghelani, D. P., Moran, L., Johnson, C., Mousa, A., & Naderpoor, N. (2020) · Frontiers in Endocrinology · doi:10.3389/fendo.2020.00412 · Cited by 125 · see citing works
Roache, S. A., & Gostin, L. O. (2017) · International Journal of Health Policy and Management · doi:10.15171/ijhpm.2017.69 · Cited by 87 · see citing works
Automatic diet monitoring: a review of computer vision and wearable sensor-based methods
Hassannejad, H., Matrella, G., Ciampolini, P., Munari, I. D., Mordonini, M., & Cagnoni, S. (2017) · International Journal of Food Sciences and Nutrition · doi:10.1080/09637486.2017.1283683 · Cited by 75 · see citing works
Unobtrusive and Wearable Systems for Automatic Dietary Monitoring
Prioleau, T., Moore, E., & Ghovanloo, M. (2017) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2016.2631246 · Cited by 69 · see citing works
Palanica, A., Thommandram, A., Lee, A., Li, M., & Fossat, Y. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0133-x · Cited by 63 · see citing works
Fung, T. T., Long, M. W., Hung, P., & Cheung, L. (2016) · Journal of the Academy of Nutrition and Dietetics · doi:10.1016/j.jand.2016.03.013 · Cited by 61 · see citing works
Empirical Analysis of Bias in Voice-based Personal Assistants
Lima, L., Furtado, V., Furtado, E., & Almeida, V. A. F. (2019) · The Web Conference · doi:10.1145/3308560.3317597 · Cited by 46 · see citing works
Speech2Health: A Mobile Framework for Monitoring Dietary Composition From Spoken Data
Hezarjaribi, N., Mazrouee, S., & Ghasemzadeh, H. (2018) · IEEE journal of biomedical and health informatics · doi:10.1109/jbhi.2017.2709333 · Cited by 45 · see citing works
Low-carbohydrate diets for the treatment of obesity and type 2 diabetes
Hall, K., & Chung, S. (2018) · Current opinion in clinical nutrition and metabolic care · doi:10.1097/mco.0000000000000470 · Cited by 43 · see citing works
Liu, Y.-C., Chen, C.-H., Lin, Y.-S., Chen, H.-Y., Irianti, D., Jen, T.-N., Yeh, J.-Y., & Chiu, S. Y.-H. (2020) · JMIR mHealth and uHealth · doi:10.2196/20317 · Cited by 41 · see citing works
Brinkel, J., May, J., Krumkamp, R., Lamshöft, M., Kreuels, B., Owusu‐Dabo, E., Mohammed, A., Marinović, A. B., Dako‐Gyeke, P., Krämer, A., et al. (2017) · Tropical Medicine & International Health · doi:10.1111/tmi.12864 · Cited by 38 · see citing works
Dodd, R., Santos, J. A., Tan, M., Campbell, N. R. C., Mhurchú, C. N., Cobb, L. K., Jacobson, M. F., He, F. J., Trieu, K., Osornprasop, S., et al. (2020) · Advances in Nutrition · doi:10.1093/advances/nmaa067 · Cited by 38 · see citing works
Potential Policy Approaches to Address Diet-Related Diseases
Jacobson, M. F., Krieger, J. W., & Brownell, K. D. (2018) · JAMA · doi:10.1001/jama.2018.7434 · Cited by 37 · see citing works
Byonanebye, D. M., Nabaggala, M. S., Naggirinya, A. B., Lamorde, M., Oseku, E., King, R. L., Owarwo, N. C., Laker, E. A. O., Orama, R., Castelnuovo, B., et al. (2021) · JMIR mhealth and uhealth · doi:10.2196/22229 · Cited by 37 · see citing works
Plante, T. B., O’Kelly, A. C., MacFarlane, Z. T., et al. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy060 · Cited by 27 · see citing works
Incidence of Obesity at 1 and 3 Years Among Community Dwelling Adults: A Population-Based Study
DeJesus, R. S., Croghan, I. T., Jacobson, D. J., Fan, C. L., & Sauver, J. L. S. (2022) · Journal of Primary Care & Community Health · doi:10.1177/21501319211068632 · Cited by 20 · see citing works
Fuchs, K., Haldimann, M. L., Vuckovac, D., & Ilic, A. (2018) · International Conference on Information and Communication Technology Convergence (ICTC) · doi:10.1109/ictc.2018.8539468 · Cited by 16 · see citing works
Deep Cooking: Predicting Relative Food Ingredient Amounts from Images
Li, J., Guerrero, R., & Pavlović, V. (2019) · doi:10.1145/3347448.3357164 · Cited by 15 · see citing works
Lim, J.-H., Lim, C.-K.-M., İbrahim, I., Syahrul, J., Zabil, M. H. M., Zakaria, N. F., & Daud, Z. A. M. (2020) · JMIR mhealth and uhealth · doi:10.2196/13808 · Cited by 15 · see citing works
Miller, H., Berger, M. B., Askew, S., Kay, M. C., Hopkins, C. M., Iragavarapu, M. S., de Leon, M., Freed, M., Barnes, C. N., Yang, Q., et al. (2021) · Contemporary Clinical Trials · doi:10.1016/j.cct.2021.106539 · Cited by 15 · see citing works
“What if it's wrong?” Ovulation and fertility understanding of menstrual app users
Hohmann‐Marriott, B. E., & Starling, L. (2022) · SSM - Qualitative Research in Health · doi:10.1016/j.ssmqr.2022.100057 · Cited by 12 · see citing works
BLIND ASSIST : A One Stop Mobile Application for the Visually Impaired
Shreya, P., Shreyas, N., Pushya, D., & Uma Maheswar Reddy, N. (2021) · IEEE Pune Section International Conference (PuneCon) · doi:10.1109/punecon52575.2021.9686476 · Cited by 11 · see citing works
Challenges and Opportunities in Automated Detection of Eating Activity
Thomaz, E., Essa, I. A., & Abowd, G. D. (2017) · Springer eBooks · doi:10.1007/978-3-319-51394-2_9 · Cited by 9 · see citing works
Towards an API for User Attention Prediction in Mobile Notification Overload
da Silva, A. V. D., & Vieira, V. (2018) · doi:10.5753/webmedia.2018.4552 · Cited by 4 · see citing works
Millard, L. A. C., Johnson, L., Neaves, S., Flach, P. A., Tilling, K. M., & Lawlor, D. A. (2023) · JMIR mhealth and uhealth · doi:10.2196/41117 · Cited by 3 · see citing works
Voice-Assisted Food Recall Using Voice Assistants
Liang, X., Batsis, J. A., Yuan, J., et al. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-17902-0_7 · Cited by 3 · see citing works
Flagged first, then by citations
Research works printed but not linked: 5
- [3]IDF Diabetes Atlas, 10th edition 2022-10-23 https://diabetesatlas.org/
- [15]Bitesnap - Photo Food Journal 2023-01-20 https://getbitesnap.com
- [20]Naphtal R Natural language processing based nutritional application (Thesis) 2015 Massachusetts Department of Electrical Engineering and Computer Science, Massachusetts Institute of Technology
- [21]He Y Hakguder Z Shi X Mazzeo PL Frontoni E Sclaroff S Distante C Smart diet management through food image cooking recipe analysis Image Analysis and Processing. ICIAP 2022 Workshops. Lecture Notes in Computer Science, Vol 13373 2022 Cham Springer 82 93
- [40]Lowdermilk T User-Centered Design: A Developer's Guide to Building User-Friendly Applications 2013 Sebastopol, California O'Reilly Media, Inc
Cited by
Sorted by citations
Mehta, T., John, T., El Zein, A., Faught, V., Nawshin, T., Chilke, T. S., Cohen, C. W., Cherrington, A., & Thirumalai, M. (2025) · JMIR Research Protocols · doi:10.2196/70271 · Cited by 2 · see citing works
- Authors
- Chikwetu, L., Daily, S., Mortazavi, B. J., & Dunn, J.
- Affiliations
- Duke University (US)
- Texas A&M University (US)
- Corresponding author
- Dunn, Jessilyn
- Venue
- JMIR Formative Research
- Publisher
- JMIR Publications Inc.
- Published
- 2023-05-16
- Volume
- 7
- Pages
- e46659
- ISSN
- 2561-326X (ISSN-L)
- PMC
- PMC10230351
- OpenAlex
- W4376643274
- Cited by
- 8
- References
- 33
- Citations by year
- 2026: 1 · 2025: 3 · 2024: 4
- FWCI
- 2.83
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- JMIR Formative Research (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Science Foundation (1851173)
- Topics
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Health Sciences › Health Professions › Health Information Management › Innovation in Digital Healthcare Systems
- Keywords
- speech recognition, natural language processing, mobile application, usability, acceptability, type 2 diabetes prevention, user attrition
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
