CoChat

Controlled and Real-Life Investigation of Optical Tracking Sensors in Smart Glasses for Monitoring Eating Behavior Using Deep Learning: Cross-Sectional Study (Preprint)

Open access

Preprint — published as 10.2196/59469

The DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.

Abstract

BACKGROUND: The increasing prevalence of obesity necessitates innovative approaches to better understand this health crisis, particularly given its strong connection to chronic diseases such as diabetes, cancer, and cardiovascular conditions. Monitoring dietary behavior is crucial for designing effective interventions that help decrease obesity prevalence and promote healthy lifestyles. However, traditional dietary tracking methods are limited by participant burden and recall bias. Exploring microlevel eating activities, such as meal duration and chewing frequency, in addition to eating episodes, is crucial due to their substantial relation to obesity and disease risk. OBJECTIVE: The primary objective of the study was to develop an accurate and noninvasive system for automatically monitoring eating and chewing activities using sensor-equipped smart glasses. The system distinguishes chewing from other facial activities, such as speaking and teeth clenching. The secondary objective was to evaluate the system’s performance on unseen test users using a combination of laboratory-controlled and real-life user studies. Unlike state-of-the-art studies that focus on detecting full eating episodes, our approach provides a more granular analysis by specifically detecting chewing segments within each eating episode. METHODS: The study uses OCO optical sensors embedded in smart glasses to monitor facial muscle activations related to eating and chewing activities. The sensors measure relative movements on the skin’s surface in 2 dimensions (X and Y). Data from these sensors are analyzed using deep learning (DL) to distinguish chewing from other facial activities. To address the temporal dependence between chewing events in real life, we integrate a hidden Markov model as an additional component that analyzes the output from the DL model. RESULTS: Statistical tests of mean sensor activations revealed statistically significant differences across all 6 comparison pairs (P<.001) involving 2 sensors (cheeks and temple) and 3 facial activities (eating, clenching, and speaking). These results demonstrate the sensitivity of the sensor data. Furthermore, the convolutional long short-term memory model, which is a combination of convolutional and long short-term memory neural networks, emerged as the best-performing DL model for chewing detection. In controlled laboratory settings, the model achieved an F1-score of 0.91, demonstrating robust performance. In real-life scenarios, the system demonstrated high precision (0.95) and recall (0.82) for detecting eating segments. The chewing rates and the number of chews evaluated in the real-life study showed consistency with expected real-life eating behaviors. CONCLUSIONS: The study represents a substantial advancement in dietary monitoring and health technology. By providing a reliable and noninvasive method for tracking eating behavior, it has the potential to revolutionize how dietary data are collected and used. This could lead to more effective health interventions and a better understanding of the factors influencing eating habits and their health implications.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.2196/preprints.59469

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 (A) The 2 types of facial muscles related to chewing (temporalis and zygomaticus); (B) monitored skin areas by the smart glasses; and (C) the placement of OCO sensors within the glasses frame.
Figure 1 (A) The 2 types of facial muscles related to chewing (temporalis and zygomaticus); (B) monitored skin areas by the smart glasses; and (C) the placement of OCO sensors within the glasses frame.CC BY · PubMed Central
Figure 2 Overview of the developed method for detection of chewing and eating segments estimation.
Figure 2 Overview of the developed method for detection of chewing and eating segments estimation.CC BY · PubMed Central
Figure 3 Processing steps for detecting the number of chews in an eating segment: (A) cheek and temple signals; (B) selection of signal with highest root mean square (RMS) value; (C) filtering the chosen signal; (D) detection of peaks and chews in the filtered signal.
Figure 3 Processing steps for detecting the number of chews in an eating segment: (A) cheek and temple signals; (B) selection of signal with highest root mean square (RMS) value; (C) filtering the chosen signal; (D) detection of peaks and chews in the filtered signal.CC BY · PubMed Central
Figure 4 Sensor signals from the sensors on the right cheek and temple recorded during eating, speaking, and teeth clenching activity performed by one participant.
Figure 4 Sensor signals from the sensors on the right cheek and temple recorded during eating, speaking, and teeth clenching activity performed by one participant.CC BY · PubMed Central
Figure 5 Wilcoxon signed-rank (paired) test with Bonferroni correction for comparing mean cheek and temple movements during activity pairs (n=28): clenching versus eating, clenching versus speaking, and eating versus speaking. Statistical significance annotations: *If P ∈ {.05, .01); **if P ∈ {.01, .001); ***if P ∈ {.001, .0001); and ****if P≤.001.
Figure 5 Wilcoxon signed-rank (paired) test with Bonferroni correction for comparing mean cheek and temple movements during activity pairs (n=28): clenching versus eating, clenching versus speaking, and eating versus speaking. Statistical significance annotations: *If P ∈ {.05, .01); **if P ∈ {.01, .001); ***if P ∈ {.001, .0001); and ****if P≤.001.CC BY · PubMed Central
Figure 6 Confusion matrices for the evaluated deep learning architectures: (A) convolutional neural network (CNN) 1D; (B) CNN 2D; (C) attention model; (D) convolutional long short-term memory (ConvLSTM).
Figure 6 Confusion matrices for the evaluated deep learning architectures: (A) convolutional neural network (CNN) 1D; (B) CNN 2D; (C) attention model; (D) convolutional long short-term memory (ConvLSTM).CC BY · PubMed Central
Figure 7 Confusion matrix for the chewing detection model evaluated on the real-life data set on window level.
Figure 7 Confusion matrix for the chewing detection model evaluated on the real-life data set on window level.CC BY · PubMed Central
Figure 8 Probability density function of the model’s output probabilities for the chewing-labeled instances.
Figure 8 Probability density function of the model’s output probabilities for the chewing-labeled instances.CC BY · PubMed Central
Figure 9 Distribution of estimated number of chews and chewing rates per eating segment in the real-life data set.
Figure 9 Distribution of estimated number of chews and chewing rates per eating segment in the real-life data set.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 1 references

Research works printed but not linked: 42

  1. 10.1016/j.nutres.2018.06.002 · DOI · Paper page
  2. 10.3945/jn.116.242552 · DOI · Paper page
  3. 10.1093/ajcn/nqab235 · DOI
  4. 10.2147/cia.s5697 · DOI · Paper page
  5. 10.1001/jama.2018.3060 · DOI · Paper page
  6. 10.1093/oxfordjournals.aje.a114086 · DOI · Paper page
  7. 10.1093/acprof:oso/9780199754038.001.0001 · DOI · Paper page
  8. 10.1017/s1368980018002951 · DOI · Paper page
  9. 10.1136/bjsports-2015-095538 · DOI
  10. 10.3389/fnut.2017.00031 · DOI
  11. 10.1038/ejcn.2015.204 · DOI · Paper page
  12. 10.1136/bmj.a2002 · DOI · Paper page
  13. 10.3389/fnut.2022.941001 · DOI
  14. 10.1016/j.ypmed.2007.07.031 · DOI
  15. 10.3390/nu12061734 · DOI
  16. 10.3390/computers6010004 · DOI · Paper page
  17. 10.3390/nu11051168 · DOI · Paper page
  18. 10.1038/s41746-020-0246-2 · DOI · Paper page
  19. 10.1145/3544794.3558476 · DOI
  20. 10.1145/3313831.3376869 · DOI · Paper page
  21. 10.1145/3491102.3502041 · DOI
  22. 10.1145/3594739.3610793 · DOI
  23. 10.3390/s16071067 · DOI · Paper page
  24. 10.1038/srep41690 · DOI
  25. 10.1109/jbhi.2017.2698523 · DOI
  26. 10.1145/2750858.2807545 · DOI · Paper page
  27. 10.1038/s41598-021-85163-z · DOI
  28. 10.1038/s41598-022-21456-1 · DOI
  29. 10.1145/3411763.3451741 · DOI
  30. 10.1145/3314410 · DOI
  31. 10.1145/3530884 · DOI
  32. 10.1145/3384657.3384801 · DOI
  33. 10.1145/3491102.3501925 · DOI
  34. 10.1145/3458709.3458945 · DOI
  35. 10.1109/mprv.2023.3276471 · DOI
  36. 10.1145/2971763.2971764 · DOI · Paper page
  37. 10.1145/3267242.3267287 · DOI · Paper page
  38. 10.1016/j.asoc.2017.09.027 · DOI · Paper page
  39. 10.3390/s21051902 · DOI
  40. 10.1038/s41592-020-0772-5 · DOI · Paper page
  41. 10.1016/j.archoralbio.2008.09.006 · DOI · Paper page
  42. 10.3390/electronics5040062 · DOI

Metadata

Authors
Stankoski, S., Kiprijanovska, I., Gjoreski, M., Panchevski, F., Sazdov, B., Sofronievski, B., Cleal, A., Fatoorechi, M., Nduka, C., & Gjoreski, H.
Affiliations
  • Emteq (United Kingdom) (GB)
  • Università della Svizzera italiana (CH)
Corresponding author
Stankoski, Simon
Venue
JMIR mhealth and uhealth
Published
2024-04-12
Volume
12
Pages
e59469-e59469
ISSN
2291-5222 (ISSN-L)
OpenAlex
W4401898631
Cited by
14
References
1
Citations by year
2026: 4 · 2025: 8 · 2024: 2
FWCI
1.96
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • JMIR mhealth and uhealth (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • National Science Foundation
  • Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (216405)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
  • Physical Sciences › Engineering › Biomedical Engineering › Non-Invasive Vital Sign Monitoring
  • Physical Sciences › Computer Science › Human-Computer Interaction › Gaze Tracking and Assistive Technology
Keywords
smart glasses, eating behavior monitoring, chewing detection, facial muscle activation, convolutional long short-term memory, hidden Markov models, obesity, dietary monitoring, non-invasive monitoring, chewing frequency, meal duration, teeth clenching, wearable sensors, health technology, eating episodes, convolutional neural networks
MeSH terms
Deep Learning, Smart Glasses, Adult, Cross-Sectional Studies, Feeding Behavior, Female, Humans, Male, Mastication, Monitoring, Physiologic

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Controlled and Real-Life Investigation of Optical Tracking Sensors in Smart Glasses for Monitoring Eating Behavior Using Deep Learning: Cross-Sectional Study (Preprint). https://app.cochat.ai/papers/10.2196/preprints.59469
The paper
Stankoski, S., Kiprijanovska, I., Gjoreski, M., Panchevski, F., Sazdov, B., Sofronievski, B., Cleal, A., Fatoorechi, M., Nduka, C., & Gjoreski, H. (2024). Controlled and Real-Life Investigation of Optical Tracking Sensors in Smart Glasses for Monitoring Eating Behavior Using Deep Learning: Cross-Sectional Study (Preprint). JMIR mhealth and uhealth. https://doi.org/10.2196/preprints.59469

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.