A comparison between audio and IMU data to detect chewing events based on an earable device
Abstract
The feasibility of collecting various data from built-in wearable sensors has enticed many researchers to use these devices for analyzing human activities and behaviors. In particular, audio, video, and motion data have been utilized for automatic dietary monitoring. In this paper, we investigate the feasibility of detecting chewing activities based on audio and inertial sensor data obtained from an ear-worn device, eSense. We process each sensor data separately and determine the accuracy of each sensing modality for chewing detection when using MFCC and Spectral Centroid as features and Logistic Regression, Decision Tree, and Random Forest as classifiers. We also measure the performance of chewing detection when fusing features extracted from both audio and inertial sensor data. We evaluate the chewing detection algorithm by running a pilot study inside a lab environment on a total of 5 participants. This consists of 130 minutes audio and inertial measurement unit (IMU) data. The results of this study indicate that an in-ear IMU with an accuracy of 95% outperforms audio data in detecting chewing and fusing both modalities improves the accuracy to 97%.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Eating behaviours are analysed using audio collected from smartwatches and earables.
“Eating behaviours are also analysed using audio collected from smartwatches and earables [47,51,85].”
From Human-centred artificial intelligence for mobile health sensing: challenges and opportunities (2023) · cites this paper as [51]
Cites
An Introduction to Statistical Learning
James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013) · Springer Texts in Statistics · doi:10.1007/978-1-4614-7138-7 · Cited by 12,106 · see citing works
Ahmed, N., Natarajan, T., & Rao, K. R. (1974) · IEEE Transactions on Computers · doi:10.1109/t-c.1974.223784 · Cited by 5,156 · see citing works
A behavior model for persuasive design
Fogg, B. (2009) · Proceedings of the 4th International Conference on Persuasive Technology · doi:10.1145/1541948.1541999 · Cited by 2,328 · see citing works
An Introduction to Statistical Learning
James, G. M., Witten, D., Hastie, T. J., Tibshirani, R. J., & Taylor, J. (2023) · Springer texts in statistics · doi:10.1007/978-3-031-38747-0 · Cited by 1,765 · see citing works
A Scale for the Measurement of the Psychological Magnitude Pitch
Stevens, S. S., Volkmann, J., & Newman, E. B. (1937) · The Journal of the Acoustical Society of America · doi:10.1121/1.1915893 · Cited by 1,107 · cited 1× · see citing works
“…as follows: (1) Partition the signal into short frames (3 seconds) (2) Calculate the Discrete Fourier Transform for each frame (3) Apply mel filter bank [27] with 45 filters on magnitude spectrum obtained from step 2 and extract a 45 dimensional feature vector (4) Take the logarithm of all…”
MethodMel Frequency Cepstral Coefficients For Music Modeling.
Logan, B. (2000) · doi:10.5281/zenodo.1416443 · Cited by 928 · see citing works
An empirical study of wearable technology acceptance in healthcare
Gao, Y., Li, H., & Luo, Y. (2015) · Industrial Management & Data Systems · doi:10.1108/imds-03-2015-0087 · Cited by 673 · see citing works
Mathematics of the Discrete Fourier Transform (DFT) with Audio Applications
Smith, & Olaniyan, J. (2007) · Cited by 218 · see citing works
Analysis of Chewing Sounds for Dietary Monitoring
Amft, O., Stäger, M., Lukowicz, P., & Tröster, G. (2005) · Lecture Notes in Computer Science · doi:10.1007/11551201_4 · Cited by 216 · see citing works
Bedri, A., Li, R. Z., Haynes, M., Kosaraju, R. P., Grover, I., Prioleau, T., Beh, M. Y., Goel, M., Starner, T. E., & Abowd, G. D. (2017) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3130902 · Cited by 185 · see citing works
Non-invasive monitoring of chewing and swallowing for objective quantification of ingestive behavior
Sazonov, E. E., Schuckers, S., Lopez‐Meyer, P., Makeyev, O., Sazonova, N. A., Melanson, E. L., & Neuman, M. R. (2008) · Physiological Measurement · doi:10.1088/0967-3334/29/5/001 · Cited by 177 · see citing works
A Sensor System for Automatic Detection of Food Intake Through Non-Invasive Monitoring of Chewing
Sazonov, E. E., & Fontana, J. M. (2011) · IEEE Sensors Journal · doi:10.1109/jsen.2011.2172411 · Cited by 163 · see citing works
Earables for Personal-Scale Behavior Analytics
Kawsar, F., Min, C., Mathur, A., & Montanari, A. (2018) · IEEE Pervasive Computing · doi:10.1109/mprv.2018.03367740 · Cited by 141 · see citing works
Television viewing is associated with an increase in meal frequency in humans
Stroebele, N., & de Castro, J. M. (2003) · Appetite · doi:10.1016/j.appet.2003.09.001 · Cited by 114 · see citing works
A Novel Chewing Detection System Based on PPG, Audio, and Accelerometry
Papapanagiotou, V., Diou, C., Zhou, L., van den Boer, J., Mars, M., & Delopoulos, A. N. (2016) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2016.2625271 · Cited by 91 · see citing works
What Should You Know about Mindful and Intuitive Eating?
Mathieu, J. J. (2009) · Journal of the American Dietetic Association · doi:10.1016/j.jada.2009.10.023 · Cited by 89 · see citing works
Monroe, J. T. (2015) · American Journal of Lifestyle Medicine · doi:10.1177/1559827615569682 · Cited by 77 · see citing works
Accelerometer-Based Detection of Food Intake in Free-Living Individuals
Farooq, M., & Sazonov, E. E. (2018) · IEEE Sensors Journal · doi:10.1109/jsen.2018.2813996 · Cited by 67 · see citing works
iHear Food: Eating Detection Using Commodity Bluetooth Headsets
Gao, Y., Zhang, N., Wang, H., Ding, X., Ye, X., Chen, G., & Cao, Y. (2016) · 2016 IEEE First International Conference on Connected Health: Applications, Systems and Engineering Technologies (CHASE) · doi:10.1109/chase.2016.14 · Cited by 63 · see citing works
Detection of chewing from piezoelectric film sensor signals using ensemble classifiers
Farooq, M., & Sazonov, E. E. (2016) · doi:10.1109/embc.2016.7591833 · Cited by 41 · see citing works
Chewing detection from an in-ear microphone using convolutional neural networks
Papapanagiotou, V., Diou, C., & Delopoulos, A. N. (2017) · doi:10.1109/embc.2017.8037060 · Cited by 33 · see citing works
Eating detection and chews counting through sensing mastication muscle contraction
Wang, S., Zhou, G., Ma, Y., Hu, L., Chen, Z., Chen, Y., Zhao, H., & Jung, W. (2018) · Smart Health · doi:10.1016/j.smhl.2018.07.004 · Cited by 30 · see citing works
Acoustic vibration method for food texture evaluation using an accelerometer sensor
Iwatani, S.-I., Akimoto, H., & Sakurai, N. (2012) · Journal of Food Engineering · doi:10.1016/j.jfoodeng.2012.09.015 · Cited by 23 · see citing works
Wang, S., Zhou, G., Hu, L., Chen, Z., & Chen, Y. (2015) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/2800835.2800884 · Cited by 22 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Röddiger, T., Clarke, C., Breitling, P., Schneegans, T., Zhao, H., Gellersen, H. W., & Beigl, M. (2022) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3550314 · Cited by 104 · see citing works
Human-centred artificial intelligence for mobile health sensing: challenges and opportunities
Dang, T., Spathis, D., Ghosh, A., & Mascolo, C. (2023) · Royal Society Open Science · doi:10.1098/rsos.230806 · Cited by 30 · see citing works
I Am an Earphone and I Can Hear My User’s Face: Facial Landmark Tracking Using Smart Earphones
Zhang, S., Lu, T., Zhou, H., Liu, Y., Liu, R., & Gowda, M. (2023) · ACM Transactions on Internet of Things · doi:10.1145/3614438 · Cited by 23 · see citing works
Gashi, S., Saeed, A., Vicini, A., Lascio, E. D., & Silvia Santini, B. (2021) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3462244.3479921 · Cited by 22 · see citing works
Khan, M. I., Acharya, B., & Chaurasiya, R. K. (2022) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2022.106843 · Cited by 22 · see citing works
EarRumble: Discreet Hands- and Eyes-Free Input by Voluntary Tensor Tympani Muscle Contraction
Röddiger, T., Clarke, C., Wolffram, D., Budde, M., & Beigl, M. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445205 · Cited by 19 · see citing works
Detecting Verbal and Non-Verbal Gestures Using Earables
Laporte, M., Baglat, P., Gashi, S., Gjoreski, M., Silvia Santini, B., & Langheinrich, M. (2021) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3460418.3479322 · Cited by 19 · see citing works
Technology to Automatically Record Eating Behavior in Real Life: A Systematic Review
Hiraguchi, H., Perone, P., Toet, A., Camps, G., & Brouwer, A.-M. (2023) · Sensors · doi:10.3390/s23187757 · Cited by 17 · see citing works
Earable Multimodal Sensing and Stimulation: A Prospective Toward Unobtrusive Closed-Loop Biofeedback
Xu, Y., Uppal, A., Lee, M. S., Mahato, K., Wuerstle, B. L., Lin, M., Djassemi, O., Chen, T., Lin, R., Paul, A., et al. (2024) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2024.3508713 · Cited by 17 · see citing works
Audio-based Active and Assisted Living: A review of selected applications and future trends
Despotović, V., Počta, P., & Žgank, A. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.106027 · Cited by 13 · see citing works
Gu, X., Ou, L., Ong, D., & Wang, Y. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548411 · Cited by 12 · see citing works
Morshed, M. B., Haresamudram, H., Bandaru, D., Abowd, G., & Ploetz, T. (2022) · International Workshop on the Semantic Web · doi:10.1145/3544794.3558469 · Cited by 17 · see citing works
A Systematic Review of Sensor-Based Methods for Measurement of Eating Behavior
Hossain, D., Thomas, J. G., McCrory, M. A., HIGGINS, J. A., & Sazonov, E. E. (2025) · Sensors · doi:10.3390/s25102966 · Cited by 11 · see citing works
GustosonicSense: Towards understanding the design of playful gustosonic eating experiences
Wang, Y., Obie, H. O., Li, Z., Salim, F. D., Grundy, J., & Mueller, F. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642182 · Cited by 10 · see citing works
BiteSense: Earable-Based Inertial Sensing for Eating Behaviour Assessment
Chugh, G., Ghosh, I., Chakraborty, S., & Chakraborty, S. (2025) · IEEE International Conference on Pervasive Computing and Communications · doi:10.1109/percom64205.2025.00030 · Cited by 9 · see citing works
Design space and usability of earable prototyping
Röddiger, T., Beigl, M., & Exler, A. (2020) · International Symposium on Wearable Computers · doi:10.1145/3410531.3414302 · Cited by 8 · see citing works
Spicher, L., Bell, C. L., Sienko, K. H., & Huan, X. (2025) · Sensors · doi:10.3390/s25092944 · Cited by 7 · see citing works
IMChew: Chewing Analysis using Earphone Inertial Measurement Units
Ketmalasiri, T., Wu, Y. Y., Butkow, K.-J., Cecilia Mascolo, C., & Liu, Y. (2024) · Workshop on Body-Centric Computing Systems (BodySys) · doi:10.1145/3662009.3662022 · Cited by 6 · see citing works
Cognitive Load Monitoring via Earable Acoustic Sensing
Quan, J., Al-Naimi, K., Wei, X., Liu, Y., Kawsar, F., Montanari, A., & Dang, T. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10887838 · Cited by 3 · see citing works
Passive Sensors for Detection of Food Intake
Ghosh, T., & Sazonov, E. E. (2021) · Elsevier eBooks · doi:10.1016/b978-0-12-822548-6.00086-8 · Cited by 3 · see citing works
Motion-resilient Heart Rate Monitoring with In-ear Microphones.Preprint
Butkow, K.-J., Dang, T., Ferlini, A., Ma, D., & Cecilia Mascolo, C. (2021) · arXiv (Cornell University) · Cited by 1 · see citing works
LEIB-RHYTHM: An Auditory-Based Intervention for Enhancing Rhythmic Chewing Perception
Zheng, R., Xu, X., & Wan, T. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-93227-4_30
ArtEARial: Arterial Pressure Waveform Reconstruction Using Earbud Audio
Christofferson, K., Lin, M., Cafazzo, J. A., & Mariakakis, A. T. (2025) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3770634
Vibrational, Acoustic and Inertial Sensing on Smart Eyewear for Physiological Monitoring
Cortese, A., Mozzini, F., Casciano, L., Teliti, A., Rossi, M., Gervasoni, G., Antonello, N., & Caiani, E. G. (2026) · International Conference on Mobile Systems, Applications, and Services · doi:10.1145/3812836.3814765
- Authors
- Lotfi, R., Tzanetakis, G., Eskicioglu, R., & Irani, P.
- Affiliations
- University of Manitoba (CA)
- University of Victoria (CA)
- Venue
- Proceedings of the 11th Augmented Human International Conference
- Publisher
- ACM
- Published
- 2020-05-27
- Pages
- 1-8
- OpenAlex
- W3030248081
- Cited by
- 24
- References
- 25
- Citations by year
- 2026: 1 · 2025: 6 · 2024: 3 · 2023: 3 · 2022: 5 · 2021: 5 · 2020: 1
- FWCI
- 1.08
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Augmented Human International Conference (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Physical Sciences › Engineering › Biomedical Engineering › Non-Invasive Vital Sign Monitoring
- Keywords
- chewing detection, inertial measurement unit, audio data, multimodal sensor fusion, dietary monitoring, Mel-frequency cepstral coefficients, spectral centroid, activity recognition, wearable sensors
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
