Multimodal Ambulatory Sleep Detection Using LSTM Recurrent Neural Networks
Open accessThe DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
Unobtrusive and accurate ambulatory methods are needed to monitor long-term sleep patterns for improving health. Previously developed ambulatory sleep detection methods rely either in whole or in part on self-reported diary data as ground truth, which is a problem, since people often do not fill them out accurately. This paper presents an algorithm that uses multimodal data from smart-phones and wearable technologies to detect sleep/wake state and sleep onset/offset using a type of recurrent neural network with long-short-term memory (LSTM) cells for synthesizing temporal information. We collected 5580 days of multimodal data from 186 participants and compared the new method for sleep/wake classification and sleep onset/offset detection to, first, nontemporal machine learning methods and, second, a state-of-the-art actigraphy software. The new LSTM method achieved a sleep/wake classification accuracy of 96.5%, and sleep onset/offset detection F1 scores of 0.86 and 0.84, respectively, with mean absolute errors of 5.0 and 5.5 min, respectively, when compared with sleep/wake state and sleep onset/offset assessed using actigraphy and sleep diaries. The LSTM results were statistically superior to those from non-temporal machine learning algorithms and the actigraphy software. We show good generalization of the new algorithm by comparing participant-dependent and participant-independent models, and we show how to make the model nearly realtime with slightly reduced performance.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
The Role of Actigraphy in the Study of Sleep and Circadian Rhythms
Ancoli-Israel, S., Cole, R., Alessi, C., Chambers, M., Moorcroft, W., & Pollak, C. P. (2003) · Sleep · doi:10.1093/sleep/26.3.342 · Cited by 2,751 · cited 1×
“Actigraphy [19], [20] is based on the observation that there is less movement during sleep and more movement during wake [21], [22].”
The consensus sleep diary: standardizing prospective sleep self-monitoring
Carney, C. E., Buysse, D. J., Ancoli-Israel, S., Edinger, J. D., Krystal, A. D., Lichstein, K. L., & Morin, C. M. (2012) · Sleep · doi:10.5665/sleep.1642 · Cited by 2,172 · cited 1×
“Sleep diaries and questionnaires [23] also have several drawbacks, namely users’ adherence and reporting bias [23], [24].”
The role and validity of actigraphy in sleep medicine: an update
Sadeh, A. (2011) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2010.10.001 · Cited by 1,415 · cited 1×
“ment disorders [37] and shift workers [38], [39]).”
Sukhbaatar, S., Szlam, A., Weston, J., & Fergus, R. (2015) · arXiv · doi:10.48550/arxiv.1503.08895 · Cited by 1,345
Sleep Deprivation and Vigilant Attention
Lim, J., & Dinges, D. F. (2008) · Annals of the New York Academy of Sciences · doi:10.1196/annals.1417.002 · Cited by 1,213 · cited 1×
“disturbs mood [5], leads to slowed reaction time [6] and attentional failures [7], [8], and compromises memory and learning [9].”
Graves, A., Wayne, G., & Danihelka, I. (2014) · arXiv · doi:10.48550/arxiv.1410.5401 · Cited by 1,058 · cited 1×
“Finally, compared with implicit knowledge learned in LSTM, RNNs with explicit memory [57], [58] can memorize facts and common sense, which are potentially useful for the task of sleep detection.”
Metabolic consequences of sleep and sleep loss
Van Cauter, E., Spiegel, K., Tasali, E., & Leproult, R. (2008) · Sleep Medicine · doi:10.1016/s1389-9457(08)70013-3 · Cited by 812
Impact of insufficient sleep on total daily energy expenditure, food intake, and weight gain
Markwald, R. R., Melanson, E. L., Smith, M. R., Higgins, J., Perreault, L., Eckel, R. H., & Wright, K. P. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1216951110 · Cited by 810 · cited 1×
“increases appetite and food intake [2], decreases insulin sensitivity and glucose tolerance [3], impairs immune function [4],”
Visual discrimination learning requires sleep after training
Stickgold, R., James, L., & Hobson, J. A. (2000) · Nature Neuroscience · doi:10.1038/81756 · Cited by 715 · cited 1×
“disturbs mood [5], leads to slowed reaction time [6] and attentional failures [7], [8], and compromises memory and learning [9].”
Research works printed but not linked: 34
- 10.1093/sleep/22.3.378 · DOI · Paper page
- lee, 2015, Consumer sleep tracking devices: A critical review, Stud Health Technol Inform, 210, 458
- 10.5664/jcsm.5498 · DOI · Paper page
- 10.1109/BHI.2017.7897306 · DOI
- 10.1093/sleep/15.4.293 · DOI · Paper page
- 10.1016/S1389-9457(00)00098-8 · DOI · Paper page
- 10.5665/sleep.4918 · DOI · Paper page
- 10.1145/2556288.2557220 · DOI · Paper page
- 10.1016/j.sleh.2014.12.008 · DOI · Paper page
- 10.1053/smrv.2001.0182 · DOI · Paper page
- 10.1016/j.brainresbull.2003.12.012 · doi:10.1016/j.brainresbull.2003.12.012
- 10.1016/0014-4886(68)90025-3 · DOI
- 10.1093/sleep/15.5.461 · DOI · Paper page
- 10.2170/jjphysiol.14.365 · DOI
- 10.1016/j.ijpsycho.2014.09.011 · DOI · Paper page
- 10.1371/journal.pone.0061142 · DOI · Paper page
- hinton, 0, Rmsprop: Divide the gradient by a running average of its recent magnitude, Neural Netw Mach Learning
- 10.1002/0471751723.ch10 · DOI
- 10.1016/j.sleh.2014.12.011 · DOI · Paper page
- sadeh, 1989, Actigraphically-based automatic bedtime sleep-wake scoring: Validity and clinical applications, J Ambulatory Monitoring, 2, 209
- 10.1378/chest.121.3.753 · DOI
- 10.1007/s10439-015-1290-y · doi:10.1007/s10439-015-1290-y
- 10.1371/journal.pone.0084139 · DOI
- 10.1145/2746285.2746303 · DOI · Paper page
- 10.1037/h0033615 · DOI
- dinges, 1997, Cumulative sleepiness, mood disturbance, and psychomotor vigilance performance decrements during a week of sleep restricted to 4-5 hours per night, Sleep, 20, 267
- doran, 2001, Sustained attention performance during sleep deprivation: Evidence of state instability, Archives Italiennes de Biologie, 139, 253
- 10.1109/TNSRE.2017.2733220 · DOI · Paper page
- sathyanarayana, 2016, Sleep quality prediction from wearable data using deep learning, JMIR Mhealth Uhealth, 4
- 10.1016/j.pmcj.2011.09.004 · DOI · Paper page
- graves, 0, Towards end-to-end speech recognition with recurrent neural networks, Proc Int Conf Mach Learn, 32, 1764
- 10.2196/jmir.6821 · DOI
- 10.3115/v1/W14-4012 · DOI · Paper page
- sutskever, 0, Sequence to sequence learning with neural networks, Proc Neural Inf Process Syst, 3104
Cited by
A survey on deep learning for multimodal data fusion
Gao, J., Li, P., Chen, Z., & Zhang, J. (2020) · Neural Computation · doi:10.1162/neco_a_01273 · Cited by 821
Bayoudh, K., Knani, R., Hamdaoui, F., & Mtibaa, A. (2022) · The Visual Computer · doi:10.1007/s00371-021-02166-7 · Cited by 456
Deep multi-view learning methods: A review
Yan, X., Hu, S., Mao, Y., Ye, Y., & Yu, H. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.03.090 · Cited by 322
Deep learning based multimodal biomedical data fusion: an overview and comparative review
Duan, J., Xiong, J., Li, Y., & Ding, W. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102536 · Cited by 284
Sleep stage classification from heart-rate variability using long short-term memory neural networks
Radha, M., Fonseca, P., Moreau, A., Ross, M., Cerny, A., Anderer, P., Long, X., & Aarts, R. M. (2019) · Scientific Reports · doi:10.1038/s41598-019-49703-y · Cited by 186
“With non-EEG data, Sano et al .”
Metadata
- Authors
- Sano, A., Chen, W., Lopez-Martinez, D., Taylor, S., & Picard, R. W.
- Venue
- IEEE Journal of Biomedical and Health Informatics
- Published
- 2019-07
- Volume
- 23
- Issue
- 4
- Pages
- 1607-1617
- ISSN
- 2168-2194 (ISSN-L)
- 2168-2208
- OpenAlex
- W2889162782
- Cited by
- 70
- References
- 19
- FWCI
- 2.87
- Type
- journal article
- Language
- en
- Open access
- green
- Licence
- Funders
- National Institutes of Health (R01GM105018)
- Samsung Electronics
- NEC
- MIT Media Lab Consortium
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Multimodal Ambulatory Sleep Detection Using LSTM Recurrent Neural Networks. https://app.cochat.ai/papers/10.1109/jbhi.2018.2867619
- The paper
- Sano, A., Chen, W., Lopez-Martinez, D., Taylor, S., & Picard, R. W. (2019). Multimodal Ambulatory Sleep Detection Using LSTM Recurrent Neural Networks. IEEE Journal of Biomedical and Health Informatics. https://doi.org/10.1109/jbhi.2018.2867619
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.