CoChat
Sign Up
Cited by 36Sleep and related disordersContext-Aware Activity Recognition SystemsNon-Invasive Vital Sign Monitoring

Multi-Night at-Home Evaluation of Improved Sleep Detection and Classification with a Memory-Enhanced Consumer Sleep Tracker

Open access
Ghorbani, Shohreh, Golkashani, Hosein Aghayan, Chee, Nicholas IYN, Teo, Teck Boon, Dicom, Andrew Roshan, Yilmaz, Gizem, Leong, Ruth LF, Ong, Ju Lynn +1 more·2022·Nature and Science of Sleep
DOI 10.2147/nss.s359789PMC PMC9015046
Open this paper in CoChat — free accountOpen PDF

Abstract

Purpose: To evaluate the benefits of applying an improved sleep detection and staging algorithm on minimally processed multi-sensor wearable data collected from older generation hardware. Patients and Methods: Generation Oura Rings equipped with additional memory to store raw data from accelerometer, infra-red photoplethysmography and temperature sensors. 2-stage and 4-stage sleep classifications using a new machine-learning algorithm (Gen3) trained on a diverse and independent dataset were compared to the existing consumer algorithm (Gen2) for whole-night and epoch-by-epoch metrics. Results: Gen 3 outperformed its predecessor with a mean (SD) accuracy of 92.6% (0.04), sensitivity of 94.9% (0.03), and specificity of 78.5% (0.11); corresponding to a 3%, 2.8% and 6.2% improvement from Gen2 across the three nights, with Cohen's d values >0.39, t values >2.69, and p values <0.01. Notably, Gen 3 showed robust performance comparable to PSG in its assessment of sleep latency, light sleep, rapid eye movement (REM), and wake after sleep onset (WASO) duration. Participants <40 years of age benefited more from the upgrade with less measurement bias for total sleep time (TST), WASO, light sleep and sleep efficiency compared to those ≥40 years. Males showed greater improvements on TST and REM sleep measurement bias compared to females, while females benefitted more for deep sleep measures compared to males. Conclusion: These results affirm the benefits of applying machine learning and a diverse training dataset to improve sleep measurement of a consumer wearable device. Importantly, collecting raw data with appropriate hardware allows for future advancements in algorithm development or sleep physiology to be retrospectively applied to enhance the value of longitudinal sleep studies.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.2147/nss.s359789

Cites

9 of 9 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 46

  1. [1]Liang J, Xian D, Liu X, et al. Usability study of mainstream wearable fitness devices: feature analysis and system usability scale evaluation. JMIR mhealth uhealth. 2018; 6(11): e11066. doi: 10.2196/11066 30409767 PMC6250954 · DOI
  2. [3]de Zambotti M, Cellini N, Goldstone A, Colrain IM, Baker FC. Wearable sleep technology in clinical and research settings. Med Sci Sports Exerc. 2019; 51(7): 1538– 1557. doi: 10.1249/MSS.0000000000001947 30789439 PMC6579636 · DOI · Paper page
  3. [4]Chong KPL, Guo JZ, Deng X, Woo BKP. Consumer perceptions of wearable technology devices: retrospective review and analysis. JMIR mhealth uhealth. 2020; 8(4): e17544. doi: 10.2196/17544 32310148 PMC7199133 · DOI
  4. [6]Grandner MA, Lujan MR, Ghani SB. Sleep-tracking technology in scientific research: looking to the future. Sleep. 2021; 44(5). doi: 10.1093/sleep/zsab071 PMC8343587 33782687 · DOI
  5. [7]de Zambotti M, Cellini N, Menghini L, Sarlo M, Baker FC. Sensors capabilities, performance, and use of consumer sleep technology. Sleep Med Clin. 2020; 15(1): 1– 30. doi: 10.1016/j.jsmc.2019.11.003 32005346 PMC7482551 · DOI
  6. [8]Van de Water ALEXANDERTM, Holmes A, Hurley DA. Objective measurements of sleep for non-laboratory settings as alternatives to polysomnography - A systematic review. J Sleep Res. 2011; 20(1pt2): 183– 200. doi: 10.1111/j.1365-2869.2009.00814.x 20374444 · DOI
  7. [9]Meltzer LJ, Hiruma LS, Avis K, Montgomery-Downs H, Valentin J. Comparison of a commercial accelerometer with polysomnography and actigraphy in children and adolescents. Sleep. 2015; 38(8): 1323– 1330. doi: 10.5665/sleep.4918 26118555 PMC4507738 · DOI
  8. [10]Schoch SF, Kurth S, Werner H. Actigraphy in sleep research with infants and young children: current practices and future benefits of standardized reporting. J Sleep Res. 2021; 30(3): e13134. doi: 10.1111/jsr.13134 32638500 PMC8244022 · DOI
  9. [11]Neikrug AB, Chen IY, Palmer JR, et al. Characterizing behavioral activity rhythms in older adults using actigraphy. Sensors. 2020; 20(2): 549. doi: 10.3390/s20020549 PMC7014517 31963889 · DOI
  10. [14]Smith MT, McCrae CS, Cheung J, et al. Use of actigraphy for the evaluation of sleep disorders and circadian rhythm sleep-wake disorders: an American academy of sleep medicine systematic review, meta-analysis, and GRADE assessment. J Clin Sleep Med. 2018; 14(7): 1209– 1230. doi: 10.5664/jcsm.7228 29991438 PMC6040804 · DOI
  11. [15]Lewis C, Lewis K, Kitchiner N, Isaac S, Jones I, Bisson JI. Sleep disturbance in post-traumatic stress disorder (PTSD): a systematic review and meta-analysis of actigraphy studies. Eur J Psychotraumatol. 2020; 11(1): 1767349. doi: 10.1080/20008198.2020.1767349 33029311 PMC7473131 · DOI
  12. [16]De Crescenzo F, Licchelli S, Ciabattini M, et al. The use of actigraphy in the monitoring of sleep and activity in ADHD: a meta-analysis. Sleep Med Rev. 2016; 26: 9– 20. doi: 10.1016/j.smrv.2015.04.002 26163053 · DOI
  13. [17]Colrain IM, Trinder J, Fraser G, Wilson GV. Ventilation during sleep onset. J Appl Physiol. 1987; 63(5): 2067– 2074. doi: 10.1152/jappl.1987.63.5.2067 3693239 · DOI
  14. [18]Burgess HJ, Kleiman J, Trinder J. Cardiac activity during sleep onset. Psychophysiology. 1999; 36(3): 298– 306. doi: 10.1017/s0048577299980198 10352553 · DOI
  15. [19]Toscani L, Gangemi PF, Parigi A, et al. Human heart rate variability and sleep stages. Ital J Neurol Sci. 1996; 17(6): 437– 439. doi: 10.1007/BF01997720 8978452 · DOI
  16. [20]Xiao M, Yan H, Song J, Yang Y, Yang X. Sleep stages classification based on heart rate variability and random forest. Biomed Signal Process Control. 2013; 8(6): 624– 633. doi: 10.1016/j.bspc.2013.06.001 · DOI
  17. [22]Cuesta M, Boudreau P, Cermakian N, Boivin DB. Skin temperature rhythms in humans respond to changes in the timing of sleep and light. J Biol Rhythms. 2017; 32(3): 257– 273. doi: 10.1177/0748730417702974 28569119 · DOI · Paper page
  18. [24]Rentz LE, Ulman HK, Galster SM. Deconstructing commercial wearable technology: contributions toward accurate and free-living monitoring of sleep. Sensors. 2021; 21(15): 5071. doi: 10.3390/s21155071 34372308 PMC8348972 · DOI
  19. [25]Khosla S, Deak MC, Gault D, et al. Consumer sleep technology: an American academy of sleep medicine position statement. J Clin Sleep Med. 2018; 14(5): 877– 880. doi: 10.5664/jcsm.7128 29734997 PMC5940440 · DOI
  20. [26]Chinoy ED, Cuellar JA, Huwa KE, et al. Performance of seven consumer sleep-tracking devices compared with polysomnography. Sleep. 2021; 44(5). doi: 10.1093/sleep/zsaa291 PMC8120339 33378539 · DOI
  21. [27]Roberts DM, Schade MM, Mathew GM, Gartenberg D, Buxton OM. Detecting sleep using heart rate and motion data from multisensor consumer-grade wearables, relative to wrist actigraphy and polysomnography. Sleep. 2020; 43(7). doi: 10.1093/sleep/zsaa045 PMC7355403 32215550 · DOI
  22. [28]Altini M, Kinnunen H. The promise of sleep: a multi-sensor approach for accurate sleep stage detection using the Oura Ring. Sensors. 2021; 21(13). doi: 10.3390/s21134302 PMC8271886 34201861 · DOI
  23. [29]Johns MW. A new method for measuring daytime sleepiness: the Epworth sleepiness scale. sleep. 1991; 14(6): 540– 545. doi: 10.1093/sleep/14.6.540 1798888 · DOI · Paper page
  24. [30]Patanaik A, Ong JL, Gooley JJ, Ancoli-Israel S, Chee MWL. An end-to-end framework for real-time automatic sleep stage classification. Sleep. 2018; 41(5). doi: 10.1093/sleep/zsy041 PMC5946920 29590492 · DOI
  25. [32]Menghini L, Cellini N, Goldstone A, Baker FC, de Zambotti M. A standardized framework for testing the performance of sleep-tracking technology: step-by-step guidelines and open-source code. Sleep. 2021; 44(2). doi: 10.1093/sleep/zsaa170 PMC7879416 32882005 · DOI
  26. [33]Lorenzo JL, Barbanoj MJ. Variability of sleep parameters across multiple laboratory sessions in healthy young subjects: the “very first night effect”. Psychophysiology. 2002; 39(4): 409– 413. doi: 10.1111/1469-8986.3940409 12212632 · DOI
  27. [34]Tamaki M, Nittono H, Hayashi M, Hori T. Examination of the first-night effect during the sleep-onset period. Sleep. 2005; 28(2): 195– 202. doi: 10.1093/sleep/28.2.195 16171243 · DOI
  28. [36]Haghayegh S, Khoshnevis S, Smolensky MH, Diller KR, Castriotta RJ. Accuracy of wristband fitbit models in assessing sleep: systematic review and meta-analysis. J Med Internet Res. 2019; 21(11): e16273. doi: 10.2196/16273 31778122 PMC6908975 · DOI · Paper page
  29. [37]Miller DJ, Lastella M, Scanlan AT, et al. A validation study of the WHOOP strap against polysomnography to assess sleep. J Sports Sci. 2020; 38(22): 2631– 2636. doi: 10.1080/02640414.2020.1797448 32713257 · DOI
  30. [39]Stone JD, Rentz LE, Forsey J, et al. Evaluations of commercial sleep technologies for objective monitoring during routine sleeping conditions. Nat Sci Sleep. 2020; 12: 821– 842. doi: 10.2147/NSS.S270705 33149712 PMC7603649 · DOI
  31. [40]Liang Z, Martell MAC. Achieving accurate ubiquitous sleep sensing with consumer wearable activity wristbands using multi-class imbalanced classification. IEEE; 2019: 768– 775.
  32. [41]Liang Z, Chapa-Martell MA. Combining resampling and machine learning to improve sleep-wake detection of Fitbit wristbands. IEEE; 2019: 1– 3.
  33. [42]Meltzer LJ, Short M, Booster GD, et al. Pediatric motor activity during sleep as measured by actigraphy. Sleep. 2019; 42(1). doi: 10.1093/sleep/zsy196 PMC6335874 30335173 · DOI
  34. [43]Short MA, Gradisar M, Lack LC, Wright H, Carskadon MA. The discrepancy between actigraphic and sleep diary measures of sleep in adolescents. Sleep Med. 2012; 13(4): 378– 384. doi: 10.1016/j.sleep.2011.11.005 22437142 · DOI · Paper page
  35. [44]Lee XK, Chee N, Ong JL, et al. Validation of a consumer sleep wearable device with actigraphy and polysomnography in adolescents across sleep opportunity manipulations. J Clin Sleep Med. 2019; 15(9): 1337– 1346. doi: 10.5664/jcsm.7932 31538605 PMC6760396 · DOI
  36. [45]Park S, Toth LP, Hibbing PR, et al. Dominant vs. non-dominant wrist placement of activity monitors: impact on steps per day. JMPB. 2019; 2(2): 118– 123. doi: 10.1123/jmpb.2018-0060 · DOI
  37. [46]Muehlroth BE, Werkle-Bergner M. Understanding the interplay of sleep and aging: methodological challenges. Psychophysiology. 2020; 57(3): e13523. doi: 10.1111/psyp.13523 31930523 · DOI
  38. [47]Basner M, Griefahn B, Penzel T. Inter-rater agreement in sleep stage classification between centers with different backgrounds. Somnologie. 2008; 12(1): 75– 84. doi: 10.1007/s11818-008-0327-y · DOI
  39. [48]Rosenberg RS, Van Hout S. The American academy of sleep medicine inter-scorer reliability program: sleep stage scoring. J Clin Sleep Med. 2013; 9(1): 81– 87. doi: 10.5664/jcsm.2350 23319910 PMC3525994 · DOI · Paper page
  40. [49]Danker‐Hopfe H, Kunz D, Gruber G, et al. Interrater reliability between scorers from eight European sleep laboratories in subjects with different sleep disorders. J Sleep Res. 2004; 13(1): 63– 69. doi: 10.1046/j.1365-2869.2003.00375.x 14996037 · DOI
  41. [50]Lucey BP, Wisch J, Boerwinkle AH, et al. Sleep and longitudinal cognitive performance in preclinical and early symptomatic Alzheimer’s disease. Brain. 2021; 144(9): 2852– 2862. doi: 10.1093/brain/awab272 34668959 PMC8536939 · DOI
  42. [52]Mikutta C, Wenke M, Spiegelhalder K, et al. Co-ordination of brain and heart oscillations during non-rapid eye movement sleep. J Sleep Res. 2021; 31(2): e13466. doi: 10.1111/jsr.13466 34467582 PMC9285890 · DOI
  43. [53]Naji M, Krishnan GP, McDevitt EA, Bazhenov M, Mednick SC. Coupling of autonomic and central events during sleep benefits declarative memory consolidation. Neurobiol Learn Mem. 2019; 157: 139– 150. doi: 10.1016/j.nlm.2018.12.008 30562589 PMC6425961 · DOI
  44. [54]Lipsitz LA, Novak V. Aging and the autonomic nervous system. In: Primer on the Autonomic Nervous System. Elsevier; 2012: 271– 273.
  45. [55]Chen PC, Sattari N, Whitehurst LN, Mednick SC. Age-related losses in cardiac autonomic activity during a daytime nap. Psychophysiology. 2021; 58(7): e13701. doi: 10.1111/psyp.13701 33048396 PMC8041919 · DOI
  46. [56]de Zambotti M, Trinder J, Silvani A, Colrain IM, Baker FC. Dynamic coupling between the central and autonomic nervous systems during sleep: a review. Neurosci Biobehav Rev. 2018; 90: 84– 103. doi: 10.1016/j.neubiorev.2018.03.027 29608990 PMC5993613 · DOI · Paper page

Cited by

Sorted by citations

Authors
Ghorbani, S., Golkashani, H. A., Chee, N. I., Teo, T. B., Dicom, A. R., Yilmaz, G., Leong, R. L., Ong, J. L., & Chee, M. W.
Affiliations
National University of Singapore (SG)
Corresponding author
Chee, Michael WL
ORCID
Venue
Nature and Science of Sleep
Publisher
Informa UK Limited
Published
2022-04
Volume
Volume 14
Pages
645-660
ISSN
1179-1608 (ISSN-L)
DOI
10.2147/nss.s359789
PMC
PMC9015046
OpenAlex
W4224004997
Cited by
36
References
9
Citations by year
2026: 9 · 2025: 6 · 2024: 11 · 2023: 8 · 2022: 2
FWCI
6.21
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature and Science of Sleep (publishedVersion, cc-by-nc)
  • PubMed Central (publishedVersion, cc-by-nc)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Dove Medical Press (Taylor and Francis Group) (submittedVersion, cc-by-nc)
Funders
  • National University of Singapore
  • Medical Research Council
  • National Medical Research Council (STAR19may-0001)
Topics
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Sleep and related disorders
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
  • Physical Sciences › Engineering › Biomedical Engineering › Non-Invasive Vital Sign Monitoring
Keywords
sleep stage classification, sleep detection, Oura Ring, wearable sensors, photoplethysmography, accelerometer, machine learning, sleep latency, wake after sleep onset, REM sleep, total sleep time, sleep efficiency, sex differences, age differences

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review