Evaluations of Commercial Sleep Technologies for Objective Monitoring During Routine Sleeping Conditions
Open accessThe DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Abstract
PURPOSE: The commercial market is saturated with technologies that claim to collect proficient, free-living sleep measurements despite a severe lack of independent third-party evaluations. Therefore, the present study evaluated the accuracy of various commercial sleep technologies during in-home sleeping conditions. MATERIALS AND METHODS: sleep from five healthy adults. Prior to bedtime, participants utilized nine popular sleep devices while concurrently wearing a previously validated electroencephalography (EEG)-based device. Data collected from the commercial devices were extracted for later comparison against EEG to determine degrees of accuracy. Sleep and wake summary outcomes as well as sleep staging metrics were evaluated, where available, for each device. RESULTS: Total sleep time (TST), total wake time (TWT), and sleep efficiency (SE) were measured with greater accuracy (lower percent errors) and limited bias by Fitbit Ionic [mean absolute percent error, bias (95% confidence interval); TST: 9.90%, 0.25 (-0.11, 0.61); TWT: 25.64%, -0.17 (-0.28, -0.06); SE: 3.49%, 0.65 (-0.82, 2.12)] and Oura smart ring [TST: 7.39%, 0.19 (0.04, 0.35); TWT: 36.29%, -0.18 (-0.31, -0.04); SE: 5.42%, 1.66 (0.17, 3.15)], whereas all other devices demonstrated a propensity to over or underestimate at least one if not all of the aforementioned sleep metrics. No commercial sleep technology appeared to accurately quantify sleep stages. CONCLUSION: Generally speaking, commercial sleep technologies displayed lower error and bias values when quantifying sleep/wake states as compared to sleep staging durations. Still, these findings revealed that there is a remarkably high degree of variability in the accuracy of commercial sleep technologies, which further emphasizes that continuous evaluations of newly developed sleep technologies are vital. End-users may then be able to determine more accurately which sleep device is most suited for their desired application(s).
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
STATISTICAL METHODS FOR ASSESSING AGREEMENT BETWEEN TWO METHODS OF CLINICAL MEASUREMENT
Martin Bland, J., & Altman, D. (1986) · The Lancet · doi:10.1016/s0140-6736(86)90837-8 · Cited by 47,965
Wickham, H., Averick, M., Bryan, J., Chang, W., McGowan, L., François, R., Grolemund, G., Hayes, A., Henry, L., Hester, J., et al. (2019) · Journal of Open Source Software · doi:10.21105/joss.01686 · Cited by 22,138
The Role of Actigraphy in the Study of Sleep and Circadian Rhythms
Ancoli-Israel, S., Cole, R., Alessi, C., Chambers, M., Moorcroft, W., & Pollak, C. P. (2003) · Sleep · doi:10.1093/sleep/26.3.342 · Cited by 2,751
The role and validity of actigraphy in sleep medicine: an update
Sadeh, A. (2011) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2010.10.001 · Cited by 1,415
Short sleep duration and health outcomes: a systematic review, meta-analysis, and meta-regression
Itani, O., Jike, M., Watanabe, N., & Kaneita, Y. (2017) · Sleep Medicine · doi:10.1016/j.sleep.2016.08.006 · Cited by 1,252
Marino, M., Li, Y., Rueschman, M. N., Winkelman, J. W., Ellenbogen, J. M., Solet, J. M., Dulin, H., Berkman, L. F., & Buxton, O. M. (2013) · Sleep · doi:10.5665/sleep.3142 · Cited by 1,056
Investigating sources of inaccuracy in wearable optical heart rate sensors
Bent, B., Goldstein, B. A., Kibbe, W. A., & Dunn, J. P. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0226-6 · Cited by 673
Wearable Sleep Technology in Clinical and Research Settings
DE ZAMBOTTI, M., CELLINI, N., GOLDSTONE, A., COLRAIN, I. M., & BAKER, F. C. (2019) · Medicine & Science in Sports & Exercise · doi:10.1249/mss.0000000000001947 · Cited by 564
(2019) · Handbook of clinical neurology · doi:10.1016/b978-0-444-64032-1.00025-4 · Cited by 555
Accuracy of Wristband Fitbit Models in Assessing Sleep: Systematic Review and Meta-Analysis
Haghayegh, S., Khoshnevis, S., Smolensky, M. H., Diller, K. R., & Castriotta, R. J. (2019) · Journal of Medical Internet Research · doi:10.2196/16273 · Cited by 446
Nelson, B. W., & Allen, N. B. (2019) · JMIR mHealth and uHealth · doi:10.2196/10828 · Cited by 310
The Sleep of the Ring: Comparison of the ŌURA Sleep Tracker Against Polysomnography
de Zambotti, M., Rosas, L., Colrain, I. M., & Baker, F. C. (2019) · Behavioral Sleep Medicine · doi:10.1080/15402002.2017.1300587 · Cited by 287
Lee, J., Matsumura, K., Yamakoshi, K.-I., Rolfe, P., Tanaka, S., & Yamakoshi, T. (2013) · 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2013.6609852 · Cited by 202 · cited 1×
“Although the current study was unable to directly assess the accuracy of heart rate measurements in the technologies assessed due to a lack worthy comparison (eg, multi-lead electrocardiography), previous research suggests PPG devices that utilize a red or infrared wavelength most accurately evaluate physiological sig…”
ResultKinnunen, H., Rantanen, A., Kenttä, T., & Koskimäki, H. (2020) · Physiological Measurement · doi:10.1088/1361-6579/ab840a · Cited by 184
De Arriba-Pérez, F., Caeiro-Rodríguez, M., & Santos-Gago, J. (2016) · Sensors · doi:10.3390/s16091538 · Cited by 153
Tedesco, S., Sica, M., Ancillao, A., Timmons, S., Barton, J., & O'Flynn, B. (2019) · JMIR mHealth and uHealth · doi:10.2196/13084 · Cited by 139
Ahmadi, N., Shapiro, G. K., Chung, S. A., & Shapiro, C. M. (2009) · Sleep and Breathing · doi:10.1007/s11325-008-0234-2 · Cited by 134
Finan, P. H., Richards, J. M., Gamaldo, C. E., Han, D., Leoutsakos, J. M., Salas, R., Irwin, M. R., & Smith, M. T. (2016) · Journal of Clinical Sleep Medicine · doi:10.5664/jcsm.6262 · Cited by 90
Sañudo, B., De Hoyo, M., Muñoz-López, A., Perry, J., & Abt, G. (2019) · Journal of Medical Systems · doi:10.1007/s10916-019-1325-2 · Cited by 48
Cited by
Waking Up to the Importance of Sleep in Type 2 Diabetes Management: A Narrative Review.
Henson, J., Covenant, A., Hall, A. P., Herring, L., Rowlands, A. V., Yates, T., & Davies, M. J. (2024) · Diabetes Care · doi:10.2337/dci23-0037 · Cited by 69
Ash, G. I., Stults-Kolehmainen, M., Busa, M. A., Gaffey, A. E., Angeloudis, K., Muniz-Pardos, B., Gregory, R., Huggins, R. A., Redeker, N. S., Weinzimer, S. A., et al. (2021) · Sports Medicine · doi:10.1007/s40279-021-01543-5 · Cited by 41
Ghorbani, S., Golkashani, H. A., Chee, N. I., Teo, T. B., Dicom, A. R., Yilmaz, G., Leong, R. L., Ong, J. L., & Chee, M. W. (2022) · Nature and Science of Sleep · doi:10.2147/nss.s359789 · Cited by 36
Metadata
- Authors
- Stone, J. D., Rentz, L. E., Forsey, J., Ramadan, J., Markwald, R. R., Finomore, V. S., Galster, S. M., Rezai, A., & Hagen, J. A.
- Venue
- Nature and Science of Sleep
- Publisher
- Informa UK Limited
- Published
- 2020-10
- Volume
- Volume 12
- Pages
- 821-842
- ISSN
- 1179-1608 (ISSN-L)
- OpenAlex
- W3097166894
- Cited by
- 82
- References
- 19
- FWCI
- 9.72
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.