Validation of Zulu Watch against Polysomnography and Actigraphy for On-Wrist Sleep-Wake Determination and Sleep-Depth Estimation
Open accessThe DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index.
Abstract
Traditional measures of sleep or commercial wearables may not be ideal for use in operational environments. The Zulu watch is a commercial sleep-tracking device designed to collect longitudinal sleep data in real-world environments. Laboratory testing is the initial step towards validating a device for real-world sleep evaluation; therefore, the Zulu watch was tested against the gold-standard polysomnography (PSG) and actigraphy. Eight healthy, young adult participants wore a Zulu watch and Actiwatch simultaneously over a 3-day laboratory PSG sleep study. The accuracy, sensitivity, and specificity of epoch-by-epoch data were tested against PSG and actigraphy. Sleep summary statistics were compared using paired samples t-tests, intraclass correlation coefficients, and Bland–Altman plots. Compared with either PSG or actigraphy, both the accuracy and sensitivity for Zulu watch sleep-wake determination were >90%, while the specificity was low (~26% vs. PSG, ~33% vs. actigraphy). The accuracy for sleep scoring vs. PSG was ~87% for interrupted sleep, ~52% for light sleep, and ~49% for deep sleep. The Zulu watch showed mixed results but performed well in determining total sleep time, sleep efficiency, sleep onset, and final awakening in healthy adults compared with PSG or actigraphy. The next step will be to test the Zulu watch’s ability to evaluate sleep in industrial operations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“sleep assessment algorithms are frequently proprietary and validated in small populations, making their measurement validity unclear5–8”
From Self-supervised learning of accelerometer data provides new insights for sleep and its association with mortality (2024) · cites this paper as [7]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
STATISTICAL METHODS FOR ASSESSING AGREEMENT BETWEEN TWO METHODS OF CLINICAL MEASUREMENT
Martin Bland, J., & Altman, D. (1986) · The Lancet · doi:10.1016/s0140-6736(86)90837-8 · Cited by 47,965
A guideline of selecting and reporting intraclass correlation coefficients for reliability research
Koo, T. K., & Li, M. Y. (2016) · Journal of Chiropractic Medicine · doi:10.1016/j.jcm.2016.02.012 · Cited by 31,709 · cited 2×
“Based on the guidelines set forth by Koo and Li [26], ICCs for time of sleep onset and time of final awakening were good (0.”
MethodSleep duration and all-cause mortality: a systematic review and meta-analysis of prospective studies
Cappuccio, F. P., D'Elia, L., Strazzullo, P., & Miller, M. A. (2010) · Sleep · doi:10.1093/sleep/33.5.585 · Cited by 2,224
The Rise of Consumer Health Wearables: Promises and Barriers
Piwek, L., Ellis, D. A., Andrews, S., & Joinson, A. (2016) · PLOS Medicine · doi:10.1371/journal.pmed.1001953 · Cited by 1,235 · cited 1×
“This necessity restricts the ability of devices to log sleep episodes in the absence of wireless connectivity to download data, and it can generate concerns about privacy among researchers and consumers alike [23,24].”
BackgroundMarino, M., Li, Y., Rueschman, M. N., Winkelman, J. W., Ellenbogen, J. M., Solet, J. M., Dulin, H., Berkman, L. F., & Buxton, O. M. (2013) · Sleep · doi:10.5665/sleep.3142 · Cited by 1,056 · cited 1×
“Low specificity is an issue across research actigraphy devices and scoring algorithms [30,31], with longer epoch lengths relating to lower specificity [9].”
BackgroundNormal Human Sleep: An Overview
Carskadon, M. A., & Dement, W. C. (2005) · Principles and Practice of Sleep Medicine · doi:10.1016/b0-72-160797-7/50009-4 · Cited by 995 · cited 1×
“Starting with NREM, the sleep stages cycle throughout the night, each playing a critical role in the maintenance of the brain and body to promote optimal health and functioning [7].”
BackgroundLittner, M., Kushida, C. A., Anderson, W. M., Bailey, D., Berry, R. B., Davila, D. G., Hirshkowitz, M., Kapen, S., Kramer, M., Loube, D., et al. (2003) · Sleep · doi:10.1093/sleep/26.3.337 · Cited by 691
Wearable Sleep Technology in Clinical and Research Settings
DE ZAMBOTTI, M., CELLINI, N., GOLDSTONE, A., COLRAIN, I. M., & BAKER, F. C. (2019) · Medicine & Science in Sports & Exercise · doi:10.1249/mss.0000000000001947 · Cited by 564
The SBSM Guide to Actigraphy Monitoring: Clinical and Research Applications
Ancoli-Israel, S., Martin, J. L., Blackwell, T., Buenaver, L., Liu, L., Meltzer, L. J., Sadeh, A., Spira, A. P., & Taylor, D. J. (2015) · Behavioral Sleep Medicine · doi:10.1080/15402002.2015.1046356 · Cited by 559 · cited 2×
“Low specificity is an issue across research actigraphy devices and scoring algorithms [30,31], with longer epoch lengths relating to lower specificity [9].”
BackgroundMethodPeake, J. M., Kerr, G., & Sullivan, J. P. (2018) · Frontiers in Physiology · doi:10.3389/fphys.2018.00743 · Cited by 523 · cited 1×
“However, while the devices collect accelerometry and other biometric data, sleep scoring is done using algorithms on a companion mobile app or software platform [14,22].”
Method
Research works printed but not linked: 1
- Berry R.B. Brooks R. Gamaldo C.E. Harding S.M. Marcus C. Vaughn B.V. The AASM manual for the scoring of sleep and associated events Rules Terminol. Techn. Specif. Darien Ill. Am. Acad. Sleep Med. 2012 176 2012
Cited by
A Systematic Review of Sensing Technologies for Wearable Sleep Staging
Imtiaz, S. A. (2021) · Sensors · doi:10.3390/s21051562 · Cited by 162
Evaluating reliability in wearable devices for sleep staging
Birrer, V., Elgendi, M., Lambercy, O., & Menon, C. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01016-9 · Cited by 164
“Anadditionaltwodevices 40,43 utilized input from accelerometers and additionally other sensors, such as ambient light, bio-impedance, or skin temperature, but did not include a PPG sensor.”
Svensson, T., Madhawa, K., Nt, H., Chung, U., & Svensson, A. K. (2024) · Sleep Medicine · doi:10.1016/j.sleep.2024.01.020 · Cited by 139
Ecological and social pressures interfere with homeostatic sleep regulation in the wild
Loftus, J., Harel, R., Nuñez, C., & Crofoot, M. (2022) · eLife · doi:10.7554/elife.73695 · Cited by 63
A systematic review of the performance of actigraphy in measuring sleep stages
Yuan, H., Hill, E. A., Kyle, S. D., & Doherty, A. (2024) · Journal of Sleep Research · doi:10.1111/jsr.14143 · Cited by 56
“Studies that used commercial150 grade devices relied on the proprietary algorithms from the vendors which do not 151 reveal the process for sleep stage classification [18, 19].”
BackgroundMethodInfluentialYuan, H., Plekhanova, T., Walmsley, R., Reynolds, A. C., Maddison, K. J., Bucan, M., Gehrman, P., Rowlands, A., Ray, D. W., Bennett, D., et al. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01065-0 · Cited by 40
The past, present, and future of sleep quality assessment and monitoring
Chen, Y., Zhou, E., Wang, Y., Wu, Y., Xu, G., & Chen, L. (2023) · Brain Research · doi:10.1016/j.brainres.2023.148333 · Cited by 45
Lee, Y. J., Lee, J. Y., Cho, J. H., Kang, Y. J., & Choi, J. H. (2024) · Journal of Clinical Sleep Medicine · doi:10.5664/jcsm.11460 · Cited by 43
Devine, J. K., Schwartz, L. P., & Hursh, S. R. (2022) · Sensors · doi:10.3390/s22072731 · Cited by 25
“Several years ago, IBR partnered with a small company to produce a wearable device that would be suitable for monitoring sleep and fatigue in shiftwork and similar operational work environments [12].”
BackgroundSelf-citationObjective sleep assessments for healthy people in environmental research: A literature review
Xu, X., & Lian, Z. (2022) · Indoor Air · doi:10.1111/ina.13034 · Cited by 22
Gabinet, N. M., & Portnov, B. A. (2022) · Environmental Research · doi:10.1016/j.envres.2022.113941 · Cited by 15
Can we rely on wearable sleep-tracker devices for fatigue management?
Reifman, J., Priezjev, N. V., & Vital-Lopez, F. G. (2023) · SLEEP · doi:10.1093/sleep/zsad288 · Cited by 14
Expert Demand for Consumer Sleep Technology Features and Wearable Devices: A Case Study
Devine, J. K., Schwartz, L. P., Choynowski, J. J., & Hursh, S. R. (2022) · IoT · doi:10.3390/iot3020018 · Cited by 13
Self-citationConsistency of chronotype measurements is affected by sleep quality, gender, longitude, and latitude
Wang, H., Wang, S., Yu, W., & Lei, X. (2023) · Chronobiology International · doi:10.1080/07420528.2023.2237118 · Cited by 12
“It can automatically determine sleep from the activity level using validated algorithms and can measure macroscopic sleepwake behaviors such as time in bed (TIB), total sleep time (TST), sleep end time (SE), and sleep onset time (SO) (Devine et al. 2020), which provides the necessary parameters for measuring chronotyp…”
MethodBiomathematical Modeling Predicts Fatigue Risk in General Surgery Residents
Schwartz, L. P., Devine, J. K., Hursh, S. R., Mosher, E., Schumacher, S., Boyle, L., Davis, J. E., Stafford‐Smith, M., & Fitzgibbons, S. (2021) · Journal of surgical education · doi:10.1016/j.jsurg.2021.04.007 · Cited by 8
Self-citationPilot Sleep Behavior across Time during Ultra-Long-Range Flights
Devine, J. K., Choynowski, J. J., Garcia, C. R., Simoes, A. S., Guelere, M. R., de Godoy, B., Silva, D. S., Pacheco, P., & Hursh, S. R. (2021) · Clocks & Sleep · doi:10.3390/clockssleep3040036 · Cited by 10
“The specificity of the Zulu watch to identify awakenings during a sleep interval compared to PSG under laboratory conditions is 26% [21].”
BackgroundMethodInfluentialSelf-citationPredictive Biomathematical Modeling Compared to Objective Sleep During COVID-19 Humanitarian Flights
Devine, J. K., Garcia, C. R., Simoes, A. S., Guelere, M. R., de Godoy, B., Silva, D. S., Pacheco, P. C., Choynowski, J., & Hursh, S. R. (2022) · Aerospace Medicine and Human Performance · doi:10.3357/amhp.5909.2022 · Cited by 8
Self-citationA Review of Benchmark Datasets and Training Loss Functions in Neural Depth Estimation
Khan, F., Hussain, S., Basak, S., Moustafa, M., & Corcoran, P. (2021) · IEEE Access · doi:10.1109/access.2021.3124978 · Cited by 8
“SoA monocular depth methods have been reviewed in [11], [17], [18], [23]–[25], which focus on both non-deep learning and deep learning methods.”
BackgroundDevine, J. K., Nichols, M. G., Schwartz, L. P., Choynowski, J. J., & Hursh, S. R. (2023) · Safety Science · doi:10.1016/j.ssci.2023.106128 · Cited by 6
Self-citationDevine, J. K., Schwartz, L. P., Hursh, S. R., Mosher, E., Schumacher, S., Boyle, L., Davis, J. E., Stafford‐Smith, M., & Fitzgibbons, S. (2020) · Journal of surgical education · doi:10.1016/j.jsurg.2020.11.010 · Cited by 6
Self-citationUsing actigraphy to assess chronotype: Simpler is better
Wei, Y., Wang, S., Wang, W., & Lei, X. (2024) · Chronobiology International · doi:10.1080/07420528.2024.2428196 · Cited by 4
“A number of validated algorithms are then used to derive sleep and activity parameters from the data, providing a detailed and objective portrait of an individual’s circadian rhythm (Depner et al. 2020; Devine et al. 2020).”
Monitoring sleep in real-world conditions using low-cost technology tools
Shama, H., Gabinet, N. M., Tzischinsky, O., & Portnov, B. A. (2022) · Biological Rhythm Research · doi:10.1080/09291016.2022.2131990 · Cited by 4
“Yet it should be kept in mind that previous studies of sleep in real-world conditions typically lasted less than a week (e.g., Asgari Choi et al. 2018; Scott et al. 2018; Devine et al. 2020; Mehrabadi et al. 2020).”
BackgroundResultInfluentialDharani, B., SUBA, A., & Abeetha, S. (2025) · Global Journal of Medical Pharmaceutical and Biomedical Update · doi:10.25259/gjmpbu_31_2025 · Cited by 3
Use of technology for real-world sleep and circadian research
Chinoy, E. D., & Markwald, R. R. (2021) · Elsevier eBooks · doi:10.1016/b978-0-12-822963-7.00200-0 · Cited by 3
Self-citationYuan, H., Plekhanova, T., Walmsley, R., Reynolds, A. C., Maddison, K. J., Bucan, M., Gehrman, P., Rowlands, A., Ray, D. W., Bennett, D., et al. (2023) · doi:10.1101/2023.07.07.23292251 · Cited by 3
Baarsen, A. M., van den Broek, N., Hilgenkamp, T. I. M., Festen, D. A. M., Elbers, R. G., & Huisman, S. A. (2025) · Journal of Sleep Research · doi:10.1111/jsr.70148 · Cited by 2
“…et al. 2007; Signal et al. 2005; Tonetti et al. 2008; Chae et al. 2009; Cook et al. 2019; Suksatit 2014; Chakar et al. 2017; Chinoy et al. 2021; Devine et al. 2020; Fonseca et al. 2017; Hsiou et al. 2022; Kawada et al. 2001; Markwald et al. 2016; Montgomery-Downs et al. 2012; Peterson et al.…”
Agostinho, M., Borges, M., Pereira, T. S., Borges, D. F., & Soares, J. I. (2026) · Behavioral Sleep Medicine · doi:10.1080/15402002.2026.2673893 · Cited by 2
A GA-SVM Model for Sleep/Wake Classification Using Single-Channel EEG Signals
Qinga, H. U., Li, M., Jiang, L., & Yu, X. (2023) · International Conference on Information Science and Education (ICISE-IE) · doi:10.1109/icise-ie60962.2023.10456532 · Cited by 2
“[6] conducted experiments with healthy adults using an activity monitor, demonstrating impressive results with a sleep-wake classification accuracy of 90.3%.”
MethodInfluentialResearch of a Thermal-Regulation System Using Electronic Integrated Textiles for a Mattress
Serdar, S. G., Değırmencı, Z., & Topalbekiroğlu, M. (2023) · Journal of Testing and Evaluation · doi:10.1520/jte20220517
Doherty, A., Yuan, H., Plekhanova, T., Walmsley, R., Reynolds, A., Maddison, K., Bucan, M., Gehrman, P., Rowlands, A., Ray, D., et al. (2023) · doi:10.21203/rs.3.rs-3275989/v1
Digital health in neurology: Advancements, applications, and impact
Taha, M., & Kedar, S. (2025) · Elsevier eBooks · doi:10.1016/b978-0-443-15728-8.00024-0
Kvalita spánku všeobecných sester pracujících v nepřetržitém provozu
Buřičová, I. (2021) · Digital Library (University of West Bohemia)
A validation study of the Withings ScanWatch compared to the gold standard polysomnography
Vöggsdóttir, S. (2021) · Skemman
Luo, J., Liu, R., Yin, J., Cao, W., Sun, S., & Chen, R. (2026) · PLoS Medicine · doi:10.1371/journal.pmed.1005213
Karahan, E., Çelik, S., Altıntaş, S., Uçar, Ö., & Yücel, M. (2026) · Current Psychology · doi:10.1007/s12144-026-10092-x
Metadata
- Authors
- Devine, J. K., Chinoy, E. D., Markwald, R. R., Schwartz, L. P., & Hursh, S. R.
- Affiliations
- Institutes for Behavior Resources (US)
- Leidos (United States) (US)
- Naval Health Research Center (US)
- Corresponding author
- Devine, Jaime K.
- Venue
- Sensors
- Publisher
- MDPI AG
- Published
- 2020-12-25
- Volume
- 21
- Issue
- 1
- Pages
- 76
- ISSN
- 1424-8220 (ISSN-L)
- PMC
- PMC7796293
- OpenAlex
- W3113553538
- Cited by
- 32
- References
- 37
- Citations by year
- 2026: 2 · 2025: 3 · 2024: 6 · 2023: 8 · 2022: 7 · 2021: 5 · 2020: 1
- FWCI
- 5.08
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Sensors (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, other-oa)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Medical Technology Enterprise Consortium (MTEC-17-08-Multi-Topic-0104)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Sleep and related disorders
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Sleep and Work-Related Fatigue
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
- Keywords
- wearable sleep trackers, sleep-wake detection, polysomnography, actigraphy, device validation, total sleep time, sleep efficiency
- MeSH terms
- Humans, Sleep, Wrist, Reproducibility of Results, Polysomnography, Young Adult, Actigraphy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.