CoChat

Validation of Zulu Watch against Polysomnography and Actigraphy for On-Wrist Sleep-Wake Determination and Sleep-Depth Estimation

Open access

The DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index.

Abstract

Traditional measures of sleep or commercial wearables may not be ideal for use in operational environments. The Zulu watch is a commercial sleep-tracking device designed to collect longitudinal sleep data in real-world environments. Laboratory testing is the initial step towards validating a device for real-world sleep evaluation; therefore, the Zulu watch was tested against the gold-standard polysomnography (PSG) and actigraphy. Eight healthy, young adult participants wore a Zulu watch and Actiwatch simultaneously over a 3-day laboratory PSG sleep study. The accuracy, sensitivity, and specificity of epoch-by-epoch data were tested against PSG and actigraphy. Sleep summary statistics were compared using paired samples t-tests, intraclass correlation coefficients, and Bland–Altman plots. Compared with either PSG or actigraphy, both the accuracy and sensitivity for Zulu watch sleep-wake determination were >90%, while the specificity was low (~26% vs. PSG, ~33% vs. actigraphy). The accuracy for sleep scoring vs. PSG was ~87% for interrupted sleep, ~52% for light sleep, and ~49% for deep sleep. The Zulu watch showed mixed results but performed well in determining total sleep time, sleep efficiency, sleep onset, and final awakening in healthy adults compared with PSG or actigraphy. The next step will be to test the Zulu watch’s ability to evaluate sleep in industrial operations.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3390/s21010076

Cites

37 of 37 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Recoding the polysomnography (PSG) sleep stages from 30-s epochs into 2-min Zulu watch sleep score equivalents. REM, rapid eye movement; SWS, slow-wave sleep.
Figure 1 Recoding the polysomnography (PSG) sleep stages from 30-s epochs into 2-min Zulu watch sleep score equivalents. REM, rapid eye movement; SWS, slow-wave sleep.CC BY · PubMed Central
Figure 2 Comparison of the participants’ epoch-by-epoch sleep-wake hypnograms by polysomnography (PSG), Zulu watch, and actigraphy. A three-panel comparison of the PSG, Zulu watch, and Actiwatch 2 sleep-wake determination across all study nights by participant. Epochs are shown in study days and graphed along the x-axis. Sleep versus wake determination is plotted in orange () for PSG, blue () for the corresponding Zulu watch data, and green () for the corresponding Actiwatch 2 data. Missing nights of data are indicated by gray blocks ().
Figure 2 Comparison of the participants’ epoch-by-epoch sleep-wake hypnograms by polysomnography (PSG), Zulu watch, and actigraphy. A three-panel comparison of the PSG, Zulu watch, and Actiwatch 2 sleep-wake determination across all study nights by participant. Epochs are shown in study days and graphed along the x-axis. Sleep versus wake determination is plotted in orange () for PSG, blue () for the corresponding Zulu watch data, and green () for the corresponding Actiwatch 2 data. Missing nights of data are indicated by gray blocks ().CC BY · PubMed Central
Figure 3 Comparison of the participants’ epoch-by-epoch sleep-wake hypnograms by polysomnography (PSG) and Zulu watch. A two-panel comparison of the paired PSG and Zulu watch sleep-scoring data from the first study night for each participant. Only the first night is depicted in order to improve readability. Epochs are graphed along the x-axis and shown in hours of sleep over the study night. Sleep stages are plotted in orange () for PSG and blue () for the corresponding Zulu watch data.
Figure 3 Comparison of the participants’ epoch-by-epoch sleep-wake hypnograms by polysomnography (PSG) and Zulu watch. A two-panel comparison of the paired PSG and Zulu watch sleep-scoring data from the first study night for each participant. Only the first night is depicted in order to improve readability. Epochs are graphed along the x-axis and shown in hours of sleep over the study night. Sleep stages are plotted in orange () for PSG and blue () for the corresponding Zulu watch data.CC BY · PubMed Central
Figure 4 Bland–Altman plots of the differences (y-axis) between polysomnography (PSG) and the Zulu watch () or PSG and the Actiwatch 2 () measures of sleep versus the mean of the two measurements (x-axis). Bias is represented by the solid blue line () for the Zulu watch and the solid green line () for the Actiwatch 2. Upper and lower limits of agreement (LOAs) are represented by the blue dashed lines () for the Zulu watch and green dashed lines () for the Actiwatch 2. Narrower LOAs indicate relatively less variability versus PSG than wider LOAs. Agreement is shown between measures for (A) total sleep time, (B) sleep efficiency, (C) time of sleep onset, and (D) time of final awakening. Time in bed (TIB) could not be visualized by the Bland–Altman plots because TIB was invariably set at 480 min for all participants across all nights for PSG and the Actiwatch 2.
Figure 4 Bland–Altman plots of the differences (y-axis) between polysomnography (PSG) and the Zulu watch () or PSG and the Actiwatch 2 () measures of sleep versus the mean of the two measurements (x-axis). Bias is represented by the solid blue line () for the Zulu watch and the solid green line () for the Actiwatch 2. Upper and lower limits of agreement (LOAs) are represented by the blue dashed lines () for the Zulu watch and green dashed lines () for the Actiwatch 2. Narrower LOAs indicate relatively less variability versus PSG than wider LOAs. Agreement is shown between measures for (A) total sleep time, (B) sleep efficiency, (C) time of sleep onset, and (D) time of final awakening. Time in bed (TIB) could not be visualized by the Bland–Altman plots because TIB was invariably set at 480 min for all participants across all nights for PSG and the Actiwatch 2.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 37 references

Research works printed but not linked: 1

  1. Berry R.B. Brooks R. Gamaldo C.E. Harding S.M. Marcus C. Vaughn B.V. The AASM manual for the scoring of sleep and associated events Rules Terminol. Techn. Specif. Darien Ill. Am. Acad. Sleep Med. 2012 176 2012

Cited by

See all 35

Metadata

Authors
Devine, J. K., Chinoy, E. D., Markwald, R. R., Schwartz, L. P., & Hursh, S. R.
Affiliations
  • Institutes for Behavior Resources (US)
  • Leidos (United States) (US)
  • Naval Health Research Center (US)
Corresponding author
Devine, Jaime K.
Venue
Sensors
Publisher
MDPI AG
Published
2020-12-25
Volume
21
Issue
1
Pages
76
ISSN
1424-8220 (ISSN-L)
OpenAlex
W3113553538
Cited by
32
References
37
Citations by year
2026: 2 · 2025: 3 · 2024: 6 · 2023: 8 · 2022: 7 · 2021: 5 · 2020: 1
FWCI
5.08
Type
journal article
Language
en
Open access
Locations
  • Sensors (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, other-oa)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Medical Technology Enterprise Consortium (MTEC-17-08-Multi-Topic-0104)
Topics
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Sleep and related disorders
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Sleep and Work-Related Fatigue
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
Keywords
wearable sleep trackers, sleep-wake detection, polysomnography, actigraphy, device validation, total sleep time, sleep efficiency
MeSH terms
Humans, Sleep, Wrist, Reproducibility of Results, Polysomnography, Young Adult, Actigraphy

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.