Cited by 310Non-Invasive Vital Sign MonitoringECG Monitoring and AnalysisContext-Aware Activity Recognition Systems

Accuracy of Consumer Wearable Heart Rate Measurement During an Ecologically Valid 24-Hour Period: Intraindividual Validation Study

Open access
Nelson, Benjamin W, Allen, Nicholas B·2019·JMIR mHealth and uHealth

Published version of a preprint

Abstract

BACKGROUND: Wrist-worn smart watches and fitness monitors (ie, wearables) have become widely adopted by consumers and are gaining increased attention from researchers for their potential contribution to naturalistic digital measurement of health in a scalable, mobile, and unobtrusive way. Various studies have examined the accuracy of these devices in controlled laboratory settings (eg, treadmill and stationary bike); however, no studies have investigated the heart rate accuracy of wearables during a continuous and ecologically valid 24-hour period of actual consumer device use conditions. OBJECTIVE: The aim of this study was to determine the heart rate accuracy of 2 popular wearable devices, the Apple Watch 3 and Fitbit Charge 2, as compared with the gold standard reference method, an ambulatory electrocardiogram (ECG), during consumer device use conditions in an individual. Data were collected across 5 daily conditions, including sitting, walking, running, activities of daily living (ADL; eg, chores, brushing teeth), and sleeping. METHODS: One participant, (first author; 29-year-old Caucasian male) completed a 24-hour ecologically valid protocol by wearing 2 popular wrist wearable devices (Apple Watch 3 and Fitbit Charge 2). In addition, an ambulatory ECG (Vrije Universiteit Ambulatory Monitoring System) was used as the gold standard reference method, which resulted in the collection of 102,740 individual heartbeats. A single-subject design was used to keep all variables constant except for wearable devices while providing a rapid response design to provide initial assessment of wearable accuracy for allowing the research cycle to keep pace with technological advancements. Accuracy of these devices compared with the gold standard ECG was assessed using mean error, mean absolute error, and mean absolute percent error. These data were supplemented with Bland-Altman analyses and concordance class correlation to assess agreement between devices. RESULTS: The Apple Watch 3 and Fitbit Charge 2 were generally highly accurate across the 24-hour condition. Specifically, the Apple Watch 3 had a mean difference of -1.80 beats per minute (bpm), a mean absolute error percent of 5.86%, and a mean agreement of 95% when compared with the ECG across 24 hours. The Fitbit Charge 2 had a mean difference of -3.47 bpm, a mean absolute error of 5.96%, and a mean agreement of 91% when compared with the ECG across 24 hours. These findings varied by condition. CONCLUSIONS: The Apple Watch 3 and the Fitbit Charge 2 provided acceptable heart rate accuracy (<±10%) across the 24 hour and during each activity, except for the Apple Watch 3 during the daily activities condition. Overall, these findings provide preliminary support that these devices appear to be useful for implementing ambulatory measurement of cardiac activity in research studies, especially those where the specific advantages of these methods (eg, scalability, low participant burden) are particularly suited to the population or research question.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.2196/10828

Cites

41 of 41 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • For dark skin pigmentation, pulse rate mean bias was -0.57 bpm (95% CI -9.44 to 8.29).

    “Mean biases were –1.24 (95% CI –5.31 to 2.83) bpm, –0.89 (95% CI –3.70 to 1.93) bpm, and –0.57 (95% CI –9.44 to 8.29) bpm for the corresponding groups (Table 4 and Figures 5-7 [41,60-62]).”

    From Impact of skin pigmentation on pulse oximetry blood oxygenation and wearable pulse rate accuracy: systematic review and meta-analysis (2024) · cites this paper as [60]

    The Methods (Recruitment; page unavailable) describe a single participant with Fitzpatrick skin tone 2, not a dark-skin group. The Results (Bland-Altman Analysis; page unavailable) report mean errors of −1.80 bpm for Apple Watch 3 and −3.47 bpm for Fitbit Charge 2 across 24 hours, not the claimed −0.57 bpm with a 95% CI of −9.44 to 8.29. (LLM, conf=0.88)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured
  • For medium skin pigmentation, pulse rate 95% limits of agreement were -18.62 to 16.84 bpm.

    “Our analysis revealed 95% LoA of –16.02 to 13.54 bpm, –18.62 to 16.84 bpm, and –33.69 to 32.54 bpm for light, medium, and dark skin pigmentation groups, respectively.”

    From Impact of skin pigmentation on pulse oximetry blood oxygenation and wearable pulse rate accuracy: systematic review and meta-analysis (2024) · cites this paper as [60]

    The Methods > Recruitment section (page unavailable) says the study involved a single participant with Fitzpatrick skin tone 2, not light, medium, and dark pigmentation groups. The Results > Bland-Altman Analysis section (page unavailable) reports limits of agreement by device and activity, not the claimed −18.62 to 16.84 bpm for a medium-pigmentation group. (LLM, conf=0.87)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured
  • For medium skin pigmentation, pulse rate mean bias was -0.89 bpm (95% CI -3.70 to 1.93).

    “Mean biases were –1.24 (95% CI –5.31 to 2.83) bpm, –0.89 (95% CI –3.70 to 1.93) bpm, and –0.57 (95% CI –9.44 to 8.29) bpm for the corresponding groups (Table 4 and Figures 5-7 [41,60-62]).”

    From Impact of skin pigmentation on pulse oximetry blood oxygenation and wearable pulse rate accuracy: systematic review and meta-analysis (2024) · cites this paper as [60]

    The Methods (Recruitment; page unavailable) describe a single participant with Fitzpatrick skin tone 2, not a medium-pigmentation group. The Results (Bland-Altman Analysis; page unavailable) report device- and activity-specific errors, but not a −0.89 bpm mean bias or the stated 95% confidence interval. Searches for medium skin pigmentation and those figures found no supporting passage. (LLM, conf=0.76)

    Experimental — accuracy not yet measured
  • For light skin pigmentation, pulse rate mean bias was -1.24 bpm (95% CI -5.31 to 2.83).

    “Mean biases were –1.24 (95% CI –5.31 to 2.83) bpm, –0.89 (95% CI –3.70 to 1.93) bpm, and –0.57 (95% CI –9.44 to 8.29) bpm for the corresponding groups (Table 4 and Figures 5-7 [41,60-62]).”

    From Impact of skin pigmentation on pulse oximetry blood oxygenation and wearable pulse rate accuracy: systematic review and meta-analysis (2024) · cites this paper as [60]

    The Results section reports mean errors by device and activity, not by skin pigmentation. A search of our copy for light skin pigmentation and the stated bias and confidence interval found no passage reporting those subgroup results; the paper’s overall device estimates do not resolve the claim. (LLM, conf=0.75)

    Experimental — accuracy not yet measured
  • For light skin pigmentation, pulse rate 95% limits of agreement were -16.02 to 13.54 bpm.

    “Our analysis revealed 95% LoA of –16.02 to 13.54 bpm, –18.62 to 16.84 bpm, and –33.69 to 32.54 bpm for light, medium, and dark skin pigmentation groups, respectively.”

    From Impact of skin pigmentation on pulse oximetry blood oxygenation and wearable pulse rate accuracy: systematic review and meta-analysis (2024) · cites this paper as [60]

    The Results section, “Bland-Altman Analysis and 95 Percent Limits of Agreement” (page unavailable), reports limits by device and activity, not by skin pigmentation. Searches for the stated −16.02 to 13.54 bpm range and for light, medium, or dark skin groups found no matching result in our copy. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. [47]Terry N Wiley LF Liability for mobile health and wearable technologies Ann Health Law 2016 02 02

Cited by

Sorted by citations

Authors
Nelson, B. W., & Allen, N. B.
Affiliations
University of Oregon (US)
Corresponding author
Nelson, Benjamin W
Venue
JMIR mHealth and uHealth
Published
2019-03-11
Volume
7
Issue
3
Pages
e10828
ISSN
2291-5222 (ISSN-L)
OpenAlex
W2900058591
Cited by
310
References
41
Citations by year
2026: 39 · 2025: 39 · 2024: 45 · 2023: 52 · 2022: 50 · 2021: 49 · 2020: 33 · 2019: 2 · 2018: 1
FWCI
8.49
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • JMIR mhealth and uhealth (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Physical Sciences › Engineering › Biomedical Engineering › Non-Invasive Vital Sign Monitoring
  • Health Sciences › Medicine › Cardiology and Cardiovascular Medicine › ECG Monitoring and Analysis
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
Keywords
wearable devices, heart rate monitoring, Holter monitoring, device accuracy, activities of daily living, sleep heart rate, single-case experimental design, Bland-Altman analysis, concordance correlation coefficient
MeSH terms
Heart Rate Determination, Fitness Trackers, Wearable Electronic Devices, Activities of Daily Living, Adult, Heart Rate, Humans, Male, Monitoring, Physiologic, Validation Studies as Topic, Accelerometry

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review