CoChat
Sign Up
Cited by 157Physical Activity and HealthContext-Aware Activity Recognition SystemsNon-Invasive Vital Sign Monitoring

Accuracy of Wrist-Worn Activity Monitors During Common Daily Physical Activities and Types of Structured Exercise: Evaluation Study

Open access
Reddy, Ravi Kondama, Pooni, Rubin, Zaharieva, Dessi P, Senf, Brian, El Youssef, Joseph, Dassau, Eyal, Doyle III, Francis J, Clements, Mark A +5 more·2018·JMIR mHealth and uHealth
DOI 10.2196/10338PMC PMC6305876

Published version of a preprint

Open this paper in CoChat — free accountOpen PDF

Abstract

BACKGROUND: Wrist-worn activity monitors are often used to monitor heart rate (HR) and energy expenditure (EE) in a variety of settings including more recently in medical applications. The use of real-time physiological signals to inform medical systems including drug delivery systems and decision support systems will depend on the accuracy of the signals being measured, including accuracy of HR and EE. Prior studies assessed accuracy of wearables only during steady-state aerobic exercise. OBJECTIVE: The objective of this study was to validate the accuracy of both HR and EE for 2 common wrist-worn devices during a variety of dynamic activities that represent various physical activities associated with daily living including structured exercise. METHODS: ; 11 females) performed a maximal oxygen uptake test, free-weight resistance circuit, interval training session, and activities of daily living. Validity was assessed using an HR chest strap (Polar) and portable indirect calorimetry (Cosmed). Accuracy of the commercial wearables versus research-grade standards was determined using Bland-Altman analysis, correlational analysis, and error bias. RESULTS: Fitbit and Garmin were reasonably accurate at measuring HR but with an overall negative bias. There was more error observed during high-intensity activities when there was a lack of repetitive wrist motion and when the exercise mode indicator was not used. The Garmin estimated HR with a mean relative error (RE, %) of -3.3% (SD 16.7), whereas Fitbit estimated HR with an RE of -4.7% (SD 19.6) across all activities. The highest error was observed during high-intensity intervals on bike (Fitbit: -11.4% [SD 35.7]; Garmin: -14.3% [SD 20.5]) and lowest error during high-intensity intervals on treadmill (Fitbit: -1.7% [SD 11.5]; Garmin: -0.5% [SD 9.4]). Fitbit and Garmin EE estimates differed significantly, with Garmin having less negative bias (Fitbit: -19.3% [SD 28.9], Garmin: -1.6% [SD 30.6], P<.001) across all activities, and with both correlating poorly with indirect calorimetry measures. CONCLUSIONS: Two common wrist-worn devices (Fitbit Charge 2 and Garmin vívosmart HR+) show good HR accuracy, with a small negative bias, and reasonable EE estimates during low to moderate-intensity exercise and during a variety of common daily activities and exercise. Accuracy was compromised markedly when the activity indicator was not used on the watch or when activities involving less wrist motion such as cycle ergometry were done.

AI snapshot

Written from the paper’s full text.

TL;DR

In 20 healthy adults, Fitbit Charge 2 and Garmin vívosmart HR+ showed reasonable heart rate (HR) accuracy with a small negative bias, but accuracy worsened during high-intensity activities, cycling, and when Garmin’s activity mode was not indicated. Energy expenditure (EE) estimates were less accurate, especially for Fitbit during cycling activities.

Key findings

  • HR across all activities: Garmin mean relative error (RE) −3.3% (SD 16.7); Fitbit −4.7% (SD 19.6). Garmin without activity mode was worse: −5.47% (SD 22.79; P<.001 vs activity mode indicated). MAPE was 10.79% (SD 13.14) for Garmin and 11.33% (SD 16.71) for Fitbit.
  • Highest HR error occurred during high-intensity intervals on bike: Fitbit −11.4% (SD 35.7), Garmin −14.3% (SD 20.5); lowest during high-intensity intervals on treadmill: Fitbit −1.7% (SD 11.5), Garmin −0.5% (SD 9.4). Activities with repetitive upper-torso motion had lower error (−1.6%, SD 9.6) than those without (−12.25%, SD 19.3; P<.001).
  • EE estimates differed significantly: Fitbit −19.3% (SD 28.9) vs Garmin −1.6% (SD 30.6; P<.001), and both correlated poorly with indirect calorimetry. MAPE was 27.0% (SD 21.8) for Garmin and 25.1% (SD 17.3) for Fitbit. Highest EE error was Fitbit during MAX-C (−39.1%, SD 30.6) and HIIT-C (−41.9%, SD 31.3); Garmin had positive bias during resistance exercise (+21.0%, SD 35.7).
  • Both devices produced spurious HR measurements when not worn on the wrist, confirmed by a bench-top nutator experiment at 22 rpm.

Methods

  • Twenty healthy adults (11 females; mean age 27.5 [SD 6.0] years; mean BMI 22.5 [SD 2.3] kg/m²; 10 at OHSU, 10 at York University) completed two laboratory visits 24 hours apart. Activities included maximal oxygen uptake tests on treadmill or cycle ergometer, free-weight resistance circuit, activities of daily living, and high-intensity interval training on treadmill or cycle ergometer.
  • Fitbit Charge 2 and Garmin vívosmart HR+ were tested simultaneously on randomized/counterbalanced wrists. HR criterion was Polar H7 chest strap; EE criterion was portable indirect calorimetry (Cosmed K4b2 or K5). Garmin was tested with and without activity mode indicated.
  • Analysis used Bland-Altman, Pearson correlation, mean relative error, mean absolute percent error, paired t tests, one-way ANOVA with Tukey post hoc, and concordance class correlation in R and GraphPad Prism.

Limitations

  • Stated: only two consumer-grade devices were tested; skin color was not accounted for, and future studies should report Fitzpatrick skin tone; exercise was conducted in a laboratory rather than real-world setting; HR was not compared against a true gold standard such as ECG.
  • Evident from the text: Fitbit data could not be exported reliably without the exercise type selected, preventing a no-button comparison for Fitbit; Garmin EE data could only be compared at low resolution as an average across each activity; proprietary algorithms can change without warning, limiting integration into medical systems.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.2196/10338

Cites

36 of 36 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

17 of 38 cited sources checked · 7 supported · 0 flagged · accuracy not yet measured

  • Findingown data

    The two devices showed reasonable accuracy in HR and EE estimations under certain exercise conditions.

    “Similar to previous studies [13,28,30-32], we found reasonable accuracy in HR and EE estimations for these 2 devices under certain exercise conditions.”
    Quote in the paperSubstantial claim5 of 5 sources checked
    • cites [13] source

      The Results (Laboratory validation; page unavailable) report activity-specific energy-expenditure errors for the Apple Watch and Fitbit Charge HR, supporting reasonable EE accuracy under some exercise conditions. The Methods (Statistical analysis; page unavailable) compare device EE estimates with criterion EE measurements, but our copy does not report a validation of the devices’ HR-estimation accuracy. (LLM, conf=0.88)

      Experimental — accuracy not yet measured
    • cites [28] source

      The Results and Discussion (Abstract; page unavailable) report that wrist-worn monitors’ HR accuracy varied with exercise intensity and was generally best at rest, supporting only the qualified HR portion. The study assessed four monitors, not the citing authors’ two devices, and our copy reports no evaluation of energy-expenditure (EE) accuracy. (LLM, conf=0.88)

      Experimental — accuracy not yet measured
    • cites [30] source

      The Discussion’s Principal Findings (page unavailable) describe reasonably accurate heart-rate readings for the Fitbit Charge HR under some conditions and accurate readings for the Garmin Forerunner 225 during vigorous activity. But the same section says energy-expenditure measurements were not accurate; the Results’ Energy Expenditure Overview reports an overestimation bias even where Fitbit estimates did not differ significantly from the criterion. The source studied three devices, so it cannot establish accuracy for the citing sentence’s unspecified ‘2 devices.’ (LLM, conf=0.87)

      Experimental — accuracy not yet measured
    • cites [31] source

      The Discussion reports that six monitors measured heart rate comparably to the criterion measure during resting, walking, and running. The Methods describe heart-rate readings and analyses, but the paper does not assess the accuracy of energy-expenditure (EE) estimates. Its introductory mention of EE does not establish that part of the claim. (LLM, conf=0.90)

      Experimental — accuracy not yet measured
    • cites [32] source

      The Discussion reports satisfactory heart-rate monitoring during exercise, with estimates within 1–9% of reference measurements. But the Results report larger energy-expenditure errors (9–43%), and the Discussion identifies inaccurate energy-expenditure estimation as a limitation. The paper tested four devices, not the citing sentence’s unspecified two, so it does not establish reasonable accuracy for both measures in those two devices under particular exercise conditions. (LLM, conf=0.87)

      Experimental — accuracy not yet measured
  • Findingown data

    Considerable energy expenditure error was noted for resistance activities for Garmin.

    “but considerable error was noted for some of the activities, particularly with cycling activities for Fitbit and resistance activities for Garmin.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The selected activities of daily living and structured exercises represent common daily physical activities and types of structured exercise relevant to users of wrist-worn activity monitors.
  • Findingown data

    Overall heart rate performance was significantly worse when activity mode was not indicated on the Garmin device compared with when activity mode was indicated (P<.001).

    “The overall performance was significantly worse if the activity mode was not indicated on the Garmin device compared with when activity mode was indicated (P<.001).”
    Quote in the paperSubstantial claim
  • Findingown data

    For resistance exercise, error was higher on the left hand versus the right hand (15.0% vs 13.5%, P<.001).

    “whereas the error was higher on the left hand versus the right hand for resistance (15.0% vs 13.5%, P<.001) and MAX-C (9.53 vs 2.97%, P<.001).”
    Quote in the paperNot a substantial claim
  • Findingown data

    When the activity mode is not initiated, the Garmin no-button trace appears shifted in time relative to the Polar.

    “Both panels show that when the activity mode is not initiated on the wearable, there is less accuracy and also a distinct phase shift whereby the Garmin with no button trace appears to be shifted in time relative to the Polar.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim. 31
  • May combine two claims. 18
  • The quote only partly states this claim. 4

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    In people with type 1 diabetes, aerobic exercise is known to cause steep drops in blood glucose levels, whereas anaerobic exercise can cause glucose levels to rise.

    “For example, in people with type 1 diabetes, aerobic exercise is known to cause steep drops in blood glucose levels, whereas anaerobic exercise can cause glucose levels to rise [3].”
    Quote in the paperNot a substantial claim1 of 1 sources checked

  • Classified as probably not a substantial claim.

    Premisecitation

    A 5% level of accuracy was adopted as the widely accepted acceptable limit.

    “We adopted the widely accepted level of accuracy of 5% to be within the acceptable limits [16].”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [16] source

      The Methods (§2.5, Error) set 5% error as within acceptable limits. But the paper calls statistical significance—not 5% measurement accuracy—a widely accepted standard, and the Discussion (§4) describes its clinically acceptable error threshold as a proposed standard. It therefore supports adopting the 5% limit, but not calling that accuracy limit widely accepted. (LLM, conf=0.87)

      Experimental — accuracy not yet measured

  • Classified as probably not a substantial claim.

    Premisecitation

    Monitoring of patient physical activity levels may be helpful in implementing insulin and/or nutritional strategies to optimize glucose control in type 1 diabetes.

    “Monitoring of patient physical activity levels may be helpful in implementing insulin and/or nutritional strategies to optimize glucose control in type 1 diabetes [4].”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [4] source

      The Introduction (page unavailable) says exercise type affects blood glucose in type 1 diabetes and that insulin adjustments or carbohydrate intake can affect the response. The Discussion (page unavailable) also reports different glucose responses to continuous and circuit-based exercise. The paper does not establish that monitoring patients’ physical activity levels helps implement insulin or nutritional strategies; its monitoring recommendation concerns blood glucose. (LLM, conf=0.78)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Activity monitors can be used in conjunction with on-body continuous glucose monitors, an insulin pump and a control algorithm to adjust insulin delivery, and perhaps glucagon delivery in real time.

    “In theory, activity monitors can be used in conjunction with on-body continuous glucose monitors, an insulin pump and a control algorithm to adjust insulin delivery, and perhaps glucagon delivery in real time [5,6].”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites [5] source
    • cites [6] source

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    A recent review by Bunn et al showed that EE was generally underestimated by physical activity devices and that HR measurements were generally more accurate at rest or on a cycle ergometer as compared with treadmill.

    “A recent review by Bunn et al [33] showed that EE was generally underestimated by physical activity devices and that HR measurements were generally more accurate at rest or on a cycle ergometer as compared with treadmill.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [33] Bunn, JA, 2018, Int J Exerc Sci source

  • Classified as probably not a substantial claim.

    Premisecitation

    Automated insulin delivery systems can potentially modify insulin dosing in response to activity monitors to reduce the risk or severity of exercise-induced hypoglycemia in people living with type 1 diabetes.

    “Automated insulin delivery systems can potentially modify insulin dosing in response to activity monitors to reduce the risk (or severity) of exercise-induced hypoglycemia in people living with type 1 diabetes [7-10].”
    Quote in the paperSubstantial claim3 of 4 sources checked
    • cites [7] source

      The Discussion reports that the system reduced or suspended insulin delivery during or after exercise, supporting the potential to respond to exercise-related hypoglycemia. But the Results explicitly state that there was no activity measurement: the controller responded to continuous glucose monitor readings, not activity monitors. Our copy provides no page number for these passages. (LLM, conf=0.91)

      Experimental — accuracy not yet measured
    • cites [8] source
    • cites [9] source
    • cites [10] source

  • May combine two claims.

    Premisecitation

    In recent studies involving predominantly steady-state aerobic activities, wrist-worn activity monitors have been shown to have reasonable accuracy in heart rate estimation (approximately 5% error) but a poor estimate of energy expenditure, where the error was found to be closer to approximately 30% with a negative bias.

    “In recent studies involving predominantly steady-state aerobic activities, wrist-worn activity monitors have been shown to have reasonable accuracy in HR estimation (approximately 5% error) but a poor estimate of EE, where the error was found to be closer to approximately 30% with a negative bias [16,17].”
    Quote in the paperSubstantial claim2 of 2 sources checked
    • cites [16] source

      Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

      Experimental — accuracy not yet measured
    • cites [17] source

      The Discussion (Principal Findings; page not available) says the Apple Watch does not substantially overestimate or underestimate heart rate, supporting the qualitative heart-rate point. It gives no approximately 5% heart-rate error or approximately 30% negatively biased energy-expenditure error. The Results report a 10-percentage-point underestimation of the moderate-intensity exercise threshold, which is a different measure. (LLM, conf=0.83)

      Experimental — accuracy not yet measured

  • Classified as probably not a substantial claim.

    Premisecitation

    Short nonsteady state exercises such as a 10-second maximal sprint have been shown to influence the rapid change in glucose response to aerobic exercise.

    “Additionally, short nonsteady state exercises such as a 10-second maximal sprint have been shown to influence the rapid change in glucose response to aerobic exercise [39], but findings from this study indicate that detecting these quick nonsteady exercises might be challenging for activity monitors to capture.”
    Quote in the paperSubstantial claim1 of 1 sources checked

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Skin color could influence the accuracy of the HR measurement.

    “It has been reported that skin color could influence the accuracy of the HR measurement [16], and future studies should report the Fitzpatrick skin tone scale to account for this limitation.”
    Quote in the paperNot a substantial claim1 of 1 sources checked

  • The quote only partly states this claim.

    Findingown data

    More heart rate error was observed when the exercise mode indicator was not used.

    “There was more error observed during high-intensity activities when there was a lack of repetitive wrist motion and when the exercise mode indicator was not used.”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data

    Fitbit Charge 2 and Garmin vívosmart HR+ were reasonably accurate at measuring heart rate across all activities.

    “Fitbit and Garmin were reasonably accurate at measuring HR but with an overall negative bias.”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data

    Fitbit and Garmin showed an overall systematic negative bias in heart rate measurement across all activities.

    “For HR data collected with the activity mode indicated, a systematic negative bias was observed in both Fitbit and Garmin devices.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The mean absolute percentage error (MAPE) for Garmin and Fitbit was 27.0% (SD 21.8) and 25.1% (SD 17.3), respectively.

    “MAPE (SD) for Garmin and Fitbit was 27.0% (SD 21.8) and 25.1% (SD 17.3), respectively.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing), S4 (based). The quote only partly states this claim.

    Findingown data

    Fitbit was significantly more accurate than Garmin at estimating EE during all treadmill activities.

    “Fitbit was significantly more accurate than Garmin at estimating EE during ADL (ADL: 20.9% [SD 21.8], ergometer: 42.7% [SD 26.8]; P=.02) and all treadmill activities (Treadmill: 16.9% [SD 10.9], ergometer: 42.7% [SD 26.8]; P=.003) compared with all activities performed on the cycle ergometer.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    HR accuracy as measured by these activity monitors is acceptable during low-intensity activities and high-intensity activities with repetitive wrist motion but is poorer when there is no repetitive wrist motion and when any activity is at a high intensity (≥70% of maximal aerobic capacity).

    “Second, we show that HR accuracy as measured by these activity monitors is acceptable during low-intensity activities and high-intensity activities with repetitive wrist motion but that HR accuracy is poorer when there is no repetitive wrist motion and when any activity is at a high intensity (ie, ≥70% of maximal aerobic capacity).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    Two common wrist-worn devices show good heart rate accuracy, with a small negative bias, and reasonable energy expenditure estimates during low to moderate-intensity exercise and during a variety of common daily activities and exercise.

    “Two common wrist-worn devices (Fitbit Charge 2 and Garmin vívosmart HR+) show good HR accuracy, with a small negative bias, and reasonable EE estimates during low to moderate-intensity exercise and during a variety of common daily activities and exercise.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The selected activities of daily living and structured exercises represent common daily physical activities and types of structured exercise relevant to users of wrist-worn activity monitors.
    • Rests on an assumption: The 20 healthy adults studied are representative of the broader population of wrist-worn activity monitor users.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    Participants were screened for cardiovascular complications using the Physical Activity Readiness Questionnaire.

    “Participants were screened for any cardiovascular complications using a Physical Activity Readiness Questionnaire [25].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [25] Eparmedx2018-10-31Physical Activity Readiness Questionnaire http://eparmedx.com/wp-content/uploads/2013/01/January2018PARQPlus.pdf

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    Two wearable devices, one of each brand, were tested at the same time on all participants using a randomized and counterbalanced method.

    “Two wearable devices (one of each brand) were tested at the same time on all participants (one on each wrist as per manufacturer’s instructions) using a randomized and counterbalanced method.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    All 20 participants recruited for the study completed the procedures.

    “All 20 participants recruited for the study completed the procedures.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Data from the Polar heart rate monitor served as the reference standard for heart rate in beats per minute.

    “Data from the Polar HR monitor served as the as the reference standard for HR (beats per minute, bpm).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    The experimental protocol conformed to the Declaration of Helsinki and was approved by the institutional review board at Oregon Health and Science University and by the research ethics board at York University.

    “The experimental protocol conformed to the standards set by the Declaration of Helsinki and was approved by the institutional review board at the Oregon Health and Science University (OHSU, Portland Oregon) and by the research ethics board at York University (Toronto, Canada).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Due to a limitation on the Garmin Connect application, energy expenditure data could only be compared at a low resolution, namely an average across each activity mode.

    “Due to the limitation on the Garmin Connect application, EE data could only be compared at a low resolution, namely an average across each activity mode (eg, ADL, HIIT-C, or HIIT-T).”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Data from indirect calorimetry served as the reference standard measurement for calculations of energy expenditure in kcal/min.

    “Data from the indirect calorimetry (VO2 and carbon dioxide consumption [VCO2]) served as the reference standard measurement for calculations of EE (kcal/min).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    All devices were synchronized to a single clock before the exercise protocol commenced.

    “synchronization of all the devices to a single clock before the exercise protocol commenced.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    PPG sensors used to measure HR are liable to poor accuracy during activities with increased physical exertion or activities involving repetitive contractions of forearm skeletal muscles.

    “Prior research has suggested that PPG sensors used to measure the HR are liable to poor accuracy during activities with increased physical exertion or activities involving repetitive contractions of forearm skeletal muscles [36-38].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [36] source
    • cites [37] source
    • cites [38] source

  • May combine two claims.

    Premisecitation

    Bourdreaux et al demonstrated that heart rate measured by nonwrist worn devices was relatively accurate, whereas wrist-worn devices showed poor correlations (R<.8) and higher error during resistance exercises (mean absolute percent error >9%).

    “Results from this study demonstrated that HR measured by nonwrist worn devices were relatively accurate, whereas wrist-worn devices showed poor correlations (R<.8) and higher error during resistance exercises (mean absolute percent error [MAPE] >9%).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [24] source

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationreasoning

    If EE estimates by Garmin included BMR, there would likely be greater accuracy in the EE values reported by these devices.

    “If EE estimates by Garmin included BMR, there would likely be greater accuracy in the EE values reported by these devices.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    During activities involving sustained muscle contractions or higher intensity exercises, the contact between the device's PPG sensor and skin is decreased, leading to a disruption in the signal quality and causing poor quality data.

    “It has been suggested that during activities involving sustained muscle contractions or higher intensity exercises, the contact between the device’s PPG sensor and skin is decreased, leading to a disruption in the signal quality and causing poor quality data [36,37].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [36] source
    • cites [37] source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Bai et al reported that energy expenditure measured during an unstructured resistance exercise protocol in which participants selected exercises and loads was inaccurate across numerous devices.

    “Bai et al [21] reported that EE measured during an unstructured resistance exercise protocol in which participants selected exercises and loads was inaccurate across numerous devices.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [21] source

  • May combine two claims.

    Premisecitation

    Presently, there are at least four studies that have investigated the accuracy of wearable devices during resistance exercises and none during high-intensity interval training.

    “Presently, there are at least 4 studies [21-24] that have investigated the accuracy of wearable devices during resistance exercises and none during HIIT training.”
    Quote in the paperSubstantial claim0 of 4 sources checked
    • cites [21] source
    • cites [22] source
    • cites [23] Jo, E, 2016, J Sports Sci Med

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [24] source
    • Rests on an assumption: In free-living conditions, many individuals often perform resistance exercise using free weights or their own body weight, followed by some form of high-intensity interval training within the same session.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Jo E et al reported poor correlation and heart rate accuracy in the Fitbit Charge HR device.

    “Jo E et al [23] reported poor correlation and HR accuracy in the Fitbit Charge HR device.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [23] Jo, E, 2016, J Sports Sci Med

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation

    Dondzila et al found that with aerobic exercise under laboratory conditions, the Fitbit Charge HR underestimated the HR compared with a Polar chest strap, with higher error at slower speeds.

    “Dondzila et al [34] also looked at the Fitbit Charge HR and found that with aerobic exercise under laboratory conditions, the Fitbit Charge HR underestimated the HR compared with a Polar chest strap, with higher error at slower speeds.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [34] Dondzila, C, 2018, Int J Exerc Sci source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Bourdreaux et al showed that energy expenditure measured by the devices was poor, with mean absolute percent error values ranging between 43% and 57%.

    “They also showed that the EE measured by the devices was poor, with MAPE values ranging between 43% and 57%.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [24] source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Jo et al found a negative bias of HR with respect to ECG measurements for the Basis Peak and the Fitbit Charge HR.

    “Jo et al [35] compared the Basis Peak and the Fitbit Charge HR with ECG and also found a negative bias of HR with respect to ECG measurements (−4.9 bpm for the Basis and −12.7 bpm for the Fitbit).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [35] Jo, E, 2016, J Sports Sci Med

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Premisereasoning

    For any medical system utilizing an activity monitor, the accuracy of the heart rate and energy expenditure estimates by the activity monitor is critical as it can influence medical dosing decisions and patient outcomes.

    “For any medical system utilizing an activity monitor, the accuracy of the HR and EE estimates by the activity monitor is critical as it can influence medical dosing decisions and patient outcomes.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Horton et al assessed the validity of heart rate only using the Polar M600 when compared with a 3-lead electrocardiogram during both aerobic and resistance exercises, and the accuracy of the wearable device was reported to be better during aerobic exercise (92%) than during resistance exercises (35%).

    “Horton et al [22] assessed the validity of HR only using the Polar M600 when compared with a 3-lead electrocardiogram (ECG) during both aerobic and resistance exercises. The accuracy of the wearable device was reported to be better during aerobic exercise (92%) as compared with only 35% accurate during the resistance exercises.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [22] source

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Premisenone given

    According to the manufacturers, Fitbit's EE estimate includes both active calories and the basal metabolic rate (BMR), whereas Garmin only reports active calories without BMR.

    “According to the manufacturers, Fitbit’s EE estimate includes both active calories and the basal metabolic rate (BMR), whereas Garmin only reports active calories without BMR.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    The findings from this study point to shortcomings that could arise in both detecting activity and distinguishing the type of activity based on the HR signal.

    “The findings from this study point to shortcomings that could arise in both detecting activity and distinguishing the type of activity based on the HR signal.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Exercise was conducted in a laboratory setting as opposed to the real world.

    “Another limitation of our study is that exercise was conducted in a laboratory setting as opposed to the real world.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    We attempted to capture several real-world ADLs to minimize the laboratory setting limitation, though these activities were also recorded within a lab.

    “However, we attempted to capture several real-world ADLs to minimize this limitation, though these activities were also recorded within a lab.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    Our current data and interpretations may be limited as we did not account for the skin color in our study.

    “Our current data and interpretations may be limited as we did not account for the skin color in our study.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisenone given

    Neither of these watches provide real-time access to their data streams.

    “Currently, neither of these watches provide real-time access to their data streams.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning

    Integrating these activity monitors into medical systems such as type 1 diabetes decision support systems or automated drug delivery systems in the future will require high fidelity data both from the HR signal and the EE estimates.

    “Integrating these activity monitors into medical systems such as type 1 diabetes decision support systems or automated drug delivery systems in the future will require high fidelity data both from the HR signal and the EE estimates.”
    Quote in the paperNot a substantial claim
43 of 43

  • The paper relies on this without support.

    The selected activities of daily living and structured exercises represent common daily physical activities and types of structured exercise relevant to users of wrist-worn activity monitors.

    “Six ADLs were performed to simulate daily chores.”

  • The paper relies on this without support.

    The 20 healthy adults studied are representative of the broader population of wrist-worn activity monitor users.

    “This study recruited 20 healthy adults (11 females; 10 subjects at OHSU; 10 at York University) who all provided informed consent before taking part in the study.”

  • The paper relies on this without support.

    In the diabetes population, patients are encouraged to perform both resistance and aerobic training all in one session.

    “In fact, in the diabetes population, patients are encouraged to perform both resistance and aerobic training all in one session.”

  • The paper relies on this without support.

    In free-living conditions, many individuals often perform resistance exercise using free weights or their own body weight, followed by some form of high-intensity interval training within the same session.

    “For example, in free-living conditions, many individuals often perform resistance exercise using free weights or their own body weight, followed by some form of high-intensity interval training (HIIT) within the same session.”

  • The authors state this as an assumption.

    The EE estimations provided by these devices utilize anthropometric characteristics and HR measurements.

    “We assume that the EE estimations provided by these devices are also utilizing this information, but these calculations are proprietary.”

  • The paper relies on this without support.

    The algorithms running on these wearables that estimate HR and EE are proprietary and can change without warning from the manufacturers.

    “Clearly, the algorithms running on these wearables that estimate HR and EE are proprietary and can change without warning from the manufacturers, which poses further challenges for those wanting to integrate these devices into medical products.”
6 of 6

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Apple and Fitbit's proprietary algorithms outperform smaller-brand or open-source counterparts, largely because of superior noise filtering and adaptive calibration.

    “Comparative evaluations indicate that Apple and Fitbit’s proprietary algorithms outperform smaller-brand or open-source counterparts, largely because of superior noise filtering and adaptive calibration [9,11].”

    From Photoplethysmography in Diverse Skin Tones: Evaluating Bias in Smartwatch Health Monitoring (2025) · cites this paper as [11]

    The Results (Heart Rate Accuracy; page unavailable) report no accuracy difference between Fitbit and Garmin when activity mode was indicated. The Results (Energy Expenditure Accuracy; page unavailable) report less overall bias for Garmin and significantly better Garmin accuracy in some activities, contrary to the claimed general superiority of Fitbit. The study did not compare Apple or open-source algorithms, or establish noise filtering and adaptive calibration as explanations. (LLM, conf=0.88)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

  • Cyclical motion can affect the accuracy of heart rate in wearable sensors.

    “Several research groups and manufacturers have identified that cyclical motion can affect accuracy of HR in wearable sensors.”

    From Investigating sources of inaccuracy in wearable optical heart rate sensors (2020) · cites this paper as [9]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • HIIT cycling heart-rate correlations are assigned the wrong values

    Major

    The text gives Fitbit and Garmin correlations of 0.46 and 0.71 for HIIT cycling, but the HIIT cycling row of Table 2 gives 0.56 and 0.47. The stated values instead match the table's maximal cycling test.

  • HIIT cycling is not Fitbit's highest heart-rate error

    Minor

    The abstract identifies HIIT cycling as having the highest error, but the paper reports a larger-magnitude mean error for Fitbit during maximal cycling: −16.4% versus −11.4%.

  • Conflicting SDs for Fitbit's HIIT cycling EE relative error

    Minor

    Table 3 reports an SD of 1.3 for Fitbit's −41.9% relative error during HIIT cycling, while the results text reports an SD of 31.3 for the same value.

  • Fitbit is called more accurate for daily activities despite higher absolute error

    Minor

    The text says Fitbit was more accurate than Garmin during activities of daily living, whereas Table 3 reports a higher mean absolute EE error for Fitbit (20.9%) than for Garmin (17.0%).

4 of 4

Cites

Flagged first, then by citations

Research works printed but not linked: 3

  1. Jo, E, 2016, J Sports Sci Med, 15, 540
  2. Borg, GA, 1982, Med Sci Sports Exerc, 14, 377
  3. Jo, E, 2016, J Sports Sci Med, 15, 540

Cited by

Sorted by citations

Related

Authors
Reddy, R. K., Pooni, R., Zaharieva, D. P., Senf, B., El Youssef, J., Dassau, E., Doyle III, F. J., Clements, M. A., Rickels, M. R., Patton, S. R., Castle, J. R., Riddell, M. C., & Jacobs, P. G.
Affiliations
  • York University (CA)
  • Oregon Health & Science University (US)
  • Harvard University (US)
  • Children's Mercy Hospital (US)
  • University of Pennsylvania (US)
  • University of Kansas Medical Center (US)
Corresponding author
Jacobs, Peter G
ORCID
Venue
JMIR mHealth and uHealth
Publisher
JMIR Publications Inc.
Published
2018-12-10
Volume
6
Issue
12
Pages
e10338
ISSN
2291-5222 (ISSN-L)
DOI
10.2196/10338
PMC
PMC6305876
OpenAlex
W2889730139
Semantic Scholar
13b617c10b60392edf8a861866c771c62c517ddd
Cited by
157
References
36
Citations by year
2026: 11 · 2025: 11 · 2024: 20 · 2023: 24 · 2022: 26 · 2021: 27 · 2020: 22 · 2019: 15 · 2016: 1
FWCI
9.54
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • JMIR mhealth and uhealth (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Europe PMC (PubMed Central) (submittedVersion, cc-by)
  • PubMed Central (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Leona M. and Harry B. Helmsley Charitable Trust
Topics
  • Health Sciences › Medicine › Physiology › Physical Activity and Health
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Context-Aware Activity Recognition Systems
  • Physical Sciences › Engineering › Biomedical Engineering › Non-Invasive Vital Sign Monitoring
Keywords
wrist-worn activity monitors, heart rate monitoring, energy expenditure estimation, Fitbit, Garmin, interval training, activities of daily living, VO2max, indirect calorimetry, high-intensity exercise, measurement bias, real-time physiological monitoring, wearable medical devices, clinical decision support systems, smartwatches

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review