Guidelines for wrist-worn consumer wearable assessment of heart rate in biobehavioral research

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Between 2008 and 2018 there was a 580% increase in total articles published that contained "wearable" and "heart rate".

“Between 2008 and 2018 there was a 580% increase in total articles published that contained “wearable” and “heart rate”, indicating that this is a burgeoning area of research that may benefit from standardization of procedures to increase generalizability and clinical utility of research findings.”

Probably not a substantial claim: may not pass S4 (based).

Researchers have increasingly used consumer wearables or wrist-worn smartwatches and fitness monitors for measurement of cardiovascular psychophysiological processes related to mental and physical health outcomes.

“Researchers have increasingly begun to use consumer wearables or wrist-worn smartwatches and fitness monitors for measurement of cardiovascular psychophysiological processes related to mental and physical health outcomes.”

Probably not a substantial claim: may not pass S4 (based).

Consumer wearables allow continuous, scalable, unobtrusive, and ecologically valid data collection of cardiac activity in big data studies.

“These devices have strong appeal because they allow for continuous, scalable, unobtrusive, and ecologically valid data collection of cardiac activity in “big data” studies.”

May combine two claims.

Heart rate is predominantly influenced by coordination of the sympathetic and parasympathetic branches of the autonomic nervous system, which can be modified by biopsychosocial factors including physical and relational stress, diet, physical fitness, medication use, and substance use.

“HR is predominantly influenced by the coordination of the sympathetic and parasympathetic branches of the autonomic nervous system, which can be modified by biopsychosocial factors including physical and relational stress, diet, physical fitness, medication use, and substance use1; thus, offering a direct and quantifiable connection to the stress diathesis model of health.”

• cites [1] Cacioppo J. T., Tassinary L. G., & Berntson G. G. (eds) Handbook of Psychophysiology 4th edn. (Cambridge University Press, 2017). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Collection of heart rate data in laboratory or medical environments lacks temporal resolution within and across days as heart rate is usually collected during a short recording within a single day, which has precluded research from comprehensively investigating the transition from acute to chronic stress.

“Second, collection of HR data in these environments lacks temporal resolution within and across days as HR is usually collected during a short recording within a single day, which has precluded research from comprehensively investigating the transition from acute to chronic stress22.”

• cites [22] Rohleder, N. Stress and inflammation—the need to address the gap in the transition between acute and chronic stress effects. Psychoneuroendocrinology https://doi.org/10.1016/j.psyneuen.2019.02.021 (2019). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based).

Heart rate research has been constrained by inability to collect temporally detailed and longitudinal heart rate measurements both within and across time.

“(2) inability to collect temporally detailed and longitudinal HR measurements both within and across time (e.g., hours, days, weeks, months)”

May combine two claims.

Researchers are increasingly using consumer wearables or wrist-worn smartwatches and fitness monitors for digital measurement of cardiovascular health and psychophysiological stress because these devices allow continuous, scalable, unobtrusive big data collection of cardiac activity in real-world conditions with large samples.

“Researchers are increasingly using consumer wearables or wrist-worn smartwatches and fitness monitors for the digital measurement of cardiovascular health and psychophysiological stress as these devices allow for continuous, scalable, and unobtrusive, “big data” collection of overall cardiac activity in real-world conditions with large samples25–30.”

• cites [25] Bradshaw, B. et al. Influenza surveillance using wearable mobile health devices. In International Society for Disease Surveillance Conference 2019 (2019).Not checked yet

  Not checked in the free run.

• cites [26] Natarajan, A., Pantelopoulos, A., Emir-Farinas, H. & Natarajan, P. Heart rate variability with photoplethysmography in 8 million individuals: results and scaling relations with age, gender, and time of day. Preprint at https://www.biorxiv.org/content/10.1101/772285v1 (2019). sourceNot checked yet

  Not checked in the free run.

• cites [27] Perez, M. V. et al. Large-scale assessment of a smartwatch to identify atrial fibrillation. N. Engl. J. Med. 381, 1909–1917 (2019). sourceNot checked yet

  Not checked in the free run.

• cites [28] Sim, I. Mobile devices and health. N. Engl. J. Med. 381, 956–968 (2019). sourceNot checked yet

  Not checked in the free run.

• cites [29] Tison, G. H. et al. Passive detection of atrial fibrillation using a commercially available smartwatch. JAMA Cardiol. 3, 409 (2018). sourceNot checked yet

  Not checked in the free run.

• cites [30] Tison, G. H. et al. Cardiovascular Risk Stratification Using Off-the-Shelf Wearables and a Multi-Task Deep Learning Algorithm. In:; 2018. sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Some recent wrist-worn wearable devices, such as the Apple Watch Series 4 and 5, have allowed for collection of heart rate via electrical impulses on par with a single-lead electrode.

“The latter of these two is the predominant method for data collected with wrist-worn wearable devices, although some more recent wrist-worn wearable devices, such as the Apple Watch Series 4 and 5, have allowed for the collection of HR via electrical impulses on par with a single-lead electrode.”

May combine two claims.

Research has historically been limited to collecting data from medical patients and research participants in novel settings that might not generalize to everyday lives, creating potential for inaccurate baselines and inaccurate measurement of heart rate dynamics.

“In other words, research has been historically limited to collecting data from medical patients and research participants in novel settings that might not generalize to their everyday lives, which creates the potential for inaccurate baselines and therefore inaccurate measurement of HR dynamics (i.e., reactivity/recovery).”

Probably not a substantial claim: may not pass S4 (based).

Heart rate research has been constrained by prohibitive financial cost of devices for everyday use.

“(3) prohibitive financial cost of devices for everyday use.”

May combine two claims.

Photoplethysmography can be used to index heart rate variability, blood pressure, oxygen saturation, and cardiac output, but as of yet has not been widely rolled out in consumer-use devices.

“Furthermore, PPG can be used to index heart rate variability (HRV), blood pressure, oxygen saturation, and cardiac output24, but as of yet have not be widely rolled out in consumer-use devices and therefore data collection other than HR will not be discussed further.”

• cites [24] Elgendi, M. On the analysis of fingertip photoplethysmogram signals. Curr. Cardiol. Rev. 8, 14–25 (2012). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Until recently, quantification and measurement of heart rate has been relegated to medical and research settings, largely precluding observation of heart rate during daily life and limiting generalizability of findings to real-world living conditions.

“Until recently, the quantification and measurement of HR has been relegated to medical and research settings, which has largely precluded the observation of HR during patient and participant daily life, thereby limiting the generalizability of findings to real-world living conditions.”

May combine two claims.

Red LEDs are more commonly used in medical settings, are not absorbed as well by the skin allowing light to transmit deeper and detect multiple biological measures, but have a lower signal-to-noise ratio and are more susceptible to motion artifacts.

“In contrast, red LEDs, which are more commonly used in medical settings, are not absorbed as well by the skin allowing the light to transmit deeper and allowing for detection of multiple biological measures, but it has a lower signal-to-noise ratio and is more susceptible to motion artifacts23.”

• cites [23] Mahloko, L. & Adebesin, F. in Responsible Design, Implementation and Use of Information and Communication Technology. Lecture Notes in Computer Science (eds Hattingh, M., et al.) Vol. 12067, 96–107 (Springer International Publishing, Cham, 2020). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Green LED light is a shorter wavelength and has good signal-to-noise ratio and resistance to motion artifacts, but is limited by the amount of light that can pass through tissue, especially darker skin tones.

“For example, green LED light is a shorter wave-length and has been shown to be convenient, because it has good signal-to-noise ratio and is resistant to motion artifacts, but is limited by the amount of light that can pass through tissue, especially darker skin tones.”

May combine two claims.

This novel electrocardiogram technology only allows for discrete, rather than continuous, collection of data, which precludes temporally detailed longitudinal ECG collection at this time.

“Therefore, this novel electrocardiogram (ECG) technology only allows for the discrete, rather than continuous, collection of data, which precludes the ability for temporally detailed longitudinal ECG collection at this time.”

May combine two claims.

Heart rate may be a transdiagnostic biomarker with clinical utility for psychology, psychiatry, and medicine to improve nosology and increase identification of those at risk for onset of psychiatric and physical health disorders.

“In other words, HR may be a transdiagnostic biomarker with clinical utility for psychology, psychiatry, and medicine to improve nosology and increase identification of those at risk for the onset of psychiatric and physical health disorders.”

Probably not a substantial claim: may not pass S4 (based).

Heart rate research has been constrained by lack of ecological validity.

“The foundation of HR research has, until recently, been largely constrained due to (1) lack of ecological validity (i.e., the degree to which a laboratory or medical environment represents actual real-world conditions)”

May combine two claims.

Wrist-worn wearables provide an opportunity to examine heart rate in real-world environments, across longer periods of time, and with high resolution at low cost, by using photoplethysmography.

“Currently, wrist-worn wearables provide an opportunity to examine HR in real-world environments, across longer periods of time, and with high resolution at a low cost, by using an optical sensor (photoplethysmography; PPG), that allows these devices to collect volumetric changes in blood profusion (i.e., pulse rate) that can serve as a surrogate for HR”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The reproducibility and replicability crisis is important for digital health and mHealth research, including wrist-worn heart rate accuracy and studies utilizing wearable heart rate data, because this new field allows for large data samples with various sensors that increase the ease for p-hacking.

“Recent information on the reproducibility and replicability crisis has swept the sciences and as such these issues are very important for digital health and mHealth research, which includes wrist-worn HR accuracy and any studies that utilize wearable HR data, as this new field will allow for large data samples31 with various sensors that will increase the ease for p-hacking.”

• cites [31] Stupple, A., Singerman, D., & Celi, L. A. The reproducibility crisis in the age of digital medicine. npjDigital Med. 2, https://doi.org/10.1038/s41746-019-0079-z (2019). sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

Questions remain as to the accuracy and validity of data collected from wearables.

“yet questions remain as to the accuracy and validity of data collected from wearables35,36.”

• cites [35] Evenson, K. R., Goto, M. M., & Furberg, R. D. Systematic review of the validity and reliability of consumer-wearable activity trackers. Int. J. Behav. Nutr. Phys. Activity. 12, https://doi.org/10.1186/s12966-015-0314-1 (2015). sourceNot checked yet

  Not checked in the free run.

• cites [36] Peake, J. M., Kerr, G., & Sullivan, J. P. A critical review of consumer wearables, mobile applications, and equipment for providing biofeedback, monitoring stress, and sleep in physically active populations. Front. Physiol. 9, https://doi.org/10.3389/fphys.2018.00743 (2018). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Some studies have used the gold-standard reference method of electrocardiogram, which directly measures heart rate via electrical impulses from the heart and is used for clinical accuracy.

“Although some studies have attempted to address wearable HR accuracy using the gold-standard reference method of ECG, which directly measures HR via electrical impulses from the heart and is used for clinical accuracy38–44”

• cites [38] Nelson, B. W. & Allen, N. B. Accuracy of consumer wearable heart rate measurement during an ecologically valid 24-hour period: intraindividual validation study. JMIR mHealth uHealth 7, e10828 (2019). sourceNot checked yet

  Not checked in the free run.

• cites [39] Boudreaux, B. D. et al. Validity of wearable activity monitors during cycling and resistance exercise. Med. Sci. Sports Exerc. 50, 624–633 (2018). sourceNot checked yet

  Not checked in the free run.

• cites [40] de Zambotti, M. et al. Measures of sleep and cardiac functioning during sleep using a multi-sensory commercially-available wristband in adolescents. Physiol. Behav. 158, 143–149 (2016). sourceNot checked yet

  Not checked in the free run.

• cites [41] Kroll, R. R., Boyd, J. G. & Maslove, D. M. Accuracy of a wrist-worn wearable device for monitoring heart rates in hospital inpatients: a prospective observational study. J. Med. Internet Res. 18, e253 (2016). sourceNot checked yet

  Not checked in the free run.

• cites [42] Shcherbina, A. et al. Accuracy in wrist-worn, sensor-based measurements of heart rate and energy expenditure in a diverse cohort. J. Personalized Med. 7, 3 (2017). sourceNot checked yet

  Not checked in the free run.

• cites [43] Wallen, M. P., Gomersall, S. R., Keating, S. E., Wisløff, U., & Coombes, J. S. Accuracy of heart rate watches: implications for weight management. PLoS ONE 11, https://doi.org/10.1371/journal.pone.0154420 (2016). sourceNot checked yet

  Not checked in the free run.

• cites [44] Wang, R. et al. Accuracy of wrist-worn heart rate monitors. JAMA Cardiol. 2, 104–106 (2017). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

While overall summary statistics after outlier detection and data cleaning may be very accurate for research purposes, any single observation in real-time may have a large degree of error, which could be significant for moment-to-moment observations in medical settings.

“This finding indicated that while overall, summary statistics after outlier detection and data cleaning may be very accurate for research purposes, any single observation in real-time may have a large degree of error, which could be significant for moment-to-moment observations in medical settings.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Some wearable devices, such as the Apple Watch 4 and 5, have gained FDA 510(k) class II clearance for the ECG feature and ability to detect arrhythmias.

“Furthermore, some wearable devices, such as the Apple Watch 4 and 5, have gained FDA 510(k) class II clearance for the ECG feature and ability to detect arrhythmias49”

• cites [49] FDA. Electrocardiographsoftware for Over-the-Counter Use. https://www.accessdata.fda.gov/cdrh_docs/pdf18/DEN180044.pdf (FDA, 2018).Not checked yet

  Not checked in the free run.

May combine two claims.

Consumer wearables such as the Apple Watch and Fitbit are more accurate than research-grade wearables, which are much more expensive yet provide the benefit of raw data.

“Furthermore, recent research has indicated that consumer wearables, such as the Apple Watch and Fitbit are more accurate than research-grade wearables, which are much more expensive yet provide the benefit of raw data50.”

• cites [50] Bent, B., Goldstein, B. A., Kibbe, W. A., & Dunn, J. P. Investigating sources of inaccuracy in wearable optical heart rate sensors. npj Digital Med. 3, https://doi.org/10.1038/s41746-020-0226-6 (2020). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

In a study comparing ambulatory electrocardiogram, Apple Watch 3, and Fitbit Charge 2 within individuals across 24 h, devices slightly underestimated heart rate when values were aggregated within and across different daily activities, indicating high heart rate accuracy.

“For example, researchers compared the accuracy of ambulatory ECG, Apple Watch 3, and Fitbit Charge 2 data within an individual across 24 h and found that while devices slightly underestimated HR when values were aggregated within and across different daily activities, which indicated high HR accuracy”

• cites [38] Nelson, B. W. & Allen, N. B. Accuracy of consumer wearable heart rate measurement during an ecologically valid 24-hour period: intraindividual validation study. JMIR mHealth uHealth 7, e10828 (2019). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Many researchers have begun to push for increased focus on reproducible and open practices in science.

“Therefore, many researchers have begun to push for increased focus on reproducible and open practices in science31,53–55.”

• cites [31] Stupple, A., Singerman, D., & Celi, L. A. The reproducibility crisis in the age of digital medicine. npjDigital Med. 2, https://doi.org/10.1038/s41746-019-0079-z (2019). sourceNot checked yet

  Not checked in the free run.

• cites [53] Bavel, J. J. V., Mende-siedlecki, P., Brady, W. J. & Reinero, D. A. Contextual sensitivity in scientific reproducibility. Proc. Natl Acad. Sci. USA 113, 6454–6459 (2016). sourceNot checked yet

  Not checked in the free run.

• cites [54] Munafò, M. R. et al. A manifesto for reproducible science. Nat. Hum. Behav. 1, https://doi.org/10.1038/s41562-016-0021 (2017). sourceNot checked yet

  Not checked in the free run.

• cites [55] Tackett, J. L., Brandes, C. M., King, K. M. & Markon, K. E. Psychology’s replication crisis and clinical psychological science. Annu. Rev. Clin. Psychol. 15, 579–604 (2019). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S1 (asserted). May combine two claims.

Wearables detect heart rate with acceptable accuracy in both laboratory and real-world settings for research, but importantly not medical settings.

“These findings have led some researchers to conclude that wearables detect HR with acceptable accuracy in both laboratory and real-world settings for research, but importantly not medical settings38,42 as more research needs to be done.”

• cites [38] Nelson, B. W. & Allen, N. B. Accuracy of consumer wearable heart rate measurement during an ecologically valid 24-hour period: intraindividual validation study. JMIR mHealth uHealth 7, e10828 (2019). sourceNot checked yet

  Not checked in the free run.

• cites [42] Shcherbina, A. et al. Accuracy in wrist-worn, sensor-based measurements of heart rate and energy expenditure in a diverse cohort. J. Personalized Med. 7, 3 (2017). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Apple Watch has lower mean difference standard deviation and higher agreement with electrocardiogram than Fitbit devices.

“lower mean difference standard deviation43, and higher agreement with ECG than Fitbit devices39,44.”

• cites [39] Boudreaux, B. D. et al. Validity of wearable activity monitors during cycling and resistance exercise. Med. Sci. Sports Exerc. 50, 624–633 (2018). sourceNot checked yet

  Not checked in the free run.

• cites [43] Wallen, M. P., Gomersall, S. R., Keating, S. E., Wisløff, U., & Coombes, J. S. Accuracy of heart rate watches: implications for weight management. PLoS ONE 11, https://doi.org/10.1371/journal.pone.0154420 (2016). sourceNot checked yet

  Not checked in the free run.

• cites [44] Wang, R. et al. Accuracy of wrist-worn heart rate monitors. JAMA Cardiol. 2, 104–106 (2017). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Hardware differences between devices determine heart rate measurement second to second, with large implications for overall interpretability of the data, such that sampling rate introduces researcher degrees of freedom for calculating bpm.

“Hardware differences between devices determine HR measurement second to second, which has large implications on overall interpretability of the data, such that sampling rate can introduces researcher degrees of freedom for calculating bpm.”

Probably not a substantial claim: may not pass S2 (checkable).

For long-term field studies, information about participant adherence and wear-time should be considered and reported since these issues can affect data quality.

“For long-term field studies, information about participant adherence and wear-time should be considered and reported since these issues can affect data quality.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Outliers in linear models may strongly impact linear estimators, and some studies may therefore adopt a modeling strategy wherein values are winsorized.

“For example, outliers in linear models may strongly impact linear estimators, and some studies may therefore adopt a modeling strategy wherein values are winsorized (e.g., extreme values can be truncated at the 2.5th and 97.5th quantiles56).”

• cites [56] Hastings, C. Jr., Mosteller, F., Tukey, J. W. & Winsor, C. P. Low moments for small samples: a comparative study of order statistics. Ann. Math. Stat. 18, 413–426 (1947). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Fitbit logs heart rate data at 1 s intervals during exercise tracking and 5 s intervals at other times, whereas the Apple Watch samples heart rate data every 10 min, except during workouts, which are continuous.

“For example, Fitbit logs HR data at 1 s intervals during exercise tracking and 5 s intervals at other times, whereas the Apple Watch samples HR data every 10 min, except during workouts, which are continuous.”

Probably not a substantial claim: may not pass S2 (checkable).

Data cleaning methods should be customized to the device being used and the models being adopted, rather than adopting a universal pre-processing pipeline, due to device-specific differences and study research questions.

“There is a strong need to customize the methods of data cleaning to the device being used and the models being adopted, rather than adopting a universal pre-processing pipeline due to device-specific differences and based on the study research questions.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

There are many researcher degrees of freedom in decisions related to study protocol design and data collection, cleaning, and processing that are important to note as standardizing this process may help ameliorate some barriers and inconsistencies in the literature.

“There are many researcher degrees of freedom when it comes to decisions related to study protocol design and data collection, cleaning, and processing that are important to note as standardizing this process may help ameliorate some barriers and inconsistencies in the literature.”

May combine two claims.

Listwise deletion is known to bias estimation and adversely impact the standard error of point estimates across modeling strategies.

“The practice of completely removing persons from studies due to proportions of missing data (i.e. listwise deletion) is known to bias estimation and adversely impact the standard error of point estimates across modeling strategies58.”

• cites [58] Graham, J. W. Missing data analysis: making it work in the real world. Annu. Rev. Psychol. 60, 549–576 (2009). sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

If a study is interested in heart rate arrhythmias using wearable assessments, extreme values may be of vital interest and should not be removed or altered.

“However, if a study were interested in HR arrythmias using wearable assessments (such as atrial fibrillation), then these extreme values may be of vital interest and should not be removed or altered27.”

• cites [27] Perez, M. V. et al. Large-scale assessment of a smartwatch to identify atrial fibrillation. N. Engl. J. Med. 381, 1909–1917 (2019). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based).

Methods researchers select to clean and process raw data prior to analyses can affect results and conclusions.

“In addition, the methods researchers select to clean and process raw data prior to analyses can also affect results and conclusions and should be reported in detail.”

May combine two claims.

Larger wrist circumference has been shown to be associated with reduced heart rate accuracy, although these outcomes have not been found in all studies.

“Larger wrist circumference has been shown to be associated with reduced HR accuracy42, although these outcomes have not been found in all studies44,46.”

• cites [42] Shcherbina, A. et al. Accuracy in wrist-worn, sensor-based measurements of heart rate and energy expenditure in a diverse cohort. J. Personalized Med. 7, 3 (2017). sourceNot checked yet

  Not checked in the free run.

• cites [44] Wang, R. et al. Accuracy of wrist-worn heart rate monitors. JAMA Cardiol. 2, 104–106 (2017). sourceNot checked yet

  Not checked in the free run.

• cites [46] Gillinov, S. et al. Variable accuracy of wearable heart rate monitors during aerobic exercise. Med. Sci. Sports Exerc. 49, 1697–1703 (2017). sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

Some devices report missing heart rate data with zero values; these values should be removed and filled in with missing data rather than winsorized, which would otherwise substantially bias parameter estimates unless the degree of missing heart rate points was trivial.

“Moreover, for some, but not all devices will report missing HR data with zero values and these values should be removed and filled in with missing data rather than winsorized (which would otherwise substantially bias parameter estimates unless the degree of missing HR points was trivial).”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

One issue reported by multiple users on the online Fitbit Community Forum is occasional logging of heart rate values (often higher and more variable than typical, e.g., 100-150 bpm) when the devices are not being worn, suggesting that supplemental non-wear-time diaries may be required for some devices or protocols.

“One issue reported by multiple users on the online Fitbit Community Forum is occasional logging of HR values (often higher and more variable than typical, e.g., 100–150 bpm) when the devices are not being worn, suggesting that supplemental non-wear-time diaries may be required for some devices or protocols.”

May combine two claims.

Some devices, such as the Apple Watch, can take multiple heart rate measurements within a minute, allowing researchers to use various calculations such as the mean or median within this minute to calculate bpm, which may alter wearable data values.

“Relatedly, some devices, such as the Apple Watch can take multiple HR measurements within a minute allowing researchers to use various calculations, such as the mean or median within this minute to calculate bpm, which may alter wearable data values.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Absolute error during activity is 30% higher than at rest.

“some research showing that absolute error during activity is 30% higher than rest50”

• cites [50] Bent, B., Goldstein, B. A., Kibbe, W. A., & Dunn, J. P. Investigating sources of inaccuracy in wearable optical heart rate sensors. npj Digital Med. 3, https://doi.org/10.1038/s41746-020-0226-6 (2020). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

Low power likely will not be an issue for mHealth research as large sample sizes will be relatively easy to collect even in samples with low base rates in the population.

“the last of which likely will not be an issue for mHealth research as large samples sizes will be relatively easy to collect even in samples with low base rates in the population.”

May combine two claims.

Failure to report or control for participant race and ethnicity in wrist-worn wearable heart rate studies undermines the ability to examine potentially important group differences in health outcomes and decreases the ability of results to generalize to large segments of the population.

“Currently, the vast majority of wrist-worn wearable studies incorporating HR measurements do not report or control for participant race and ethnicity, which undermines the ability to look at potentially important group differences in health outcomes and therefore decreases the ability of results to generalize to large segments of the population.”

• AssumptionRests on an assumption: The vast majority of wrist-worn wearable studies incorporating heart rate measurements do not report or control for participant race and ethnicity.

May combine two claims.

Darker skin tone (and tattoos) has been found to be associated with reduced wrist-worn heart rate accuracy as they absorb more green light, yet this is not found in all studies.

“Evidence is starting to accumulate on the importance of skin tone when it comes to accuracy and generalizability of wrist-worn wearable studies with darker skin tone (and tattoos) being found to be associated with reduced wrist-worn HR accuracy as they absorb more green light67, yet this is not found in all studies43,50.”

• cites [67] Weiler, D. T., Villajuan, S. O., Edkins, L., Cleary, S. & Saleem, J. J. Wearable heart rate monitor technology accuracy in research: a comparative study between PPG and ECG technology. Proc. Hum. Factors Ergonomics Soc. Annu. Meet. 61, 1292–1296 (2017). sourceNot checked yet

  Not checked in the free run.

• cites [43] Wallen, M. P., Gomersall, S. R., Keating, S. E., Wisløff, U., & Coombes, J. S. Accuracy of heart rate watches: implications for weight management. PLoS ONE 11, https://doi.org/10.1371/journal.pone.0154420 (2016). sourceNot checked yet

  Not checked in the free run.

• cites [50] Bent, B., Goldstein, B. A., Kibbe, W. A., & Dunn, J. P. Investigating sources of inaccuracy in wearable optical heart rate sensors. npj Digital Med. 3, https://doi.org/10.1038/s41746-020-0226-6 (2020). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Wearables that use red light will likely be more effective on darker skin tones because red light is not absorbed by melanin, which may allow for more accurate heart rate measurements.

“wearables that use red light will likely be more effective on darker skin tones as it is not absorbed by melanin, which may allow for more accurate HR measurements23.”

• cites [23] Mahloko, L. & Adebesin, F. in Responsible Design, Implementation and Use of Information and Communication Technology. Lecture Notes in Computer Science (eds Hattingh, M., et al.) Vol. 12067, 96–107 (Springer International Publishing, Cham, 2020). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Higher body mass index, which likely covaries with larger wrist circumference, has been shown to be associated with reduced heart rate accuracy, although these outcomes have not been found in all studies.

“Higher body mass index, which likely covaries with larger wrist circumference as described above, has been shown to be associated with reduced HR accuracy42,62, although again these outcomes have not been found in all studies44,46.”

• cites [42] Shcherbina, A. et al. Accuracy in wrist-worn, sensor-based measurements of heart rate and energy expenditure in a diverse cohort. J. Personalized Med. 7, 3 (2017). sourceNot checked yet

  Not checked in the free run.

• cites [62] Menghini, L. et al. Stressing the accuracy: wrist-worn wearable sensor validation over different conditions. Psychophysiology. 56, https://doi.org/10.1111/psyp.13441 (2019). sourceNot checked yet

  Not checked in the free run.

• cites [44] Wang, R. et al. Accuracy of wrist-worn heart rate monitors. JAMA Cardiol. 2, 104–106 (2017). sourceNot checked yet

  Not checked in the free run.

• cites [46] Gillinov, S. et al. Variable accuracy of wearable heart rate monitors during aerobic exercise. Med. Sci. Sports Exerc. 49, 1697–1703 (2017). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Hair follicle density and sweat can influence device measurement.

“some research indicates that hair follicle density and sweat can influence device measurement69.”

• cites [69] Cosoli, G., Spinsante, S. & Scalise, L. Wrist-worn and chest-strap wearable devices: systematic review on accuracy and metrological characteristics. Measurement 159, 107789 (2020). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Historically, science and medicine have paid less attention to recruiting minority groups, which has resulted in blunted health gains in these populations.

“The importance of this point cannot be overstated as historically, science and medicine have paid less attention to recruiting minority groups68, which has resulted in blunted health gains in these populations”

• cites [68] Hall, G. C. N., Yip, T. & Zárate, M. A. On becoming multicultural in a monocultural research world: a conceptual approach to studying ethnocultural diversity. Am. Psychol. 71, 40–51 (2016). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Wearables that use red light will likely be more effective on darker skin tones as it is not absorbed by melanin.

“Furthermore, wearables that use red light will likely be more effective on darker skin tones as it is not absorbed by melanin, which may allow for more”

May combine two claims.

Consumer wearable devices can be used to supplement medical-grade devices and continuously collect heart rate data on research subjects when risk for acute cardiac events is not of immediate concern.

“these devices can be used to supplement medical-grade devices and continuously collect HR data on research subjects when risk for acute cardiac events is not of immediate concern.”

Probably not a substantial claim: may not pass S3 (load-bearing).

A second study found that accuracy of heart rate measurement was highest during running.

“a second study found that the accuracy of HR measurement was highest during running51”

• cites [51] Xie, J. et al. Evaluating the validity of current mainstream wearable devices in fitness tracking under various physical activities: comparative study. JMIR Mhealth Uhealth 6, e94 (2018). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable).

The recent introduction of consumer wearables has the potential to improve the study of cardiovascular health in medicine and psychophysiological stress in psychiatry and psychology.

“This has the potential to improve the study of cardiovascular health in medicine and psychophysiological stress in psychiatry and psychology.”

• AssumptionRests on an assumption: Consumer wearables allow continuous, scalable, unobtrusive, big data collection of cardiac activity in real-world conditions with large samples.

Probably not a substantial claim: may not pass S2 (checkable).

Using consumer wearables to supplement medical-grade devices has the potential to advance research into cardiovascular health and psychophysiological stress in order to better understand how heart rate influences overall physical and psychological health.

“This has the potential to advance research into cardiovascular health and psychophysiological stress in order to better understand how HR influences overall physical and psychological health.”

May combine two claims.

PPG green light is less susceptible to motion artifacts, whereas PPG red light is more susceptible to motion artifacts.

“one strength of PPG green light is that it is less susceptible to motion artifacts, whereas PPG red light is more susceptible to motion artifacts”

• cites [23] Mahloko, L. & Adebesin, F. in Responsible Design, Implementation and Use of Information and Communication Technology. Lecture Notes in Computer Science (eds Hattingh, M., et al.) Vol. 12067, 96–107 (Springer International Publishing, Cham, 2020). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Across four consumer wearables and two research-grade devices, all were reasonably accurate during rest or during prolonged elevated heart rate, and differences in accuracy tended to arise during changes in activity.

“Other research found that across four consumer wearables and two research-grade devices all were “reasonably accurate” during rest or during prolonged HR that was elevated and that differences in accuracy tended to arise during changes in activity50.”

• cites [50] Bent, B., Goldstein, B. A., Kibbe, W. A., & Dunn, J. P. Investigating sources of inaccuracy in wearable optical heart rate sensors. npj Digital Med. 3, https://doi.org/10.1038/s41746-020-0226-6 (2020). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Some researchers have worked with industry partners to derive measurements of heart rate variability.

“although some researchers have worked with industry partners to derive measurements of HRV26.”

• cites [26] Natarajan, A., Pantelopoulos, A., Emir-Farinas, H. & Natarajan, P. Heart rate variability with photoplethysmography in 8 million individuals: results and scaling relations with age, gender, and time of day. Preprint at https://www.biorxiv.org/content/10.1101/772285v1 (2019). sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Activity intensity may be less important to device accuracy than the degree of erratic wrist movements and corresponding position of the wearable device during an activity, which cause motion artifacts and may or may not co-occur with more vigorous physical activity.

“Therefore, it is possible that activity intensity may be less important to device accuracy than the degree of erratic wrist movements and corresponding position of the wearable device during an activity74, which cause motion artifacts and may or may not co-occur with”

• cites [74] Parak, J. & Korhonen, I. Evaluation of wearable consumer heart rate monitors based on photopletysmography. Conf. Proc. IEEE Eng. Med. Biol. Soc. 3670-3673 2014, 3670–3673 (2014).Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Some studies found that accuracy of heart rate measurement was comparable across resting baseline or vigorous activity.

“some studies finding that accuracy of HR measurement was comparable across resting baseline or vigorous activity45”

• cites [45] Dooley, E. E., Golaszewski, N. M. & Bartholomew, J. B. Estimating accuracy at exercise intensities: a comparative study of self-monitoring heart rate and physical activity wearable devices. JMIR mHealth uHealth 5, e34 (2017). sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

Utilizing these guidelines and referencing them in methods sections has the potential to decrease non-standard reporting, increase reproducibility and replicability, and lead to greater generalizability of results, which may increase clinical utility.

“Utilizing these guidelines and referencing them in methods sections has the potential to decrease non-standard reporting, increase reproducibility and replicability, while also leading to greater generalizability of results, which may increase the clinical utility of findings in this field.”