Accuracy of Vital Signs Measurements by a Smartwatch and a Portable Health Device: Validation Study
Open accessAbstract
Background: New consumer health devices are being developed to easily monitor multiple physiological parameters on a regular basis. Many of these vital sign measurement devices have yet to be formally studied in a clinical setting but have already spread widely throughout the consumer market. Objective: The aim of this study was to investigate the accuracy and precision of heart rate (HR), systolic blood pressure (SBP), diastolic blood pressure (DBP), and oxygen saturation (SpO2) measurements of 2 novel all-in-one monitoring devices, the BodiMetrics Performance Monitor and the Everlast smartwatch. Methods: We enrolled 127 patients (>18 years) from the Thomas Jefferson University Hospital Preadmission Testing Center. SBP and HR were measured by both investigational devices. In addition, the Everlast watch was utilized to measure DBP, and the BodiMetrics Performance Monitor was utilized to measure SpO2. After 5 min of quiet sitting, four hospital-grade standard and three investigational vital sign measurements were taken, with 60 seconds in between each measurement. The reference vital sign measurements were calculated by determining the average of the two standard measurements that bounded each investigational measurement. Using this method, we determined three comparison pairs for each investigational device in each subject. After excluding data from 42 individuals because of excessive variation in sequential standard measurements per prespecified dropping rules, data from 85 subjects were used for final analysis. Results: Of 85 participants, 36 (42%) were women, and the mean age was 53 (SD 21) years. The accuracy guidelines were only met for the HR measurements in both devices. SBP measurements deviated 16.9 (SD 13.5) mm Hg and 5.3 (SD 4.7) mm Hg from the reference values for the Everlast and BodiMetrics devices, respectively. The mean absolute difference in DBP measurements for the Everlast smartwatch was 8.3 (SD 6.1) mm Hg. The mean absolute difference between BodiMetrics and reference SpO2 measurements was 3.02%. Conclusions: Both devices we investigated met accuracy guidelines for HR measurements, but they failed to meet the predefined accuracy guidelines for other vital sign measurements. Continued sale of consumer physiological monitors without prior validation and approval procedures is a public health concern.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The few existing studies do not investigate the effects of individual characteristics such as skin tone, compare across multiple devices, thoroughly examine performance in low ranges, or explore patterns of error and bias or missingness.
“The few studies that do exist do not investigate the effects of individual characteristics (e.g., skin tone), compare across multiple devices or thoroughly examine performance in low ranges, or explore patterns of error and bias (e.g., consistent over- or underestimation of measurements) or missingness [10–13].”
From Investigating the accuracy of blood oxygen saturation measurements in common consumer smartwatches (2023) · cites this paper as [10]
The Methods (Data and Statistical Analysis; page not provided) describe comparisons with reference values, Bland–Altman error plots, and assessment of abnormal ranges. The Discussion (Principal Findings; page not provided) explicitly compares two devices, reports detection of low heart rates, and quantifies failed measurements. The paper does not appear to examine skin tone, but it contradicts several of the sentence’s other claims about what these studies do not investigate. (LLM, conf=0.90)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
No references could be resolved yet.
Research works printed but not linked: 13
- [1]Torkamani A Andersen KG Steinhubl SR Topol EJ High-definition medicine Cell 2017 8 24 170 5 828 43 10.1016/j.cell.2017.08.007 28841416 28841416 PMC5714316 · DOI
- [2]Klasko SK Chen AF Digital 'Snake Oil' or the future of healthcare? Healthc Transform 2016 1 4 234 9 10.1089/heat.2016.29032.dso · DOI
- [3]Plante TB Urrea B MacFarlane ZT Blumenthal RS Miller ER Appel LJ Martin SS Validation of the instant blood pressure smartphone app JAMA Intern Med 2016 5 1 176 5 700 2 10.1001/jamainternmed.2016.0157 26938174 26938174 PMC4922794 · DOI
- [7]US Food & Drug Administration Food and Drug Administration 2001 Premarket notification 510(k) Datex Ohmeda Cardiocap 5 (Rev B) and accessories https://onlinelibrary.wiley.com/doi/pdf/10.1002/9780471462422.eoct546 · DOI
- [8]Pacagnella RC Souza JP Durocher J Perel P Blum J Winikoff B Gülmezoglu AM A systematic review of the relationship between blood loss and clinical signs PLoS One 2013 8 3 e57594 10.1371/journal.pone.0057594 23483915 23483915 PMC3590203 · DOI
- [10]Stergiou GS Alpert BS Mieke S Wang J O'Brien E Validation protocols for blood pressure measuring devices in the 21st century J Clin Hypertens (Greenwich) 2018 7 20 7 1096 9 10.1111/jch.13294 10.1111/jch.13294 30003697 30003697 PMC8030852 · DOI · Paper page
- [11]Sheshadri V Tiwari AK Nagappa M Venkatraghavan L Accuracy in blood pressure monitoring: the effect of noninvasive blood pressure cuff inflation on intra-arterial blood pressure values Anesth Essays Res 2017 11 1 169 73 10.4103/0259-1162.181430 28298779 28298779 PMC5341678 · DOI
- [12]van Helmond N Freeman CG Hahnen C Haldar N Hamati JN Bard DM Murali V Merli GJ Joseph JI The accuracy of blood pressure measurement by a smartwatch and a portable health device Hosp Pract (1995) 2019 10 47 4 211 5 10.1080/21548331.2019.1656991 31423912 31423912 · DOI
- [13]O'Brien E Waeber B Parati G Staessen J Myers M G Blood pressure measuring devices: recommendations of the European Society of Hypertension BMJ 2001 3 03 322 7285 531 6 10.1136/bmj.322.7285.531 11230071 11230071 PMC1119736 · DOI
- [14]O'Brien E Petrie J Littler W de Swiet M Padfield PL Altman DG Bland M Coats A Atkins N An outline of the revised British Hypertension Society protocol for the evaluation of blood pressure measuring devices J Hypertens 1993 6 11 6 677 9 10.1097/00004872-199306000-00013 8397248 8397248 · DOI
- [18]Weenk M van Goor H van Acht M Engelen LJ van de Belt TH Bredie SJ A smart all-in-one device to measure vital signs in admitted patients PLoS One 2018 13 2 e0190138 10.1371/journal.pone.0190138 29432461 29432461 PMC5809012 · DOI
- [19]Ogink PA de Jong JM Koeneman M Weenk M Engelen LJ van Goor H van de Belt TH Bredie SJ Feasibility of a new cuffless device for ambulatory blood pressure measurement in patients with hypertension: mixed methods study J Med Internet Res 2019 6 19 21 6 e11164 10.2196/11164 31219050 31219050 PMC6607776 · DOI
- [20]IEEE Engineering in Medicine and Biology Society - IEEE Standard for Wearable, Cuffless, Blood Pressure Measuring Devices 2014 New York The Institue of Electrical and Electronis Engineers Inc
Cited by
Sorted by citations
Cao, R., Azimi, I., Sarhaddi, F., Niela-Vilen, H., Axelin, A., Liljeberg, P., & Rahmani, A. M. (2022) · Journal of Medical Internet Research · doi:10.2196/27487 · Cited by 159 · see citing works
Investigating the accuracy of blood oxygen saturation measurements in common consumer smartwatches
Jiang, Y., Spies, C., Magin, J., Bhosai, S. J., Snyder, L., & Dunn, J. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000296 · Cited by 36 · see citing works
- Authors
- Hahnen, C., Freeman, C. G., Haldar, N., Hamati, J. N., Bard, D. M., Murali, V., Merli, G. J., Joseph, J. I., & van Helmond, N.
- Affiliations
- Radboud University Nijmegen (NL)
- Thomas Jefferson University (US)
- ORCID
- Venue
- JMIR mHealth and uHealth
- Publisher
- JMIR Publications Inc.
- Published
- 2020-02-12
- Volume
- 8
- Issue
- 2
- Pages
- e16811
- ISSN
- 2291-5222 (ISSN-L)
- PMC
- PMC7055753
- OpenAlex
- W3006639564
- Cited by
- 55
- References
- 0
- Citations by year
- 2026: 6 · 2025: 12 · 2024: 6 · 2023: 7 · 2022: 15 · 2021: 9
- FWCI
- 2.44
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- JMIR mhealth and uhealth (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- The Medicine Forum (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Radboud Repository (Radboud University) (submittedVersion)
- Radboud Repository (Radboud University) (submittedVersion, other-oa)
- Topics
- Physical Sciences › Engineering › Biomedical Engineering › Non-Invasive Vital Sign Monitoring
- Health Sciences › Medicine › Surgery › Hemodynamic Monitoring and Therapy
- Health Sciences › Health Professions › General Health Professions › Mobile Health and mHealth Applications
- Keywords
- vital signs, smartwatches, consumer health devices, device accuracy, heart rate measurement, blood pressure measurement, systolic blood pressure, diastolic blood pressure, oxygen saturation, preadmission testing, device validation
- MeSH terms
- Wearable Electronic Devices, Blood Pressure, Blood Pressure Determination, Female, Heart Rate, Humans, Male, Middle Aged, Monitoring, Physiologic, Oximetry, Reproducibility of Results, Blood Pressure Monitors, Mobile Applications
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
