CoChat
Sign Up
Cited by 36Non-Invasive Vital Sign MonitoringHemodynamic Monitoring and TherapyCardiovascular Health and Disease Prevention

Impact of skin pigmentation on pulse oximetry blood oxygenation and wearable pulse rate accuracy: systematic review and meta-analysis

Open access
Singh, Sanidhya, Bennett, Miles Romney, Chen, Chen, Shin, Sooyoon, Ghanbari, Hamid, Nelson, Benjamin W·2024·Journal of Medical Internet Research
DOI 10.2196/62769PMC PMC11502980

Published version of a preprint

Open this paper in CoChat — free accountOpen PDF

Abstract

BACKGROUND Photoplethysmography (PPG) is a technology routinely used in clinical practice to assess blood oxygenation (SpO2) and pulse rate (PR). Skin pigmentation may influence accuracy, leading to health outcomes disparities. OBJECTIVE This meta-analysis primarily aimed to evaluate the accuracy of PPG-derived SpO2 and PR by skin pigmentation. Secondarily, we aimed to evaluate statistical biases and the clinical relevance of PPG-derived SpO2 and PR according to skin pigmentation. METHODS We identified 23 pulse oximetry studies (N=59,684; 197,353 paired SpO2-arterial blood observations) and 4 wearable PR studies (N=176; 140,771 paired photoplethysmography-electrocardiography observations). We evaluated accuracy according to skin pigmentation group by comparing SpO2 accuracy root-mean-square (Arms) values to the regulatory threshold of 3% and PR 95% limits of agreement (LoA) values to ±5 bpm, according to the standards of the American National Standards Institute, Advancing Safety in Medical Technology, and the International Electrotechnical Commission. We evaluated biases and clinical relevance using mean bias and 95% confidence intervals (CI). RESULTS For SpO2, Arms were 3.96%, 4.71%, and 4.15% and pooled mean biases were 0.70% (95% CI: 0.17 to 1.22), 0.27% (95% CI: -0.64 to 1.19), and 1.27% (95% CI: 0.58 to 1.95) for light, medium, and dark pigmentation, respectively. For PR, 95% LoAs were -16.02 to 13.54, -18.62 to 16.84, and -33.69 to 32.54 and pooled mean biases were -1.24 bpm (95% CI: -5.31-2.83), -0.89 bpm (95% CI: -3.70-1.93), and -0.57 bpm (95% CI: -9.44-8.29) for light, medium, and dark pigmentation, respectively. CONCLUSIONS SpO2 and PR measurements may be inaccurate across all skin pigmentation groups, breaching FDA guidance and industry standards thresholds. Pulse oximeters significantly overestimate SpO2 for both light and dark skin pigmentation, but this overestimation may not be clinically relevant. PRs obtained from wearables exhibit no statistically or clinically significant bias based on skin pigmentation.

AI snapshot

Written from the paper’s full text.

TL;DR

This systematic review and meta-analysis found that pulse oximetry SpO2 and wearable pulse rate measurements were inaccurate across all skin pigmentation groups, breaching FDA and ANSI/AAMI/IEC thresholds. Pulse oximeters significantly overestimated SpO2 for light and dark skin pigmentation, but this bias may not be clinically relevant; wearable pulse rate showed no statistically or clinically significant bias by skin pigmentation.

Key findings

  • For SpO2, pooled accuracy root-mean-square values were 3.96%, 4.71%, and 4.15% for light, medium, and dark pigmentation, respectively, exceeding the FDA threshold of ≤3%; pooled mean biases were 0.70% (95% CI 0.17%-1.22%), 0.27% (95% CI –0.64% to 1.19%), and 1.27% (95% CI 0.58%-1.95%).
  • For wearable pulse rate, 95% limits of agreement were –16.02 to 13.54 bpm, –18.62 to 16.84 bpm, and –33.69 to 32.54 bpm for light, medium, and dark pigmentation, respectively, exceeding ±5 bpm; pooled mean biases were –1.24 (95% CI –5.31 to 2.83) bpm, –0.89 (95% CI –3.70 to 1.93) bpm, and –0.57 (95% CI –9.44 to 8.29) bpm.
  • Subgroup analysis found a statistically significant difference in SpO2 bias between light and dark skin pigmentation (estimate=0.596, SE 0.240; 95% CI 0.069-1.123), but no statistically significant difference between light and medium pigmentation; no statistically significant PR bias differences were found between groups.
  • In sensitivity analysis excluding Crooks et al, pooled SpO2 Arms dropped to 3.00%, 2.91%, and 3.40% for light, medium, and dark pigmentation, leaving only dark pigmentation inaccurate.

Methods

  • PRISMA systematic review and meta-analysis with MEDLINE searches performed between April 2022 and June 2023, plus a final ad hoc search in June 2023; 23 pulse oximetry studies (n=59,684; 197,353 paired SpO2-arterial blood observations) and 4 wearable pulse rate studies (n=176; 140,771 paired PPG-ECG observations) were included.
  • Studies were included if they investigated PPG-derived SpO2 or PR, used arterial blood gas (SaO2) or ECG as reference, and reported mean bias and SD, SE, 95% LoA, or 95% CI by race, ethnicity, or skin tone; skin tone, race, and ethnicity were mapped into light, medium, and dark pigmentation groups.
  • Pooled mean biases and SDs were estimated using correlated and hierarchical effect models with robust variance estimation (correlation coefficient 0.9; sensitivity analyses at 0.3 and 0.6); accuracy was assessed against Arms≤3% for SpO2 and 95% LoA bounded by ±5 bpm for PR, with statistical significance based on 95% CI excluding 0 and clinical relevance based on FDA/ANSI/AAMI/IEC thresholds.

Limitations

  • The paper states that its skin pigmentation categorization reclassified race, ethnicity, and skin tone into a universal schema, but this method classifies most White people in the United States as light rather than medium skin pigmentation; only 4 prior PR studies collected participant skin tone, and only 1 had participants with dark pigmentation.
  • The paper states that it used stated FDA and ANSI/AAMI/IEC thresholds that may change in the future, potentially limiting applicability; it also grouped varied patient populations and testing methodologies and did not examine moderators such as sensor type, transmittance versus reflectance, wavelength, sensor placement, SpO2 level, or participant activity level.
  • Evident from the text: the PR dark pigmentation analysis rested on a single study (1 study, 6 evaluations, 16 participants, 6,922 paired observations), and the pulse oximetry sample overrepresented light skin pigmentation (40,416/59,684 participants, 67.72%; 131,008/197,353 paired observations, 66.38%).

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.2196/62769

Cites

55 of 55 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

50 of 67 cited sources checked · 13 supported · 8 flagged · accuracy not yet measured

  • Findingown data

    For light skin pigmentation, pulse rate 95% limits of agreement were -16.02 to 13.54 bpm.

    “Our analysis revealed 95% LoA of –16.02 to 13.54 bpm, –18.62 to 16.84 bpm, and –33.69 to 32.54 bpm for light, medium, and dark skin pigmentation groups, respectively.”
    Quote in the paperNot a substantial claim4 of 4 sources checked
    • cites [41] source

      The Results section on wearable accuracy across skin tones reports mean directional and absolute errors by Fitzpatrick skin-tone group, but not 95% limits of agreement for light pigmentation. Searches for the stated endpoints and for ‘limits of agreement’ found no passage giving −16.02 to 13.54 bpm. The citing sentence calls these ‘our analysis’ results, and our copy cannot verify those numbers. (LLM, conf=0.76)

      Experimental — accuracy not yet measured
    • cites [60] source

      The Results section, “Bland-Altman Analysis and 95 Percent Limits of Agreement” (page unavailable), reports limits by device and activity, not by skin pigmentation. Searches for the stated −16.02 to 13.54 bpm range and for light, medium, or dark skin groups found no matching result in our copy. (LLM, conf=0.76)

      Experimental — accuracy not yet measured
    • cites [61] source

      The Results section (page not available in our copy) reports 95% limits of agreement of −8 to 5 bpm during exercise for Fitzpatrick type II, the lightest skin group studied, rather than −16.02 to 13.54 bpm. Its reported resting limits for type II are −5 to 4 bpm. (LLM, conf=0.88)

      Checked by a reasoning model — may be wrong.

      Experimental — accuracy not yet measured
    • cites [62] source

      The Methods and Results report limits of agreement by age group, activity, and device, not by light, medium, and dark skin-pigmentation groups. Searches for “−16.02 to 13.54,” “16.02,” and skin-pigmentation subgroup results found no passage reporting the claimed limits for light skin. (LLM, conf=0.78)

      Experimental — accuracy not yet measured
  • Findingown data

    For dark skin pigmentation, pulse rate 95% limits of agreement were -33.69 to 32.54 bpm.

    “Our analysis revealed 95% LoA of –16.02 to 13.54 bpm, –18.62 to 16.84 bpm, and –33.69 to 32.54 bpm for light, medium, and dark skin pigmentation groups, respectively.”
    Quote in the paperSubstantial claim4 of 4 sources checked
    • cites [41] source

      The Results section on accuracy across skin tones reports mean directional and absolute errors for six pigmentation groups, but not 95% limits of agreement of −33.69 to 32.54 bpm for a dark-skin group. Searches for those limits and for Bland–Altman limits of agreement found no matching passage in our copy. (LLM, conf=0.76)

      Experimental — accuracy not yet measured
    • cites [60] source

      The Results section, ‘Bland-Altman Analysis and 95 Percent Limits of Agreement’ (page unavailable), reports limits by wearable device and activity, not by skin-pigmentation group. Searches for the stated limits and for dark-skin pigmentation found no passage reporting −33.69 to 32.54 bpm for that group. (LLM, conf=0.75)

      Experimental — accuracy not yet measured
    • cites [61] source

      The Results (section 3; page unavailable) report 95% limits of agreement of −6 to 4 bpm for the darkest skin type studied (type IV) during exercise, and −5 to 4 bpm at rest—not −33.69 to 32.54 bpm. The paper also did not include participants with Fitzpatrick skin types V or VI. (LLM, conf=0.90)

      Checked by a reasoning model — may be wrong.

      Experimental — accuracy not yet measured
    • cites [62] source

      The Results (Bland-Altman Analysis and Table 1; page unavailable) report limits of agreement by age group, device, and activity, not by light, medium, or dark skin pigmentation. Searches for the stated limits (−33.69 to 32.54 bpm) and skin-pigmentation subgroups found no matching result in our copy. (LLM, conf=0.78)

      Experimental — accuracy not yet measured
  • Findingown data

    For medium skin pigmentation, pulse rate mean bias was -0.89 bpm (95% CI -3.70 to 1.93).

    “Mean biases were –1.24 (95% CI –5.31 to 2.83) bpm, –0.89 (95% CI –3.70 to 1.93) bpm, and –0.57 (95% CI –9.44 to 8.29) bpm for the corresponding groups (Table 4 and Figures 5-7 [41,60-62]).”
    Quote in the paperNot a substantial claim4 of 4 sources checked
    • cites [41] source

      The Results section on accuracy across skin tones reports heart-rate errors by six pigmentation categories, but does not report a mean bias of −0.89 bpm or a 95% CI of −3.70 to 1.93 for medium pigmentation. Searches for those numbers and for a medium-pigmentation mean bias found no matching passage in our copy. (LLM, conf=0.73)

      Experimental — accuracy not yet measured
    • cites [60] source

      The Methods (Recruitment; page unavailable) describe a single participant with Fitzpatrick skin tone 2, not a medium-pigmentation group. The Results (Bland-Altman Analysis; page unavailable) report device- and activity-specific errors, but not a −0.89 bpm mean bias or the stated 95% confidence interval. Searches for medium skin pigmentation and those figures found no supporting passage. (LLM, conf=0.76)

      Experimental — accuracy not yet measured
    • cites [61] source

      The Results (page unavailable) report a mean bias of −2 bpm for skin type III during exercise, with 95% limits of agreement of −8 to 4 bpm; at rest, the reported bias is 0 bpm. The cited paper does not report the claimed −0.89 bpm bias or its stated 95% confidence interval for this group. (LLM, conf=0.84)

      Checked by a reasoning model — may be wrong.

      Experimental — accuracy not yet measured
    • cites [62] source

      The Results (Bland-Altman Analysis and Table 1; page unavailable) report heart-rate biases by age group, device, and activity, not a bias for a medium-pigmentation group. Searches for −0.89 bpm and the stated confidence interval found no matching result in our copy. (LLM, conf=0.75)

      Experimental — accuracy not yet measured
  • Findingown data

    For medium skin pigmentation, pulse rate 95% limits of agreement were -18.62 to 16.84 bpm.

    “Our analysis revealed 95% LoA of –16.02 to 13.54 bpm, –18.62 to 16.84 bpm, and –33.69 to 32.54 bpm for light, medium, and dark skin pigmentation groups, respectively.”
    Quote in the paperNot a substantial claim4 of 4 sources checked
    • cites [41] source

      The Results section, “Wearables accuracy across skin tones” (page unavailable), reports mean directional and absolute heart-rate errors by Fitzpatrick skin-tone group, not 95% limits of agreement for a medium-pigmentation group. Searches for the stated endpoints and for limits of agreement found no passage verifying −18.62 to 16.84 bpm. (LLM, conf=0.76)

      Experimental — accuracy not yet measured
    • cites [60] source

      The Methods > Recruitment section (page unavailable) says the study involved a single participant with Fitzpatrick skin tone 2, not light, medium, and dark pigmentation groups. The Results > Bland-Altman Analysis section (page unavailable) reports limits of agreement by device and activity, not the claimed −18.62 to 16.84 bpm for a medium-pigmentation group. (LLM, conf=0.87)

      Checked by a reasoning model — may be wrong.

      Experimental — accuracy not yet measured
    • cites [61] source

      The Results section reports 95% limits of agreement of −8 to 4 beats/min for skin type III during exercise, not −18.62 to 16.84 bpm. It reports −5 to 5 beats/min for type III at rest. Our copy does not provide a page number for this passage. (LLM, conf=0.89)

      Checked by a reasoning model — may be wrong.

      Experimental — accuracy not yet measured
    • cites [62] source

      The Results section (Bland-Altman Analysis; page unavailable) reports limits of agreement by age group, device, and activity, not by light, medium, and dark skin pigmentation. A search for the stated limits of −18.62 to 16.84 bpm and for skin-pigmentation subgroup analyses found no passage verifying that figure. The citing sentence also describes the figures as the citing authors’ own analysis. (LLM, conf=0.72)

      Experimental — accuracy not yet measured
  • Findingown data

    For dark skin pigmentation, pulse rate mean bias was -0.57 bpm (95% CI -9.44 to 8.29).

    “Mean biases were –1.24 (95% CI –5.31 to 2.83) bpm, –0.89 (95% CI –3.70 to 1.93) bpm, and –0.57 (95% CI –9.44 to 8.29) bpm for the corresponding groups (Table 4 and Figures 5-7 [41,60-62]).”
    Quote in the paperNot a substantial claim4 of 4 sources checked
    • cites [41] source

      The Results section (‘Wearables accuracy across skin tones’; page unavailable) reports heart-rate measurement error by skin tone, but it does not report a mean bias of −0.57 bpm or the stated 95% confidence interval for dark skin. Its reported values include a mean directional error of −4.25 bpm for FP5 at rest and a mean absolute error of 10.6 bpm for FP6 at rest; these do not establish the claimed estimate. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
    • cites [60] source

      The Methods (Recruitment; page unavailable) describe a single participant with Fitzpatrick skin tone 2, not a dark-skin group. The Results (Bland-Altman Analysis; page unavailable) report mean errors of −1.80 bpm for Apple Watch 3 and −3.47 bpm for Fitbit Charge 2 across 24 hours, not the claimed −0.57 bpm with a 95% CI of −9.44 to 8.29. (LLM, conf=0.88)

      Checked by a reasoning model — may be wrong.

      Experimental — accuracy not yet measured
    • cites [61] source

      The Results section (page unavailable) reports a mean bias of −1 bpm for the darkest skin type studied (type IV), with 95% limits of agreement of −6 to 4 bpm. It does not report the claimed −0.57 bpm or a 95% confidence interval of −9.44 to 8.29 bpm; the reported limits of agreement are a different measure from a confidence interval. (LLM, conf=0.88)

      Checked by a reasoning model — may be wrong.

      Experimental — accuracy not yet measured
    • cites [62] source

      The Results (Reliability of Examined Devices, Table 1) report heart-rate differences by age group, activity, and device, not a mean bias for dark skin pigmentation. The Introduction (Aim of the Study) describes participants with Fitzpatrick skin types III or IV. Searches for the quoted bias and interval found no matching result in our copy. (LLM, conf=0.75)

      Experimental — accuracy not yet measured
  • May combine two claims. 34
  • Classified as probably not a substantial claim. 22
  • The quote only partly states this claim. 7

  • May combine two claims.

    Premisecitation

    Only 1 meta-analysis focused on pulse oximetry for this topic is available, and an additional 7 studies have been published recently, providing an additional 50,980 participants and 182,369 paired observations.

    “Only 1 meta-analysis focused on pulse oximetry for this topic is available [36], and an additional 7 studies have been published recently, providing an additional 50,980 participants and 182,369 paired observations.”
    Quote in the paperSubstantial claim1 of 1 sources checked

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    Funnel plots evaluated publication bias.

    “Funnel plots evaluated publication bias using the metafor package [63] (Figures S2 and S3 in Multimedia Appendix 1).”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [63] Viechtbauer, W, 2010, J Stat Soft source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    The study followed PRISMA guidelines.

    “We followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines [39]”
    Quote in the paperNot a substantial claim1 of 1 sources checked

  • May combine two claims.

    Premisecitation

    Overestimated SpO2 readings in patients with darker skin pigmentation lead to lower hospital admission, higher occult hypoxemia, delayed or no access to dexamethasone, therapeutic oxygen, and COVID-19 therapies, resulting in increased hospital readmission, organ dysfunction, and mortality.

    “leading to lower hospital admission; higher occult hypoxemia; and delayed or no access to dexamethasone, therapeutic oxygen, and COVID-19 therapies, resulting in increased hospital readmission, organ dysfunction, and mortality [11-18].”
    Quote in the paperSubstantial claim7 of 8 sources checked
    • cites [11] source

      Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

      Experimental — accuracy not yet measured
    • cites [12] source

      Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

      Experimental — accuracy not yet measured
    • cites [13] source
    • cites [14] source

      Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

      Experimental — accuracy not yet measured
    • cites [15] source

      Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

      Experimental — accuracy not yet measured
    • cites [16] source

      Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

      Experimental — accuracy not yet measured
    • cites [17] source

      Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

      Experimental — accuracy not yet measured
    • cites [18] source

      Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

      Experimental — accuracy not yet measured

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    No established methodologies exist for providing pooled 95% LoA and Arms, so the study followed an analytical approach used in prior meta-analyses.

    “When it comes to providing pooled 95% LoA and Arms, no established methodologies exist, so we followed an analytical approach used in prior meta-analyses within the discipline [36].”
    Quote in the paperSubstantial claim1 of 1 sources checked

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Sensitivity analyses for pulse oximetry SpO2 were performed by removing Crooks et al [48] as an outlier due to substantially large standard error.

    “Sensitivity analyses were performed for pulse oximetry SpO2 by removing Crooks et al [48] as an outlier due to substantially large SE (Tables S6 and S7 in Multimedia Appendix 1).”
    Quote in the paperNot a substantial claim1 of 1 sources checked

  • May combine two claims.

    Findingown data

    In sensitivity analyses without the outlier study of Crooks et al, all pooled Arms values dropped, resulting in inaccurate pulse oximeter SpO2 only for dark skin pigmentation and no group exceeding international thresholds.

    “Additionally, in the sensitivity analyses without the outlier study of Crooks et al [48], all pooled Arms values dropped, resulting in inaccurate pulse oximeter SpO2 only for dark skin pigmentation and no group exceeding international thresholds.”
    Quote in the paperSubstantial claim1 of 1 sources checked

  • May combine two claims.

    Findingown data

    For PR, 95% limits of agreement were -16.02 to 13.54 for light, -18.62 to 16.84 for medium, and -33.69 to 32.54 for dark skin pigmentation.

    “For PR, 95% limits of agreement values were from –16.02 to 13.54, from –18.62 to 16.84, and from –33.69 to 32.54”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    For SpO2, pooled accuracy root-mean-square values were 3.6% combined, 3.96% light, 4.71% medium, and 4.15% dark skin pigmentation.

    “The pooled Arms across different skin pigmentation groups was 3.6%, 3.96%, 4.71%, and 4.15% for combined, light, medium, and dark skin pigmentation, respectively”
    • Also stated in: § Results · plain
      “For SpO2, accuracy root-mean-square values were 3.96%, 4.71%, and 4.15%”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The systematic review included 23 pulse oximetry studies with 59,684 participants and 197,353 paired SpO2-SaO2 observations.

    “We identified 23 pulse oximetry studies (n=59,684; 197,353 paired SpO2-arterial blood observations)”
    • Also stated in: § Study Selection and Characteristics · plain
      “A total of 23 pulse oximetry studies involving 59,684 participants with 197,353 paired SpO2-SaO2 observations were included.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The systematic review included 4 wearable pulse rate studies with 176 participants and 140,771 paired PPG-ECG observations.

    “4 wearable PR studies (n=176; 140,771 paired PPG-electrocardiography observations)”
    • Also stated in: § Study Selection and Characteristics · plain
      “Additionally, 4 PR studies with 176 participants and 140,771 paired PR-ECG observations were analyzed (Table 3).”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Findingown data

    27 studies were selected for full review (23 pulse oximetry, 4 PR).

    “We selected 27 studies for full review (pulse oximetry: n=23, PR: n=4; Figure 1, Table 1, and Table 2).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    In pulse oximetry studies, there were 40,416 (67.72%) patients with light skin pigmentation and 131,008 (66.38%) paired observations; 9967 (16.70%) patients with medium skin pigmentation and 32,095 (16.26%) paired observations; and 9301 (15.58%) patients with dark skin pigmentation and 34,250 (17.35%) paired observations.

    “Of the 59,684 patients with 197,353 paired observations, there were a total of 40,416 (67.72%) patients with light skin pigmentation with 131,008 (66.38%) paired observations, 9967 (16.70%) patients with medium skin pigmentation with 32,095 (16.26%) paired observations, and 9301 (15.58%) patients with dark skin pigmentation with 34,250 (17.35%) paired observations.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    In PR studies, there were 31 (17.61%) participants with light skin pigmentation and 43,116 (30.60%) paired observations; 129 (73.30%) participants with medium skin pigmentation and 90,733 (64.50%) paired observations; and 16 (9.09%) participants with dark skin pigmentation and 6922 (4.92%) paired observations.

    “Of the 176 patients with 140,771 paired observations, there were a total of 31 (17.61%) participants with light skin pigmentation with 43,116 (30.60%) paired observations, 129 (73.30%) participants with medium skin pigmentation with 90,733 (64.50%) paired observations, and 16 (9.09%) participants with dark skin pigmentation with 6922 (4.92%) paired observations.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Skin pigmentation was classified by race in 13 (57%) of 23 pulse oximetry studies, ethnicity in 3 (13%), skin tone in 4 (17%), and both race and ethnicity in 3 (13%).

    “Skin pigmentation was classified by race in 13 (57%) out of 23 studies, ethnicity in 3 (13%) studies, skin tone in 4 (17%) studies, and both race and ethnicity in 3 (13%) studies (Table 3).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    22 (96%) of 23 pulse oximeter studies used a transmittance device and only one used an ear clip.

    “We used this threshold as 22 (96%) out of 23 of pulse oximeter studies used a transmittance device and only one used an ear clip”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    In the pulse rate meta-analysis, 26.76% of the variation in bias stemmed from between-study differences and 73.34% originated from within-study variation.

    “Heterogeneity analysis demonstrated that 26.76% of the variation in bias (I2Level3) stemmed from between-study differences, while 73.34% (I2Level2) originated from within-study variation.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The search strategy resulted in 8582 records.

    “Search strategy resulted in 8582 records.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    For PR, pooled mean biases were -1.24 (95% CI -5.31 to 2.83) bpm for light, -0.89 (95% CI -3.70 to 1.93) bpm for medium, and -0.57 (95% CI -9.44 to 8.29) bpm for dark skin pigmentation.

    “pooled mean biases were –1.24 (95% CI –5.31 to 2.83) bpm, –0.89 (95% CI –3.70 to 1.93) bpm, and –0.57 (95% CI –9.44 to 8.29) bpm for light, medium, and dark pigmentation, respectively.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    In PR studies, skin pigmentation was classified by skin tone in 4 (100%) studies and by race or ethnicity in 0 (0%) studies.

    “Skin pigmentation was classified by race or ethnicity in 0 (0%) out of 4 studies and skin tone in 4 (100%) studies (Table 3).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Across all pulse rate studies, the pooled mean bias was -0.29 bpm (95% CI -3.87 to 3.29) and the 95% limits of agreement were -17.23 to 16.65 bpm.

    “Analysis using the CHE model revealed LoA values of –17.23 to 16.65 bpm and a mean bias of –0.29 (95% CI –3.87 to 3.29) bpm across all studies.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Between-study heterogeneity accounted for 14.53% of total variation, and within-study heterogeneity explained 84.02% of total variation.

    “Between-study heterogeneity (I2Level3) accounted for 14.53% of the total variation, while within-study heterogeneity (I2Level2) explained 84.02% of total variation.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Subgroup analysis found a statistically significant difference in SpO2 bias between light and dark skin pigmentation (estimate=0.596, SE 0.240; 95% CI 0.069-1.123), with higher pooled bias for darker skin.

    “but there was a statistically significant difference between light and dark skin pigmentation (estimate=0.596, SE 0.240; 95% CI 0.069-1.123), such that pooled bias for those with darker skin was higher as compared to those with lighter skin”
    Quote in the paperSubstantial claim

  • May combine two claims. The quote only partly states this claim.

    Findingown data

    For dark skin pigmentation, pulse oximetry Arms was 4.15%, exceeding FDA guidance.

    “| | Dark | 19 (36) | 9,301 (34,250) | Percent | 1.27 (0.58 to 1.95) | 3.96 (1.30) | –6.49 to 9.02 | 4.15e | 98.46% (0.00% and 98.46%) |”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims. The quote only partly states this claim.

    Findingown data

    For light skin pigmentation, pulse oximetry Arms was 3.96%, exceeding FDA guidance.

    “| | Light | 20 (44) | 40,416 (131,008) | Percent | 0.70 (0.17 to 1.22) d | 3.90 (1.36) | –6.94 to 8.34 | 3.96e | 97.45% (46.44% and 51.01%) |”
    Quote in the paperNot a substantial claim

  • May combine two claims. The quote only partly states this claim.

    Findingown data

    For dark skin pigmentation, pulse oximetry pooled mean bias was 1.27% (95% CI 0.58 to 1.95), and this was statistically significant.

    “| | Dark | 19 (36) | 9,301 (34,250) | Percent | 1.27 (0.58 to 1.95) | 3.96 (1.30) | –6.49 to 9.02 | 4.15e | 98.46% (0.00% and 98.46%) |”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning

    Both SpO2 and pulse rate studies had lower between-study heterogeneity and higher within-study heterogeneity.

    “Lastly, it should be noted that both SpO2 and PR studies had lower between-study heterogeneity and higher within-study heterogeneity, indicating that the studies that were included in these analyses were largely consistent with one another and that most of the variation came from within studies potentially due to higher variability across devices used or participants enrolled within each study.”
    Quote in the paperNot a substantial claim

  • May combine two claims. The quote only partly states this claim.

    Findingown data

    For light skin pigmentation, pulse oximetry pooled mean bias was 0.70% (95% CI 0.17 to 1.22), and this was statistically significant.

    “| | Light | 20 (44) | 40,416 (131,008) | Percent | 0.70 (0.17 to 1.22) d | 3.90 (1.36) | –6.94 to 8.34 | 3.96e | 97.45% (46.44% and 51.01%) |”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    No statistically significant or clinically relevant bias was found in wearable pulse rate devices.

    “We did not find statistically significant or clinically relevant bias in wearable PR devices.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims. The quote only partly states this claim.

    Findingown data

    For medium skin pigmentation, pulse oximetry Arms was 4.71%, exceeding FDA guidance.

    “| | Medium | 9 (15) | 9,967 (32,095) | Percent | 0.27 (–0.64 to 1.19) | 4.71 (1.71) | –8.95 to 9.50 | 4.71e | 95.31% (81.16% and 14.14%) |”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    Devices with accuracy measures breaching these accuracy thresholds can produce questionable results.

    “Devices with accuracy measures breaching these accuracy thresholds can produce questionable results.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    ANSI/AAMI/IEC recommend that ECG devices have mean bias of +5 or -5 bpm or mean absolute percentage error≤10%, whichever is greater.

    “The American National Standards Institute (ANSI), Association for the Advancement of Medical Instrumentation (AAMI), and International Electrotechnical Commission (IEC) have set the recommendation that electrocardiography (ECG) devices have mean bias of +5 or –5 beats per minute (bpm) or mean absolute percentage error≤10%, whichever is greater [33,34].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [33] ANSI/AAMI/IECIEC 60601-2-27 particular requirements for the basic safety and essential performance of electro cardiographic monitoringITC India2024-08-24https://www.itcindia.org/iec-60601-2-27-particular-requirements-for-the-basic-safety-and-essential-performance-of-electro-cardiographic-monitoring/#:~:text=What%20is%20IEC%2060601%2D2,processing%2C%20alarms%2C%20and%20displays

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [34] ANSI/AAMI EC13-2002 cardiac monitors, heart rate meters, and alarmsANSI20022023-10-30https://webstore.ansi.org/standards/aami/ansiaamiec132002

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation

    FDA guidances since 2013 have called to address darker skin pigmentation bias in pulse oximeters but lacked standards for assessing skin tone.

    “Starting in 2013, a series of U.S. Food and Drug Administration (FDA) guidances, safety communications, and more recently, statements from attorneys general have called to address darker skin pigmentation bias in pulse oximeters [19]; however, these guidances lacked standards for assessing skin tone [20].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [19] Pulse oximeters - premarket notification submissions [510(k)s] guidance for industry and Food and Drug Administration staffU.S. Food and Drug Administration201303042024-08-24https://www.fda.gov/media/72470/download

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [20] Review of pulse oximeters and factors that can impact their accuracyU.S. Food and Drug Administration20222024-08-24https://www.fda.gov/media/162709/download

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The first author verified the quality of manuscripts.

    “The first author verified the quality of manuscripts.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    FDA guidance has a threshold of Arms≤3% for transmittance devices and Arms≤3.5% for ear clip and reflectance devices, while international thresholds are set at Arms≤4%.

    “FDA guidance has a threshold of Arms≤3% [19] for transmittance devices and a threshold of Arms≤3.5% [19] for ear clip and reflectance devices, while international thresholds are set at Arms≤4% [32].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [19] Pulse oximeters - premarket notification submissions [510(k)s] guidance for industry and Food and Drug Administration staffU.S. Food and Drug Administration201303042024-08-24https://www.fda.gov/media/72470/download

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [32] Medical electrical equipment particular requirements for basic safety and essential performance of pulse oximeter equipment (British standard)ANSI2023-10-23https://tinyurl.com/ayfrdpau

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    Data were independently extracted by two reviewers, with a third and fourth reviewer adjudicating differences.

    “Data were independently extracted from published manuscripts (BWN and SS). A third and fourth reviewer (MRB and HG) adjudicated any differences between the initial 2 reviewers and resolved any disagreements by checking manuscripts.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    SpO2 and PR measurements may be inaccurate across all skin pigmentation groups, breaching FDA guidance and industry standard thresholds.

    “SpO2 and PR measurements may be inaccurate across all skin pigmentation groups, breaching U.S. Food and Drug Administration guidance and industry standard thresholds.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    For heart rate or PR devices, overall accuracy can be assessed with 95% limits of agreement.

    “For heart rate or PR devices, overall accuracy can be assessed with 95% limits of agreement (LoAs).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    QUADAS-2 tool was used to evaluate risk of bias and applicability.

    “QUADAS-2 tool was used to evaluate risk of bias and applicability (Figure S1 in Multimedia Appendix 1).”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Overall accuracy of pulse oximeters can be assessed with accuracy root-mean-square (Arms), which combines mean bias and precision into a single metric.

    “Overall accuracy of pulse oximeters can be assessed with accuracy root-mean-square (Arms), which combines mean bias and precision (SD of bias) into a single metric [32].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [32] Medical electrical equipment particular requirements for basic safety and essential performance of pulse oximeter equipment (British standard)ANSI2023-10-23https://tinyurl.com/ayfrdpau

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    The level of SpO2 was not examined given the study's goal of gathering the largest sample upon which to study the effect of skin tone.

    “The level of SpO2 was not examined given this study’s goal of gathering the largest sample upon which to study the effect of skin tone.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    Despite not meeting FDA guidance across all groups, pulse oximeter SpO2 was inaccurate only across medium and dark skin pigmentation groups when compared to the more liberal international thresholds.

    “Despite not meeting FDA guidance across all groups, pulse oximeter SpO2 was inaccurate only across medium and dark skin pigmentation groups when compared to the more liberal international thresholds.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    Most of the variation in SpO2 and pulse rate studies came from within studies, potentially due to higher variability across devices used or participants enrolled within each study.

    “Lastly, it should be noted that both SpO2 and PR studies had lower between-study heterogeneity and higher within-study heterogeneity, indicating that the studies that were included in these analyses were largely consistent with one another and that most of the variation came from within studies potentially due to higher variability across devices used or participants enrolled within each study.”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Interpretationown data

    Pulse oximeters may overestimate SpO2 significantly for light skin pigmentation.

    “Pulse oximeters may also overestimate SpO2 significantly for light and dark skin pigmentation, but without clinically relevant bias.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    The potential moderating effects of sensor type, transmittance versus reflectance sensors, wavelength of light used by sensors, and location of sensor placement were not examined because the core sensor technology studied was PPG across all included studies.

    “We did not examine the potential moderating effects of sensor type, transmittance versus reflectance sensors, wavelength of light used by sensors, and location of sensor placement given the core sensor technology studied was PPG across all included studies.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    Participant activity level was not evaluated as a moderator variable because many included studies did not delineate participant activity level explicitly.

    “Participant activity level was not evaluated as a moderator variable because many included studies did not delineate participant activity level explicitly.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    Subgroup analyses for pulse rate were not conducted because most studies did not specify whether subjects were healthy or part of medical populations.

    “We did not conduct these analyses for pulse rate because most studies did not specify whether subjects were healthy or part of medical populations.”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Interpretationown data

    Pulse oximeters may overestimate SpO2 significantly for dark skin pigmentation.

    “Pulse oximeters may also overestimate SpO2 significantly for light and dark skin pigmentation, but without clinically relevant bias.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    This systematic review and meta-analysis found that pulse oximeter SpO2 and wearable pulse rate were inaccurate across all skin pigmentation groups as the resulting accuracy values breached FDA guidance and ANSI/AAMI/IEC standard thresholds, respectively, although pulse oximeter SpO2 was only found to be inaccurate for dark skin pigmentation in sensitivity analyses.

    “This systematic review and meta-analysis found that pulse oximeter SpO2 and wearable PR were inaccurate across all skin pigmentation groups as the resulting accuracy values breached FDA guidance and ANSI/AAMI/IEC standard thresholds, respectively, although pulse oximeter SpO2 was only found to be inaccurate for dark skin pigmentation in sensitivity analyses.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Subgroup analyses were conducted to examine the potential moderating variables of medical versus healthy populations for pulse oximeters.

    “Despite this, we conducted subgroup analyses to examine the potential moderating variables of medical versus healthy populations for pulse oximeters (see Table S8 in Multimedia Appendix 1).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    Results suggest that SpO2 and pulse rate measurements are inaccurate across all skin pigmentation groups because they breach FDA and ANSI/AAMI/IEC standards, respectively.

    “Results suggest inaccurate SpO2 and PR measurements across all skin pigmentation groups as they breach FDA and ANSI/AAMI/IEC standards, respectively, with wearable accuracy varying considerably depending on the model, which may be due to date of model production or algorithm development.”
    Quote in the paperSubstantial claim
51 of 51

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Darker skin, with higher melanin levels, absorbs more light, reducing signal strength and limiting light penetration to blood vessels.

    “Darker skin, with higher melanin levels, absorbs more light, reducing signal strength and limiting light penetration to blood vessels [19].”

    From Photoplethysmography in Diverse Skin Tones: Evaluating Bias in Smartwatch Health Monitoring (2025) · cites this paper as [19]

    The Discussion’s Strengths and Limitations section says skin tone is determined by the amount of melanin in the epidermis. But neither that section nor the Principal Findings discussion states that darker skin absorbs more light, weakens the signal, or limits light penetration to blood vessels; the discussion only suggests that melanin levels could affect the PPG signal. No page numbers are provided in our copy. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • Weaker signals and increased scattering from darker skin degrade measurement accuracy.

    “This results in weaker signals and increased scattering, which degrade measurement accuracy [19,20].”

    From Photoplethysmography in Diverse Skin Tones: Evaluating Bias in Smartwatch Health Monitoring (2025) · cites this paper as [19]

    The Results (PPG Accuracy and Bias by Skin Pigmentation; page not provided) report higher pooled pulse-oximetry bias for darker than lighter skin, supporting a difference in measurement accuracy. The paper does not establish weaker signals or increased scattering as the cause of that difference. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • Socioeconomic disparities in access to smartwatches may interact with skin-tone-related bias, raising additional equity concerns.

    “However, regional variations may exist, and socioeconomic disparities in access to these devices may further interact with skin-tone-related bias, raising additional equity concerns [17-23].”

    From Photoplethysmography in Diverse Skin Tones: Evaluating Bias in Smartwatch Health Monitoring (2025) · cites this paper as [19]

    The Introduction (page unavailable) links potential skin-pigmentation bias in wearable devices to concerns about inequity and discrimination. It does not discuss socioeconomic disparities in smartwatch access or whether those disparities interact with skin-tone-related bias. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Medical-combined setting percentage is miscalculated

    Minor

    Table 3 lists one medical-combined study among 23 pulse-oximetry studies as 5%; 1 out of 23 is approximately 4.35%, which rounds to 4% at the table's whole-percent precision.

    Reported 5 · recomputed 4.347826

  • Pulse-rate within-study heterogeneity conflicts with Table 4

    Minor

    The results text attributes 73.34% of pulse-rate variation to within-study differences, whereas the combined pulse-rate row of Table 4 reports within-study heterogeneity of 3.90% and overall heterogeneity of 30.66%.

  • Pulse-rate clinical-relevance rule contradicts the stated conclusion

    Minor

    Table 2 calls a pulse-rate bias clinically relevant if the upper 95% CI bound is at most 5 bpm. The reported upper bounds for the light and medium groups are 2.83 and 1.93 bpm, so those groups meet the stated rule despite the conclusion of no clinically relevant bias.

3 of 3

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. Bothma, P A, 1996, S Afr Med J, 86, 594

Cited by

Sorted by citations

Related

Authors
Singh, S., Bennett, M. R., Chen, C., Shin, S., Ghanbari, H., & Nelson, B. W.
Affiliations
  • University of Michigan (US)
  • Verily Life Sciences LLC (United States) (US)
  • Beth Israel Deaconess Medical Center (US)
ORCID
Venue
Journal of Medical Internet Research
Publisher
JMIR Publications Inc.
Published
2024-10-10
Volume
26
Pages
e62769
ISSN
1438-8871 (ISSN-L)
DOI
10.2196/62769
PMC
PMC11502980
OpenAlex
W4401647034
Cited by
36
References
54
Citations by year
2026: 17 · 2025: 19
FWCI
1.68
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Journal of Medical Internet Research (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • PubMed Central (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
Topics
  • Physical Sciences › Engineering › Biomedical Engineering › Non-Invasive Vital Sign Monitoring
  • Health Sciences › Medicine › Surgery › Hemodynamic Monitoring and Therapy
  • Health Sciences › Medicine › Cardiology and Cardiovascular Medicine › Cardiovascular Health and Disease Prevention
Keywords
pulse oximetry, photoplethysmography, oxygen saturation, wearable devices, measurement bias, health disparities, dark skin, arterial oxygenation, electrocardiography comparison, limits of agreement, root mean square error, measurement reliability, measurement validity
MeSH terms
Wearable Electronic Devices, Female, Heart Rate, Humans, Male, Oximetry, Oxygen, Skin Pigmentation, Photoplethysmography

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-17.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-17.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review