CoChat
Sign Up
PreprintCited by 158Non-Invasive Vital Sign MonitoringHemodynamic Monitoring and TherapyChronic Obstructive Pulmonary Disease (COPD) Research

The effects of skin pigmentation on the accuracy of pulse oximetry in measuring oxygen saturation: a systematic review and meta-analysis

Open access
Shi, Chunhu, Goodall, Mark, Dumville, Jo, Hill, James, Norman, Gill, Hamer, Oliver, Clegg, Andrew, Watkins, Caroline Leigh +5 more·2022·BMC Medicine
DOI 10.1101/2022.02.16.22271062PMC PMC9377806
Open this paper in CoChat — free accountOpen PDF

Abstract

Background: Pulse oximetry was widely used in hospitals and at home to monitor blood oxygen during the COVID-19 pandemic. There have been concerns regarding potential bias in pulse oximetry measurements for people with dark skin. We aimed to assess the effects of skin pigmentation on the accuracy of oxygen saturation measurement by pulse oximetry (SpO 2 ) compared with the gold standard SaO 2 measured by CO-oximetry. Methods: We searched Ovid MEDLINE, Ovid Embase, and EBSCO CINAHL Plus (up to December 2021), as well as ClinicalTrials.gov and World Health Organization International Clinical Trials Registry Platform (up to August 2021). We identified studies comparing SpO 2 values in any population, in any care setting, using any type of pulse oximeter, with SaO 2 by standard CO-oximetry; and measuring the impact of skin pigmentation or ethnicity on pulse oximetry accuracy. We performed meta-analyses for mean bias (the primary outcome in this review) and its standard deviations (SDs) across studies included for each subgroup of level of skin pigmentation and ethnicity. We calculated accuracy root-mean-square ( A rms ) and 95% limits of agreement based on pooled mean bias and pooled SDs for each subgroup. Results: We included 32 studies (6505 participants); 27/32 (84.38%) in hospitals and none in people’s homes. Findings of 14/32 studies (43.75%) were judged, via QUADAS-2, at high overall risk of bias. Fifteen studies measured skin pigmentation and 22 referred only to ethnicity. Compared with standard SaO 2 measurement, pulse oximetry probably overestimates oxygen saturation in people with dark skin (pooled mean bias 1.11%; 95% confidence interval 0.29% to 1.93%) and people described as Black/African American (pooled mean bias 1.52%; 0.95% to 2.09%) (moderate- and low-certainty evidence). These results suggest that, for people with dark skin, pulse oximetry may overestimate blood oxygen saturation by around 1% on average compared with SaO 2 . The bias of pulse oximetry measurements for people with other levels of skin pigmentation, or those from the White/Caucasian group is more uncertain. The data do not suggest overestimation in people from other ethnic groups such as those described as Asian, Hispanic, or mixed ethnicity (pooled mean bias 0.31%, 0.09% to 0.54%), but this evidence is low certainty. Whilst the extent of mean bias is small or negligible for all the subgroups of population evaluated, the associated imprecision is unacceptably large (with the pooled SDs > 1%). Nevertheless, when the extents of measurement bias and precision are considered jointly in A rms , pulse oximetry measurements for all the subgroups appear acceptably accurate (with A rms < 4%). Conclusions: Low-certainty evidence suggests that pulse oximetry may overestimate oxygen saturation in people with dark skin and people whose ethnicity is reported as Black/African American, compared with SaO 2 , although the overestimation may be quite small in hospital settings. The clinical importance of any overestimation will depend on the particular clinical circumstance. Pulse oximetry measurements appear accurate but imprecise for all levels of skin pigmentation. The evidence relates to clinician-measured oximetry in health care environments and may not be reflected in home pulse oximetry where other factors may also influence accuracy. Author summary: Why was this study done?: Pulse oximetry was widely used in hospital and at home to measure blood oxygen levels during the COVID-19 pandemic. There was uncertainty as to whether skin pigmentation affects the accuracy of pulse oximetry measurements. What did the researchers do and find?: We assessed, via systematic review, the effects of skin pigmentation on the accuracy of pulse oximetry measurement (SpO 2 ) compared with SaO 2 measured by standard CO-oximetry. In people with dark skin, oxygen saturation measured in hospital by pulse oximetry may be overestimated by an average of 1% compared with gold standard SaO 2 , however, the evidence is of low certainty. The accuracy of pulse oximetry measured compared with standard SaO 2 is quite uncertain for people with light or medium levels of skin pigmentation and for people from ethnic groups other than those described in papers as Black or African American. Pulse oximetry measurements appear to have acceptable overall accuracy (with A rms < 4%) for all subgroups of population evaluated whilst the variation of oximetry readings appear unacceptably wide (with the pooled SDs > 1%). What do these findings mean?: Hospital measured pulse oximetry may overestimate oxygen saturation in people with dark skin compared with SaO 2 by approximately 1%. The implications of this finding in different clinical scenarios will vary but could be clinically important. Impacts are likely to be at the thresholds of being diagnosed as having hypoxaemia where even a small SpO 2 overestimation could lead to clinically important hypoxaemia remaining undetected and untreated. How these findings extrapolate to community and home care settings is unclear.

Verified 2026-10-09 · 1 reference not checked yet

AI snapshot

Written from the paper’s full text.

TL;DR

This systematic review and meta-analysis found that pulse oximetry probably overestimates oxygen saturation in people with high skin pigmentation and in people described as Black/African American compared with CO-oximetry SaO2. Although mean bias was small and accuracy root-mean-square remained below recommended thresholds, measurement precision was poor across subgroups.

Key findings

  • The review included 32 studies with 6505 participants; 15 studies measured skin pigmentation and 22 referred to ethnicity.
  • Pooled mean bias was 1.11% (95% CI 0.29 to 1.93%) for high skin pigmentation (moderate-certainty evidence) and 1.52% (95% CI 0.95 to 2.09%) for people described as Black/African American (low-certainty evidence), indicating overestimation by pulse oximetry.
  • For medium pigmentation, low pigmentation, Asian/Hispanic/mixed ethnicity, and White/Caucasian groups, evidence was uncertain or did not suggest clinically important systematic overestimation.
  • Pooled SDs were generally >1%, indicating unacceptably large imprecision, but calculated Arms values were <4% for all subgroups, which the authors describe as acceptably accurate when bias and precision are considered jointly.

Methods

  • Systematic review and meta-analysis of methods-comparison studies comparing SpO2 from commercially available pulse oximeters with SaO2 measured by standard CO-oximetry; searches covered Ovid MEDLINE, Ovid Embase, EBSCO CINAHL, ClinicalTrials.gov, and WHO ICTRP up to December 2021.
  • Data were pooled separately for skin pigmentation level and ethnicity using a correlated hierarchical effects model with robust variance estimation; mean bias was the primary outcome, with pooled SDs used to calculate Arms and 95% limits of agreement.
  • Risk of bias was assessed with an adapted QUADAS-2, and certainty of evidence for mean bias was assessed using GRADE.

Limitations

  • The authors state that some included studies reported only significance tests or diagnostic accuracy proportions rather than SpO2–SaO2 data by subgroup, limiting meta-analysis; they also note that race and ethnicity were combined for simplicity, that Fitzpatrick-type skin pigmentation scales are blunt, that device, age, condition, and pigmentation-measurement differences were not analysed, that only English-language peer-reviewed publications were searched, and that no risk-of-bias or GRADE approach was specific to this topic.
  • The paper states that most included studies were hospital-based and had limited information on whether devices were appropriate for home self-monitoring, so findings may not reflect home pulse oximetry by patients or carers.
  • Evident from the text: included studies spanned 1985–2021 and some may have used discontinued devices, and meta-analyses showed substantial between-study heterogeneity (e.g., overall I² values often >90% for several pooled estimates).

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1101/2022.02.16.22271062

Cites

67 of 68 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 38 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    For all levels of skin pigmentation, the Arms values are around 2% or lower.

    “For all the levels of skin pigmentation, the Arms values are around 2% or lower (95% CI non-estimable), and the pooled SD values are around 1.50% on average (Table 2).”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: SaO2 measured by CO-oximetry is the gold standard for blood oxygen saturation.
  • Findingown data

    The extent of mean bias is small or negligible for all subgroups evaluated.

    “Whilst the extent of mean bias is small or negligible for all subgroups evaluated, the associated imprecision is unacceptably large (pooled SDs > 1%).”
    Quote in the paperSubstantial claim
    • Rests on an assumption: SaO2 measured by CO-oximetry is the gold standard for blood oxygen saturation.
  • Findingown data

    For people with low levels of skin pigmentation, the pooled mean bias is -0.35% (95% CI -1.36 to 0.67).

    “The evidence for people with low levels of skin pigmentation does not suggest clinically important systematic bias (pooled mean bias -0.35, 95% CI − 1.36 to 0.67), but the finding is of low certainty.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: SaO2 measured by CO-oximetry is the gold standard for blood oxygen saturation.
  • Findingown data

    For all levels of skin pigmentation, the pooled SD values are around 1.50% on average.

    “For all the levels of skin pigmentation, the Arms values are around 2% or lower (95% CI non-estimable), and the pooled SD values are around 1.50% on average (Table 2).”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: SaO2 measured by CO-oximetry is the gold standard for blood oxygen saturation.
  • Findingown data

    Pulse oximetry probably overestimates oxygen saturation in people described as Black/African American compared with standard SaO2, with a pooled mean bias of 1.52% (95% CI 0.95 to 2.09%).

    “and people described as Black/African American (1.52%; 0.95 to 2.09%) (moderate- and low-certainty evidence).”
    • Also stated in: § Conclusions · hedged
      “Pulse oximetry may overestimate oxygen saturation in people with high levels of skin pigmentation and people whose ethnicity is reported as Black/African American, compared with SaO2.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: SaO2 measured by CO-oximetry is the gold standard for blood oxygen saturation.
  • May combine two claims. 43
  • Classified as probably not a substantial claim. 45

  • May combine two claims.

    Findingown data

    Pulse oximetry probably overestimates oxygen saturation in people with high levels of skin pigmentation compared with standard SaO2, with a pooled mean bias of 1.11% (95% CI 0.29 to 1.93%).

    “Compared with standard SaO2 measurement, pulse oximetry probably overestimates oxygen saturation in people with the high level of skin pigmentation (pooled mean bias 1.11%; 95% confidence interval 0.29 to 1.93%)”
    • Also stated in: § Pulse oximetry accuracy by levels of skin pigmentation · hedged
      “Hospital-based pulse oximetry probably overestimates oxygen saturation for people with high levels of skin pigmentation compared with standard SaO2 (8 studies, 24 comparisons, 3270 SpO2-SaO2 pairs from 221 participants): pooled mean bias 1.11% (95% CI 0.29 to 1.93%), moderate-certainty evidence.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: SaO2 measured by CO-oximetry is the gold standard for blood oxygen saturation.

  • May combine two claims.

    Findingown data

    The associated imprecision is unacceptably large, with pooled SDs greater than 1%.

    “Whilst the extent of mean bias is small or negligible for all subgroups evaluated, the associated imprecision is unacceptably large (pooled SDs > 1%).”
    Quote in the paperSubstantial claim
    • Rests on an assumption: SaO2 measured by CO-oximetry is the gold standard for blood oxygen saturation.

  • May combine two claims.

    Findingown data

    The bias of pulse oximetry measurements for people with other levels of skin pigmentation or those from other ethnic groups is either more uncertain or suggests no overestimation.

    “The bias of pulse oximetry measurements for people with other levels of skin pigmentation or those from other ethnic groups is either more uncertain or suggests no overestimation.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: SaO2 measured by CO-oximetry is the gold standard for blood oxygen saturation.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The pooled SD values are around 1.50% on average for all these subgroups.

    “The Arms values are around 2% or lower (95% CI non-estimable) for all these subgroups, and the pooled SD values are around 1.50% on average (Table 2).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    Pulse oximetry readings appear unacceptably imprecise for all subgroups, with pooled SDs greater than the recommended criterion of 1%.

    “For all the subgroups of populations evaluated, whilst the degree of mean bias is small or negligible over the ranges of SaO2 reported (median minimum value of 76% and maximum value of 100%), pulse oximetry readings appear unacceptably imprecise (pooled SDs > the recommended criterion of 1%) [10, 55].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [10] source

      Not checked in the free run.

    • cites [55] source

      Not checked in the free run.

  • May combine two claims.

    Findingown data

    The review included 32 studies with 6505 participants.

    “We included 32 studies (6505 participants): 15 measured skin pigmentation and 22 referred to ethnicity.”
    • Also stated in: § Study selection and characteristics · plain
      “The 32 studies (6505 participants) reported SpO2-SaO2 comparison evaluations of 54 different pulse oximeters (26 manufacturers) cf. standard SaO2 (Additional file 7: Table S3).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Sensitivity analyses showed that Arms and SD values were generally consistent, but there was increased uncertainty for mean bias findings.

    “We tested the sensitivity of the findings: Arms and SD values were generally consistent but there was increased uncertainty for mean bias findings.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: SaO2 measured by CO-oximetry is the gold standard for blood oxygen saturation.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    Of the 16 studies reporting SaO2 ranges, four had ranges in line with the recommended range of 70 to 100%, eight had narrower ranges, and four had wider ranges.

    “Of the 16 studies, four had SaO2 ranges that were in line with the recommended range of 70 to 100%; eight had narrower ranges such as 80 or 90 to 100%; and four had wider ranges such as 50 or 60 to 100%.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    Of the 32 studies, 16 (50%) reported the ranges of SaO2 over which the accuracy of pulse oximeters was evaluated; the minimum values had a median of 76% and the maximum values had a median of 100%.

    “Of the 32 studies, 16 (50%) reported the ranges of SaO2 over which the accuracy of pulse oximeters was evaluated: the minimum values of these ranges had a median of 76% whilst the maximum values had a median of 100%.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingnone given

    The evidence is uncertain for groups described in papers as White/Caucasian, meaning further research is likely to alter findings, with very low certainty evidence.

    “The evidence is uncertain for groups described in papers as White/Caucasian, meaning further research is likely to alter findings (very low certainty evidence).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    Of the 32 included studies, 15 measured skin pigmentation and 22 referred to ethnicity.

    “We included 32 studies (6505 participants): 15 measured skin pigmentation and 22 referred to ethnicity.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The evidence for people described in studies as Asian, Hispanic or of mixed ethnicity does not indicate a clinically important systematic bias, with a mean bias of 0.31% (95% CI 0.09 to 0.54%), but it is of low certainty.

    “The evidence for people described in studies as Asian, Hispanic or of mixed ethnicity does not indicate a clinically important systematic bias (mean bias 0.31%, 0.09 to 0.54%), but it is of low certainty.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The 32 studies reported SpO2-SaO2 comparison evaluations of 54 different pulse oximeters from 26 manufacturers.

    “The 32 studies (6505 participants) reported SpO2-SaO2 comparison evaluations of 54 different pulse oximeters (26 manufacturers) cf. standard SaO2 (Additional file 7: Table S3).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The remaining seven studies (503 participants) were excluded from meta-analysis due to lack of mean bias data by levels of skin pigmentation.

    “The remaining seven studies (503 participants) were excluded from meta-analysis due to lack of mean bias data by levels of skin pigmentation (Additional file 9: Table S5).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The review identified 33 publications of 32 studies published between 1985 and 2021 as eligible for inclusion.

    “Of these records, we identified 33 publications of 32 studies—published between 1985 and 2021—as eligible for inclusion (Fig. 1) [21–53].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [21–53]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Before this review, a scoping exercise using a simple search of Ovid Medline with pulse oximetry terms identified one systematic review in this area published by Jensen and colleagues in 1995.

    “Before this review, our scoping exercise using a simple search of Ovid Medline with ‘pulse oximetry’ terms identified one systematic review in this area published by Jensen and colleagues in 1995 [8].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [8] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    A recent rapid review by the NHS Race and Health Observatory had an unclear methodology.

    “We also identified a recent rapid review by the NHS Race and Health Observatory that had an unclear methodology [6].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [6] NHS Race and Health Observatory. Pulse oximetry and racial bias: Recommendations for national healthcare, regulatory and research bodies. March 2021. https://www.nhsrho.org/wp-content/uploads/2021/03/Pulse-oximetry-racial-bias-report.pdf

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    The calculated Arms values are generally around 2% or less over the ranges of SaO2 reported, far below the Arms threshold of 4% required by current international and UK standards.

    “It is interesting to note that, despite clinically important mean bias and unacceptably large imprecision identified, the calculated Arms values are generally around 2% or less over the ranges of SaO2 reported, that is, the Arms values are far below the Arms threshold of 4% required by the current international and UK standards [10, 55].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [10] source

      Not checked in the free run.

    • cites [55] source

      Not checked in the free run.

  • May combine two claims.

    Findingreasoning

    Whilst the extent of measurement bias and overall accuracy meet current international thresholds, the variation of pulse oximetry measurements appears unacceptably wide.

    “Whilst the extent of measurement bias and overall accuracy meet current international thresholds, the variation of pulse oximetry measurements appears unacceptably wide.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingnone given

    Most included studies in this review were based in hospital settings and had limited information whether the pulse oximeters evaluated were appropriate for home self-monitoring.

    “However, most included studies in this review were based in hospital settings and had limited information whether the pulse oximeters evaluated were appropriate for home self-monitoring.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingreasoning

    The included studies were published between 1985 and 2021 and some of the older studies may have used discontinued devices.

    “The included studies were published between 1985 and 2021 and some of the older studies may have used discontinued devices.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationnone given

    The extent of overestimation may be small in hospital settings but unknown in community settings.

    “The extent of overestimation may be small in hospital settings but unknown in community settings.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: SaO2 measured by CO-oximetry is the gold standard for blood oxygen saturation.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    The review was registered with the Open Science Framework.

    “The review was registered with the Open Science Framework (https://osf.io/gm7ty).”
    • Also stated in: § Search strategy and selection criteria · plain
      “The methods used were described in the registered protocol (https://osf.io/gm7ty).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductown data

    Using QUADAS-2, 14/32 studies (43.75%) were at unclear risk of bias for all four domains or high risk of bias for at least one domain, and the remaining 18 (56.25%) were at low risk of bias for at least one of the four domains.

    “Using QUADAS-2, we considered 14/32 studies (43.75%) to be at unclear risk of bias for all four domains or high risk of bias for at least one domain, and the remaining 18 (56.25%) to be at low risk of bias for at least one of the four domains (Fig. 2).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    The review is reported in accordance with the PRISMA statement.

    “We report this review in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement [9].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [9] Higgins JPT , Thomas J , Chandler J , Cumpston M , Li T , Page MJ , Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.2 (updated February 2021). Cochrane, 2021. Available from http://www.training.cochrane.org/handbook.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationnone given

    These results are for clinician-measured oximetry in controlled clinical environments and do not necessarily reflect the measurement bias of home pulse oximetry by patients or carers.

    “These results are for clinician-measured oximetry in controlled clinical environments and do not necessarily reflect the measurement bias of home pulse oximetry by patients or carers.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    Pre-planned sensitivity analyses were undertaken excluding studies with similar skin pigmentation or same ethnicity, studies with no data available for meta-analysis without transformation, and studies at high overall risk of bias.

    “We undertook pre-planned sensitivity analyses through (1) excluding studies where all participants had similar skin pigmentation or the same ethnicity, (2) excluding studies with no data available for meta-analysis without transformation, and (3) excluding studies at high overall risk of bias.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Pulse oximetry for people with other levels of skin pigmentation is less likely to be overestimated, but the evidence is uncertain.

    “Pulse oximetry for people with other levels of skin pigmentation is less likely to be overestimated but the evidence is uncertain.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Separate analysis of studies reporting level of skin pigmentation and ethnicity was pre-specified.

    “We pre-specified separate analysis of studies reporting level of skin pigmentation and ethnicity.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    One reviewer assessed the certainty of evidence on mean bias using the GRADE approach, checked by another reviewer.

    “One reviewer (CS) assessed the certainty of evidence on mean bias using the GRADE approach developed for the test accuracy topic, checked by another reviewer (GN) [18, 19].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [18] source

      Not checked in the free run.

    • cites [19] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    Two reviewers independently assessed titles and abstracts and full texts, with disagreements resolved through discussion or involving a third reviewer.

    “Two reviewers (CS and MG, or JH, OH) independently assessed titles and abstracts of the search results for relevance and the full texts of all potentially eligible studies for inclusion, with disagreements resolved through discussion or involving a third reviewer (GN) where necessary.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    In 2 studies, the flow and timing domain had time intervals between SpO2 readings and arterial blood sampling for SaO2 measurement that were too long or participants excluded from analysis without rationale.

    “for the flow and timing domain, where the time intervals between SpO2 readings and the arterial blood sampling for SaO2 measurement were too long or participants were excluded from the analysis without rationale (2 studies).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductown data

    Applicability concern was judged high for one study, moderate for 13 studies, and low for the remaining 18 studies.

    “We judged the applicability concern as high for one study, moderate for 13 studies, and low in terms of all three applicability considerations for the remaining 18 studies.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Even though the internationally recommended thresholds were met in terms of measurement bias, the relatively small amount of mean bias identified could impact on clinical decision-making at threshold values for diagnosis of hypoxaemia.

    “Even though our estimates suggest that the internationally recommended thresholds were met in terms of measurement bias [10, 55], the relatively small amount of mean bias identified could impact on clinical decision-making at threshold values for diagnosis of hypoxaemia.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [10] source

      Not checked in the free run.

    • cites [55] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    Publication bias was assessed following a qualitative approach.

    “We assessed publication bias following a qualitative approach given funnel plots or Egger’s tests were not considered appropriate for this review [20].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [20] 2017, A framework for the meta-analysis of Bland–Altman studies based on a limits of agreement approach, Stat Med source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    One reviewer extracted data and assessed risk of bias using an adapted QUADAS-2, all checked by another reviewer.

    “One reviewer (CS, or OH or JH) independently extracted data from included studies for items in Additional file 3: Box S2 and assessed the risk of bias for the included studies using an adapted QUADAS-2 (Additional file 4: Box S3) [12], all checked by another reviewer (JH, MG, OH, GN).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [12] British Standards Institution (BSI). BS EN ISO 80601-2-61:2019: Medical electrical equipment – Part 2-61: Particular requirements for basic safety and essential performance of pulse oximeter equipment (ISO 80601-2-61:2017, Corrected version 2018-02). London: British Standards Institution. 2019.

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    A recent US study found that people described as Black had nearly three times the frequency of occult hypoxemia not detected by pulse oximetry as those described as White.

    “Results showed people described as Black had ‘nearly three times the frequency of occult hypoxemia that was not detected by pulse oximetry’ as those described as White [7].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [7] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    In 30 studies, there was no blinding information for CO-oximeter SaO2 readings.

    “or CO-oximeter SaO2 readings (30 studies)”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    Underestimated SpO2 readings could also be harmful, resulting in unnecessary treatment with oxygen (and the risk of hyperoxaemia) and wider impacts such as delayed hospital discharge.

    “Underestimated SpO2 readings could also be harmful, resulting in unnecessary treatment with oxygen (and the risk of hyperoxaemia) and wider impacts such as delayed hospital discharge.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    A 1995 systematic review included three studies that explicitly considered the impact of skin pigmentation on pulse oximetry accuracy and suggested that pulse oximeters may overestimate blood oxygen saturation in people with dark skin.

    “The only current relevant systematic review, published in 1995, included three studies that explicitly considered the impact of skin pigmentation on pulse oximetry accuracy [8].”
    • Also stated in: § Background · hedged
      “The review suggested that pulse oximeters may overestimate blood oxygen saturation in people with dark skin [8].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [8] source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    The low certainty for much of the data presented means that further research could overturn these conclusions.

    “The low certainty for much of the data presented means that further research could overturn these conclusions.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductown data

    Sensitivity analyses showed Arms and SD values were generally consistent, but there was increased uncertainty for mean bias findings.

    “We tested the sensitivity of the findings: Arms and SD values were generally consistent but there was increased uncertainty for mean bias findings.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    The recent rapid review by the NHS Race and Health Observatory came to similar conclusions but used a non-systematic review process.

    “The recent rapid review by the NHS Race and Health Observatory came to similar conclusions but used a non-systematic review process, i.e., no comprehensive search, risk of bias assessment or meta-analysis [6].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [6] NHS Race and Health Observatory. Pulse oximetry and racial bias: Recommendations for national healthcare, regulatory and research bodies. March 2021. https://www.nhsrho.org/wp-content/uploads/2021/03/Pulse-oximetry-racial-bias-report.pdf

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductcitation

    This approach deals with correlations of multiple effect size estimates within a repeated-measures design study.

    “This approach deals with correlations of multiple effect size estimates within a repeated-measures design study [14, 15].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [14] 2004, Skin color is not the same thing as race, Arch Dermatol

      Not checked in the free run.

    • cites [15] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    Currently recommended thresholds may need re-evaluation, and use of the more conservative criterion of 3% applied by the US FDA guidance may have merit.

    “Given these, currently recommended thresholds may need re-evaluation, and use of the more conservative criterion of 3% applied by the US FDA guidance may have merit [56].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [56] 2018, Comparison of nasal and forehead oximetry accuracy and pressure injury in critically ill patients, Heart Lung source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationcitation

    Using scales like the Fitzpatrick scale to measure levels of skin pigmentation is a limitation; such scales are criticised as being too blunt, which impacts on the findings of this review.

    “We also acknowledge the limitation of using scales like the Fitzpatrick scale to measure levels of skin pigmentation [59]. Such scales are criticised as being too blunt—an issue that impacts on the findings of this review and should be considered in future research.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [59] 1994, Accuracy of pulse oximetry during hypoxemia, South Med J source

      Not checked in the free run.

    • The authors state this as an assumption: Scales such as the Fitzpatrick scale used in the included studies provide valid measures of levels of skin pigmentation for assessing pulse oximetry bias.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    We did not consider the differences between specific pulse oximeter devices, the differences between children and adults and their health conditions or the difference between skin pigmentation measurement methods.

    “Thirdly, we did not consider the differences between specific pulse oximeter devices, the differences between children and adults and their health conditions or the difference between skin pigmentation measurement methods.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    This review only addresses skin pigmentation and ethnicity; therefore, little is known for pulse oximetry undertaken by untrained people at home where other factors such as movement need to be considered.

    “This review only addresses skin pigmentation and ethnicity. Therefore, little is known for the case of pulse oximetry undertaken by untrained people at home where other factors such as movement need to be considered.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductcitation

    We developed a correlated hierarchical effects model and used the novel RVE approaches to meta-analyse not only independent data (of 11 studies) but also data from studies (n = 21) with repeated-measures design.

    “We developed a correlated hierarchical effects model and used the novel RVE approaches to meta-analyse not only independent data (of 11 studies) but also data from studies (n = 21) with repeated-measures design [15].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [15] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given

    No available approach to risk of bias and GRADE assessment is specific to the topic of this review.

    “Finally, no available approach to risk of bias and GRADE assessment is specific to the topic of this review.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    In the NHS rapid review, a summary of narrative findings suggested the overestimation of blood oxygen saturation levels in people with darker skin.

    “In this rapid review, a summary of narrative findings suggested the overestimation of blood oxygen saturation levels in people with darker skin.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [6] NHS Race and Health Observatory. Pulse oximetry and racial bias: Recommendations for national healthcare, regulatory and research bodies. March 2021. https://www.nhsrho.org/wp-content/uploads/2021/03/Pulse-oximetry-racial-bias-report.pdf

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Findings support calls for better calibrating algorithms used in oximeter device software to inherently address possible measurement bias.

    “Findings also support calls for better calibrating algorithms used in oximeter device software to inherently address possible measurement bias.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductreasoning

    We were only able to use the relevant approaches developed for the test accuracy topic, and the GRADE approach used was only applicable to assess the certainty of evidence for mean bias, rather than precision, Arms and limits of agreement.

    “We were only able to use the relevant approaches developed for the test accuracy topic, and the GRADE approach used was only applicable to assess the certainty of evidence for mean bias, rather than precision, Arms and limits of agreement.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    We considered the possible publication bias in assessing the certainty of evidence using GRADE approach.

    “We considered the possible publication bias in assessing the certainty of evidence using GRADE approach.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given

    We used the gold standard CO-oximetry as the comparator for pulse oximetry, and accuracy outcomes as recommended in the British Standards Institution standards for pulse oximetry.

    “We used the gold standard CO-oximetry as the comparator for pulse oximetry, and accuracy outcomes as recommended in the British Standards Institution standards for pulse oximetry.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationnone given

    Such a small overestimation may be crucial for some patients, particularly at the threshold that informs clinical decision-making.

    “Such a small overestimation may be crucial for some patients: particularly at the threshold that informs clinical decision-making.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Our findings indicate that sub-population specific recommendations would be needed for future updates.

    “Our findings indicate that sub-population specific recommendations would be needed for future updates.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Study design and conductnone given

    Our search for research is comprehensive and identified more studies.

    “Our search for research is comprehensive and identified more studies.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    We only searched for English language peer-reviewed publications, without considering preprints.

    “Fourthly, we only searched for English language peer-reviewed publications, without considering preprints.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    The Jensen review included only one study with data on the impact of skin pigmentation, and findings were inconclusive.

    “It included only one study with data on the impact of skin pigmentation, and findings were inconclusive.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [8] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    If more data were received, then the review results could change.

    “If more data were received, then the review results could change.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    The 2021 WHO living guidance recommends using pulse oximetry monitoring at home as part of care package for symptomatic people in community settings but does not note the potential impact of level of skin pigmentation.

    “In terms of COVID-19 management, the 2021 WHO living guidance recommends using pulse oximetry monitoring at home as part of care package for symptomatic people in community settings but does not note the potential impact of level of skin pigmentation [3].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [3] World Health Organization (WHO). Covid-19 clinical management: living guideline (updated 25.1.21). 2021. https://apps.who.int/iris/handle/10665/338882

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Of the nine studies identified in the NHS rapid review, seven had appropriate SpO2-SaO2 comparison data but the other two used inappropriate designs for the question being addressed.

    “Of the nine studies identified in this rapid review, seven had appropriate SpO2–SaO2 comparison data but the other two used inappropriate designs for the question being addressed.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [6] NHS Race and Health Observatory. Pulse oximetry and racial bias: Recommendations for national healthcare, regulatory and research bodies. March 2021. https://www.nhsrho.org/wp-content/uploads/2021/03/Pulse-oximetry-racial-bias-report.pdf

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given

    If we had treated race and ethnicity data separately, the evidence base would change; however, we would not expect the overall conclusion to change.

    “If we had treated race and ethnicity data separately, the evidence base would change; however, we would not expect the overall conclusion to change.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    For simplicity, we chose to use the term of ethnicity throughout this review given race and ethnicity are context/country-specific concepts and there is no globally accepted classification approach to distinguishing them.

    “For simplicity, we chose to use the term of ‘ethnicity’ throughout this review given race and ethnicity are context/country-specific concepts and there is no globally accepted classification approach to distinguishing them [58].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [58] source

      Not checked in the free run.

    • The authors state this as an assumption: Ethnicity categories used in the included studies are valid proxies for levels of skin pigmentation when pooling data on pulse oximetry bias.

  • May combine two claims.

    Premisecitation

    There are probably no major differences between summary treatment effects in English-language restricted meta-analyses and other language-inclusive meta-analyses, and the exclusion of non-English language publications from systematic reviews had no impacts on overall findings.

    “However, there are probably no major differences between summary treatment effects in English-language restricted meta-analyses and other language-inclusive meta-analyses [60], and the exclusion of non-English language publications from systematic reviews had no impacts on overall findings [61].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [60] 2021, Racial bias in pulse oximetry measurement among patients about to undergo ECMO in 2019-2020, a retrospective cohort study, Chest

      Not checked in the free run.

    • cites [61] 2021, Racial discrepancy in pulse oximeter accuracy in preterm infants, J Perinatol

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Study design and conductreasoning

    Following prespecified methods to minimise the risk of bias in the review process, this review has important strengths.

    “Following prespecified methods to minimise the risk of bias in the review process, this review has important strengths.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    There may be differences between devices for the use of health professionals in hospitals and those for home self-monitoring.

    “Regarding pulse oximeters evaluated, there may be differences between devices for the use of health professionals in hospitals and those for home self-monitoring.”
    Quote in the paperNot a substantial claim
68 of 68

  • The paper relies on this without support.

    SaO2 measured by CO-oximetry is the gold standard for blood oxygen saturation.

    “The gold standard measure of blood oxygen saturation levels (SaO2) requires a sample of arterial blood and measurement using CO-oximetry.”

  • The paper relies on this without support.

    Pulse oximetry works by beaming light through skin into the blood and inferring an SpO2 reading from the amount of light absorbed.

    “Pulse oximetry works by beaming light through skin into the blood and inferring an SpO2 reading from the amount of light absorbed.”

  • The authors state this as an assumption.

    Ethnicity categories used in the included studies are valid proxies for levels of skin pigmentation when pooling data on pulse oximetry bias.

    “For simplicity, we chose to use the term of ‘ethnicity’ throughout this review given race and ethnicity are context/country-specific concepts and there is no globally accepted classification approach to distinguishing them [58].”

  • The authors state this as an assumption.

    Scales such as the Fitzpatrick scale used in the included studies provide valid measures of levels of skin pigmentation for assessing pulse oximetry bias.

    “We also acknowledge the limitation of using scales like the Fitzpatrick scale to measure levels of skin pigmentation [59]. Such scales are criticised as being too blunt—an issue that impacts on the findings of this review and should be considered in future research.”
4 of 4

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Jensen review publication year differs from its reference

    Minor

    The text dates the Jensen review to 1995, whereas reference 8 gives its publication year as 1998.

  • Preprint cited among included studies despite stated exclusion

    Minor

    The paper says it did not consider preprints, but its list of included ethnicity studies cites reference 26, which the references explicitly label a preprint.

2 of 2

Cites

Flagged first, then by citations

Research works printed but not linked: 2

  1. 2016, An investigation into the bias of pulse oximeters, Respirology, 21, 6
  2. 2021, Accuracy of pulse oximetry (SpO2) with different oximeters. Oxygap study, Intensive Care Medicine Experimental, 9, 51

Cited by

Sorted by citations

Related

Authors
Shi, C., Goodall, M., Dumville, J., Hill, J., Norman, G., Hamer, O., Clegg, A., Watkins, C. L., Georgiou, G., Hodkinson, A., Lightbody, C. E., Dark, P., & Cullum, N.
Affiliations
  • Manchester Academic Health Science Centre (GB)
  • University of Liverpool (GB)
  • University of Lancashire (GB)
  • University of Manchester (GB)
Corresponding author
Shi, Chunhu
ORCID
Venue
BMC Medicine
Publisher
openRxiv
Published
2022-02-17
Volume
20
Issue
1
Pages
267-267
ISSN
1741-7015 (ISSN-L)
DOI
10.1101/2022.02.16.22271062
PMC
PMC9377806
OpenAlex
W4291754627
Cited by
158
References
68
Citations by year
2026: 32 · 2025: 46 · 2024: 41 · 2023: 33 · 2022: 6
FWCI
2.81
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • BMC Medicine (publishedVersion, cc-by)
  • medRxiv (acceptedVersion, cc-by-nc)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Research Explorer (The University of Manchester) (publishedVersion, cc-by)
  • CLOK (University of Central Lancashire) (acceptedVersion)
  • Research Explorer (The University of Manchester) (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Edge Hill University Research Information Repository (Edge Hill University) (submittedVersion, cc-by)
Funders
  • National Institute for Health and Care Research
  • Department of Health and Social Care
Topics
  • Physical Sciences › Engineering › Biomedical Engineering › Non-Invasive Vital Sign Monitoring
  • Health Sciences › Medicine › Surgery › Hemodynamic Monitoring and Therapy
  • Health Sciences › Medicine › Pulmonary and Respiratory Medicine › Chronic Obstructive Pulmonary Disease (COPD) Research
Keywords
pulse oximetry, oxygen saturation, skin pigmentation, measurement accuracy, co-oximetry, arterial oxygen saturation, SpO2, measurement bias, COVID-19 pandemic
MeSH terms
COVID-19, Oxygen Saturation, Humans, Oximetry, Oxygen, Skin Pigmentation, Pandemics

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review