CoChat
Sign Up
Cited by 153Medical Coding and Health InformationElectronic Health Records SystemsData Quality and Management

The quality of social determinants data in the electronic health record: a systematic review

Open access
Cook, Lily A, Sachs, Jonathan, Weiskopf, Nicole G·2021·Journal of the American Medical Informatics Association
DOI 10.1093/jamia/ocab199PMC PMC8714289
Open this paper in CoChat — free accountOpen PDF

Abstract

Objective: The aim of this study was to collect and synthesize evidence regarding data quality problems encountered when working with variables related to social determinants of health (SDoH). Materials and Methods: We conducted a systematic review of the literature on social determinants research and data quality and then iteratively identified themes in the literature using a content analysis process. Results: The most commonly represented quality issue associated with SDoH data is plausibility (n = 31, 41%). Factors related to race and ethnicity have the largest body of literature (n = 40, 53%). The first theme, noted in 62% (n = 47) of articles, is that bias or validity issues often result from data quality problems. The most frequently identified validity issue is misclassification bias (n = 23, 30%). The second theme is that many of the articles suggest methods for mitigating the issues resulting from poor social determinants data quality. We grouped these into 5 suggestions: avoid complete case analysis, impute data, rely on multiple sources, use validated software tools, and select addresses thoughtfully. Discussion: The type of data quality problem varies depending on the variable, and each problem is associated with particular forms of analytical error. Problems encountered with the quality of SDoH data are rarely distributed randomly. Data from Hispanic patients are more prone to issues with plausibility and misclassification than data from other racial/ethnic groups. Conclusion: Consideration of data quality and evidence-based quality improvement methods may help prevent bias and improve the validity of research conducted with SDoH data.

AI snapshot

Written from the paper’s full text.

TL;DR

This systematic review of 76 articles synthesizes data-quality problems in social determinants of health (SDoH) data from clinical sources, finding plausibility issues most common (n = 31, 41%) and race/ethnicity data the largest literature category (n = 40, 53%). Bias or validity concerns appeared in 62% (n = 47) of articles, especially misclassification bias (n = 23, 30%), with Hispanic patients’ data more prone to plausibility and misclassification problems.

Key findings

  • Across 76 included articles, primary quality issues were plausibility (n = 31, 41%), completeness (n = 29, 38%), and conformance (n = 16, 21%). Race/ethnicity/country-of-origin/language was the largest SDoH category (40/76; 53%), followed by geocoded address (16; 21%), occupation (7; 9%), environmental factors (5; 7%), insurance status (1; 1%), and nonspecific SDoH (7; 9%).
  • Bias/validity problems were identified or noted as potential in 47/76 articles (62%); 24 (32%) did not evaluate for bias, and 5 (7%) tested but found none. Misclassification bias was noted in 23 articles (30%), and differential incompleteness/Missing Not at Random in 16 (21%).
  • For race/ethnicity data, plausibility was discussed in 26/40 articles (65%), with misclassification noted in 18/26 (69%); 14 studies reported Hispanic patients were more likely misclassified, missing ethnicity, or grouped as “Other,” 4 found Asian patients more likely missing race, and 6 found disproportionately high misclassification for American Indians. For geocoded address data, relational conformance was examined in 10/16 articles (63%), 7 studies (43%) acknowledged potential bias, and one study reported community-level data accuracy of 48.0% for identifying patients with and without social risks.
  • Recommendations appeared in 47/76 articles (62%) and were grouped into 5 suggestions: avoid complete case analysis, impute data, rely on multiple sources, use validated software tools, and select addresses thoughtfully. Three articles on ICD Z/V-codes concluded clinicians used these codes to represent SDoH around 2% of the time.

Methods

  • Systematic review with iterative deductive and inductive content analysis; PubMed and Ovid MEDLINE were searched using MeSH-derived keywords and an adjacency search, with no date range specified, followed by snowball citation searching.
  • Eligibility required English-language original peer-reviewed research using clinical data sources in the United States and Canada (EHR, registries, administrative databases compiled from EHR data, or observational studies using clinical records); excluded nonclinical sources, vital records, behaviors such as smoking/exercise, and clinical sources outside the US/Canada.
  • Each included article was categorized by specific social determinant and primary data quality issue, then themes were identified and tabulated; 76 articles were included.

Limitations

  • Stated by the authors: data quality concepts are not completely orthogonal or distinct; snowball sampling may under-represent some research areas; publication bias may affect findings if negative bias evaluations were not reported; a larger literature for a variable may reflect accessibility rather than poorer quality; and the search for solutions focused on researchers using observational databases, ignoring point-of-care data collection improvements.
  • Evident from the text: the review is restricted to English-language studies from US/Canada clinical sources, so generalizability to other settings is unclear; it is a qualitative synthesis without meta-analysis; and 24 articles (32%) did not evaluate their datasets for bias, leaving the extent of bias in those studies unknown.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1093/jamia/ocab199

Cites

82 of 82 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

78 of 144 cited sources checked · 46 supported · 2 flagged · accuracy not yet measured

  • Findingcitation

    Fourteen studies reported that Hispanic patients were more likely to be misclassified, either that information about their ethnicity was missing or they had been mistakenly grouped into the "Other" category.

    “Fourteen studies reported that Hispanic patients were more likely to be misclassified, either that information about their ethnicity was missing or they had been mistakenly grouped into the “Other” category.11,23–25,29,30,32,33,36–41”
    Quote in the paperNot a substantial claim4 of 10 sources checked
    • cites [11] Magana Lopez, 2016, Discrepancies in race and ethnicity documentation: a potential barrier in identifying racial and ethnic disparities, J Racial Ethn Health Disparities source
    • cites [23] Lee, 2016, Assessing race and ethnicity data quality across cancer registries and EMRs in two hospitals, J Am Med Inform Assoc source

      The Discussion reports mismatches between “White” and “Hispanic” and poor-quality records coded “Other” or “Unknown.” It does not establish that Hispanic patients were more likely than other patients to have missing ethnicity or to be placed in “Other.” The count of fourteen studies belongs to the citing paper’s collective statement, not to this individual source. (LLM, conf=0.85)

      Experimental — accuracy not yet measured
    • cites [24] Grafova, 2021, Beyond black and white: mapping misclassification of Medicare beneficiaries race and ethnicity, Med Care Res Rev source
    • cites [25] Webster, 2013, Conflicting race/ethnicity reports: lessons for improvement in data quality, J Registry Manag
    • cites [29] Pinheiro, 2009, Validation of ethnicity in cancer data: which Hispanics are we misclassifying?, J Registry Manag
    • cites [30] Eicheldinger, 2008, More accurate racial and ethnic codes for Medicare administrative data, Health Care Financ Rev
    • cites [32] West, 2005, Race and ethnicity: comparing medical records to self-reports, J Natl Cancer Inst Monogr source

      The Results and Table 2 (page not supplied) support a higher misclassification rate for Latinas: medical records correctly identified 32 of 50 self-reported Latinas. But Table 2 records 17 as white and one as Asian; it does not support the claimed explanation that their ethnicity was missing or that they were grouped as “Other.” This paper is one study, not a report of fourteen studies. (LLM, conf=0.88)

      Experimental — accuracy not yet measured
    • cites [33] Waldo, 2004, Accuracy and bias of race ethnicity codes in the Medicare enrollment database, Health Care Financ Rev
    • cites [36] Polednak, 2001, Agreement in race-ethnicity coding between a hospital discharge database and another database, Ethn Dis
    • cites [37] Morgan, 2004, Improving identification of Hispanic males in Medicare: use of surname matching, Med Care source
  • Findingcitation

    Misclassification bias was noted as a problem or potential problem in 18 of the 26 (69%) articles about the plausibility of race/ethnicity data.

    “Misclassification bias, that is, incorrect assignment, was noted as a problem or a potential problem in 18 of the 26 (69%) articles about the plausibility of race/ethnicity data.11,13,23–38”
    Quote in the paperNot a substantial claim6 of 10 sources checked
    • cites [11] Magana Lopez, 2016, Discrepancies in race and ethnicity documentation: a potential barrier in identifying racial and ethnic disparities, J Racial Ethn Health Disparities source

      The Introduction (page unavailable) describes incorrect race and ethnicity data and discrepancies between self-reports and records, supporting the general concern about misclassification. Neither it nor the Discussion (page unavailable) reports a review of 26 articles or the figure of 18 articles (69%); that numerical claim is unsupported by this source. (LLM, conf=0.85)

      Experimental — accuracy not yet measured
    • cites [13] Zingmond, 2015, Improving hospital reporting of patient race and ethnicity—approaches to data auditing, Health Serv Res source
    • cites [23] Lee, 2016, Assessing race and ethnicity data quality across cancer registries and EMRs in two hospitals, J Am Med Inform Assoc source
    • cites [24] Grafova, 2021, Beyond black and white: mapping misclassification of Medicare beneficiaries race and ethnicity, Med Care Res Rev source
    • cites [25] Webster, 2013, Conflicting race/ethnicity reports: lessons for improvement in data quality, J Registry Manag
    • cites [26] Smith, 2010, Health plan administrative records versus birth certificate records: quality of race and ethnicity information in children, BMC Health Serv Res source
    • cites [27] Gomez, 2006, Misclassification of race/ethnicity in a population-based cancer registry (United States), Cancer Causes Control source
    • cites [28] Bigback, 2015, Using record linkage to improve race data quality for American Indians and Alaska Natives in two Pacific Northwest State Hospital discharge databases, Health Serv Res source
    • cites [29] Pinheiro, 2009, Validation of ethnicity in cancer data: which Hispanics are we misclassifying?, J Registry Manag
    • cites [30] Eicheldinger, 2008, More accurate racial and ethnic codes for Medicare administrative data, Health Care Financ Rev
  • Findingcitation

    Four studies found that Asian patients were more likely to be missing information identifying their race than white patients.

    “Four of these studies also found that Asian patients were more likely to be missing information identifying their race than white patients.24,30,33,36”
    Quote in the paperNot a substantial claim1 of 4 sources checked
    • cites [24] Grafova, 2021, Beyond black and white: mapping misclassification of Medicare beneficiaries race and ethnicity, Med Care Res Rev source

      The Results section ('Misclassification of Asians and Pacific Islander beneficiaries'; page unavailable) reports substantial misclassification of Asian American/Pacific Islander Medicare beneficiaries, and the Discussion says Asians are under-identified in Medicare administrative data. The paper does not establish that their race information was missing, rather than incorrectly classified, or give the claimed missing-information comparison with White patients. (LLM, conf=0.83)

      Experimental — accuracy not yet measured
    • cites [30] Eicheldinger, 2008, More accurate racial and ethnic codes for Medicare administrative data, Health Care Financ Rev
    • cites [33] Waldo, 2004, Accuracy and bias of race ethnicity codes in the Medicare enrollment database, Health Care Financ Rev
    • cites [36] Polednak, 2001, Agreement in race-ethnicity coding between a hospital discharge database and another database, Ethn Dis
  • Findingcitation

    Three studies in the review found that casewise deletion decreased the quality of race/ethnicity data.

    “However, 3 studies in our review found that casewise deletion decreased the quality of race/ethnicity data.23,50,65”
    Quote in the paperSubstantial claim3 of 3 sources checked
    • cites [23] Lee, 2016, Assessing race and ethnicity data quality across cancer registries and EMRs in two hospitals, J Am Med Inform Assoc source

      The Concordance section (page unavailable) reports that excluding records with 'Unknown' race/ethnicity values eliminated over 44% of the pairs, supporting a loss of information from deletion. But the Discussion says the exclusion substantially increased agreement; the paper does not conclude that deletion decreased data quality overall. The count of three studies belongs to the citing review, not to this individual source. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
    • cites [50] Brown, 2016, Using Bayesian imputation to assess racial and ethnic disparities in pediatric performance measures, Health Serv Res source

      The HEDIS Analysis and Discussion report that dropping cases with missing race or ethnicity data biased disparity estimates and sometimes reduced their statistical significance. That supports the concern about casewise deletion, but the paper does not show that deletion decreased the quality of the race/ethnicity data themselves. The count of three studies describes the citing authors’ review, not a finding attributed to this paper. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
    • cites [65] Grundmeier, 2015, Imputing missing race/ethnicity in pediatric electronic health records: reducing bias with use of U.S. Census location and surname data, Health Serv Res source

      The Performance of Imputation Methods section reports that complete-case analysis, which excludes records with missing values, produced highly biased estimates of associations between race/ethnicity and outcomes. This supports the concern about casewise deletion, but the paper measures bias in analysis results rather than a decrease in the quality of the race/ethnicity data itself. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
  • Findingcitation

    Several studies looked at the use of imputation to increase the completeness of datasets and avoid casewise deletion.

    “Several studies looked at the use of imputation, also referred to as indirect estimation, to increase the completeness of datasets and avoid casewise deletion.14,22,30,45,46,50,61,65,67–70”
    Quote in the paperNot a substantial claim7 of 10 sources checked
    • cites [14] Ma, 2018, The HCUP SID imputation project: improving statistical inferences for health disparities research by imputing missing race data, Health Serv Res source
    • cites [22] Derose, 2013, Race and ethnicity data quality and imputation using U.S. Census Data in an integrated health system: the Kaiser Permanente Southern California Experience, Med Care Res Rev source
    • cites [30] Eicheldinger, 2008, More accurate racial and ethnic codes for Medicare administrative data, Health Care Financ Rev
    • cites [45] Haas, 2019, Imputation of race/ethnicity to enable measurement of HEDIS performance by race/ethnicity, Health Serv Res source

      The Conclusion (page unavailable in our copy) says the study supports indirect methods for reporting race/ethnicity when Medicare administrative data are unavailable or improperly reported. That supports using imputation to address incomplete data, but the paper does not say it avoids casewise deletion; its Methods section says respondents missing self-reported race/ethnicity were omitted from the validation sample. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
    • cites [46] Dembosky, 2019, Indirect estimation of race ethnicity for survey respondents who do not report race/ethnicity, Med Care source
    • cites [50] Brown, 2016, Using Bayesian imputation to assess racial and ethnic disparities in pediatric performance measures, Health Serv Res source
    • cites [61] Scott, 2021, Using hospitalization data for injury surveillance in agriculture, forestry and fishing: a crosswalk between ICD10CM external cause of injury coding and the Occupational Injury and Illness Classification System, Inj Epidemiol source

      The Methods section describes mapping injury codes and appending OIICS codes to hospital records, while the Limitations section discusses incomplete E-code reporting. Neither addresses imputation, indirect estimation, or avoiding casewise deletion; searches for those terms found no supporting passage in our copy. Page numbers were not provided. (LLM, conf=0.75)

      Experimental — accuracy not yet measured
    • cites [65] Grundmeier, 2015, Imputing missing race/ethnicity in pediatric electronic health records: reducing bias with use of U.S. Census location and surname data, Health Serv Res source
    • cites [67] Curriero, 2010, Using imputation to provide location information for nongeocoded addresses, PLoS One source

      The Introduction (page unavailable) says the paper develops imputation strategies for nongeocoded addresses so analysis can use all case records. The Discussion (page unavailable) describes completing the data with geocoded and imputed records rather than excluding nongeocoded cases. Our copy does not identify imputation as “indirect estimation,” so that part of the attributed sentence is unsupported. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
    • cites [68] Kim, 2018, RIDDLE: Race and ethnicity Imputation from Disease history with Deep LEarning, PLoS Comput Biol source

      The Introduction’s “Imputation” section (page unavailable) describes estimating missing race and ethnicity values and using them to generate an imputed dataset, supporting the dataset-completeness part of the claim. Our copy does not say that the study used imputation to avoid casewise deletion. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
  • Classified as probably not a substantial claim. 36
  • May combine two claims. 27

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Two studies concluded that address at birth was adequate for research on the effect of early exposures.

    “while the other 2 concluded that address at birth was adequate for research on the effect of early exposures.15,57”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites [15] Chen, 2010, Residential mobility during pregnancy and the potential for ambient air pollution exposure misclassification, Environ Res source
    • cites [57] Ling, 2019, Residential mobility in early childhood and the impact on misclassification in pesticide exposures, Environ Res source

      The Discussion conclusion says that a single residential address should be used with caution to estimate early-childhood exposures, especially after the first year of life. Although it reports moderate-to-good agreement for pesticide exposures assessed around birth and alternative addresses, it does not conclude that the birth address is adequate for the stated research purpose. (LLM, conf=0.86)

      Checked by a reasoning model — may be wrong.

      Experimental — accuracy not yet measured
    • Rests on an assumption: Patients move over addresses over time.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Three studies concluded that a single patient address was sufficient when patients may have moved several times over the study period.

    “When confronted with data from patients who may have moved several times over the study period, 3 studies concluded that a single patient address was sufficient;15,56,57”
    Quote in the paperNot a substantial claim2 of 3 sources checked
    • cites [15] Chen, 2010, Residential mobility during pregnancy and the potential for ambient air pollution exposure misclassification, Environ Res source
    • cites [56] Hughes, 2017, The utility of EMR address histories for assessing neighborhood exposures, Ann Epidemiol source

      The Introduction (page unavailable) says life-course neighborhood exposures cannot be fully assessed through a single measurement in space and time. The Discussion (page unavailable) also cautions that exposure misclassification may be greater over longer periods or with more moves. This paper does not conclude that one address is sufficient for patients who may have moved several times. (LLM, conf=0.88)

      Checked by a reasoning model — may be wrong.

      Experimental — accuracy not yet measured
    • cites [57] Ling, 2019, Residential mobility in early childhood and the impact on misclassification in pesticide exposures, Environ Res source

      The Discussion (page unavailable) reports moderate-to-strong agreement between pesticide exposure estimates based on a single address and residential histories, and says misclassification may be acceptable for the first year of life. It does not conclude that one address is generally sufficient when patients have moved several times: agreement varied with distance moved and buffer size, and the paper cautions that misclassification remains an issue. (LLM, conf=0.86)

      Experimental — accuracy not yet measured
    • Rests on an assumption: Patients move over addresses over time.

  • May combine two claims.

    Premisereasoning

    Few social determinants of health data elements are routinely stored in the medical record, and these data have remained sparse despite widespread efforts to collect more.

    “SDoH are, by their very nature, nonmedical. As such, few SDoH data elements are routinely stored in the medical record.”
    • Also stated in: § BACKGROUND · plain
      “although there has been a widespread effort to collect more SDoH from patients these data have remained sparse.3”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [3] Hatef, 2019, Assessing the availability of data on social and behavioral determinants in structured and unstructured electronic health records: a retrospective analysis of a multilevel health care system, JMIR Med Inform source

  • May combine two claims.

    Premisecitation

    Diagnostic codes indicating social determinants of health are standardized across systems but are rarely used by clinicians.

    “Diagnostic codes (eg , ICD Z- codes) indicating SDoH such as “Problems relating to housing and economic circumstances” have the benefit of being standardized across systems, but in practice are rarely used by clinicians.4–6”
    Quote in the paperSubstantial claim2 of 3 sources checked
    • cites [4] Torres, 2017, ICD social codes: an underutilized resource for tracking social needs, Med Care source
    • cites [5] Guo, 2020, International Classification of Diseases, Tenth Revision, Clinical Modification social determinants of health codes are poorly used in electronic health records, Medicine (Baltimore) source
    • cites [6] Truong, 2020, Utilization of social determinants of health ICD-10 Z-codes among hospitalized patients in the United States, 2016–2017, Med Care source

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingcitation

    A study used natural language processing (NLP) to comb through unstructured text fields in clinical notes and used the results to augment race/ethnicity data missing from structured fields, improving identification of race and ethnicity in EHR data.

    “Another study used natural language processing (NLP) to “improve the identification of race and ethnicity in EHR data.”49 Those researchers used NLP to comb through the unstructured text fields in clinical notes, then used the results to augment race/ethnicity data missing from structured fields.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [49] Sholle, 2019, Underserved populations with missing race ethnicity data differ significantly from those with structured race/ethnicity documentation, J Am Med Inform Assoc source

  • May combine two claims.

    Findingcitation

    Two of the articles mentioned the potential for bias, in both cases due to data missing nonrandomly.

    “Two of the articles mentioned the potential for bias, in both cases due to data missing nonrandomly.6,59”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites [6] Truong, 2020, Utilization of social determinants of health ICD-10 Z-codes among hospitalized patients in the United States, 2016–2017, Med Care source
    • cites [59] Cottrell, 2019, Variation in electronic health record documentation of social determinants of health across a national network of community health centers, Am J Prev Med source

  • May combine two claims.

    Findingcitation

    Ma et al compared 4 imputation methods and found that conditional multiple imputation substantially improved statistical inferences for racial health disparities research.

    “Ma et al14 compared 4 imputation methods and found that conditional multiple imputation, “substantially improved statistical inferences for racial health disparities research.””
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [14] Ma, 2018, The HCUP SID imputation project: improving statistical inferences for health disparities research by imputing missing race data, Health Serv Res source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    The accuracy of community-level data for identifying patients with and without social risks was 48.0%.

    “They found that “the accuracy of the community-level data for identifying patients with and without social risks was 48.0%.””
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [52] Cottrell, 2020, Comparison of community-level and patient-level social risk data in a network of community health centers, JAMA Netw Open source

  • May combine two claims.

    Findingcitation

    Four studies found that occupational data were not missing at random and that male patients were more likely to have occupational information in their record.

    “Four studies found that data were not missing at random and that male patients were more likely to have occupational information in their record.60,62–64”
    Quote in the paperNot a substantial claim2 of 4 sources checked
    • cites [60] McClure, 2019, Availability and accuracy of occupation in cancer registry data among Florida firefighters, PLoS One source

      The Discussion (page not specified) says occupation fields are missing disproportionately by case characteristics, including gender. The Results (page not specified) find that male Florida firefighters were more likely to have a firefighting-related occupation code, but do not establish that male patients generally were more likely to have any occupational information in their records. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
    • cites [62] Silver, 2018, Codability of industry and occupation information from cancer registry records: differences by patient demographics, casefinding source, payor, and cancer type, Am J Ind Med source

      Table I reports codable industry information for 40.4% of male patients versus 34.3% of female patients; the paper says occupation results were similar. The Discussion says availability varies by demographic, reporting, and outcome characteristics, but frames the resulting selection bias as a possibility rather than establishing that these data were not missing at random. This paper also cannot establish the citing sentence’s count of four studies. (LLM, conf=0.86)

      Experimental — accuracy not yet measured
    • cites [63] Armenti, 2010, Improving the quality of industry and occupation data at a central cancer registry, Am J Ind Med source
    • cites [64] Polednak, 2005, Obtaining occupation as an indicator of patients' socioeconomic status in a population-based cancer registry, J Registry Manag

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    Ten of these (66%) identified that the incomplete data led to validity issues, most commonly that these data were not missing at random and had the potential to introduce bias.

    “Ten of these (66%) identified that the incomplete data led to validity issues, most commonly that these data were not missing at random and had the potential to introduced bias.12,34,43–50”
    Quote in the paperSubstantial claim5 of 10 sources checked
    • cites [12] Nasir, 2019, Disparities in patient record completeness with respect to the health care utilization project, Health Informatics J source
    • cites [34] Klinger, 2015, Accuracy of race, ethnicity, and language preference in an electronic health record, J Gen Intern Med source

      The Discussion reports inaccuracies in EHR demographic data, including a possible undercount of patients identifying as African American or Hispanic, and says accurate data are needed to monitor health disparities. That supports a concern about validity and potential bias, but the paper does not establish that incomplete data were missing not at random. The count of ten studies (66%) is the citing authors’ synthesis, not a finding attributed to this individual paper. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
    • cites [43] Gomez, 2004, Bias in completeness of birthplace data for Asian groups in a population-based cancer registry (United States), Cancer Causes Control source
    • cites [44] Lin, 2001, Factors associated with missing birthplace information in a population-based cancer registry, Ethn Dis
    • cites [45] Haas, 2019, Imputation of race/ethnicity to enable measurement of HEDIS performance by race/ethnicity, Health Serv Res source

      The paper says in its introductory discussion that systematic race/ethnicity misclassification leads to biased disparity estimates. Section 2.3.1 also says missing-data quality might correlate with race/ethnicity, but it does not establish that the missing data were not at random. The “ten (66%)” figure is the citing authors’ synthesis, not a finding attributed to this individual paper. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
    • cites [46] Dembosky, 2019, Indirect estimation of race ethnicity for survey respondents who do not report race/ethnicity, Med Care source
    • cites [47] Gomez, 2005, Quality of cancer registry birthplace data for Hispanics living in the United States, Cancer Causes Control source
    • cites [48] Chen, 2018, Racial differences in data quality and completeness: spinal cord injury model systems’ experiences, Top Spinal Cord Inj Rehabil source

      The Abstract says racial variation in data quality could introduce bias into spinal cord injury health-disparity research, and the Conclusion calls for further study of whether non-response bias affects that research. The paper does not establish that the data were missing not at random. The ‘ten of these (66%)’ count is the citing authors’ synthesis, not a result reported by this source. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
    • cites [49] Sholle, 2019, Underserved populations with missing race ethnicity data differ significantly from those with structured race/ethnicity documentation, J Am Med Inform Assoc source
    • cites [50] Brown, 2016, Using Bayesian imputation to assess racial and ethnic disparities in pediatric performance measures, Health Serv Res source

      The Discussion reports that dropping records with missing race and ethnicity can bias disparity estimates, supporting the concern about validity and bias. The Summary Statistics section also finds differences between children with and without reported race and ethnicity, but the paper does not establish that the data were ‘not missing at random.’ The ten-study (66%) figure is the citing authors’ tally, not a result attributed to this individual paper. (LLM, conf=0.83)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingcitation

    In all of these articles, bias was characterized as misclassification of exposure to contaminants, an issue noted elsewhere to be a source of Type II error in environmental health research.

    “In all of these articles, bias was characterized as misclassification of exposure to contaminants, an issue that has been noted elsewhere to be a source of Type II error in environmental health research.58”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [58] Armstrong, 1998, Effect of measurement error on epidemiological studies of environmental and occupational exposures, Occup Environ Med source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Smith et al. recommended supplementing race/ethnicity data from the EHR with birth certificate data to increase completeness and plausibility.

    “For example, Smith et al.26 recommended supplementing race/ethnicity data from the EHR with birth certificate data to increase completeness and plausibility.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Smith et al. [26] Smith, 2010, Health plan administrative records versus birth certificate records: quality of race and ethnicity information in children, BMC Health Serv Res source

  • May combine two claims.

    Findingcitation

    Ling et al noticed significant differences between patients with complete address information and those who were missing information; Hispanic women born in Mexico and people living in poor neighborhoods were more likely to have missing addresses.

    “In a study on using patient addresses to determine pesticide exposures, Ling et al57 noticed that there were significant differences between patients with complete address information and those who were missing information. In particular, Hispanic women born in Mexico and people living in poor neighborhoods were more likely to have missing addresses.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [57] Ling, 2019, Residential mobility in early childhood and the impact on misclassification in pesticide exposures, Environ Res source

  • Classified as probably not a substantial claim. May combine two claims.

    Findingcitation

    Imputation was recommended not only as a solution for missing data but also to improve the accuracy of implausible data.

    “Imputation was recommended not only as a solution for missing data but also to improve the accuracy of implausible data.29,30,36,37,66,71”
    Quote in the paperSubstantial claim1 of 6 sources checked
    • cites [29] Pinheiro, 2009, Validation of ethnicity in cancer data: which Hispanics are we misclassifying?, J Registry Manag
    • cites [30] Eicheldinger, 2008, More accurate racial and ethnic codes for Medicare administrative data, Health Care Financ Rev
    • cites [36] Polednak, 2001, Agreement in race-ethnicity coding between a hospital discharge database and another database, Ethn Dis
    • cites [37] Morgan, 2004, Improving identification of Hispanic males in Medicare: use of surname matching, Med Care source
    • cites [66] Wei, 2006, Using a Spanish surname match to improve identification of Hispanic women in Medicare administrative data, Health Serv Res source

      The Conclusions recommend adding a Spanish-surname-based proxy race code to improve the accuracy of racial and ethnic representation in Medicare data. This supports using a proxy to address inaccurate coding, but the paper does not recommend imputation for missing data or describe the codes as implausible data. (LLM, conf=0.77)

      Experimental — accuracy not yet measured
    • cites [71] Hsieh, 2011, Using NAPIIA to improve the accuracy of Asian race codes in registry data, J Registry Manag

  • May combine two claims.

    Findingcitation

    Most of these studies examined methods for imputing race/ethnicity data, although several looked at imputing geocoded patient addresses, and one looked at imputing occupational data.

    “Most of these studies examined methods for imputing race/ethnicity data, although several looked at imputing geocoded patient addresses,67,69,70 and one looked at imputing occupational data.61”
    Quote in the paperNot a substantial claim4 of 4 sources checked
    • cites [67] Curriero, 2010, Using imputation to provide location information for nongeocoded addresses, PLoS One source
    • cites [69] Henry, 2008, Estimating the accuracy of geographical imputation, Int J Health Geogr source
    • cites [70] Hibbert, 2009, Evaluating geographic imputation approaches for zip code level data: an application to a study of pediatric diabetes, Int J Health Geogr source
    • cites [61] Scott, 2021, Using hospitalization data for injury surveillance in agriculture, forestry and fishing: a crosswalk between ICD10CM external cause of injury coding and the Occupational Injury and Illness Classification System, Inj Epidemiol source

      The Methods section (‘Testing the crosswalk on hospital data’; page unavailable) describes using injury codes to flag hospital records and append classification codes, which supports the narrower point that the study inferred work-related injury categories from records. It does not describe imputing patients’ occupational data. The Discussion (‘Limitations’; page unavailable) says hospital data cannot definitively assign occupational status; its explicit imputation in Methods concerns median age, not occupation. (LLM, conf=0.82)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Use of community-level data for patient risk stratification may heighten the risk of ecologic fallacy, wherein incorrect assumptions are made about an individual based on aggregate-level information about a group.

    “The authors noted that the use of these data for patient risk stratification “may heighten the risk of ecologic fallacy, wherein incorrect assumptions are made about an individual based on aggregate-level information about a group.”52”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [52] Cottrell, 2020, Comparison of community-level and patient-level social risk data in a network of community health centers, JAMA Netw Open source

  • May combine two claims.

    Findingcitation

    Six studies found disproportionately high rates of misclassification for American Indians in comparison to other racial/ethnic groups; most often, these patients were misidentified as white.

    “Six studies found disproportionately high rates of misclassification for American Indians in comparison to other racial/ethnic groups; most often, these patients were misidentified as white.27,28,30,32,41,42”
    Quote in the paperNot a substantial claim2 of 6 sources checked
    • cites [27] Gomez, 2006, Misclassification of race/ethnicity in a population-based cancer registry (United States), Cancer Causes Control source
    • cites [28] Bigback, 2015, Using record linkage to improve race data quality for American Indians and Alaska Natives in two Pacific Northwest State Hospital discharge databases, Health Serv Res source
    • cites [30] Eicheldinger, 2008, More accurate racial and ethnic codes for Medicare administrative data, Health Care Financ Rev
    • cites [32] West, 2005, Race and ethnicity: comparing medical records to self-reports, J Natl Cancer Inst Monogr source
    • cites [41] Gomez, 2005, Inconsistencies between self-reported ethnicity and ethnicity recorded in a health maintenance organization, Ann Epidemiol source
    • cites [42] Fiscella, 2008, Race and ethnicity coding agreement between hospitals and between hospital and death data, Med Sci Monit

  • May combine two claims.

    Findingcitation

    Six of the 7 studies (86%) that looked at the quality of occupational data primarily examined completeness, all noting that occupational information is frequently missing from the health record.

    “Six of the 7 studies (86%) that looked at the quality of occupational data primarily examined completeness, all noting that occupational information is frequently missing from the health record.10,60–64”
    Quote in the paperNot a substantial claim4 of 6 sources checked
    • cites [10] Freeman, 2017, Capture and coding of industry and occupation measures: findings from eight National Program of Cancer Registries states, Am J Ind Med source
    • cites [60] McClure, 2019, Availability and accuracy of occupation in cancer registry data among Florida firefighters, PLoS One source
    • cites [61] Scott, 2021, Using hospitalization data for injury surveillance in agriculture, forestry and fishing: a crosswalk between ICD10CM external cause of injury coding and the Occupational Injury and Illness Classification System, Inj Epidemiol source

      The Background (page unavailable) says occupational variables are missing from many injury surveillance systems, and the Discussion’s Limitations section says hospital data alone cannot definitively establish occupational status. The paper does not report the six-of-seven (86%) tally or primarily examine the completeness of occupational information in health records; it tests a coding crosswalk for injury surveillance. (LLM, conf=0.83)

      Experimental — accuracy not yet measured
    • cites [62] Silver, 2018, Codability of industry and occupation information from cancer registry records: differences by patient demographics, casefinding source, payor, and cancer type, Am J Ind Med source
    • cites [63] Armenti, 2010, Improving the quality of industry and occupation data at a central cancer registry, Am J Ind Med source
    • cites [64] Polednak, 2005, Obtaining occupation as an indicator of patients' socioeconomic status in a population-based cancer registry, J Registry Manag

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Findingcitation

    Multiple studies recommended comparing and linking data across an individual's record to fill in or correct missing data fields, either to increase completeness and/or to check the accuracy of implausible data.

    “This method compares and links data across an individual’s record in order to fill in or correct missing data fields and was recommended by several studies as a way to either increase completeness26,28,49,60,62,64,72 and/or check the accuracy of implausible data.13,26,28”
    Quote in the paperSubstantial claim3 of 3 sources checked
    • cites [26] Smith, 2010, Health plan administrative records versus birth certificate records: quality of race and ethnicity information in children, BMC Health Serv Res source
    • cites [28] Bigback, 2015, Using record linkage to improve race data quality for American Indians and Alaska Natives in two Pacific Northwest State Hospital discharge databases, Health Serv Res source
    • cites [49] Sholle, 2019, Underserved populations with missing race ethnicity data differ significantly from those with structured race/ethnicity documentation, J Am Med Inform Assoc source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Thirteen studies speculated that poor data quality for Hispanic patients may be due to the fluid, debatable, and problematic nature of the definition of race and ethnicity.

    “Thirteen studies speculated that this may be due to the “fluid, debatable, and problematic”34 nature of the definition of race and ethnicity.11,22,24–26,34,37,40,41,45,46,80,81”
    Quote in the paperNot a substantial claim2 of 3 sources checked
    • cites [34] Klinger, 2015, Accuracy of race, ethnicity, and language preference in an electronic health record, J Gen Intern Med source
    • cites [11] Magana Lopez, 2016, Discrepancies in race and ethnicity documentation: a potential barrier in identifying racial and ethnic disparities, J Racial Ethn Health Disparities source

      The Introduction describes racial and ethnic identification as fluid, and the Discussion links Hispanic/Latino reporting inconsistencies to difficulty fitting existing race and ethnicity categories. The paper does not use the quoted phrase “fluid, debatable, and problematic,” so it supports the explanation but not the quotation attributed to it. The count of thirteen studies is the citing authors’ aggregation, not a finding attributed to this individual source. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
    • cites [22] Derose, 2013, Race and ethnicity data quality and imputation using U.S. Census Data in an integrated health system: the Kaiser Permanente Southern California Experience, Med Care Res Rev source

  • May combine two claims.

    Findingcitation

    Several studies found that the plausibility of patient race/ethnicity information in their dataset was questionable because the data were incomplete; plausibility was low because a patient's race was reported correctly in one area but as 'other' in another.

    “For example, several studies found that the plausibility of the patient race/ethnicity information in their dataset was questionable because the data were incomplete; in other words, plausibility was low because a patient’s race was reported correctly in one area, but as “other” in another.21,86,87”
    Quote in the paperNot a substantial claim2 of 3 sources checked
    • cites [21] Hamilton, 2009, Concordance between self-reported race ethnicity and that recorded in a veteran affairs electronic medical record, N C Med J source
    • cites [86] Kressin, 2003, Agreement between administrative data and patients’ self-reports of race/ethnicity, Am J Public Health source

      Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

      Experimental — accuracy not yet measured
    • cites [87] Maizlish, 2006, Race/ethnicity in medical charts and administrative databases of patients served by community health centers, Ethn Dis source

      Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    One study recommended using the most recent address.

    “one study recommended using the most recent address,56”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [56] Hughes, 2017, The utility of EMR address histories for assessing neighborhood exposures, Ann Epidemiol source

      The Discussion (page unavailable) finds that more recent EMR address entries are more accurate, which supports the rationale for using the most recent address. The paper does not explicitly recommend using only that address; its Conclusions instead describe the value of address histories for longitudinal exposure research. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
    • Rests on an assumption: Patients move over addresses over time.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingcitation

    The NIOSH Industry and Occupation Computerized Coding System, a free web-based system, was evaluated and recommended by several studies for increasing the quality of occupation data prior to analysis.

    “For researchers seeking to increase the quality of occupation data prior to analysis, the National Institute of Occupational Safety and Health (NIOSH) has created a free, web-based system called NIOSH Industry and Occupation Computerized Coding System78 that was evaluated and recommended by several studies.10,60,62,76”
    Quote in the paperNot a substantial claim2 of 3 sources checked
    • cites [78] 2021

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [10] Freeman, 2017, Capture and coding of industry and occupation measures: findings from eight National Program of Cancer Registries states, Am J Ind Med source

      The Materials and Methods section (page unavailable) describes NIOCCS as a freely available web-based system and evaluates its coding of occupation and industry text. The Discussion (page unavailable) says researchers can use its confidence levels to balance capture against coding confidence, but does not explicitly recommend it as a way to increase the quality of occupation data before analysis. (LLM, conf=0.83)

      Experimental — accuracy not yet measured
    • cites [60] McClure, 2019, Availability and accuracy of occupation in cancer registry data among Florida firefighters, PLoS One source

      The Discussion (page unavailable) describes NIOCCS as a web-based tool and reports that an eight-state study found it useful for improving occupation fields. It also calls such tools valuable, but this paper did not itself evaluate NIOCCS, and it cautions that the tool cannot remedy missing occupation information. (LLM, conf=0.86)

      Experimental — accuracy not yet measured

  • Classified as probably not a substantial claim. May combine two claims.

    Findingreasoning

    Little information is available to assist researchers making choices about social determinants of health data, and there are no agreed-upon best practices for working with such data.

    “However, little information is available to assist researchers making these choices, and there are no agreed-upon best practices for working with SDoH data.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Findingown data

    Many articles suggest methods for mitigating issues resulting from poor social determinants of health data quality, grouped into five suggestions: avoid complete case analysis, impute data, rely on multiple sources, use validated software tools, and select addresses thoughtfully.

    “The second theme is that many of the articles suggest methods for mitigating the issues resulting from poor social determinants data quality. We grouped these into 5 suggestions: avoid complete case analysis, impute data, rely on multiple sources, use validated software tools, and select addresses thoughtfully.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    A total of 76 articles were included in this review.

    “A total of 76 articles were included in this review.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    Quality problems found with geocoded patient address data were not randomly distributed; relational conformance (match rates) tend to be poorer for rural areas and certain parts of the country.

    “It should also be noted that the quality problems found with geocoded patient address data were not randomly distributed; for example, relational conformance (ie, match rates) tend to be poorer for rural areas and certain parts of the country.53–55”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [53] Lin, 2010, Improving geocoding outcomes for the Nebraska Cancer Registry: learning from proven practices, J Registry Manag
    • cites [54] Dilekli, 2020, Improved geocoding of cancer registry addresses in urban and rural Oklahoma, J Registry Manag
    • cites [55] Wilkins, 1993, Use of postal codes and addresses in the analysis of health data, Health Rep

  • May combine two claims.

    Findingcitation

    All of these studies confirmed that these techniques reduced misclassification errors and enhanced data quality for their respective populations.

    “All of these studies confirmed that these techniques reduced misclassification errors and enhanced data quality for their respective populations.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Six of the 7 articles which addressed social determinants data as a broad, general category assessed the completeness of this information (86%); one addressed plausibility.

    “Six of the 7 articles which addressed social determinants data as a broad, general category assessed the completeness of this information (86%); one addressed plausibility.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Findingreasoning

    In all 3 studies, imputation was recommended as preferable to casewise deletion of missing data.

    “In all 3 studies, imputation was recommended as preferable to casewise deletion of missing data.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingcitation

    Several articles evaluated a specific software tool, usually one developed by the authors, for either assessing or increasing the quality of the data.

    “Several articles evaluated a specific software tool, usually one developed by the authors, for either assessing or increasing the quality of the data.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingcitation

    Dembosky et al found that BISG did not substantially alter the estimated overall racial/ethnic distribution, but it did modestly increase sample size and statistical power.

    “In their study on the use of BISG, Dembosky et al46 found that it “did not substantially alter the estimated overall racial/ethnic distribution, but it did modestly increase sample size and statistical power.””
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [46] Dembosky, 2019, Indirect estimation of race ethnicity for survey respondents who do not report race/ethnicity, Med Care source

  • May combine two claims.

    Findingcitation

    One additional study about geocoding of patient addresses did not evaluate for bias, but noted that unmatched addresses tend to be unevenly distributed-more likely to occur in rural areas and newly developed suburban areas, and less likely to occur in inner-city areas.

    “One additional study about geocoding of patient addresses did not evaluate for bias, but did note that “unmatched addresses tend to be unevenly distributed—more likely to occur in rural areas and newly developed suburban areas, and less likely to occur in inner-city areas.”53”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [53] Lin, 2010, Improving geocoding outcomes for the Nebraska Cancer Registry: learning from proven practices, J Registry Manag

  • Probably not a substantial claim: may not pass S4 (based).

    Findingnone given

    Several studies have documented that data quality from Hispanic patients can be particularly implausible and is especially prone to misclassification bias when compared to data from other racial/ethnic groups.

    “Several studies have documented that data quality from Hispanic patients can be particularly implausible and is especially prone to misclassification bias when compared to data from other racial/ethnic groups.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Findingnone given

    Evidence-based solutions are available for researchers who want to improve the quality of social determinants data ex post facto.

    “Fortunately, evidence-based solutions are available for researchers who want to improve the quality of social determinants data ex post facto.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingnone given

    Race/ethnicity data from the health record can be implausible or incomplete.

    “race/ethnicity data from the health record can be implausible or incomplete,”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    The accuracy of race/ethnicity data for Hispanic or Latino patients is lower than the accuracy of data from white patients in the same dataset.

    “However, problems with plausibility not at random—for example, the accuracy of data for Hispanic or Latino patients being lower than the accuracy of data from white patients in the same dataset—has similarly profound implications.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Several studies speculate that the quality of data about ethnicity has been impaired by variations in how healthcare systems record and handle this information.

    “At the institutional level, several studies speculate that the quality of data about ethnicity has been impaired by variations in how healthcare systems record and handle this information.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Inconsistent classification of Hispanics is likely attributable, in part, to differences in the definition of being Hispanic across classification schemes.

    “As one study noted, “inconsistent classification of Hispanics is likely attributable, in part, to differences in the definition of being Hispanic across classification schemes.”27”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [27] Gomez, 2006, Misclassification of race/ethnicity in a population-based cancer registry (United States), Cancer Causes Control source

  • Classified as probably not a substantial claim.

    Premisecitation

    Poor-quality data lead to poor-quality results.

    “It is a long-established tenet of information science that poor-quality data lead to poor-quality results.1”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [1] Work with new electronic ‘brains’ opens field for Army math experts, The Hammond Times

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationreasoning

    The type of data quality problem varies depending on the variable, and each problem is associated with particular forms of analytical error.

    “The type of data quality problem varies depending on the variable, and each problem is associated with particular forms of analytical error.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    In 2014, the Institute of Medicine suggested using "neighborhood and community composition" as a proxy for individual-level indicators that cannot be directly collected from patients.

    “In 2014, the Institute of Medicine suggested using “neighborhood and community composition” as a proxy for individual-level indicators that cannot be directly collected from patients.7”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [7] 2014

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    The terminology of race shifts over time.

    “and thus its terminology shifts over time.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    Because researchers require ex post facto methods for improving secondary use data, the review ignored recommendations from the literature about improving data collection practices at the point of care.

    “Because researchers require ex post facto methods for improving secondary use data, we ignored any recommendations from in the literature about improving data collection practices at the point of care.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Researchers require ex post facto methods for improving secondary use data.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    A larger body of research may indicate simply that the variables are more accessible to researchers and therefore easier to study.

    “A larger body of research may indicate simply that these variables are more accessible to researchers and therefore easier to study.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Publication bias may have affected the findings of this review if authors did not report negative findings when evaluating their datasets for bias.

    “In addition, publication bias may have affected our findings if authors did not report negative findings when evaluating their datasets for bias.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Consideration of data quality by researchers prior to analysis, along with thoughtfully applied quality improvement methods, may help prevent bias and improve the validity of research conducted with SDoH data.

    “Consideration of data quality by researchers prior to analysis, along with thoughtfully applied quality improvement methods, may help prevent bias and improve the validity of research conducted with SDoH data.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    The data quality concepts used in this review are not completely orthogonal or distinct.

    “The data quality concepts used here are not completely orthogonal or distinct.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationnone given

    Citation searching is inherently exponential; if the initial search turned up few articles within a certain domain, then that domain may appear to have a smaller body of literature than is in fact present.

    “Citation searching is inherently exponential; if our initial search turned up few articles within a certain domain, then that domain may appear to have a smaller body of literature than is in fact present.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    The search for solutions to these data quality problems is focused on the needs of researchers using observational databases.

    “Another limitation is that our search for solutions to these data quality problems is focused on the needs of researchers using observational databases.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationreasoning

    While complete case analysis has the potential for bias, imputation techniques can avoid these shortcomings.

    “While complete case analysis has the potential for bias, imputation techniques can avoid these shortcomings.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    Race is not a biological category but a social and political one.

    “Race is, after all, not a biological category but a social and political one,82 and thus its terminology shifts over time.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [82] 2018

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    The finding that articles about race/ethnicity data were a large proportion of the literature may reflect that this information is more readily available in the structured fields of the health record than other SDoH variables.

    “For example, our finding that articles about race/ethnicity data were such a large proportion of the literature may reflect that this information is more readily available in the structured fields of the health record than other SDoH variables.”
    Quote in the paperNot a substantial claim
53 of 53

  • The paper relies on this without support.

    Patients move over addresses over time.

    “Patients move over addresses over time, which means that researchers often have decisions to make about which address to use.”

  • The paper relies on this without support.

    Many social determinants data elements most commonly used in research were originally collected as demographic data for administrative purposes.

    “Many of the social determinants data elements most commonly used in research, such as race, ethnicity, insurance status, and address, were originally collected as demographic data for administrative purposes.”

  • The paper relies on this without support.

    Researchers require ex post facto methods for improving secondary use data.

    “Because researchers require ex post facto methods for improving secondary use data, we ignored any recommendations from in the literature about improving data collection practices at the point of care.”

  • The paper relies on this without support.

    Privacy-preserving record linkage techniques often rely heavily on patient address along with name and date of birth.

    “These techniques often rely heavily on patient address along with name and date of birth.”
4 of 4

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Completeness row totals 31, not 29

    Minor

    The six completeness counts in Table 2 sum to 31, but the row reports 29 articles.

    Reported 29 · recomputed 31

  • Conformance row totals 14, not 16

    Minor

    The six conformance counts in Table 2 sum to 14, but the row reports 16 articles.

    Reported 16 · recomputed 14

  • Environmental misclassification count differs from Table 3

    Minor

    The text says all four articles about residential mobility characterized bias as misclassification, but Table 3 reports three environmental articles with that finding.

3 of 3

Cites

Flagged first, then by citations

Research works printed but not linked: 3

  1. Work with new electronic ‘brains’ opens field for Army math experts, The Hammond Times
  2. 2014, Capturing Social and Behavioral Domains and Measures in Electronic Health Records: Phase 2
  3. Polednak, 2005, Obtaining occupation as an indicator of patients' socioeconomic status in a population-based cancer registry, J Registry Manag, 32, 176

Cited by

Sorted by citations

Related

Authors
Cook, L. A., Sachs, J., & Weiskopf, N. G.
Affiliations
Oregon Health & Science University (US)
Corresponding author
Cook, Lily A
ORCID
Venue
Journal of the American Medical Informatics Association
Publisher
Oxford University Press (OUP)
Published
2021-10-19
Volume
29
Issue
1
Pages
187-196
ISSN
  • 1067-5027 (ISSN-L)
  • 1527-974X
DOI
10.1093/jamia/ocab199
PMC
PMC8714289
OpenAlex
W3207894119
Semantic Scholar
1f7f87f2ac2a1040dc64ac66d6de7a5f55f89502
Cited by
153
References
82
Citations by year
2026: 15 · 2025: 38 · 2024: 51 · 2023: 35 · 2022: 14
FWCI
4.42
Type
journal article
Language
en
Open access
Licence
Locations
  • Journal of the American Medical Informatics Association (publishedVersion, cc-by-nc)
  • PubMed Central (publishedVersion, cc-by-nc)
  • PubMed (publishedVersion)
Funders
  • National Library of Medicine (T15LM007088)
  • National Library of Medicine Award (K01LM012738, R21LM013645)
Topics
  • Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Data Quality and Management
Keywords
social determinants of health, electronic health records, data quality, plausibility, misclassification bias, racial and ethnic disparities, Hispanic patients, missing data imputation, measurement validity, research bias
MeSH terms
Ethnicity, Hispanic or Latino, Humans, Racial Groups, Electronic Health Records, Social Determinants of Health

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-28.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-28.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review