CoChat

Data Quality–Driven Improvement in Health Care: Systematic Literature Review

Open access
Lighterness, Anthony, Adcock, Michael, Scanlon, Lauren Abigail, Price, Gareth · 2024 · Journal of Medical Internet Research

Published version of a preprint

The DOI matches its Crossref record; 114 of the 114 references it cites were found in a registry or an index.

Abstract

Background: The promise of real-world evidence and the learning health care system primarily depends on access to high-quality data. Despite widespread awareness of the prevalence and potential impacts of poor data quality (DQ), best practices for its assessment and improvement are unknown. Objective: This review aims to investigate how existing research studies define, assess, and improve the quality of structured real-world health care data. Methods: A systematic literature search of studies in the English language was implemented in the Embase and PubMed databases to select studies that specifically aimed to measure and improve the quality of structured real-world data within any clinical setting. The time frame for the analysis was from January 1945 to June 2023. We standardized DQ concepts according to the Data Management Association (DAMA) DQ framework to enable comparison between studies. After screening and filtering by 2 independent authors, we identified 39 relevant articles reporting DQ improvement initiatives. Results: The studies were characterized by considerable heterogeneity in settings and approaches to DQ assessment and improvement. Affiliated institutions were from 18 different countries and 18 different health domains. DQ assessment methods were largely manual and targeted completeness and 1 other DQ dimension. Use of DQ frameworks was limited to the Weiskopf and Weng (3/6, 50%) or Kahn harmonized model (3/6, 50%). Use of standardized methodologies to design and implement quality improvement was lacking, but mainly included plan-do-study-act (PDSA) or define-measure-analyze-improve-control (DMAIC) cycles. Most studies reported DQ improvements using multiple interventions, which included either DQ reporting and personalized feedback (24/39, 61%), IT-related solutions (21/39, 54%), training (17/39, 44%), improvements in workflows (5/39, 13%), or data cleaning (3/39, 8%). Most studies reported improvements in DQ through a combination of these interventions. Statistical methods were used to determine significance of treatment effect (22/39, 56% times), but only 1 study implemented a randomized controlled study design. Variability in study designs, approaches to delivering interventions, and reporting DQ changes hindered a robust meta-analysis of treatment effects. Conclusions: There is an urgent need for standardized guidelines in DQ improvement research to enable comparison and effective synthesis of lessons learned. Frameworks such as PDSA learning cycles and the DAMA DQ framework can facilitate this unmet need. In addition, DQ improvement studies can also benefit from prioritizing root cause analysis of DQ issues to ensure the most appropriate intervention is implemented, thereby ensuring long-term, sustainable improvement. Despite the rise in DQ improvement studies in the last decade, significant heterogeneity in methodologies and reporting remains a challenge. Adopting standardized frameworks for DQ assessment, analysis, and improvement can enhance the effectiveness, comparability, and generalizability of DQ improvement initiatives.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.2196/57615

Cites

114 of 114 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flowchart for study selection to evaluate the current literature targeting assessment, analysis, and improvement of the quality of structured real-world data in health care. This PRISMA flowchart illustrates the selection process for reports included in the review, detailing the steps of inclusion and exclusion needed to accurately achieve the intended scope of the study.
Figure 1 PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flowchart for study selection to evaluate the current literature targeting assessment, analysis, and improvement of the quality of structured real-world data in health care. This PRISMA flowchart illustrates the selection process for reports included in the review, detailing the steps of inclusion and exclusion needed to accurately achieve the intended scope of the study.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 114 references

Research works printed but not linked: 10

  1. Earley, S, 2017, The DAMA Guide to the Data Management Body of Knowledge (DAMA-DM BOK)
  2. Horsky, J, 2018, AMIA Annu Symp Proc, 2017, 912
  3. Botsis, T, 2010, Summit Transl Bioinform, 2010, 1
  4. Sokovic, M, 2010, J Achiev Mater Manuf Eng, 43, 476
  5. Kiogou, SD, 2022, AMIA Jt Summits Transl Sci Proc, 2022, 293
  6. Ahlbrandt, J, 2012, Stud Health Technol Inform, 180, 393
  7. Soto, M, 2015, Stud Health Technol Inform, 216, 885
  8. Wallace, CJ, 2002, Proc AMIA Symp, 840
  9. Benard, M, 2015, Stud Health Technol Inform, 210, 364
  10. Amoroso, CL, 2010, Stud Health Technol Inform, 160, 337

Cited by

Metadata

Authors
Lighterness, A., Adcock, M., Scanlon, L. A., & Price, G.
Affiliations
  • The Christie Hospital (GB)
  • University of Manchester (GB)
Corresponding author
Lighterness, Anthony
Venue
Journal of Medical Internet Research
Published
2024-08-22
Volume
26
Pages
e57615
ISSN
1438-8871 (ISSN-L)
OpenAlex
W4399154252
Cited by
26
References
114
Citations by year
2026: 18 · 2025: 8
FWCI
2.19
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Journal of Medical Internet Research (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Research Explorer (The University of Manchester) (publishedVersion, other-oa)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Physical Sciences › Computer Science › Information Systems › Data Quality and Management
  • Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
Keywords
data quality, real-world data, health data, data quality improvement, learning healthcare systems, real-world evidence, PDSA cycle, DMAIC, data completeness, data cleaning, health information technology, data governance, data sharing, data security
MeSH terms
Data Accuracy, Delivery of Health Care, Humans, Quality Improvement

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.