Defining and measuring completeness of electronic health records for secondary use
Open accessAbstract
We demonstrate the importance of explicit definitions of electronic health record (EHR) data completeness and how different conceptualizations of completeness may impact findings from EHR-derived datasets. This study has important repercussions for researchers and clinicians engaged in the secondary use of EHR data. We describe four prototypical definitions of EHR completeness: documentation, breadth, density, and predictive completeness. Each definition dictates a different approach to the measurement of completeness. These measures were applied to representative data from NewYork-Presbyterian Hospital's clinical data warehouse. We found that according to any definition, the number of complete records in our clinical database is far lower than the nominal total. The proportion that meets criteria for completeness is heavily dependent on the definition of completeness used, and the different definitions generate different subsets of records. We conclude that the concept of completeness in EHR is contextual. We urge data consumers to be explicit in how they define a complete record and transparent about the limitations of their data.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The current DQ literature is inconsistent in the use of terms that describe the complex multidimensional aspects of DQ.
“The current DQ literature is inconsistent in the use of terms that describe the complex multidimensional aspects of DQ.42–51”
From A harmonized data quality assessment terminology and framework for the secondary use of electronic health record data (2016) · cites this paper as [48]
The Materials and methods section identifies several dimensions of EHR data quality, and the Conclusions describe multiple definitions of completeness that produce different results. These passages support the complexity of defining data quality, but they do not establish that the DQ literature uses terminology inconsistently across its dimensions. Page numbers are not available in our copy. (LLM, conf=0.78)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
No citation lists for this paper
No citation lists for this paper yet — checked again daily.
References 42 · Cited by 367
Research works printed but not linked: 4
- [12]Juran JM Gryna FM Juran’s quality control handbook 1988 McGraw-Hill New York 4th ed
- [13]Wang RY Strong DM Beyond accuracy: what data quality means to data consumers Journal of Management Information Systems 1996 12 5 34
- [35]Hao T Rusanov A Weng C Extracting and normalizing temporal expressions in clinical data requests from researchers 2013 International Health Informatics Conference Beijing, China
- [43]Micallef L Rodgers P Drawing area-proportional venn-3 diagrams using ellipses 2012 12th Annual Grace Hopper Celebration of Women in Computing, ACM Student Research Competition and Poster Session Baltimore, MD
- Authors
- Weiskopf, N. G., Hripcsak, G., Swaminathan, S., & Weng, C.
- Affiliations
- Columbia University (US)
- Corresponding author
- Weiskopf, Nicole G.
- Venue
- Journal of Biomedical Informatics
- Publisher
- Elsevier BV
- Published
- 2013-10
- Volume
- 46
- Issue
- 5
- Pages
- 830-836
- ISSN
- 1532-0464 (ISSN-L)
- PMC
- PMC3810243
- OpenAlex
- W2018025378
- Cited by
- 367
- References
- 42
- Citations by year
- 2026: 21 · 2025: 30 · 2024: 24 · 2023: 27 · 2022: 31 · 2021: 50 · 2020: 33 · 2019: 42 · 2018: 29 · 2017: 30 · 2016: 22 · 2015: 12 · 2014: 12 · 2013: 2 · 2012: 1
- FWCI
- 27.61
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Journal of Biomedical Informatics (publishedVersion, cc-by-nc-sa)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- Wellcome Trust (091593)
- National Institutes of Health (UL1TR000040)
- U.S. National Library of Medicine (R01LM010815, R01LM006910, R01LM009886, T15LM007079)
- National Center for Advancing Translational Sciences
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- electronic health records, data completeness, secondary use of health data, documentation completeness, clinical data warehouse, data quality
- MeSH terms
- Documentation, New York City, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
