The DOI matches its Crossref record; 113 of the 114 references it cites were found in a registry or an index; 9 of the 41 sources its claims cite were checked for support.
Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research
Open accessAbstract
OBJECTIVE: To review the methods and dimensions of data quality assessment in the context of electronic health record (EHR) data reuse for research. MATERIALS AND METHODS: A review of the clinical research literature discussing data quality assessment methodology for EHR data was performed. Using an iterative process, the aspects of data quality being measured were abstracted and categorized, as well as the methods of assessment used. RESULTS: Five dimensions of data quality were identified, which are completeness, correctness, concordance, plausibility, and currency, and seven broad categories of data quality assessment methods: comparison with gold standards, data element agreement, data source agreement, distribution comparison, validity checks, log review, and element presence. DISCUSSION: Examination of the methods by which clinical researchers have investigated the quality and suitability of EHR data for research shows that there are fundamental features of data quality, which may be difficult to measure, as well as proxy dimensions. Researchers interested in the reuse of EHR data for clinical research are recommended to consider the adoption of a consistent taxonomy of EHR data quality, to remain aware of the task-dependence of data quality, to integrate work on data quality assessment from other fields, and to adopt systematic, empirically driven, statistically based methods of data quality assessment. CONCLUSION: There is currently little consistency or potential generalizability in the methods used to assess EHR data quality. If the reuse of EHR data for clinical research is to become accepted, researchers should adopt validated, systematic methods of EHR data quality assessment.
Overview
CoChat Summary
TL;DR
This review identifies five dimensions of electronic health record (EHR) data quality—completeness, correctness, concordance, plausibility, and currency—and seven broad categories of assessment methods. It concludes that EHR data quality assessment for research lacks consistency and generalizability, and recommends validated, systematic, statistically based methods.
Key findings
- Five data quality dimensions were derived: completeness, correctness, concordance, plausibility, and currency. Seven assessment method categories were identified: comparison with gold standards, data element agreement, data source agreement, distribution comparison, validity checks, log review, and element presence.
- Completeness was the most commonly assessed dimension, in 61 (64%) of 95 articles, followed by correctness in 57 (60%), concordance in 16 (17%), plausibility in 7 (7%), and currency in 4 (4%).
- Gold standard comparison was used in 37 of 95 articles; element presence was used only for completeness, while currency was assessed only through log review.
- The authors argue that only correctness, completeness, and currency are fundamental dimensions; concordance and plausibility appear to serve as proxies when fundamental dimensions cannot be assessed directly.
Methods
- Literature search performed on PubMed in February 2012, using data quality terms and EHR-related terms; the search produced 230 articles, which were manually reviewed.
- Inclusion criteria retained original research using data quality assessment methods, focused on EHR-derived data, and published in peer-reviewed journals; administrative claims databases were excluded. This yielded 44 articles, and an ancestor search added 51 more, for a final pool of 95 articles.
- From each article, the authors abstracted data quality features, assessment methods, and descriptive information; through iterative review, they inductively derived dimensions and method categories, then recategorized all 95 articles by dimension and assessment strategy.
Limitations
- The paper states the search was limited: lack of MeSH terms for data quality and variation in terminology may have caused relevant articles to be missed, though reference review was believed to improve saturation.
- The paper states classification was largely subjective and performed by only one author, so original researchers might disagree with the interpretations.
- The paper states that underreporting likely affected results, because routine data checking such as distributions and summary statistics is rarely described in publications; evident from the text, the review is descriptive and does not empirically validate the proposed taxonomy.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures

Verification
Claims the paper makes
8 Valid · 1 Not valid · 16 Not checked · 59 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“Thiru et al,9 in calculating the sensitivity of different types of EHR data in the literature, found values ranging between 0.26 and 1.00.”
§ Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research > Abstract > Conclusion
What the sources say
- Supportedcites [9] Thiru K Hassey A Sullivan F Systematic review of scope and quality of electronic patient record data in primary care. BMJ 2003; 326: 1070 12750210 PMC155692 source
Table 3 (Results) lists sensitivities for different types of electronic patient record data. The values range from 26% for socioeconomic data to 100% for several categories, matching 0.26–1.00. (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Hogan and Wagner8 suggested specific statistical measures of the correctness and completeness of EHR elements that have been adopted by other researchers.”
§ Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research > Discussion > Data quality assessment methodology
What the sources say
- Partly supportedcites [8] Hogan WR Wagner MM Accuracy of data in computer-based patient records. J Am Med Inform Assoc 1997; 4: 342– 55 9292840 PMC61252 sourceExperimental — accuracy not yet measured
The Methodological Recommendations section (page not provided) defines correctness and completeness as specific proportions and recommends that future studies report numerical measures of both. The paper does not establish the claim that other researchers subsequently adopted those measures. (LLM, conf=0.88)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Some researchers looking at the quality of research databases pulled from general practices in the UK have adopted relatively consistent approaches to comparing the distributions of data concerning specific clinical phenomena to information from registries and surveys.31 98 99 102”
§ Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research > Discussion > Data quality assessment methodology
What the sources say
- Not checked — no full text of the sourcecites [31] Lewis JD Brensinger C Agreement between GPRD smoking data: a survey of general practitioners and a population-based survey. Pharmacoepidemiol Drug Saf 2004; 13: 437– 41 15269927 source
- Not checked — no full text of the sourcecites [98] Haynes K Forde KA Schinnar R Cancer incidence in The Health Improvement Network. Pharmacoepidemiol Drug Saf 2009; 18: 730– 6 19479713 source
- Supportedcites [99] Iyen-Omofoman B Hubbard RB Smith CJ The distribution of lung cancer across sectors of society in the United Kingdom: a study using national primary care data. BMC Public Health 2011; 11: 857 22074413 PMC3282683 source
The Methods and Discussion > Lung cancer Incidence describe comparing lung cancer data from the UK general-practice database THIN with national cancer registry and National Lung Cancer Audit data to assess its representativeness. This paper therefore exemplifies the comparison approach attributed to the cited group of researchers; the sentence does not require each cited study to use both registries and surveys. (LLM, conf=0.85)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“In three of the four, researchers reviewed whether desired data were entered into the EHR within a set time limit.36 46 52”
§ Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research > Results > Currency
What the sources say
- Not checked — no full text of the sourcecites [36] Ndira SP Rosenberger KD Wetter T Assessment of data quality of and staff satisfaction with an electronic health record system in a developing country (Uganda): a qualitative and quantitative comparative study. Methods Inf Med 2008; 47: 489– 98 19057805 source
- Partly supportedcites [46] Vawdrey DK Gardner RM Evans RS Assessing data quality in manual entry of ventilator settings. J Am Med Inform Assoc 2007; 14: 295– 303 17329731 PMC2244881 sourceExperimental — accuracy not yet measured
The Methods (Data Analysis; page unavailable) say the researchers measured charting delay—the interval between observing ventilator settings and entering them into the computer—to assess timeliness. The Results report an average delay of 6.1 minutes, but the study does not assess whether entries met a set time limit. The 30-minute limit in the Background describes an earlier study by East et al., not this study's measure. (LLM, conf=0.85)
- Not checked — no full text of the sourcecites [52] Faulconer ER de Lusignan S An eight-step method for assessing diagnostic data quality in practice: chronic obstructive pulmonary disease as an exemplar. Inform Prim Care 2004; 12: 243– 54 15808026 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For example, 37 of the 95 articles in our sample relied on a gold standard to assess data quality.”
§ Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research > Discussion > Data quality assessment methodology
plainn = 95- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
40 more claims
Create a free account to see all 84 claims and every source verdict.
- It may state two separate claims as one. 23
- Probably not a substantial claim. 24
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“While these developments have advanced the understanding of data quality in the context of medical applications, frameworks for EHR frequently focus on the data quality of individual patients86,87, neglecting data quality aspects for the overall population.”
From The METRIC-framework for assessing data quality for trustworthy AI in medicine: A systematic review (2024) · cites this paper as [87]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“The current DQ literature is inconsistent in the use of terms that describe the complex multidimensional aspects of DQ.42–51”
From A harmonized data quality assessment terminology and framework for the secondary use of electronic health record data (2016) · cites this paper as [45]
The cited source says what the claim attributes to it.
In the citing paper
“In particular, representativeness is often not a factor86,87 while it is a crucial property for secondary use of data in clinical studies88 or when reusing medical data as training data for ML applications.”
From The METRIC-framework for assessing data quality for trustworthy AI in medicine: A systematic review (2024) · cites this paper as [87]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research > Results > Plausibility
“The most common approach to assessing the plausibility of EHR data was to perform some sort of validity check to determine if specific elements within the EHR were likely to be true or not.”
§ Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research > Results
“| Distribution comparison | 97–100 | | 31 57 97 | 50 101 102 103 | | 10 | | Validity checks | | 32 62 104 105 106 | | 73 89 | | 7 |”
Citations
Cites
A General Inductive Approach for Analyzing Qualitative Evaluation Data
Thomas, D. R. (2006) · American Journal of Evaluation · doi:10.1177/1098214005283748 · Cited by 10,858 · cited 1×
“Furthermore, using an existing model to guide analysis is a deductive approach, which has the potential to obscure information contained in the data.(11) By imposing an existing model from a different discipline, we would have run the risk of missing important findings.”
MethodBeyond Accuracy: What Data Quality Means to Data Consumers
Wang, R. Y., & Strong, D. M. (1996) · Journal of Management Information Systems · doi:10.1080/07421222.1996.11518099 · Cited by 4,672
(2020) · Journal of Medical Education · doi:10.5812/jme · Cited by 3,490
The American Journal of Managed Care
(2020) · The American Journal of Managed Care · doi:10.37765/ajmc · Cited by 3,237
Pipino, L. L., Lee, Y. W., & Wang, R. Y. (2002) · Communications of the ACM · doi:10.1145/505248.506010 · Cited by 1,724
Juran's Quality Control Handbook
Ziegel, E. R. (1990) · Technometrics · doi:10.1080/00401706.1990.10484602 · Cited by 1,384
Jick, H., Jick, S. S., & Derby, L. E. (1991) · BMJ · doi:10.1136/bmj.302.6779.766 · Cited by 839
Safran, C., Bloomrosen, M., Hammond, W. E., Labkoff, S., Markel-Fox, S., Tang, P. C., & Detmer, D. E. (2007) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2273 · Cited by 705 · cited 1×
“As the adoption of electronic health records (EHRs) has made it easier to access and aggregate clinical data, there has been growing interest in conducting research with data collected during the course of clinical care.(1) 2 The Natonal Institutes of Health has called for increasing the reuse of electronic records fo…”
MethodInformatics in Primary Care: Strategies in Information Management for the Healthcare Provider
(2002) · Health Informatics · doi:10.1007/978-1-4613-0069-4 · Cited by 422
Accuracy of Data in Computer-based Patient Records
Hogan, W. R., & Wagner, M. M. (1997) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.1997.0040342 · Cited by 389 · cited 5×
“As observed by Hogan and Wagner in their literature review, there is no medical subheadings (MeSH) term fi ‘ ’ ‘ ’ ‘ ’ ‘ consistency ’ , ‘ data completeness ’ , and ‘ data error ’ .”
BackgroundInfluential
Research works printed but not linked: 15
- Hersh, 2007, The American journal of managed care, 13, 277
- 2005, AM J EVAL, 27, 237
- 1993, 75, 393
- Kaboli, 2004, The American journal of managed care, 10, 872
- Whitelaw, 1996, British Journal of General Practise, 46, 181
- Pyper, 2004, British Journal of General Practise, 54, 38
- Powell, 2006, Informatics in primary care, 14, 55
- 2002, INFORMAT PRIM CARE, 10, 227
- Pringle, 1995, British Journal of General Practise, 45, 537
- Harabagiu, 2010, 2010, 1
- Brown, 2003, Informatics in primary care, 11, 229
- Scobie, 1995, Journal of Public Health, 17, 475
- Asche, 2008, 3, 323
- Haerian, 2009, AMIA ... Annual Symposium proceedings [electronic resource] / AMIA Symposium. AMIA Symposium, 2009, 218
- de Lusignan, 2006, Informatics in primary care, 14, 133
Cited by
Benchimol, E. I., Smeeth, L., Guttmann, A., Harron, K., Moher, D., Petersen, I., Sørensen, H. T., von Elm, E., Langan, S. M., & RECORD Working Committee (2015) · PLOS Medicine · doi:10.1371/journal.pmed.1001885 · Cited by 5,749
“The description of data cleaning methods at different stages of the study should include those used to screen for erroneous and missing data, including range checks, checks for duplicate records, and handling of repeated measures [47,48].”
MethodOpportunities and obstacles for deep learning in biology and medicine
Ching, T., Himmelstein, D. S., Beaulieu-Jones, B. K., Kalinin, A. A., Do, B. T., Way, G. P., Ferrero, E., Agapow, P.-M., Zietz, M., Hoffman, M. M., et al. (2018) · Journal of The Royal Society Interface · doi:10.1098/rsif.2017.0387 · Cited by 2,310
“literature around EHR data quality and the impact on research [98], we focus on three types of challenges: local bias, wider standards, and legal issues.”
BackgroundMiotto, R., Li, L., Kidd, B. A., & Dudley, J. T. (2016) · Scientific Reports · doi:10.1038/srep26094 · Cited by 1,710
“EHR data is challenging to represent and model due to its high dimensionality, noise, heterogeneity, sparseness, incompleteness, random errors, and systematic biases 7,10,11 .”
BackgroundNext-generation phenotyping of electronic health records
Hripcsak, G., & Albers, D. J. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2012-001145 · Cited by 795
“CHALLENGES Unfortunately, the EHR carries many challenges.(2)”
Electronic health records to facilitate clinical research
Cowie, M. R., Blomster, J. I., Curtis, L. H., Duclaux, S., Ford, I., Fritz, F., Goldman, S., Janmohamed, S., Kreuzer, J., Leenay, M., et al. (2017) · Clinical Research in Cardiology · doi:10.1007/s00392-016-1025-6 · Cited by 651
“and completeness have not yet been adopted [46], but validation is a critical factor for the regulatory acceptance of EHR data.”
Background
Metadata
- Authors
- Weiskopf, N. G., & Weng, C.
- Affiliations
- Columbia University (US)
- Corresponding author
- Weiskopf, N. G.
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2013-01-01
- Volume
- 20
- Issue
- 1
- Pages
- 144-151
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC3555312
- OpenAlex
- W2158939484
- Cited by
- 1,216
- References
- 114
- Citations by year
- 2026: 153 · 2025: 102 · 2024: 97 · 2023: 92 · 2022: 88 · 2021: 108 · 2020: 88 · 2019: 106 · 2018: 102 · 2017: 88 · 2016: 70 · 2015: 54 · 2014: 38 · 2013: 23
- FWCI
- 6.18
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc
- Locations
- Journal of the American Medical Informatics Association (publishedVersion, cc-by-nc)
- Europe PMC (PubMed Central) (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- Funders
- U.S. National Library of Medicine (R01LM010815, R01LM009886, T15LM007079)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
- Physical Sciences › Computer Science › Information Systems › Data Quality and Management
- Keywords
- electronic health records, data quality assessment, clinical research data, data completeness, data correctness, concordance, plausibility, data currency, gold standard comparison, data source agreement, validity testing
- MeSH terms
- Data Collection, Humans, Quality Control, Research Design, Reproducibility of Results, Information Dissemination, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-30, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-30 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research. https://app.cochat.ai/papers/10.1136/amiajnl-2011-000681
- The paper
- Weiskopf, N. G., & Weng, C. (2013). Methods and dimensions of electronic health record data quality assessment: enabling reuse for clinical research. Journal of the American Medical Informatics Association. https://doi.org/10.1136/amiajnl-2011-000681
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.