The DOI matches its Crossref record; 68 of the 68 references it cites were found in a registry or an index.
Caveats for the Use of Operational Electronic Health Record Data in Comparative Effectiveness Research
Open accessAbstract
The growing amount of data in operational electronic health record systems provides unprecedented opportunity for its reuse for many tasks, including comparative effectiveness research. However, there are many caveats to the use of such data. Electronic health record data from clinical settings may be inaccurate, incomplete, transformed in ways that undermine their meaning, unrecoverable for research, of unknown provenance, of insufficient granularity, and incompatible with research protocols. However, the quantity and real-world nature of these data provide impetus for their use, and we develop a list of caveats to inform would-be users of such data as well as provide an informatics roadmap that aims to insure this opportunity to augment comparative effectiveness research can be best leveraged.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“However, a major concern is that data not collected systematically for research will be poorer quality, which could have negative impacts on findings generated from these data.24–27”
From A harmonized data quality assessment terminology and framework for the secondary use of electronic health record data (2016) · cites this paper as [27]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Natural language processing: an introduction
Nadkarni, P. M., Lucila Ohno-machado, A., & Chapman, W. W. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000464 · Cited by 2,127
The “Meaningful Use” Regulation for Electronic Health Records
Blumenthal, D., & Tavenner, M. (2010) · New England Journal of Medicine · doi:10.1056/nejmp1006114 · Cited by 2,059
A review of uses of health care utilization databases for epidemiologic research on therapeutics
Schneeweiß, S., & Avorn, J. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2004.10.012 · Cited by 1,290
Weiskopf, N. G., & Weng, C. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000681 · Cited by 1,216 · cited 1×
“A review of the literature of studies evaluating the data quality of EHRs for clinical research identified 5 dimensions of data quality assessed: completeness, correctness, concordance, plausibility, and currency.(26) The authors identified many studies with a wide variety of techniques to assess these dimensions and,…”
Measuring Diagnoses: ICD Code Accuracy
O’Malley, K. J., Cook, K. F., Price, M. D., Wildes, K. R., Hurdle, J. F., & Ashton, C. M. (2005) · Health Services Research · doi:10.1111/j.1475-6773.2005.00444.x · Cited by 1,156
Assessing Quality Using Administrative Data
Iezzoni, L. I. (1997) · Annals of Internal Medicine · doi:10.7326/0003-4819-127-8_part_2-199710151-00048 · Cited by 981
Next-generation phenotyping of electronic health records
Hripcsak, G., & Albers, D. J. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2012-001145 · Cited by 795
Self-citationSafran, C., Bloomrosen, M., Hammond, W. E., Labkoff, S., Markel-Fox, S., Tang, P., & Detmer, D. (2007) · JAMIA Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2273 · Cited by 714
the eMERGE Team, McCarty, C. A., Chisholm, R. L., Chute, C. G., Kullo, I. J., Jarvik, G. P., Larson, E. B., Li, R., Masys, D. R., Ritchie, M. D., et al. (2011) · BMC Medical Genomics · doi:10.1186/1755-8794-4-13 · Cited by 701
Desiderata for Controlled Medical Vocabularies in the Twenty-First Century
Cimino, J. J. (1998) · Methods of Information in Medicine · doi:10.1055/s-0038-1634558 · Cited by 656
Self-citation
Cited by
Sorted by citations
Rajkomar, A., Dean, J., & Kohane, I. (2019) · New England Journal of Medicine · doi:10.1056/nejmra1814259 · Cited by 4,579
Key challenges for delivering clinical impact with artificial intelligence
Kelly, C. J., Karthikesalingam, A., Suleyman, M., Corrado, G., & King, D. (2019) · BMC Medicine · doi:10.1186/s12916-019-1426-2 · Cited by 3,136
“Adoption of unified data formats, such as Fast Healthcare Interoperability Resources [84], offer the potential for better aggregation of data, although improved interoperability does not necessarily fix the problem of inconsistent semantic coding in EHR data [85].”
Ahmed, Z., Mohamed, K., Zeeshan, S., & Dong, X. (2020) · Database · doi:10.1093/database/baaa010 · Cited by 1,035
“Use of EHR in comparative effectiveness research (46) Reporting caveats in existing healthcare systems.”
BackgroundMethodGoldstein, B. A., Navar, A. M., Pencina, M. J., & Ioannidis, J. P. A. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw042 · Cited by 936
“As others have noted, EHR data come with many challenges.(4) EHRs include all patients that touch a medical system, primarily capture data only when patients are ill, and collect metrics that clinicians deem to be necessary at each clinic visit.”
BackgroundAutomated machine learning: Review of the state-of-the-art and opportunities for healthcare
Waring, J., Lindvall, C., & Umeton, R. (2020) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2020.101822 · Cited by 834
Metadata
- Authors
- Hersh, W. R., Weiner, M. G., Embi, P. J., Logan, J. R., Payne, P. R. O., Bernstam, E. V., Lehmann, H. P., Hripcsak, G., Hartzog, T. H., Cimino, J. J., & Saltz, J. H.
- Affiliations
- Oregon Health & Science University (US)
- AstraZeneca (United Kingdom) (GB)
- The Ohio State University Wexner Medical Center (US)
- The University of Texas Health Science Center at Houston (US)
- Johns Hopkins University (US)
- Columbia University (US)
- Medical University of South Carolina (US)
- National Institutes of Health (US)
- Emory University (US)
- Corresponding author
- Hersh, William R.
- Logan, Judith R.
- Cimino, James J.
- ORCID
- Venue
- Medical Care
- Published
- 2013-08
- Volume
- 51
- Issue
- Supplement 8Suppl 3
- Pages
- S30-S37
- ISSN
- 0025-7079 (ISSN-L)
- PMC
- PMC3748381
- OpenAlex
- W2137767404
- Cited by
- 567
- References
- 68
- Citations by year
- 2026: 43 · 2025: 44 · 2024: 30 · 2023: 43 · 2022: 43 · 2021: 58 · 2020: 50 · 2019: 59 · 2018: 50 · 2017: 45 · 2016: 45 · 2015: 27 · 2014: 24 · 2013: 6
- FWCI
- 58.67
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Medical Care (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institutes of Health (UL1TR000454, UL1TR000090)
- Centers for Disease Control and Prevention
- National Center for Research Resources (UL1RR025008, UL1RR024156, UL1RR025005, UL1RR024148, UL1RR024140)
- National Center for Advancing Translational Sciences
- H2020 European Research Council
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- electronic health records, comparative effectiveness research, data quality, data completeness, data provenance, data granularity, data reuse
- MeSH terms
- Data Collection, Data Interpretation, Statistical, Humans, Insurance Claim Review, Research Design, Comparative Effectiveness Research, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
