Bias Associated with Mining Electronic Health Records
Open accessAbstract
Large-scale electronic health record research introduces biases compared to traditional manually curated retrospective research. We used data from a community-acquired pneumonia study for which we had a gold standard to illustrate such biases. The challenges include data inaccuracy, incompleteness, and complexity, and they can produce in distorted results. We found that a naïve approach approximated the gold standard, but errors on a minority of cases shifted mortality substantially. Manual review revealed errors in both selecting and characterizing the cohort, and narrowing the cohort improved the result. Nevertheless, a significantly narrowed cohort might contain its own biases that would be difficult to estimate.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
A substantial body of research suggests that data collected in EHRs and other operational systems may not be of sufficient quality for research.
“A substantial body of research suggests that data collected in EHRs and other operational systems may not be of sufficient quality for research.28–37”
From A harmonized data quality assessment terminology and framework for the secondary use of electronic health record data (2016) · cites this paper as [35]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
No references could be resolved yet.
Research works printed but not linked: 5
- [1]Schappert SM Rechtsteiner EA Ambulatory medical care utilization estimates for 2007. National Center for Health Statistics Vital Health Stat 2011 13 169 http://www.cdc.gov/nchs/data/series/sr_13/sr13_169.pdf 21614897
- [2]Friedman C Wong A Blumenthal D Achieving a nationwide learning health system Sci Transl Med 2010 2 57 57 cm29 10.1126/scitranslmed.3001456 21068440 · DOI · Paper page
- [3]Hogan WR Wagner MM Accuracy of data in computer-based patient records J Am Med Inform Assoc 1997 4 5 342 355 9292840 PMC61252
- [4]Fine M J Auble T E Yealy D M Hanusa B H Weissfeld L A Singer D E Coley C M Marrie T J Kapoor W N A prediction rule to identify low-risk patients with community-acquired pneumonia N Engl J Med 1997 336 4 243 50 10.1056/NEJM199701233360402 8995086 · DOI
- [5]Hripcsak G Knirsch C Zhou L Wilcox A Melton GB Using discordance to improve classification in narrative clinical databases: an application to community-acquired pneumonia Comput Biol Med 2007 4 18 37 3 296 304 10.1016/j.compbiomed.2006.02.001 16620802 · DOI
Cited by
Sorted by citations
Kahn, M. G., Callahan, T. J., Barnard, J., Bauck, A. E., Brown, J., Davidson, B. N., Estiri, H., Goerg, C., Holve, E., Johnson, S. G., et al. (2016) · eGEMs (Generating Evidence & Methods to improve patient outcomes) · doi:10.13063/2327-9214.1244 · Cited by 636 · see citing works
Hersh, W. R., Weiner, M. G., Embi, P. J., Logan, J. R., Payne, P. R. O., Bernstam, E. V., Lehmann, H. P., Hripcsak, G., Hartzog, T. H., Cimino, J. J., et al. (2013) · Medical Care · doi:10.1097/mlr.0b013e31829b1dbd · Cited by 567 · see citing works
Self-citationMining Electronic Health Records (EHRs): A Survey
Yadav, P., Steinbach, M., Kumar, V., & Simon, G. (2018) · ACM Computing Surveys · doi:10.1145/3127881 · Cited by 299 · see citing works
Natural Language Processing in Oncology: A Review
Yim, W.-W., Yetisgen, M., Harris, W. P., & Kwan, S. W. (2016) · JAMA Oncology · doi:10.1001/jamaoncol.2016.0213 · Cited by 298 · see citing works
Clinical Data Reuse or Secondary Use: Current Status and Potential Future Progress
Meystre, S. M., Lovis, C., Bürkle, T., Tognola, G., Budrionis, A., & Lehmann, C. U. (2017) · Yearbook of Medical Informatics · doi:10.15265/iy-2017-007 · Cited by 204 · see citing works
A Data Quality Assessment Guideline for Electronic Health Record Data Reuse
Weiskopf, N. G., Bakken, S., Hripcsak, G., & Weng, C. (2017) · eGEMs (Generating Evidence & Methods to improve patient outcomes) · doi:10.5334/egems.218 · Cited by 152 · see citing works
Self-citationRusanov, A., Weiskopf, N. G., Wang, S., & Weng, C. (2014) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-14-51 · Cited by 137 · see citing works
Spiranovic, C., Matthews, A., Scanlan, J., & Kirkby, K. C. (2016) · Advances in Mental Health · doi:10.1080/18387357.2015.1063635 · Cited by 37 · see citing works
- Authors
- Hripcsak, G., Knirsch, C., Zhou, L., Wilcox, A., & Melton, G.
- Affiliations
- Columbia University (US)
- Pfizer (United States) (US)
- University of Minnesota (US)
- Venue
- Journal of Biomedical Discovery and Collaboration
- Publisher
- University of Illinois Libraries
- Published
- 2011-06-06
- Volume
- 6
- Pages
- 48-52
- ISSN
- 1747-5333 (ISSN-L)
- PMC
- PMC3149555
- OpenAlex
- W2171285465
- Cited by
- 83
- References
- 0
- Citations by year
- 2026: 1 · 2025: 3 · 2024: 4 · 2023: 4 · 2022: 5 · 2021: 6 · 2020: 4 · 2019: 5 · 2018: 9 · 2017: 9 · 2016: 7 · 2015: 15 · 2014: 2 · 2013: 7 · 2012: 1
- FWCI
- 5.66
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- Journal of Biomedical Discovery and Collaboration (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- Pfizer
- U.S. National Library of Medicine (LM06910, R01LM006910)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- electronic health records, research bias, community-acquired pneumonia, data quality, data incompleteness, data accuracy, cohort selection, mortality estimation, gold standard comparison
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
