Informative presence bias in analyses of electronic health records-derived data: a cautionary note
Open accessThe DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Abstract
Objective: Electronic health record (EHR)-derived data are extensively used in health research. However, the pattern of patient interaction with the healthcare system can result in informative presence bias if those who have poorer health have more data recorded than healthier patients. We aimed to determine how informative presence affects bias across multiple scenarios informed by real-world healthcare utilization patterns. Materials and methods: We conducted an analysis of EHR data from a pediatric healthcare system as well as simulation studies to characterize conditions under which informative presence bias is likely to occur. This analysis extends prior work by examining a variety of scenarios for the relationship between a biomarker and a health event of interest and the healthcare visit process. Results: Using biomarker values gathered at both informative and noninformative visits when estimating the effect of the biomarker on the event of interest resulted in minimal bias when the biomarker was relatively stable over time but produced substantial bias when the biomarker was more volatile. Adjusting analyses for the number of prior visits within a fixed look-back window was able to reduce but not eliminate this bias. Discussion: These results suggest that bias may arise frequently in commonly encountered scenarios and may not be eliminated by adjusting for prior visit intensity. Conclusion: Depending on the context, the estimated effect from analyses using data from all visits available may diverge from the true effect. Sensitivity analyses using only visits likely to be informative or noninformative based on visit type may aid in the assessment of the magnitude of potential bias.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“An important limitation of any genetic study using electronic health record data is the possibility of detection bias and informative presence bias.38”
From Rare Genetic Variants in LDLR , APOB , and PCSK9 Are Associated With Aortic Stenosis (2024) · cites this paper as [38]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Tsiatis, A. A., Degruttola, V., & Wulfsohn, M. S. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476485 · Cited by 575
Goldstein, B. A., Bhavsar, N. A., Phelan, M., & Pencina, M. J. (2016) · American Journal of Epidemiology · doi:10.1093/aje/kww112 · Cited by 176
Altman, D. G., & De Stavola, B. L. (1994) · Statistics in Medicine · doi:10.1002/sim.4780130402 · Cited by 173
Analysis of Longitudinal Data with Irregular, Outcome-Dependent Follow-Up
Lin, H., Scharfstein, D. O., & Rosenheck, R. A. (2004) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2004.b5543.x · Cited by 161
Sick patients have more data: the non-random completeness of electronic health records.
Weiskopf, N. G., Rusanov, A., & Weng, C. (2013) · Europe PMC (PubMed Central) · Cited by 132
Haneuse, S., & Daniels, M. (2016) · eGEMs (Generating Evidence & Methods to improve patient outcomes) · doi:10.13063/2327-9214.1203 · Cited by 119
Joint Modeling and Analysis of Longitudinal Data with Informative Observation Times
Liang, Y., Lu, W., & Ying, Z. (2008) · Biometrics · doi:10.1111/j.1541-0420.2008.01104.x · Cited by 117
Parameter Estimation in Longitudinal Studies with Outcome-Dependent Follow-Up
Lipsitz, S. R., Fitzmaurice, G. M., Ibrahim, J. G., Gelber, R. D., & Lipshultz, S. E. (2002) · Biometrics · doi:10.1111/j.0006-341x.2002.00621.x · Cited by 99
Phelan, M., Bhavsar, N., & Goldstein, B. A. (2017) · eGEMs (Generating Evidence & Methods to improve patient outcomes) · doi:10.5334/egems.243 · Cited by 96
Pullenayegum, E., & Lim, L. S. H. (2014) · Statistical Methods in Medical Research · doi:10.1177/0962280214536537 · Cited by 82
Statistical analysis of current status data with informative observation times
Zhang, Z., Sun, J., & Sun, L. (2004) · Statistics in Medicine · doi:10.1002/sim.2001 · Cited by 74
On the Nature of Informative Presence Bias in Analyses of Electronic Health Records
McGee, G., Haneuse, S., Coull, B. A., Weisskopf, M. G., & Rotem, R. S. (2021) · Epidemiology · doi:10.1097/ede.0000000000001432 · Cited by 63
BU̇rŽKOVÁ, P., & Lumley, T. (2007) · Canadian Journal of Statistics · doi:10.1002/cjs.5550350402 · Cited by 50
Lange, J. M., Hubbard, R. A., Inoue, L. Y. T., & Minin, V. N. (2015) · Biometrics · doi:10.1111/biom.12252 · Cited by 49
Self-citationGoldstein, B. A., Phelan, M., Pagidipati, N. J., & Peskoe, S. B. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz148 · Cited by 42
Neuhaus, J. M., McCulloch, C. E., & Boylan, R. D. (2018) · Statistics in Medicine · doi:10.1002/sim.7932 · Cited by 40
Biased and Unbiased Estimation in Longitudinal Studies with Informative Visit Processes
McCulloch, C. E., Neuhaus, J. M., & Olin, R. L. (2016) · Biometrics · doi:10.1111/biom.12501 · Cited by 36
Regression analysis of longitudinal data with irregular and informative observation times
Chen, Y., Ning, J., & Cai, C. C. (2015) · Biostatistics · doi:10.1093/biostatistics/kxv008 · Cited by 26
Joint modeling of longitudinal data with informative observation times and dropouts
Han, M., Song, X., & Sun, L. (2013) · Statistica Sinica · doi:10.5705/ss.2013.063 · Cited by 14
Multiple outputation for the analysis of longitudinal data subject to irregular observation
Pullenayegum, E. (2015) · Statistics in Medicine · doi:10.1002/sim.6829 · Cited by 12
Clustered longitudinal data subject to irregular observation
Pullenayegum, E., Birken, C. S., & Maguire, J. L. (2021) · Statistical Methods in Medical Research · doi:10.1177/0962280220986193 · Cited by 10
Joint Modelling of Survival and Longitudinal Data with Informative Observation Times
Dai, H., & Pan, J. (2018) · Scandinavian Journal of Statistics · doi:10.1111/sjos.12314 · Cited by 7
Qu, L., Sun, L., & Song, X. (2018) · Statistics in Biosciences · doi:10.1007/s12561-018-9221-8 · Cited by 5
Cited by
Predicting polycystic ovary syndrome with machine learning algorithms from electronic health records
Zad, Z., Jiang, V. S., Wolf, A. T., Wang, T., Cheng, J. J., Paschalidis, I. C., & Mahalingaiah, S. (2024) · Frontiers in Endocrinology · doi:10.3389/fendo.2024.1298628 · Cited by 97
“This is an important consideration as a potential source of bias for interpreting predictive models using EHR data (34, 35).”
BackgroundMethodBiases in Electronic Health Records Data for Generating Real-World Evidence: An Overview
Al‐Sahab, B., Leviton, A., Loddenkemper, T., Paneth, N., & Zhang, B. (2023) · Journal of Healthcare Informatics Research · doi:10.1007/s41666-023-00153-2 · Cited by 88
Inherent Bias in Electronic Health Records: A Scoping Review of Sources of Bias
Perets, O., Stagno, E., Yehuda, E. B., McNichol, M. C., Celi, L. A., Rappoport, N., & Dorotić, M. (2026) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3757924 · Cited by 75
Chavez‐Yenter, D., Goodman, M. S., Chen, Y., Chu, X., Bradshaw, R. L., Chambers, R. L., Chan, P. A., Daly, B. M., Flynn, M. C., Gammon, A., et al. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.34574 · Cited by 56
Zhou, Y., Shi, J., Stein, R., Liu, X., Baldassano, R. N., Forrest, C. B., Chen, Y., & Huang, J. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad066 · Cited by 33
Metadata
- Authors
- Harton, J., Mitra, N., & Hubbard, R. A.
- Affiliations
- University of Pennsylvania (US)
- Corresponding author
- Harton, Joanna
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2022-04-19
- Volume
- 29
- Issue
- 7
- Pages
- 1191-1199
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC9196698
- OpenAlex
- W4224247272
- Cited by
- 29
- References
- 23
- Citations by year
- 2026: 9 · 2025: 7 · 2024: 10 · 2023: 2 · 2022: 1
- FWCI
- 1.65
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model (vor)
- Locations
- Journal of the American Medical Informatics Association (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Cancer Institute of the National Institutes of Health (R21CA227613)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Keywords
- electronic health records, health care utilization patterns, bias in observational studies, pediatric health care, sensitivity analysis
- MeSH terms
- Child, Computer Simulation, Humans, Bias, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.