Informative presence bias in analyses of electronic health records-derived data: a cautionary note
Open accessThe DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Abstract
Objective: Electronic health record (EHR)-derived data are extensively used in health research. However, the pattern of patient interaction with the healthcare system can result in informative presence bias if those who have poorer health have more data recorded than healthier patients. We aimed to determine how informative presence affects bias across multiple scenarios informed by real-world healthcare utilization patterns. Materials and methods: We conducted an analysis of EHR data from a pediatric healthcare system as well as simulation studies to characterize conditions under which informative presence bias is likely to occur. This analysis extends prior work by examining a variety of scenarios for the relationship between a biomarker and a health event of interest and the healthcare visit process. Results: Using biomarker values gathered at both informative and noninformative visits when estimating the effect of the biomarker on the event of interest resulted in minimal bias when the biomarker was relatively stable over time but produced substantial bias when the biomarker was more volatile. Adjusting analyses for the number of prior visits within a fixed look-back window was able to reduce but not eliminate this bias. Discussion: These results suggest that bias may arise frequently in commonly encountered scenarios and may not be eliminated by adjusting for prior visit intensity. Conclusion: Depending on the context, the estimated effect from analyses using data from all visits available may diverge from the true effect. Sensitivity analyses using only visits likely to be informative or noninformative based on visit type may aid in the assessment of the magnitude of potential bias.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“An important limitation of any genetic study using electronic health record data is the possibility of detection bias and informative presence bias.38”
From Rare Genetic Variants in LDLR , APOB , and PCSK9 Are Associated With Aortic Stenosis (2024) · cites this paper as [38]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Tsiatis, A. A., Degruttola, V., & Wulfsohn, M. S. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476485 · Cited by 575
Goldstein, B. A., Bhavsar, N. A., Phelan, M., & Pencina, M. J. (2016) · American Journal of Epidemiology · doi:10.1093/aje/kww112 · Cited by 176
Altman, D. G., & De Stavola, B. L. (1994) · Statistics in Medicine · doi:10.1002/sim.4780130402 · Cited by 173
Analysis of Longitudinal Data with Irregular, Outcome-Dependent Follow-Up
Lin, H., Scharfstein, D. O., & Rosenheck, R. A. (2004) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2004.b5543.x · Cited by 161
Sick patients have more data: the non-random completeness of electronic health records.
Weiskopf, N. G., Rusanov, A., & Weng, C. (2013) · Europe PMC (PubMed Central) · Cited by 132
Haneuse, S., & Daniels, M. (2016) · eGEMs (Generating Evidence & Methods to improve patient outcomes) · doi:10.13063/2327-9214.1203 · Cited by 119
Joint Modeling and Analysis of Longitudinal Data with Informative Observation Times
Liang, Y., Lu, W., & Ying, Z. (2008) · Biometrics · doi:10.1111/j.1541-0420.2008.01104.x · Cited by 117
Parameter Estimation in Longitudinal Studies with Outcome-Dependent Follow-Up
Lipsitz, S. R., Fitzmaurice, G. M., Ibrahim, J. G., Gelber, R. D., & Lipshultz, S. E. (2002) · Biometrics · doi:10.1111/j.0006-341x.2002.00621.x · Cited by 99
Phelan, M., Bhavsar, N., & Goldstein, B. A. (2017) · eGEMs (Generating Evidence & Methods to improve patient outcomes) · doi:10.5334/egems.243 · Cited by 96
Pullenayegum, E., & Lim, L. S. H. (2014) · Statistical Methods in Medical Research · doi:10.1177/0962280214536537 · Cited by 82
Cited by
Predicting polycystic ovary syndrome with machine learning algorithms from electronic health records
Zad, Z., Jiang, V. S., Wolf, A. T., Wang, T., Cheng, J. J., Paschalidis, I. C., & Mahalingaiah, S. (2024) · Frontiers in Endocrinology · doi:10.3389/fendo.2024.1298628 · Cited by 97
“This is an important consideration as a potential source of bias for interpreting predictive models using EHR data (34, 35).”
BackgroundMethodBiases in Electronic Health Records Data for Generating Real-World Evidence: An Overview
Al‐Sahab, B., Leviton, A., Loddenkemper, T., Paneth, N., & Zhang, B. (2023) · Journal of Healthcare Informatics Research · doi:10.1007/s41666-023-00153-2 · Cited by 89
Inherent Bias in Electronic Health Records: A Scoping Review of Sources of Bias
Perets, O., Stagno, E., Yehuda, E. B., McNichol, M. C., Celi, L. A., Rappoport, N., & Dorotić, M. (2026) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3757924 · Cited by 75
Chavez‐Yenter, D., Goodman, M. S., Chen, Y., Chu, X., Bradshaw, R. L., Chambers, R. L., Chan, P. A., Daly, B. M., Flynn, M. C., Gammon, A., et al. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.34574 · Cited by 56
Zhou, Y., Shi, J., Stein, R., Liu, X., Baldassano, R. N., Forrest, C. B., Chen, Y., & Huang, J. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad066 · Cited by 33
Schneeweiß, S., Desai, R. J., & Ball, R. (2024) · American Journal of Epidemiology · doi:10.1093/aje/kwae226 · Cited by 34
Rare Genetic Variants in LDLR , APOB , and PCSK9 Are Associated With Aortic Stenosis
Rämö, J. T., Jurgens, S. J., Kany, S., Choi, S. H., Wang, X., Smirnov, A. N., Friedman, S. F., Maddah, M., Khurshid, S., Ellinor, P. T., et al. (2024) · Circulation · doi:10.1161/circulationaha.124.070982 · Cited by 26
“38 Patients may be diagnosed with AS through cardiovascular imaging because of symptoms that may be explained by AS or through imaging that was performed because of other underlying conditions such as CAD.”
Silva, E. L., Lane, K., Cheng, J. J., Popp, Z., van Loenen, B. D., Coull, B. A., Hart, J. E., James-Todd, T. M., & Mahalingaiah, S. (2024) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/clinem/dgae705 · Cited by 26
“Additionally, EHR data is always subject to informative presence bias as patients with poorer health are more likely to be seen frequently in the hospital (60).”
Salvatore, M., Kundu, R., Shi, X., Friese, C. R., Lee, S., Fritsche, L. G., Mondul, A. M., Hanauer, D. A., Pearce, C. L., & Mukherjee, B. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae098 · Cited by 19
Detection Bias in EHR-Based Research on Clinical Exposures and Dementia
Wang, J., Choi, M., Buto, P. T., Kelly, J. D., Joie, R. L., Kornak, J., Zimmerman, S. C., Chen, R., Raphael, E., Schaefer, C. A., et al. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.6637 · Cited by 19
Optimization Techniques for Asthma Exacerbation Prediction Models: A Systematic Literature Review
Aliyu, D. A., Akhir, E. A. P., Saidu, Y., Adamu, S., Umar, K. I., Bunu, A. S., & Mamman, H. (2024) · IEEE Access · doi:10.1109/access.2024.3440502 · Cited by 17
“Studies relying on retrospective data analysis or electronic health records may be susceptible to information bias, as the accuracy and completeness of the recorded data could vary [113], [114].”
Use of AI as a Research Recruitment Tool: Ethical Elements, Considerations, and Recommendations
Rentzepis, P. J., Snyder, D. L., Baker, T. J., & Bierer, B. E. (2026) · The American Journal of Bioethics · doi:10.1080/15265161.2026.2649234 · Cited by 14
“Bias is engrained in EHRs, the most common data source for AI recruitment, by who receives care at the healthcare entity, insurance status, location, and choice of language, all of which will impact AI-based natural language processing (Verheij et al. 2018; Harton et al. 2022; Chen et al. 2024).”
Screening Familial Risk for Hereditary Breast and Ovarian Cancer
Kiser, D., Elhanan, G., Bolze, A., Neveux, I., Schlauch, K. A., Metcalf, W. J., Cirulli, E. T., McCarthy, C. A., Greenberg, L. A., Grime, S., et al. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.35901 · Cited by 16
Using routinely collected data for research purposes: challenges and mitigation strategies
Hoffmann, S., Morris, T. P., Herrmann, M., Heinze, G., Wynants, L., CALSTER, B. V., Bischl, B., Schmid, M., Shaw, P. A., Mathes, T., et al. (2026) · BMJ · doi:10.1136/bmj-2025-087812 · Cited by 12
Arena, P. J., Sun, Y., Jaksa, A. A., Kao, Y., Yoon, L. S., Ke, M., Marks‐Anglin, A., & Turzhitsky, V. M. (2025) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.70105 · Cited by 4
“Assess the impact of observability on the availability of key data elements 6.”
InfluentialSelection processes, transportability, and failure time analysis in life history studies
Cook, R. J., & Lawless, J. F. (2024) · Biostatistics · doi:10.1093/biostatistics/kxae039 · Cited by 4
Lotspeich, S. C., Kedar, S., Tahir, R., Keleghan, A., Miranda, A., Duda, S. N., Bancks, M. P., Wells, B. J., Khanna, A. K., & Rigdon, J. (2025) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2025.104904 · Cited by 3
“EHR data are susceptible to “informative presence bias,” wherein the process through which patients engage in the healthcare system can bias associations [54, 55, 56].”
Razzaghi, H., Wieand, K., Dickinson, K., Kahn, M. G., Roy, J., Blacketer, C., Christakis, D., Forrest, C. B., Greenberg, J., Lehmann, H. P., et al. (2026) · Journal of Medical Internet Research · doi:10.2196/76398 · Cited by 2
Yin, Y., Workman, T. E., Blosnich, J. R., Brandt, C., Skanderson, M., Shao, Y., Goulet, J. L., & Zeng‐Treitler, Q. (2024) · International Journal of Public Health · doi:10.3389/ijph.2024.1606855 · Cited by 2
“Known LGBT status suggests a higher level of engagement than unknown status, which may suggest an informed presence bias [48, 49], i.e., more engagement in the health systems increases the opportunities for SO/GI documentation and more engagement likely means greater service utilization and more…”
BackgroundMethodological challenges in studying disease processes using observational cohort data
Cook, R. J., & Lawless, J. F. (2024) · Japanese Journal of Statistics and Data Science · doi:10.1007/s42081-024-00276-9 · Cited by 2
“Common problems include selective populations (e.g. insured persons) or selective representation of a population (e.g. less well persons are over-represented) and selective data collection (again, biased towards less well persons); see for example Harton et al. (2022) and Hubbard et al. (2021).”
Hendrix, N., Parikh, R., Taskier, M., Walter, G., Phillips, R. L., & Rehkopf, D. H. (2025) · American Journal of Epidemiology · doi:10.1093/aje/kwaf162 · Cited by 1
Unveiling health disparities: Diagnostic prevalences in a transgender cohort versus matched controls
Hiatt, L., Ofori-Atta, B. S., Bakian, A. V., Mihalopoulos, N. L., Keeshin, B. R., Docherty, A. R., Staley, M., Fraser, A. M., Sullivan, E. J., Kaufman, E. A., et al. (2025) · PLoS ONE · doi:10.1371/journal.pone.0329849 · Cited by 1
Environmental Exposures and Reproductive Health
Silva, E. L. (2024) · Digital Access to Scholarship at Harvard (DASH) (Harvard University)
León, T. M., Muehling, L. M., Baccile, R., & Morozova, O. (2026) · Influenza and Other Respiratory Viruses · doi:10.1111/irv.70253
Illustrating Poststratification Methods in Medical Claims Data: A Korean Example
Oh, Y. J. (2026) · Epidemiology · doi:10.1097/ede.0000000000001980
Hennessy, C., Swartz, A. Z., Harrell, F. E., & Silver, H. J. (2026) · BMC Medical Research Methodology · doi:10.1186/s12874-026-02881-6
“[ 22] In the current project, we aimed to extend prior work by exploring a variable commonly measured and recorded in EHR data, regardless of the reason for a health care encounter, and one that is strongly associated with several chronic disease states including cardiovascular diseases which…”
Childhood Sexual Abuse and Long-Term Risk of Self-Harm, Overdose, and Cardiovascular DiseasePreprint
Akinyemi, O. A., Eze, O., Fasokun, M., Olaosebikan, I., Ogundipe, T., Singleton, D., Ogunsakin, A., Khalil, S., Gordon, K., Micheal, M., et al. (2026) · medRxiv · doi:10.64898/2026.07.09.26357627
Yang, C., Salvatore, M., Lu, H., Zhu, Z., Tennant, P. W. G., Shi, X., Ohno-Machado, L., Khera, R., Gross, C. P., Li, F., et al. (2026) · medRxiv · doi:10.64898/2026.08.12.26360236
Fernández, E. I., Chen, K. C., Fernández-Llatas, C., Seoane, F., & Abtahi, F. (2026) · Research Square · doi:10.21203/rs.3.rs-10371372/v1
Youssef, S. B., Trabelsi, A., Boudiaf, K., & Jebali, N. (2025) · International Journal Of Pharmaceutical Research And Allied Sciences · doi:10.51847/twb6heb1rz
Ouyang, Z., Lu, Y., Feng, W., Cao, T., Xiao, W., & Lin, W. (2026) · Research Square · doi:10.21203/rs.3.rs-10826050/v1
Jagadeesan, S. K., Khan, E., Sandarage, R. V., Chalil, A., & Tsai, E. C. (2026) · BMJ Open · doi:10.1136/bmjopen-2026-125558
Zhang, J., Zhang, Y., & Wei, L. (2026) · Journal of Clinical Medicine · doi:10.3390/jcm15208002
“Clinically determined monitoring frequency may reflect illness severity [19].”
Metadata
- Authors
- Harton, J., Mitra, N., & Hubbard, R. A.
- Affiliations
- University of Pennsylvania (US)
- Corresponding author
- Harton, Joanna
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2022-04-19
- Volume
- 29
- Issue
- 7
- Pages
- 1191-1199
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC9196698
- OpenAlex
- W4224247272
- Cited by
- 29
- References
- 23
- Citations by year
- 2026: 9 · 2025: 7 · 2024: 10 · 2023: 2 · 2022: 1
- FWCI
- 1.65
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model (vor)
- Locations
- Journal of the American Medical Informatics Association (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Cancer Institute of the National Institutes of Health (R21CA227613)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Keywords
- electronic health records, health care utilization patterns, bias in observational studies, pediatric health care, sensitivity analysis
- MeSH terms
- Child, Computer Simulation, Humans, Bias, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.