From real-world electronic health record data to real-world results using artificial intelligence
Open accessThe DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Abstract
With the worldwide digitalisation of medical records, electronic health records (EHRs) have become an increasingly important source of real-world data (RWD). RWD can complement traditional study designs because it captures almost the complete variety of patients, leading to more generalisable results. For rheumatology, these data are particularly interesting as our diseases are uncommon and often take years to develop. In this review, we discuss the following concepts related to the use of EHR for research and considerations for translation into clinical care: EHR data contain a broad collection of healthcare data covering the multitude of real-life patients and the healthcare processes related to their care. Machine learning (ML) is a powerful method that allows us to leverage a large amount of heterogeneous clinical data for clinical algorithms, but requires extensive training, testing, and validation. Patterns discovered in EHR data using ML are applicable to real life settings, however, are also prone to capturing the local EHR structure and limiting generalisability outside the EHR(s) from which they were developed. Population studies on EHR necessitates knowledge on the factors influencing the data available in the EHR to circumvent biases, for example, access to medical care, insurance status. In summary, EHR data represent a rapidly growing and key resource for real-world studies. However, transforming RWD EHR data for research and for real-world evidence using ML requires knowledge of the EHR system and their differences from existing observational data to ensure that studies incorporate rigorous methods that acknowledge or address factors such as access to care, noise in the data, missingness and indication bias.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856
He, H., & Garcia, E. A. (2009) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2008.239 · Cited by 10,359
The UK Biobank resource with deep phenotyping and genomic data
Bycroft, C., Freeman, C., Petkova, D., Band, G., Elliott, L. T., Sharp, K., Motyer, A., Vukcevic, D., Delaneau, O., O’Connell, J., et al. (2018) · Nature · doi:10.1038/s41586-018-0579-z · Cited by 8,876 · cited 5×
“46 In addition, biobanks linked with EHR data and samples for immuno and genotyping have further extended the reach and potential for translational research with RWD. 47 48 Using EHR for population health studies does require special attention and caution to ensure high quality results.”
BackgroundInfluentialGulshan, V., Peng, L., Coram, M., Stumpe, M. C., Wu, D., Narayanaswamy, A., Venugopalan, S., Widner, K., Madams, T., Cuadros, J., et al. (2016) · JAMA · doi:10.1001/jama.2016.17216 · Cited by 7,697
Social conditions as fundamental causes of disease.
Link, B. G., & Phelan, J. (1995) · Journal of Health and Social Behavior · doi:10.2307/2626958 · Cited by 5,722
Saito, T., & Rehmsmeier, M. (2015) · PLOS ONE · doi:10.1371/journal.pone.0118432 · Cited by 5,309
PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies
Wolff, R. F., Moons, K. G. M., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., Mallett, S., & for the PROBAST Group† (2019) · Annals of Internal Medicine · doi:10.7326/m18-1376 · Cited by 2,984 · cited 1×
“49 The length of the patients’ trajectories influences the chance of being captured in case-control studies.”
BackgroundA comparison of observational studies and randomized, controlled trials.
Benson, K., & Hartz, A. J. (2000) · New England Journal of Medicine · doi:10.1056/nejm200006223422506 · Cited by 2,347
Bjornevik, K., Cortese, M., Healy, B. C., Kuhle, J., Mina, M. J., Leng, Y., Elledge, S. J., Niebuhr, D. W., Scher, A. I., Munger, K. L., et al. (2022) · Science · doi:10.1126/science.abj8222 · Cited by 2,049
The rise of artificial intelligence in healthcare applications
Bohr, A., & Memarzadeh, K. (2020) · Artificial Intelligence in Healthcare · doi:10.1016/b978-0-12-818438-7.00002-2 · Cited by 1,636 · cited 1×
“34 As with any test or prediction, clinical application necessitates an even more rigorous model assessment.”
Method
Cited by
Using artificial intelligence to improve public health: a narrative review
Olawade, D. B., Wada, O. J., David-Olawade, A. C., Kunonga, E., Abaire, O., & Ling, J. (2023) · Frontiers in Public Health · doi:10.3389/fpubh.2023.1196397 · Cited by 274
Liu, T., Krentz, A., Lu, L., & Curcin, V. (2025) · European Heart Journal - Digital Health · doi:10.1093/ehjdh/ztae080 · Cited by 177
Ruksakulpiwat, S., Thorngthip, S., Niyomyart, A., Benjasirisan, C., Phianhasin, L., Aldossary, H., Ahmed, B., & Samai, T. (2024) · Journal of Multidisciplinary Healthcare · doi:10.2147/jmdh.s459946 · Cited by 98
“20”
Data Quality–Driven Improvement in Health Care: Systematic Literature Review
Lighterness, A., Adcock, M., Scanlon, L. A., & Price, G. (2024) · Journal of Medical Internet Research · doi:10.2196/57615 · Cited by 26
“Despite its potential, EHR-RWD, and by extension RWE, grapple with formidable barriers, and chief among these is data quality (DQ) [1,2,9,10,13,14].”
Metadata
- Authors
- Knevel, R., & Liao, K. P.
- Venue
- Annals of the Rheumatic Diseases
- Publisher
- Elsevier BV
- Published
- 2023-03
- Volume
- 82
- Issue
- 3
- Pages
- 306-311
- ISSN
- 0003-4967 (ISSN-L)
- OpenAlex
- W4297014655
- Cited by
- 154
- References
- 25
- FWCI
- 9.13
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: From real-world electronic health record data to real-world results using artificial intelligence. https://app.cochat.ai/papers/10.1136/ard-2022-222626
- The paper
- Knevel, R., & Liao, K. P. (2023). From real-world electronic health record data to real-world results using artificial intelligence. Annals of the Rheumatic Diseases. https://doi.org/10.1136/ard-2022-222626
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.