Feature Robustness in Non-stationary Health Records: Caveats to Deployable Model Performance in Common Clinical Machine Learning Tasks
Open accessThe DOI matches its OpenAlex record; 14 of the 14 references it cites were found in a registry or an index.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 3×
“To illustrate the effects of non-stationarity, we benchmark four commonly used models in the machine learning literature for clinical prediction from time series data: logistic regression (LR), random forests (RF), long short-term memory (LSTM) network (Hochreiter and Schmidhuber, 1997) and a gated recurrent unit with…”
BackgroundMethodAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction
McInnes, L., Healy, J., & Melville, J. (2018) · arXiv · doi:10.48550/arxiv.1802.03426 · Cited by 7,101
Approximate Statistical Tests for Comparing Supervised Classification Learning Algorithms
Dietterich, T. G. (1998) · Neural Computation · doi:10.1162/089976698300017197 · Cited by 3,840 · cited 2×
“For example, RF models report a year-agnostic mortality AUROC of 0.82± 0.02 (5 x 2 fold CV splits (Dietterich, 1998)), as compared to their true year-averaged AUROC under the raw representation of 0.76± 0.13.”
BackgroundInfluentialThe Parable of Google Flu: Traps in Big Data Analysis
Lazer, D., Kennedy, R., King, G., & Vespignani, A. (2014) · Science · doi:10.1126/science.1248506 · Cited by 2,523 · cited 2×
“, 2017) all use time-agnostic evaluation protocols, which do not account for a significant source of error that would affect models during true deployment: namely, the evolution of care practices over time and the resultant concept drift (Žliobaitė, 2010), which are known to induce significant differences in clinical…”
BackgroundPollard, T. J., Johnson, A. E. W., Raffa, J. D., Celi, L. A., Mark, R. G., & Badawi, O. (2018) · Scientific Data · doi:10.1038/sdata.2018.178 · Cited by 1,993
Savova, G. K., Masanz, J. J., Ogren, P. V., Zheng, J., Sohn, S., Kipper-Schuler, K. C., & Chute, C. G. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2009.001560 · Cited by 1,967 · cited 2×
“Gong et al., for example, uses the clinical Text Analysis Knowledge Extraction System (cTAKES) (Savova et al., 2010) to identify UMLS concept unique identifiers (CUIs) (Bodenreider, 2004) from human-readable feature descriptions, and aggregates features into higher level buckets based on this CUI…”
MethodMultitask learning and benchmarking with clinical time series data
Harutyunyan, H., Khachatrian, H., Kale, D. C., Ver Steeg, G., & Galstyan, A. (2019) · Scientific Data · doi:10.1038/s41597-019-0103-9 · Cited by 958 · cited 6×
“The wide-spread adoption of electronic health records (EHRs) in modern healthcare systems has enabled the secondary use of these records to develop machine learning models for mortality risk (Harutyunyan et al., 2017), sepsis treatment (Raghu et al.”
BackgroundMethodResultInfluentialArtificial intelligence, bias and clinical safety
Challen, R., Denny, J., Pitt, M., Gompels, L., Edwards, T., & Tsaneva-Atanasova, K. (2019) · BMJ Quality & Safety · doi:10.1136/bmjqs-2018-008370 · Cited by 956 · cited 1×
“Overall, the issue of distributional shift is still considered to be a major barrier in the safety and robustness of ML/AI systems, especially in healthcare (Challen et al., 2019).”
BackgroundGRAM: Graph-based Attention Model for Healthcare Representation Learning
Choi, E., Bahadori, M. T., Song, L., Stewart, W. F., & Sun, J. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098126 · Cited by 637 · cited 3×
“As a result, the wide literature of prior work on MIMIC-III (Harutyunyan et al., 2017; Purushotham et al., 2018; Choi et al., 2017) all use time-agnostic evaluation protocols, which do not account for a significant source of error that would affect models during true deployment: namely, the evolution of care practices…”
BackgroundResult
Cited by
The myth of generalisability in clinical research and machine learning in health care
Futoma, J., Simons, M., Panch, T., Doshi-Velez, F., & Celi, L. A. (2020) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30186-2 · Cited by 467
Feng, J., Phillips, R. V., Malenica, I., Bishara, A., Hubbard, A. E., Celi, L. A., & Pirracchio, R. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00611-y · Cited by 423
“Indeed, many have documented ML performance decay due to patient case mix, clinical practice patterns, treatment options, and more 9 – 11 .”
BackgroundShifting machine learning for healthcare from development to deployment and from models to data
Zhang, A., Xing, L., Zou, J., & Wu, J. C. (2022) · Nature Biomedical Engineering · doi:10.1038/s41551-022-00898-y · Cited by 420
Underspecification Presents Challenges for Credibility in Modern Machine Learning
D'Amour, A., Heller, K., Moldovan, D., Adlam, B., Alipanahi, B., Beutel, A., Chen, C., Deaton, J., Eisenstein, J., Hoffman, M. D., et al. (2020) · arXiv · doi:10.48550/arxiv.2011.03395 · Cited by 398
Cutillo, C. M., Sharma, K. R., Foschini, L., Kundu, S., Mackintosh, M., Mandl, K. D., MI in Healthcare Workshop Working Group, Beck, T., Collier, E., Colvis, C., et al. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0254-2 · Cited by 337
Data drift in medical machine learning: implications and potential remedies
Sahiner, B., Chen, W., Samala, R. K., & Petrick, N. (2023) · The British Journal of Radiology · doi:10.1259/bjr.20220878 · Cited by 279
WILDS: A Benchmark of in-the-Wild Distribution Shifts
Koh, P. W., Sagawa, S., Marklund, H., Xie, S. M., Zhang, M., Balsubramani, A., Hu, W., Yasunaga, M., Phillips, R. L., Gao, I., et al. (2020) · arXiv · doi:10.48550/arxiv.2012.07421 · Cited by 518
“Finally, changes over time in care settings can also produce distribution shifts and drops in algorithmic performance: Nestor et al. (2019) shows that switching between two electronic health record (EHR) systems produced a drop in algorithmic performance.”
There is no such thing as a validated prediction model
Van Calster, B., Steyerberg, E. W., Wynants, L., & van Smeden, M. (2023) · BMC Medicine · doi:10.1186/s12916-023-02779-w · Cited by 286
“In the fast-changing and developing world of medicine, patient populations, standards of care, available treatment options, and patient preferences, measurement and data registration procedures change over time [30].”
BackgroundMcCoy, L. G., Brenna, C. T. A., Chen, S. S., Vold, K., & Das, S. (2022) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.11.001 · Cited by 118
“There are specific and valid reasons for concern regarding the challenges of empirically validating MLHC tools, such as concerns that shifts in the underlying population may render a model inaccurate over time [64] , or concerns that a continuously updating model may develop aberrant characteristics.”
BackgroundGuo, L. L., Pfohl, S. R., Fries, J., Johnson, A. E. W., Posada, J., Aftandilian, C., Shah, N., & Sung, L. (2022) · Scientific Reports · doi:10.1038/s41598-022-06484-1 · Cited by 103
Machine learning for real-time aggregated prediction of hospital admission for emergency patients
King, Z., Farrington, J., Utley, M., Kung, E., Elkhodair, S., Harris, S., Sekula, R., Gillham, J., Li, K., & Crowe, S. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00649-y · Cited by 87
“…of incomplete visits from historic data in which visits have been completed 22 , making decisions about the temporal framing of the model (for example, at what point in the visit to check if the outcome of interest has occurred) 23 , and planning for a drift in model performance over time 24 .”
InfluentialDetecting and Remediating Harmful Data Shifts for the Responsible Deployment of Clinical AI Models
Subasri, V., Krishnan, A., Kore, A., Dhalla, A., Pandya, D., Wang, B., Malkin, D., Razak, F., Verma, A. A., Goldenberg, A., et al. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.13685 · Cited by 87
London, A. J. (2022) · Cell Reports Medicine · doi:10.1016/j.xcrm.2022.100622 · Cited by 76
“42,43 This was illustrated during the current pandemic by an increase in sepsis alerts in health systems using AI systems to monitor clinical care.”
Chen, S., Yu, J., Chamouni, S., Wang, Y., & Li, Y. (2024) · BMC Medicine · doi:10.1186/s12916-024-03566-x · Cited by 73
“Consequently, these changes necessitate the periodic retraining and re-evaluation of AI systems to maintain their relevance and efficacy [69, 70].”
BackgroundSecure and robust machine learning for healthcare: A survey
Qayyum, A., Qadir, J., Bilal, M., & Al-Fuqaha, A. (2021) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2020.3013489 · Cited by 59
Secure and Robust Machine Learning for Healthcare: A Survey
Qayyum, A., Qadir, J., Bilal, M., & Al-Fuqaha, A. (2020) · arXiv · doi:10.48550/arxiv.2001.08103 · Cited by 59
D’Hondt, E., Ashby, T. J., Chakroun, I., Koninckx, T., & Wuyts, R. (2022) · Communications Medicine · doi:10.1038/s43856-022-00225-1 · Cited by 29
A comprehensive EHR timeseries pre-training benchmark
McDermott, M., Nestor, B., Kim, E., Zhang, W., Goldenberg, A., Szolovits, P., & Ghassemi, M. (2021) · Proceedings of the Conference on Health, Inference, and Learning · doi:10.1145/3450439.3451877 · Cited by 24
Self-citationMore Generalizable Models For Sepsis Detection Under Covariate Shift
Gao, J., Mar, P. L., & Chen, G. (2021) · arXiv · doi:10.48550/arxiv.2105.09385 · Cited by 3
Metadata
- Authors
- Nestor, B. A., McDermott, M. B. A., Boag, W., Berner, G., Naumann, T., Hughes, M. C., Goldenberg, A., & Ghassemi, M.
- Venue
- arXiv (Cornell University)
- Publisher
- Cornell University
- Published
- 2019-08-02
- Pages
- 381-405
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1908.00690
- OpenAlex
- W2965485927
- Cited by
- 71
- References
- 14
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.