Domain Shifts in Machine Learning Based Covid-19 Diagnosis From Blood Tests
Open accessPublished version of a preprint
Abstract
Many previous studies claim to have developed machine learning models that diagnose COVID-19 from blood tests. However, we hypothesize that changes in the underlying distribution of the data, so called domain shifts, affect the predictive performance and reliability and are a reason for the failure of such machine learning models in clinical application. Domain shifts can be caused, e.g., by changes in the disease prevalence (spreading or tested population), by refined RT-PCR testing procedures (way of taking samples, laboratory procedures), or by virus mutations. Therefore, machine learning models for diagnosing COVID-19 or other diseases may not be reliable and degrade in performance over time. We investigate whether domain shifts are present in COVID-19 datasets and how they affect machine learning methods. We further set out to estimate the mortality risk based on routinely acquired blood tests in a hospital setting throughout pandemics and under domain shifts. We reveal domain shifts by evaluating the models on a large-scale dataset with different assessment strategies, such as temporal validation. We present the novel finding that domain shifts strongly affect machine learning models for COVID-19 diagnosis and deteriorate their predictive performance and credibility. Therefore, frequent re-training and re-assessment are indispensable for robust models enabling clinical utility.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Concept drift arises when the correlation between input variables and output predictions evolves due to fluctuations in data such as variations in image acquisition hardware or protocols, shifts in disease prevalence, or advancements in gold-standard technologies.
“Specifically, it arises when the correlation between input variables, such as images, and output predictions, such as diagnoses, evolves due to fluctuations in data, such as variations in image acquisition hardware or protocols, shifts in disease prevalence, or advancements in gold-standard technologies.38”
From Bias in artificial intelligence for medical imaging: fundamentals, detection, avoidance, mitigation, challenges, ethics, and prospects (2024) · cites this paper as [38]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
A novel coronavirus from patients with pneumonia in China, 2019
Zhu, N., Zhang, D., Wang, W., Li, X., Yang, B., Song, J., Zhao, X., Huang, B., Shi, W., Lu, R., et al. (2020) · New England Journal of Medicine · doi:10.1056/nejmoa2001017 · Cited by 28,244 · see citing works
DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L. (1988) · Biometrics · doi:10.2307/2531595 · Cited by 22,649 · see citing works
Detection of 2019 novel coronavirus (2019-nCoV) by real-time RT-PCR
Corman, V. M., Landt, O., Kaiser, M., Molenkamp, R., Meijer, A., Chu, D. K. W., Bleicker, T., Brünink, S., Schneider, J. A., Schmidt, M. L., et al. (2020) · Eurosurveillance · doi:10.2807/1560-7917.es.2020.25.3.2000045 · Cited by 7,266 · see citing works
The relationship between Precision-Recall and ROC curves
Davis, J., & Goadrich, M. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143874 · Cited by 6,559 · see citing works
Continual lifelong learning with neural networks: A review
Parisi, G. I., Kemker, R., Part, J. L., Kanan, C., & Wermter, S. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.01.012 · Cited by 3,117 · see citing works
Self-Normalizing Neural NetworksPreprint
Klambauer, G., Unterthiner, T., Mayr, A., & Hochreiter, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.02515 · Cited by 2,031 · see citing works
Self-citationA Survey of Predictive Modeling on Imbalanced Domains
Branco, P., Torgo, L., & Ribeiro, R. P. (2017) · ACM Computing Surveys · doi:10.1145/2907070 · Cited by 1,129 · see citing works
Roberts, M., Driggs, D., Thorpe, M., Gilbey, J., Yeung, M., Ursprung, S., Aviles-Rivero, A. I., Etmann, C., McCague, C., Beer, L., et al. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00307-0 · Cited by 1,044 · see citing works
Fibrinogen as a key regulator of inflammation in disease
Davalos, D., & Akassoglou, K. (2011) · Seminars in Immunopathology · doi:10.1007/s00281-011-0290-8 · Cited by 926 · see citing works
Increased mortality in community-tested cases of SARS-CoV-2 lineage B.1.1.7
CMMID COVID-19 Working Group, Davies, N. G., Jarvis, C. I., Edmunds, W. J., Jewell, N. P., Diaz-Ordaz, K., & Keogh, R. H. (2021) · Nature · doi:10.1038/s41586-021-03426-1 · Cited by 921 · see citing works
Roberts, M. T., Driggs, D., Thorpe, M., Gilbey, J., Yeung, M., Ursprung, S., Avilés-Rivero, A. I., Etmann, C., McCague, C., Beer, L., et al. (2020) · Apollo (University of Cambridge) · doi:10.17863/cam.64000 · Cited by 902 · see citing works
Collins, G. S., Dhiman, P., Andaur Navarro, C. L., Ma, J., Hooft, L., Reitsma, J. B., Logullo, P., Beam, A. L., Peng, L., Van Calster, B., et al. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-048008 · Cited by 852 · see citing works
Test sensitivity is secondary to frequency and turnaround time for COVID-19 screening
Larremore, D. B., Wilder, B., Lester, E., Shehata, S., Burke, J., Hay, J. A., Tambe, M., Mina, M., & Parker, R. (2020) · Science Advances · doi:10.1126/sciadv.abd5393 · Cited by 812 · see citing works
Rethinking Covid-19 Test Sensitivity — A Strategy for Containment
Mina, M. J., Parker, R., & Larremore, D. B. (2020) · New England Journal of Medicine · doi:10.1056/nejmp2025631 · Cited by 715 · see citing works
COVID-19 Artificial Intelligence Diagnosis Using Only Cough Recordings
Laguarta, J., Hueto, F., & Subirana, B. (2020) · IEEE Open Journal of Engineering in Medicine and Biology · doi:10.1109/ojemb.2020.3026928 · Cited by 589 · see citing works
WILDS: A Benchmark of in-the-Wild Distribution Shifts
Koh, P. W., Sagawa, S., Marklund, H., Xie, S. M., Zhang, M., Balsubramani, A., Hu, W., Yasunaga, M., Phillips, R. L., Gao, I., et al. (2020) · arXiv · doi:10.48550/arxiv.2012.07421 · Cited by 518 · cited 3× · see citing works
“The domain shifts can lead to degrading of predictive performance over time , because standard ML approaches are unable to cope with domain shifts over time [54, 59, 60].”
BackgroundCOVID-19 Coronavirus Vaccine Design Using Reverse Vaccinology and Machine Learning
Ong, E., Wong, M. U., Huffman, A., & He, Y. (2020) · Frontiers in Immunology · doi:10.3389/fimmu.2020.01581 · Cited by 499 · see citing works
Evaluation of rapid antigen test for detection of SARS-CoV-2 virus
Mak, G. C. K., Cheng, P. K. C., Lau, S. S. Y., et al. (2020) · Journal of Clinical Virology · doi:10.1016/j.jcv.2020.104500 · Cited by 495 · see citing works
COVIDGR Dataset and COVID-SDNet Methodology for Predicting COVID-19 Based on Chest X-Ray Images
Tabik, S., Gómez-Ríos, A., Rodríguez, J. L. M., Sevillano-García, I., Rey-Area, M., Charte, D., Guirado, E., Suárez, J. L., Luengo, J., González, M. Á. V., et al. (2020) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.3037127 · Cited by 376 · see citing works
Detection of COVID-19 Infection from Routine Blood Exams with Machine Learning: A Feasibility Study
Brinati, D., Campagner, A., Ferrari, D., Locatelli, M., Banfi, G., & Cabitza, F. (2020) · Journal of Medical Systems · doi:10.1007/s10916-020-01597-4 · Cited by 365 · see citing works
Machine learning based early warning system enables accurate mortality risk prediction for COVID-19
Gao, Y., Cai, G., Fang, W., Li, H., Wang, S., Chen, L., Yu, Y., Liu, D., Xu, S., Cui, P., et al. (2020) · Nature Communications · doi:10.1038/s41467-020-18684-2 · Cited by 349 · see citing works
Measuring Domain Shift for Deep Learning in Histopathology
Stacke, K., Eilertsen, G., Unger, J., & Lundström, C. F. (2020) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.3032060 · Cited by 334 · see citing works
Test sensitivity is secondary to frequency and turnaround time for COVID-19 surveillancePreprint
Larremore, D. B., Wilder, B., Lester, E., Shehata, S., Burke, J., Hay, J. A., Tambe, M., Mina, M., & Parker, R. (2020) · medRxiv · doi:10.1101/2020.06.22.20136309 · Cited by 309 · see citing works
Albahri, A. S., Hamid, R. A., Alwan, J. K., Al-qays, Z. T., Zaidan, A. A., Zaidan, B. B., Albahri, O. S., Alamoodi, A. H., Khlaf, J. M., Almahdi, E. M. A., et al. (2020) · Journal of Medical Systems · doi:10.1007/s10916-020-01582-x · Cited by 284 · see citing works
Artificial Intelligence for COVID-19 Drug Discovery and Vaccine Development
Arshadi, A. K., Webb, J., Salem, M., Cruz, E., Calad-Thomson, S., Ghadirian, N., Collins, J., Diez‐Cecilia, E., Kelly, B., Goodarzi, H., et al. (2020) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2020.00065 · Cited by 245 · see citing works
An introduction to domain adaptation and transfer learningPreprint
Kouw, W. M., & Loog, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.11806 · Cited by 244 · see citing works
Deep Learning on Chest X-ray Images to Detect and Evaluate Pneumonia Cases at the Era of COVID-19
Hammoudi, K., Benhabiles, H., Melkemi, M., Dornaika, F., Arganda‐Carreras, I., Collard, D., & Scherpereel, A. (2021) · Journal of Medical Systems · doi:10.1007/s10916-021-01745-4 · Cited by 232 · see citing works
Vaid, A., Somani, S. S., Russak, A. J., Freitas, J. K. D., Chaudhry, F. F., Paranjpe, I., Johnson, K. W., Lee, S. J., Miotto, R., Zhao, S., et al. (2020) · Journal of Medical Internet Research · doi:10.2196/24018 · Cited by 230 · see citing works
How artificial intelligence and machine learning can help healthcare systems respond to COVID-19
van der Schaar, M., Alaa, A. M., Floto, R. A., Gimson, A. E., Scholtes, S., Wood, A., McKinney, E. F., Jarrett, D. P., Lió, P., & Ercole, A. (2020) · Machine Learning · doi:10.1007/s10994-020-05928-x · Cited by 211 · see citing works
Increased mortality in community-tested cases of SARS-CoV-2 lineage B.1.1.7Preprint
Davies, N. G., Jarvis, C. I., CMMID COVID-19 Working Group, Edmunds, W. J., Jewell, N. P., Diaz‐Ordaz, K., & Keogh, R. H. (2021) · medRxiv · doi:10.1101/2021.02.01.21250959 · Cited by 211 · see citing works
Wang, G., Liu, X., Shen, J., Wang, C., Li, Z., Ye, L., Wu, X., Chen, T., Wang, K., Zhang, X., et al. (2021) · Nature Biomedical Engineering · doi:10.1038/s41551-021-00704-1 · Cited by 192 · see citing works
Classification of COVID-19 chest X-rays with deep learning: new models or fine tuning?
Pham, T. D. (2020) · Health Information Science and Systems · doi:10.1007/s13755-020-00135-3 · Cited by 190 · see citing works
Development of a prognostic model for mortality in COVID-19 infection using machine learning
Booth, A. L., Abels, E. A., & McCaffrey, P. (2020) · Modern Pathology · doi:10.1038/s41379-020-00700-x · Cited by 182 · see citing works
Review on Diagnosis of COVID-19 from Chest CT Images Using Artificial Intelligence
Özşahin, İ., Şekeroğlu, B., Musa, M. S., Mustapha, M. T., & Ozsahin, D. U. (2020) · Computational and Mathematical Methods in Medicine · doi:10.1155/2020/9756518 · Cited by 179 · see citing works
Banerjee, A., Ray, S., Vorselaars, B., Kitson, J., Mamalakis, M., Weeks, S., Baker, M., & Mackenzie, L. S. (2020) · International Immunopharmacology · doi:10.1016/j.intimp.2020.106705 · Cited by 174 · see citing works
Cabitza, F., Campagner, A., Ferrari, D., Resta, C. D., Ceriotti, D., Sabetta, E., Colombini, A., Vecchi, E. D., Banfi, G., Locatelli, M., et al. (2020) · Clinical Chemistry and Laboratory Medicine (CCLM) · doi:10.1515/cclm-2020-1294 · Cited by 168 · see citing works
Alafif, T., Tehame, A. M., Bajaba, S. M., Barnawi, A., & Zia, S. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18031117 · Cited by 162 · see citing works
Saha, P., Sadi, M. S., & Islam, M. M. (2020) · Informatics in Medicine Unlocked · doi:10.1016/j.imu.2020.100505 · Cited by 158 · see citing works
Soltan, A. A. S., Kouchaki, S., Zhu, T., Kiyasseh, D., Taylor, T., Hussain, Z. B., Peto, T., Brent, A. J., Eyre, D. W., & Clifton, D. A. (2021) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30274-0 · Cited by 150 · see citing works
Ensemble learning model for diagnosing COVID-19 from routine blood tests
AlJame, M., Ahmad, I., Imtiaz, A., & Mohammed, A. (2020) · Informatics in Medicine Unlocked · doi:10.1016/j.imu.2020.100449 · Cited by 145 · see citing works
Adly, A. S., Adly, A. S., & Adly, M. S. (2020) · Journal of Medical Internet Research · doi:10.2196/19104 · Cited by 138 · see citing works
Yao, H., Zhang, N., Zhang, R., Duan, M., Xie, T., Pan, J., Peng, E., Huang, J., Zhang, Y., Xu, X., et al. (2020) · Frontiers in Cell and Developmental Biology · doi:10.3389/fcell.2020.00683 · Cited by 134 · see citing works
Schuller, B. W., Schuller, D., Qian, K., Liu, J., Zheng, H., & Li, X. (2021) · Frontiers in Digital Health · doi:10.3389/fdgth.2021.564906 · Cited by 126 · see citing works
Routine Laboratory Blood Tests Predict SARS-CoV-2 Infection Using Machine Learning
Yang, H. S., Hou, Y., Vasović, L. V., Steel, P. A. D., Chadburn, A., Racine‐Brzostek, S., Velu, P., Cushing, M. M., Loda, M. F., Kaushal, R., et al. (2020) · Clinical Chemistry · doi:10.1093/clinchem/hvaa200 · Cited by 125 · see citing works
Incremental Adversarial Domain Adaptation for Continually Changing Environments
Wulfmeier, M., Bewley, A., & Posner, I. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8460982 · Cited by 123 · see citing works
Chin, E. T., Huynh, B. Q., Chapman, L. A. C., et al. (2020) · Clinical Infectious Diseases · doi:10.1093/cid/ciaa1383 · Cited by 109 · see citing works
Li, X., Ge, P., Zhu, J., Li, H., Graham, J., Singer, A. J., Richman, P. S., & Duong, T. Q. (2020) · PeerJ · doi:10.7717/peerj.10337 · Cited by 109 · see citing works
Robust Detection of COVID-19 in Cough Sounds
Mouawad, P. G., Dubnov, T., & Dubnov, S. (2021) · SN Computer Science · doi:10.1007/s42979-020-00422-6 · Cited by 109 · see citing works
Early risk assessment for COVID-19 patients from emergency department data using machine learning
Heldt, F. S., Vizcaychipi, M. P., Peacock, S., Cinelli, M., McLachlan, L., Andreotti, F., Jovanović, S., Dürichen, R., Lipunova, N., Fletcher, R. A., et al. (2021) · Scientific Reports · doi:10.1038/s41598-021-83784-y · Cited by 97 · see citing works
Utility of Artificial Intelligence Amidst the COVID 19 Pandemic: A Review
Bansal, A., Padappayil, R. P., Garg, C., Singal, A., Gupta, M., & Klein, A. (2020) · Journal of Medical Systems · doi:10.1007/s10916-020-01617-3 · Cited by 87 · see citing works
To Annotate or Not? Predicting Performance Drop under Domain Shift
Elsahar, H., & Gallé, M. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1222 · Cited by 86 · see citing works
Heg.IA: an intelligent system to support diagnosis of Covid-19 based on blood tests
Barbosa, V. A. D. F., Gomes, J. C., de Santana, M. A., Albuquerque, J. E. D. A., de Souza, R. G., de Souza, R. E., & dos Santos, W. P. (2021) · Research on Biomedical Engineering · doi:10.1007/s42600-020-00112-5 · Cited by 78 · see citing works
Ko, H., Chung, H., Kang, W. S., Park, C., Kim, D., Kim, S. E., Chung, C. R., Ko, R., Lee, H., Seo, J. H., et al. (2020) · Journal of Medical Internet Research · doi:10.2196/25442 · Cited by 75 · see citing works
A multipurpose machine learning approach to predict COVID-19 negative prognosis in São Paulo, Brazil
Fernandes, F. T., de Oliveira, T. A., Teixeira, C. E., Batista, A. F. D. M., Costa, G. D., & Filho, A. D. P. C. (2021) · Scientific Reports · doi:10.1038/s41598-021-82885-y · Cited by 75 · see citing works
Using Artificial Intelligence for COVID-19 Chest X-ray Diagnosis
Borkowski, A., Viswanadham, N. A., Thomas, L. B., Guzman, R. D., Deland, L. A., & Mastorides, S. (2020) · Federal Practitioner · doi:10.12788/fp.0045 · Cited by 74 · see citing works
A machine learning algorithm to increase COVID-19 inpatient diagnostic capacity
Goodman‐Meza, D., Rudas, Á., Chiang, J. N., Adamson, P. C., Ebinger, J. E., Sun, N., Botting, P. G., Fulcher, J. A., Saab, F. G., Brook, R., et al. (2020) · PLoS ONE · doi:10.1371/journal.pone.0239474 · Cited by 68 · see citing works
Covid-19 Imaging Tools: How Big Data is Big?
Santosh, K., & Ghosh, S. (2021) · Journal of Medical Systems · doi:10.1007/s10916-021-01747-2 · Cited by 67 · see citing works
Harrell, F. E., Lee, K. L., & Mark, D. B. (2004) · doi:10.1002/0470023678.ch2b(i · Cited by 53 · see citing works
Large-Scale Ligand-Based Virtual Screening for SARS-CoV-2 Inhibitors Using Deep Neural NetworksPreprint
Hofmarcher, M., Mayr, A., Rumetshofer, E., Ruch, P., Renz, P., Schimunek, J., Seidl, P., Vall, A., Widrich, M., Hochreiter, S., et al. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3561442 · Cited by 52 · see citing works
Self-citationCoVA: An Acuity Score for Outpatient Screening that Predicts Coronavirus Disease 2019 Prognosis
Sun, H., Jain, A., Leone, M. J., Alabsi, H. S., Brenner, L. N., Ye, E. M., Ge, W., Shao, Y., Boutros, C. L., Wang, R., et al. (2020) · The Journal of Infectious Diseases · doi:10.1093/infdis/jiaa663 · Cited by 48 · see citing works
Plante, T. B., Blau, A. M., Berg, A. N., Weinberg, A. S., Jun, I. C., Tapson, V. F., Kanigan, T. S., & Adib, A. A. (2020) · Journal of Medical Internet Research · doi:10.2196/24048 · Cited by 46 · see citing works
Dorn, M., Avila, E., Alho, C. S., & Kahmann, A. (2020) · PeerJ · doi:10.7717/peerj.9482 · Cited by 45 · see citing works
Länger, T., Favarato, M., Giudici, R., Bassi, G., Garberi, R., Villa, F., Gay, H., Zeduri, A., Bragagnolo, S., Molteni, A., et al. (2020) · Scandinavian Journal of Trauma Resuscitation and Emergency Medicine · doi:10.1186/s13049-020-00808-8 · Cited by 43 · see citing works
Crawford, V. L. S. (2000) · QJM · doi:10.1093/qjmed/93.11.745 · Cited by 41 · see citing works
Formica, V., Minieri, M., Bernardini, S., Ciotti, M., D’Agostini, C., Roselli, M., Andreoni, M., Morelli, M. C., Parisi, G., Federici, M., et al. (2020) · Clinical Medicine · doi:10.7861/clinmed.2020-0373 · Cited by 41 · see citing works
Machine Learning Prediction of SARS-CoV-2 Polymerase Chain Reaction Results with Routine Blood Tests
Tschoellitsch, T., Dünser, M. W., Böck, C., Schwarzbauer, K., & Meier, J. M. (2020) · Laboratory Medicine · doi:10.1093/labmed/lmaa111 · Cited by 38 · see citing works
Self-citationCross-Domain Few-Shot Learning by Representation FusionPreprint
Adler, T., Brandstetter, J., Widrich, M., Mayr, A., Kreil, D. P., Kopp, M., Klambauer, G., & Hochreiter, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.06498 · Cited by 34 · see citing works
Self-citationApplications of Machine Learning and High-Performance Computing in the Era of COVID-19
Majeed, A., & Lee, S. (2021) · Applied System Innovation · doi:10.3390/asi4030040 · Cited by 25 · see citing works
Prediction of COVID-19 From Hemogram Results and Age Using Machine Learning
Silveira, E. C. (2020) · Frontiers in Health Informatics · doi:10.30699/fhi.v9i1.234 · Cited by 20 · see citing works
Artificial Intelligence and a Pandemic: an Analysis of the Potential Uses and Drawbacks
Williams, C. M., Chaturvedi, R., Urman, R. D., Waterman, R. S., & Gabriel, R. A. (2021) · Journal of Medical Systems · doi:10.1007/s10916-021-01705-y · Cited by 17 · see citing works
Heber, S., Pereyra, D., Schrottmaier, W. C., Kammerer, K., Santol, J., Rumpf, B., Pawelka, E., Hanna, M., Scholz, A., Liu, M., et al. (2022) · Frontiers in Cellular and Infection Microbiology · doi:10.3389/fcimb.2021.795026 · Cited by 17 · see citing works
Exploring the Relation between Blood Tests and Covid-19 Using Machine Learning
Almansoor, M., & Hewahi, N. M. (2020) · International Conference on Data Analytics for Business and Industry (ICDABI) · doi:10.1109/icdabi51230.2020.9325673 · Cited by 12 · see citing works
Understanding Behavior of Clinical Models under Domain ShiftsPreprint
Thiagarajan, J. J., Rajan, D., & Sattigeri, P. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.07806 · Cited by 8 · see citing works
The Predictive Effectiveness of Blood Biochemical Indexes for the Severity of COVID-19
Zhou, Y., Li, B., Liu, J., et al. (2020) · Canadian Journal of Infectious Diseases and Medical Microbiology · doi:10.1155/2020/7320813 · Cited by 6 · see citing works
Schneider, M., Wang, L., & Marr, C. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-30484-3_53 · Cited by 2 · see citing works
Harrell, F., Jr. (1996) · doi:10.1002/(sici)1097-0258(19960229)15:4
Flagged first, then by citations
Research works printed but not linked: 2
- [49]S. Heber, D. Pereyra, W. C. Schrottmaier, K. Kammerer, J. Santol, E. Pawelka, M. Hana, A. Scholz, M. Liu, A. Hell, K. Heiplik, B. Lickefett, S. Havervall, M. T. Traugott, M. Neuböck, C. Schörgenhofer, T. Seitz, C. Firbas, M. Karolyi, G. Weiss, B. Jilma, C. Thralin, R. Bellmann-Weiler, H. J. F. Salzer, M. J. M. Fischer, A. Zoufaly and A. Assinger, ”Development and external validation of a logistic regression derived formula based on repeated routine hematological measurements predicting survival of hospitalized Covid-19 patients“, medRxiv, 2020. 10.1101/2020.12.20.20248563 · DOI
- [67]J. H. Zar, “Biostatistical Analysis”, 5th edition, Prentice Hall, Upper Saddle River, New Jersey USA, 2010, ISBN: 9780321656865
Cited by
Sorted by citations
Data drift in medical machine learning: implications and potential remedies
Sahiner, B., Chen, W., Samala, R. K., & Petrick, N. (2023) · The British Journal of Radiology · doi:10.1259/bjr.20220878 · Cited by 279 · see citing works
Koçak, B., Ponsiglione, A., Stanzione, A., Bluethgen, C., Santinha, J., Ugga, L., Huisman, M., Klontzas, M. E., Cannella, R., & Cuocolo, R. (2024) · Diagnostic and Interventional Radiology · doi:10.4274/dir.2024.242854 · Cited by 168 · see citing works
“38 Behavioral bias pertains to the potential distortions in user behavior seen across various platforms, contexts, or datasets.”
Drukker, K., Chen, W., Gichoya, J. W., Gruszauskas, N. P., Kalpathy-Cramer, J., Koyejo, S., Myers, K. J., Sá, R. C., Sahiner, B., Whitney, H. M., et al. (2023) · Journal of Medical Imaging · doi:10.1117/1.jmi.10.6.061104 · Cited by 149 · see citing works
- Authors
- Roland, T., Böck, C., Tschoellitsch, T., Maletzky, A., Hochreiter, S., Meier, J., & Klambauer, G.
- Affiliations
- Johannes Kepler University of Linz (AT)
- RISC Software (Austria) (AT)
- Corresponding author
- Roland, Theresa
- ORCID
- Venue
- Journal of Medical Systems
- Published
- 2022-03-29
- Volume
- 46
- Issue
- 5
- Pages
- 23-23
- ISSN
- 0148-5598 (ISSN-L)
- 1573-689X
- PMC
- PMC8960704
- OpenAlex
- W4221013844
- Cited by
- 42
- References
- 76
- Citations by year
- 2026: 9 · 2025: 6 · 2024: 9 · 2023: 10 · 2022: 5 · 2021: 3
- FWCI
- 4.26
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Journal of Medical Systems (publishedVersion, cc-by)
- medRxiv (acceptedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Zenodo (CERN European Organization for Nuclear Research) (submittedVersion, cc-by)
- University Library Linz repository (Johannes Kepler Universitat Linz) (submittedVersion, cc-by)
- Funders
- Linz Institute of Technology (LIT-2018-6-YOU-212)
- State of Upper Austria (Wi-2018-439843)
- Johannes Kepler University Linz
- Topics
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › COVID-19 diagnosis using AI
- Health Sciences › Medicine › Infectious Diseases › SARS-CoV-2 detection and testing
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- domain shift, COVID-19 diagnosis, blood analysis, mortality prediction, temporal validation, model drift, RT-PCR testing, SARS-CoV-2 mutations, model retraining
- MeSH terms
- Machine Learning, COVID-19, COVID-19 Testing, Hematologic Tests, Humans, Reproducibility of Results
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
