The DOI matches its Crossref record, with a small difference in title or year; 472 of the 472 references it cites were found in a registry or an index.
Systematic review and critical appraisal of prediction models for diagnosis and prognosis of COVID-19 infection
Open accessPreprint — published as 10.1136/bmj.m1328 · Paper page
Abstract
Objective: To review and critically appraise published and preprint reports of models that aim to predict either (i) presence of existing COVID-19 infection, (ii) future complications in individuals already diagnosed with COVID-19, or (iii) models to identify individuals at high risk for COVID-19 in the general population. Design: Rapid systematic review and critical appraisal of prediction models for diagnosis or prognosis of COVID-19 infection. Data sources: PubMed, EMBASE via Ovid, Arxiv, medRxiv and bioRxiv until 24 th March 2020. Study selection: Studies that developed or validated a multivariable COVID-19 related prediction model. Two authors independently screened titles, abstracts and full text. Data extraction: Data from included studies were extracted independently by at least two authors based on the CHARMS checklist, and risk of bias was assessed using PROBAST. Data were extracted on various domains including the participants, predictors, outcomes, data analysis, and prediction model performance. Results: 2696 titles were screened. Of these, 27 studies describing 31 prediction models were included for data extraction and critical appraisal. We identified three models to predict hospital admission from pneumonia and other events (as a proxy for covid-19 pneumonia) in the general population; 18 diagnostic models to detect COVID-19 infection in symptomatic individuals (13 of which were machine learning utilising computed tomography (CT) results); and ten prognostic models for predicting mortality risk, progression to a severe state, or length of hospital stay. Only one of these studies used data on COVID-19 cases outside of China. Most reported predictors of presence of COVID-19 in suspected patients included age, body temperature, and signs and symptoms. Most reported predictors of severe prognosis in infected patients included age, sex, features derived from CT, C-reactive protein, lactic dehydrogenase, and lymphocyte count. Estimated C-index estimates for the prediction models ranged from 0.73 to 0.81 in those for the general population (reported for all 3 general population models), from 0.81 to > 0.99 in those for diagnosis (reported for 13 of the 18 diagnostic models), and from 0.85 to 0.98 in those for prognosis (reported for 6 of the 10 prognostic models). All studies were rated at high risk of bias, mostly because of non-representative selection of control patients, exclusion of patients who had not experienced the event of interest by the end of the study, and poor statistical analysis, including high risk of model overfitting. Reporting quality varied substantially between studies. A description of the study population and intended use of the models was absent in almost all reports, and calibration of predictions was rarely assessed. Conclusion: COVID-19 related prediction models are quickly entering the academic literature, to support medical decision making at a time where this is urgently needed. Our review indicates proposed models are poorly reported and at high risk of bias. Thus, their reported performance is likely optimistic and using them to support medical decision making is not advised. We call for immediate sharing of the individual participant data from COVID-19 studies to support collaborative efforts in building more rigorously developed prediction models and validating (evaluating) existing models. The aforementioned predictors identified in multiple included studies could be considered as candidate predictors for new models. We also stress the need to follow methodological guidance when developing and validating prediction models, as unreliable predictions may cause more harm than benefit when used to guide clinical decisions. Finally, studies should adhere to the TRIPOD statement to facilitate validating, appraising, advocating and clinically using the reported models. Systematic review registration protocol: osf.io/ehc47/ , registration: osf.io/wy245 Summary boxes: What is already known on this topic: - The sharp recent increase in COVID-19 infections has put a strain on healthcare systems worldwide, necessitating efficient early detection, diagnosis of patients suspected of the infection and prognostication of COVID-19 confirmed cases. - Viral nucleic acid testing and chest CT are standard methods for diagnosing COVID-19, but are time-consuming. - Earlier reports suggest that the elderly, patients with comorbidity (COPD, cardiovascular disease, hypertension), and patients presenting with dyspnoea are vulnerable to more severe morbidity and mortality after COVID-19 infection. What this study adds: - We identified three models to predict hospital admission from pneumonia and other events (as a proxy for COVID-19 pneumonia) in the general population. - We identified 18 diagnostic models for COVID-19 detection in symptomatic patients. - 13 of these were machine learning models based on CT images. - We identified ten prognostic models for COVID-19 infected patients, of which six aimed to predict mortality risk in confirmed or suspected COVID-19 patients, two aimed to predict progression to a severe or critical state, and two aimed to predict a hospital stay of more than 10 days from admission. - Included studies were poorly reported compromising their subsequent appraisal, and recommendation for use in daily practice. All studies were appraised at high risk of bias, raising concern that the models may be flawed and perform poorly when applied in practice, such that their predictions may be unreliable.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Medical guidelines rely on comprehensive systematic reviews, but the medical literature is growing at breakneck pace and the quality of the research is not universally adequate for summarization into policy 38 .”
From An open source machine learning framework for efficient and transparent systematic reviews (2021) · cites this paper as [38]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Medical guidelines rely on comprehensive systematic reviews, but the medical literature is growing at breakneck pace and the quality of the research is not universally adequate for summarization into policy 38 .”
From An open source machine learning framework for efficient and transparent systematic reviews (2021) · cites this paper as [38]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“within the first 15 months of the covid-19 pandemic, 381 prognostic prediction models for the disease were published.”
From PROBAST+AI: an updated quality, risk of bias, and applicability assessment tool for prediction models using regression or artificial intelligence methods (2025) · cites this paper as [5]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Medical guidelines rely on comprehensive systematic reviews, but the medical literature is growing at breakneck pace and the quality of the research is not universally adequate for summarization into policy 38 .”
From An open source machine learning framework for efficient and transparent systematic reviews (2021) · cites this paper as [38]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“At least 731 diagnostic and prognostic prediction model studies on covid-19 were published during the first 12 months of the pandemic.9”
From TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods (2024) · cites this paper as [9]
Experimental — accuracy not yet measured
In the citing paper
“Inadequate efforts were made to prioritise the health and safety of these workers, first in requiring precautions in their workplaces, and later in reducing structural barriers to accessing vaccines when they became available.”
From The Lancet Commission on lessons for the future from the COVID-19 pandemic (2022) · cites this paper as [262]
In the citing paper
“Other groups that have been disproportionately affected include people with comorbid medical conditions, particularly those with immunocompromised status; people living with physical disabilities or mental disorders; residents of congregant settings; and essential workers.”
From The Lancet Commission on lessons for the future from the COVID-19 pandemic (2022) · cites this paper as [262]
In the citing paper
“The inputs to the CDS model were Chest X-Ray (CXR) images, vital signs, demographic data, and lab values that were shown in previous publications to be predictive of COVID-19 patient outcomes18–21.”
From Federated learning for predicting clinical outcomes in patients with COVID-19 (2021) · cites this paper as [19]
In the citing paper
“older adults are particularly vulnerable to COVID-19 infection (4), especially if they are affected by comorbidities such as Alzheimer's disease (AD) (5).”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
1 retracted
Machine learning predictive model for severe COVID-19
Kang, J., Chen, T., Luo, H., Luo, Y., Du, G., & Yang, M. J. (2021) · Infection Genetics and Evolution · doi:10.1016/j.meegid.2021.104737 · Cited by 42
RetractedLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164
Zhou, F., Yu, T., Du, R., Fan, G., Liu, Y., Liu, Z., Xiang, J., Wang, Y., Song, B., Gu, X., et al. (2020) · The Lancet · doi:10.1016/s0140-6736(20)30566-3 · Cited by 27,263
An interactive web-based dashboard to track COVID-19 in real time
Dong, E., Du, H., & Gardner, L. (2020) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(20)30120-1 · Cited by 10,048
Moons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0698 · Cited by 5,147
Self-citationSeymour, C. W., Liu, V. X., Iwashyna, T. J., Brunkhorst, F. M., Rea, T. D., Scherag, A., Rubenfeld, G., Kahn, J. M., Shankar-Hari, M., Singer, M., et al. (2016) · JAMA · doi:10.1001/jama.2016.0288 · Cited by 3,939
Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating
Steyerberg, E. W. (2019) · Statistics for Biology and Health · doi:10.1007/978-3-030-16399-0 · Cited by 3,281
Self-citationLim, W. S. (2003) · Thorax · doi:10.1136/thorax.58.5.377 · Cited by 3,066
Calculating the sample size required for developing a clinical prediction model
Riley, R. D., Ensor, J., Snell, K. I. E., Harrell, F. E., Martin, G. P., Reitsma, J. B., Moons, K. G. M., Collins, G., & van Smeden, M. (2020) · BMJ · doi:10.1136/bmj.m441 · Cited by 2,917
Self-citationIntroduction to the Analysis of Survival Data in the Presence of Competing Risks
Austin, P. C., Lee, D. S., & Fine, J. P. (2016) · Circulation · doi:10.1161/circulationaha.115.017719 · Cited by 2,653
Cited by
Sorted by citations
Collins, G. S., Moons, K. G. M., Dhiman, P., Riley, R. D., Beam, A. L., Van Calster, B., Ghassemi, M., Liu, X., Reitsma, J. B., van Smeden, M., et al. (2024) · BMJ · doi:10.1136/bmj-2023-078378 · Cited by 4,718
Self-citationCoronavirus Disease 2019–COVID-19
Dhama, K., Sharun, K., Tiwari, R., Sircar, S., Bhat, S., Malik, Y. S., Singh, K. P., Chaicumpa, W., Bonilla‐Aldana, D. K., & Rodriguez-Morales, A. J. (2020) · Clinical Microbiology Reviews · doi:10.1128/cmr.00028-20 · Cited by 1,993
Immunology of COVID-19: Current State of the Science
Vabret, N., Britton, G. J., Gruber, C., Hegde, S., Kim, J., Kuksin, M., Levantovsky, R., Malle, L., Moreira, A., Park, M. D., et al. (2020) · Immunity · doi:10.1016/j.immuni.2020.05.002 · Cited by 1,623
Digital technologies in the public-health response to COVID-19
Budd, J., Miller, B. S., Manning, E. M., Lampos, V., Zhuang, M., Edelstein, M., Rees, G., Emery, V. C., Stevens, M. M., Keegan, N., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-1011-4 · Cited by 1,195
An open source machine learning framework for efficient and transparent systematic reviews
van de Schoot, R., de Bruin, J., Schram, R., Zahedi, P., de Boer, J., Weijdema, F., Kramer, B., Huijts, M., Hoogerwerf, M., Ferdinands, G., et al. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-020-00287-7 · Cited by 1,157
“NATuRe MAchiNe iNTelligeNce comparison with some of the existing systems, as shown in the last column of Table 1.”
Recent advances and clinical applications of deep learning in medical image analysis
Chen, X., Wang, X., Zhang, K., Fung, K.-M. A., Thai, T., Moore, K. N., Mannel, R. S., Liu, H., Zheng, B., & Qiu, Y. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102444 · Cited by 1,180
Moons, K. G. M., Damen, J. A. A., Kaul, T., Hooft, L., Andaur Navarro, C., Dhiman, P., Beam, A. L., Van Calster, B., Celi, L. A., Denaxas, S., et al. (2025) · BMJ · doi:10.1136/bmj-2024-082505 · Cited by 1,069
Self-citationRoberts, M., Driggs, D., Thorpe, M., Gilbey, J., Yeung, M., Ursprung, S., Aviles-Rivero, A. I., Etmann, C., McCague, C., Beer, L., et al. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00307-0 · Cited by 1,044
Knight, S. R., Ho, A. Y. W., Pius, R., Buchan, I. E., Carson, G., Drake, T. M., Dunning, J. W., Fairfield, C. J., Gamble, C. L., Green, C., et al. (2020) · BMJ · doi:10.1136/bmj.m3339 · Cited by 1,039
COVID-19 Image Data Collection: Prospective Predictions are the Future
Cohen, J., Morrison, P., Dao, L., Roth, K., Duong, T. Q., & Ghassemi, M. (2020) · The Journal of Machine Learning for Biomedical Imaging · doi:10.59275/j.melba.2020-48g7 · Cited by 1,110
Roberts, M. T., Driggs, D., Thorpe, M., Gilbey, J., Yeung, M., Ursprung, S., Avilés-Rivero, A. I., Etmann, C., McCague, C., Beer, L., et al. (2020) · Apollo (University of Cambridge) · doi:10.17863/cam.64000 · Cited by 902
Collins, G. S., Dhiman, P., Andaur Navarro, C. L., Ma, J., Hooft, L., Reitsma, J. B., Logullo, P., Beam, A. L., Peng, L., Van Calster, B., et al. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-048008 · Cited by 852
Self-citationFederated learning for predicting clinical outcomes in patients with COVID-19
Dayan, I., Roth, H. R., Zhong, A., Harouni, A., Gentili, A., Abidin, A. Z., Liu, A., Costa, A. B., Wood, B. J., Tsai, C.-S., et al. (2021) · Nature Medicine · doi:10.1038/s41591-021-01506-3 · Cited by 842
The Lancet Commission on lessons for the future from the COVID-19 pandemic
Sachs, J. D., Karim, S. S. A., Aknin, L., Allen, J., Brosbøl, K., Colombo, F., Barron, G. C., Espinosa, M. F., Gaspar, V., Gaviria, A., et al. (2022) · The Lancet · doi:10.1016/s0140-6736(22)01585-9 · Cited by 842
Collider bias undermines our understanding of COVID-19 disease risk and severity
Griffith, G. J., Morris, T. T., Tudball, M. J., Herbert, A., Mancano, G., Pike, L., Sharp, G. C., Sterne, J., Palmer, T. M., Davey Smith, G., et al. (2020) · Nature Communications · doi:10.1038/s41467-020-19478-2 · Cited by 826
Considerations for diagnostic COVID-19 tests
Vandenberg, O., Martiny, D., Rochas, O., van Belkum, A., & Kozlakidis, Z. (2020) · Nature Reviews Microbiology · doi:10.1038/s41579-020-00461-z · Cited by 820
Evaluation of clinical prediction models (part 1): from development to external validation
Collins, G. S., Dhiman, P., Ma, J., Schlussel, M. M., Archer, L., Van Calster, B., Harrell, F. E., Martin, G. P., Moons, K. G. M., van Smeden, M., et al. (2024) · BMJ · doi:10.1136/bmj-2023-074819 · Cited by 798
Self-citationCOVID-19 Neutralizing Antibodies Predict Disease Severity and Survival
García-Beltrán, W. F., Lam, E. C., Astudillo, M. G., Yang, D., Miller, T. E., Feldman, J. E., Hauser, B. M., Caradonna, T. M., Clayton, K., Nitido, A. D., et al. (2020) · Cell · doi:10.1016/j.cell.2020.12.015 · Cited by 708
Izcovich, A., Ragusa, M. A., Tortosa, F. G., Marzio, M. A. L., Agnoletti, C., Bengolea, A., Ceirano, A., Espinosa, F., Saavedra, E., Sanguine, V., et al. (2020) · PLoS ONE · doi:10.1371/journal.pone.0241955 · Cited by 676
Fairness of artificial intelligence in healthcare: review and recommendations
Ueda, D., Kakinuma, T., Fujita, S., Kamagata, K., Fushimi, Y., Ito, R., Matsui, Y., Nozaki, T., Nakaura, T., Fujima, N., et al. (2024) · Japanese Journal of Radiology · doi:10.1007/s11604-023-01474-3 · Cited by 607
Pijls, B. G. C. W., Jolani, S., Atherley, A., Derckx, R. T., Dijkstra, J. I. R., Franssen, G. H. L. M., Hendriks, S., Richters, A., Jellema, A., Zalpuri, S., et al. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-044640 · Cited by 596
Dennis, A., Wamil, M., Alberts, J., Oben, J. A., Cuthbertson, D. J., Wootton, D. G., Crooks, M. G., Gabbay, M., Brady, M., Hishmeh, L., et al. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-048391 · Cited by 587
Booth, A., Reed, A. B., Ponzo, S., Yassaee, A., Aral, M., Plans, D., Labrique, A. B., & Mohan, D. (2021) · PLoS ONE · doi:10.1371/journal.pone.0247461 · Cited by 582
Towards a standard for identifying and managing bias in artificial intelligence
Schwartz, R., Vassilev, A., Greene, K., Perine, L., Burt, A., & Hall, P. (2022) · doi:10.6028/nist.sp.1270 · Cited by 548
“An analysis that examined hundreds of AI algorithms for identifying COVID found that few of them were effective [193].”
MethodDeveloping clinical prediction models: a step-by-step guide
Efthimiou, O., Seo, M., Chalkou, K., Debray, T., Egger, M., & Salanti, G. (2024) · BMJ · doi:10.1136/bmj-2023-078276 · Cited by 544
Self-citationClift, A. K., Coupland, C. A. C., Keogh, R. H., Diaz‐Ordaz, K., Williamson, E., Harrison, E. M., Hayward, A. C., Hemingway, H., Horby, P., Mehta, N., et al. (2020) · BMJ · doi:10.1136/bmj.m3731 · Cited by 551
JCS: An Explainable COVID-19 Diagnosis System by Joint Classification and Segmentation
Wu, Y.-H., Gao, S.-H., Mei, J., Xu, J., Fan, D.-P., Zhang, R.-G., & Cheng, M.-M. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3058783 · Cited by 516
The myth of generalisability in clinical research and machine learning in health care
Futoma, J., Simons, M., Panch, T., Doshi-Velez, F., & Celi, L. A. (2020) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30186-2 · Cited by 467
The current and future state of AI interpretation of medical images
Rajpurkar, P., & Lungren, M. P. (2023) · New England Journal of Medicine · doi:10.1056/nejmra2301725 · Cited by 443
Chest CT in COVID-19: What the Radiologist Needs to Know
Kwee, T. C., & Kwee, R. M. (2020) · Radiographics · doi:10.1148/rg.2020200159 · Cited by 451
Andaur Navarro, C. L., Damen, J. A. A., Takada, T., Nijman, S. W. J., Dhiman, P., Ma, J., Collins, G. S., Bajpai, R., Riley, R. D., Moons, K. G. M., et al. (2021) · BMJ · doi:10.1136/bmj.n2281 · Cited by 415
Self-citationMinimum sample size for external validation of a clinical prediction model with a binary outcome
Riley, R. D., Debray, T. P. A., Collins, G. S., Archer, L., Ensor, J., van Smeden, M., & Snell, K. I. E. (2021) · Statistics in Medicine · doi:10.1002/sim.9025 · Cited by 400
Self-citationMultiplexed Nanomaterial-Based Sensor Array for Detection of COVID-19 in Exhaled Breath
Shan, B., Broza, Y. Y., Li, W., Wang, Y., Wu, S., Liu, Z., Wang, J., Gui, S., Wang, L., Zhang, Z., et al. (2020) · ACS Nano · doi:10.1021/acsnano.0c05657 · Cited by 384
Lim, Z. J., Subramaniam, A., Reddy, M. P., Blecher, G., Kadam, U., Afroz, A., Billah, B., Ashwin, S., Kubicki, M. A., Bilotta, F., et al. (2020) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.202006-2405oc · Cited by 372
Detection of COVID-19 Infection from Routine Blood Exams with Machine Learning: A Feasibility Study
Brinati, D., Campagner, A., Ferrari, D., Locatelli, M., Banfi, G., & Cabitza, F. (2020) · Journal of Medical Systems · doi:10.1007/s10916-020-01597-4 · Cited by 365
Low testosterone levels predict clinical adverse outcomes in SARS‐CoV‐2 pneumonia patients
Rastrelli, G., Stasi, V. D., Inglese, F., Beccaria, M., Garuti, M., Costanzo, D. D., Spreafico, F., Greco, G. F., Cervi, G., Pecoriello, A., et al. (2020) · Andrology · doi:10.1111/andr.12821 · Cited by 363
A model of disparities: risk factors associated with COVID-19 infection
Rozenfeld, Y., Beam, J., Maier, H., Haggerson, W., Boudreau, K., Carlson, J., & Medows, R. (2020) · International Journal for Equity in Health · doi:10.1186/s12939-020-01242-z · Cited by 360
Comparison of deep learning approaches to predict COVID-19 infection
Alakuş, T. B., & Türkoğlu, İ. (2020) · Chaos Solitons & Fractals · doi:10.1016/j.chaos.2020.110120 · Cited by 357
Machine learning based early warning system enables accurate mortality risk prediction for COVID-19
Gao, Y., Cai, G., Fang, W., Li, H., Wang, S., Chen, L., Yu, Y., Liu, D., Xu, S., Cui, P., et al. (2020) · Nature Communications · doi:10.1038/s41467-020-18684-2 · Cited by 358
COVID-19: clinical course and outcomes of 36 hemodialysis patients in Spain
Goicoechea, M., Cámara, L. A. S., Macías, N., de Morales, A. M., Rojas, Á. G., Bascuñana, A., Arroyo, D., Vega, A., Abad, S., Verde, E., et al. (2020) · Kidney International · doi:10.1016/j.kint.2020.04.031 · Cited by 337
Mohamadou, Y., Halidou, A., & Kapen, P. T. (2020) · Applied Intelligence · doi:10.1007/s10489-020-01770-9 · Cited by 341
A blood atlas of COVID-19 defines hallmarks of disease severity and specificity
Ahern, D. J., Ai, Z., Ainsworth, M., Allan, C., Allcock, A., Angus, B., Ansari, M. A., Arancibia-Cárcamo, C. V., Aschenbrenner, D., Attar, M., et al. (2022) · Cell · doi:10.1016/j.cell.2022.01.012 · Cited by 328
Supervised Machine Learning Models for Prediction of COVID-19 Infection using Epidemiology Dataset
Muhammad, L. J., Algehyne, E. A., Usman, S. S., Ahmad, A., Chakraborty, C., & Mohammed, I. A. (2020) · SN Computer Science · doi:10.1007/s42979-020-00394-7 · Cited by 326
Long-term consequences of COVID-19: research needs
Yelin, D., Wirtheim, E., Vetter, P., Kalil, A. C., Bruchfeld, J., Runold, M., Guaraldi, G., Mussini, C., Gudiol, C., Pujol, M., et al. (2020) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(20)30701-5 · Cited by 316
Grivas, P. D., Khaki, A. R., Wise-Draper, T. M., French, B. C., Hennessy, C., Hsu, C., Shyr, Y., Li, X., Choueiri, T. K., Painter, C., et al. (2021) · Annals of Oncology · doi:10.1016/j.annonc.2021.02.024 · Cited by 307
Clinical features of COVID-19 mortality: development and validation of a clinical prediction model
Yadaw, A. S., Li, Y. C., Bose, S., Iyengar, R., Bunyavanich, S., & Pandey, G. S. (2020) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30217-x · Cited by 320
Uncertainty and COVID-19: how are we to respond?
Koffman, J., Gross, J., Etkind, S. N., & Selman, L. (2020) · Journal of the Royal Society of Medicine · doi:10.1177/0141076820930665 · Cited by 305
Haimovich, A. D., Ravindra, N. G., Stoytchev, S., Young, H. P., Wilson, F. P., van Dijk, D., Schulz, W. L., & Taylor, R. A. (2020) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2020.07.022 · Cited by 294
A Review on Deep Learning Techniques for the Diagnosis of Novel Coronavirus (COVID-19)
Islam, M. M., Karray, F. O., Alhajj, R., & Zeng, J. (2021) · IEEE Access · doi:10.1109/access.2021.3058537 · Cited by 299
Artificial intelligence in surgery
Varghese, C., Harrison, E. M., O’Grady, G., & Topol, E. J. (2024) · Nature Medicine · doi:10.1038/s41591-024-02970-3 · Cited by 287
Cabitza, F., Campagner, A., Soares, F., García de Guadiana-Romualdo, L., Challa, F., Sulejmani, A., Seghezzi, M., & Carobene, A. (2021) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2021.106288 · Cited by 284
Viral Cultures for Coronavirus Disease 2019 Infectivity Assessment: A Systematic Review
Jefferson, T., Spencer, E. A., Brassey, J., & Heneghan, C. J. (2020) · Clinical Infectious Diseases · doi:10.1093/cid/ciaa1764 · Cited by 278
COVID-19 and Cancer: Current Challenges and Perspectives
Bakouny, Z., Hawley, J. E., Choueiri, T. K., Peters, S., Rini, B. I., Warner, J. L., & Painter, C. (2020) · Cancer Cell · doi:10.1016/j.ccell.2020.09.018 · Cited by 276
COVID-19 open source data sets: a comprehensive survey
Shuja, J., Alanazi, E., Alasmary, W., & Alashaikh, A. S. (2020) · Applied Intelligence · doi:10.1007/s10489-020-01862-6 · Cited by 278
Leveraging Data Science to Combat COVID-19: A Comprehensive Review
Latif, S., Usman, M., Manzoor, S., Iqbal, W., Qadir, J., Gareth Tyson, A., Castro, I., Razi, A., Boulos, M. N. K., Weller, A., et al. (2020) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2020.3020521 · Cited by 273
Pandemic publishing poses a new COVID-19 challenge
Palayew, A., Norgaard, O., Safreed-Harmon, K., Andersen, T. H., Rasmussen, L. N., & Lazarus, J. V. (2020) · Nature Human Behaviour · doi:10.1038/s41562-020-0911-0 · Cited by 258
Predicting COVID-19 Pneumonia Severity on Chest X-ray With Deep Learning
Cohen, J., Dao, L., Roth, K., Morrison, P., Bengio, Y., Abbasi, A., Shen, B., Mahsa, H. K., Ghassemi, M., Li, H., et al. (2020) · Cureus · doi:10.7759/cureus.9448 · Cited by 261
Manson, J. J., Crooks, C. J., Naja, M., Ledlie, A., Goulden, B., Liddle, T., Khan, E., Mehta, P., Martin‐Gutierrez, L., Waddington, K. E., et al. (2020) · The Lancet Rheumatology · doi:10.1016/s2665-9913(20)30275-7 · Cited by 289
Clinical update on COVID-19 for the emergency clinician: Presentation and evaluation
Long, B., Carius, B. M., Chavez, S., Liang, S. Y., Brady, W. J., Koyfman, A., & Gottlieb, M. A. (2022) · The American Journal of Emergency Medicine · doi:10.1016/j.ajem.2022.01.028 · Cited by 277
The Fallacy of AI Functionality
Raji, I. D., Kumar, I. E., Horowitz, A., & Selbst, A. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533158 · Cited by 251
SARS-CoV-2 was already spreading in France in late December 2019
Deslandes, A., Berti, V., Tandjaoui-Lambiotte, Y., Alloui, C., Carbonnelle, É., Zahar, J., Brichler, S., & Cohen, Y. (2020) · International Journal of Antimicrobial Agents · doi:10.1016/j.ijantimicag.2020.106006 · Cited by 243
Kim, M. S., An, M. H., Kim, W. J., & Hwang, T.-H. (2020) · PLoS Medicine · doi:10.1371/journal.pmed.1003501 · Cited by 251
The promise of digital healthcare technologies
Yeung, A. W. K., Torkamani, A., Butte, A. J., Glicksberg, B. S., Schuller, B. W., Rodríguez, B., Ting, D. S. W., Bates, D. W., Schaden, E., Peng, H., et al. (2023) · Frontiers in Public Health · doi:10.3389/fpubh.2023.1196596 · Cited by 275
Binuya, M. A. E., Engelhardt, E. G., Schats, W., Schmidt, M. K., & Steyerberg, E. W. (2022) · BMC Medical Research Methodology · doi:10.1186/s12874-022-01801-8 · Cited by 246
Self-citationCOVID-19 in older people: a rapid clinical review
Lithander, F. E., Neumann, S., Tenison, E., Lloyd, K., Welsh, T. J., Rodrigues, J. C. L., Higgins, J. P. T., Scourfield, L., Christensen, H., Haunton, V. J., et al. (2020) · Age and Ageing · doi:10.1093/ageing/afaa093 · Cited by 250
Symptoms in Health Care Workers during the COVID-19 Epidemic. A Cross-Sectional Survey
Magnavita, N., Tripepi, G. L., & Prinzio, R. R. D. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17145218 · Cited by 234
Webb, G. J., Marjot, T., Cook, J. A., Aloman, C., Armstrong, M. J., Brenner, E. J., Catana, M.-A., Cargill, T. N., Dhanasekaran, R., García‐Juárez, I., et al. (2020) · The Lancet. Gastroenterology & hepatology · doi:10.1016/s2468-1253(20)30271-5 · Cited by 237
Vaid, A., Somani, S. S., Russak, A. J., Freitas, J. K. D., Chaudhry, F. F., Paranjpe, I., Johnson, K. W., Lee, S. J., Miotto, R., Zhao, S., et al. (2020) · Journal of Medical Internet Research · doi:10.2196/24018 · Cited by 238
Manca, R., De Marco, M., & Venneri, A. (2020) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2020.585540 · Cited by 227
Digital health and care in pandemic times: impact of COVID-19
Peek, N. B., Sujan, M. A., & Scott, P. (2020) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2020-100166 · Cited by 225
Methylprednisolone in adults hospitalized with COVID-19 pneumonia
Corral-Gudino, L., Bahamonde, A., Arnaiz-Revillas, F., Barquero, J. G., Abadía-Otero, J., García‐Ibarbia, C., Mora, V., Cerezo-Hernández, A., Cerezo-Hernández, A., López-Muñíz, G., et al. (2021) · Wiener klinische Wochenschrift · doi:10.1007/s00508-020-01805-8 · Cited by 218
Stability of clinical prediction models developed using statistical or machine learning methods
Riley, R. D., & Collins, G. S. (2023) · Biometrical Journal · doi:10.1002/bimj.202200302 · Cited by 211
Self-citationGupta, R. K., Harrison, E. M., Ho, A. Y. W., Docherty, A. B., Knight, S. R., van Smeden, M., Abubakar, I. I., Lipman, M. C. I., Quartagno, M., Pius, R., et al. (2021) · The Lancet Respiratory Medicine · doi:10.1016/s2213-2600(20)30559-2 · Cited by 223
International electronic health record-derived COVID-19 clinical course profiles: the 4CE consortium
Brat, G. A., Weber, G. M., Gehlenborg, N., Avillach, P., Palmer, N., Chiovato, L., Cimino, J. J., Waitman, L. R., Omenn, G. S., Malovini, A. L., et al. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-00308-0 · Cited by 211
Galloway, J. B., Norton, S., Barker, R. D., Brookes, A., Carey, I., Clarke, B. D., Jina, R., Reid, C., Russell, M. D., Sneep, R., et al. (2020) · Journal of Infection · doi:10.1016/j.jinf.2020.05.064 · Cited by 206
Trends in Using IoT with Machine Learning in Health Prediction System
Aldahiri, A., Alrashed, B., & Hussain, W. (2021) · Forecasting · doi:10.3390/forecast3010012 · Cited by 222
Snell, K. I. E., Levis, B., Damen, J. A., Dhiman, P., Debray, T. P. A., Hooft, L., Reitsma, J. B., Moons, K. G. M., Collins, P. G. S., & Riley, R. D. (2023) · BMJ · doi:10.1136/bmj-2022-073538 · Cited by 214
Self-citationRecommendations for antibacterial therapy in adults with COVID-19 – an evidence based guideline
Sieswerda, E., de Boer, M. G. J., Bonten, M. J. M., Boersma, W. G., Jonkers, R. E., Aleva, R. M., Kullberg, B. J., Schouten, J. A., van de Garde, E. M. W., Verheij, T. J., et al. (2020) · Clinical Microbiology and Infection · doi:10.1016/j.cmi.2020.09.041 · Cited by 199
Self-citationA time-resolved proteomic and prognostic map of COVID-19
Demichev, V., Tober-Lau, P., Lemke, O., Nazarenko, T., Thibeault, C., Whitwell, H., Röhl, A., Freiwald, A., Szyrwiel, L., Ludwig, D., et al. (2021) · Cell Systems · doi:10.1016/j.cels.2021.05.005 · Cited by 200
Gupta, R. K., Marks, M., Samuels, T. H. A., Luintel, A., Rampling, T., Chowdhury, H., Quartagno, M., Nair, A., Lipman, M. C. I., Abubakar, I. I., et al. (2020) · European Respiratory Journal · doi:10.1183/13993003.03498-2020 · Cited by 197
Minimum sample size for external validation of a clinical prediction model with a continuous outcome
Archer, L., Snell, K. I. E., Ensor, J., Hudda, M. T., Collins, G. S., & Riley, R. D. (2021) · Statistics in Medicine · doi:10.1002/sim.8766 · Cited by 198
Self-citationCOVID-19 diagnosis prediction in emergency care patients: a machine learning approachPreprint
Batista, A. F. D. M., Miraglia, J. L., Donato, T. H. R., & Filho, A. D. P. C. (2020) · medRxiv · doi:10.1101/2020.04.04.20052092 · Cited by 199
Lim, Z. J., Reddy, M. P., Afroz, A., Billah, B., Shekar, K., & Subramaniam, A. (2020) · Resuscitation · doi:10.1016/j.resuscitation.2020.10.025 · Cited by 196
Lassau, N., Ammari, S., Chouzenoux, É., Gortais, H., Hérent, P., Devilder, M., Soliman, S., Meyrignac, O., Talabard, M.-P., Lamarque, J.-P., et al. (2021) · Nature Communications · doi:10.1038/s41467-020-20657-4 · Cited by 187
Health Care 4.0: A vision for smart and connected health care
Li, J., & Carayon, P. (2021) · IISE Transactions on Healthcare Systems Engineering · doi:10.1080/24725579.2021.1884627 · Cited by 190
Machine learning in precision diabetes care and cardiovascular risk prediction
Oikonomou, E. K., & Khera, R. (2023) · Cardiovascular Diabetology · doi:10.1186/s12933-023-01985-3 · Cited by 190
Wehbe, R. M., Sheng, J., Dutta, S., Chai, S., Dravid, A., Barutcu, S., Wu, Y., Cantrell, D. R., Xiao, N., Allen, B. D., et al. (2021) · Radiology · doi:10.1148/radiol.2020203511 · Cited by 190
Cecconi, M., Piovani, D., Brunetta, E., Aghemo, A., Greco, M., Ciccarelli, M., Angelini, C., Voza, A., Omodei, P., Vespa, E., et al. (2020) · Journal of Clinical Medicine · doi:10.3390/jcm9051548 · Cited by 194
Blood molecular markers associated with COVID‐19 immunopathology and multi‐organ damage
Chen, Y., Zheng, Y., Yu, Y., Wang, Y., Huang, Q., Qian, F., Sun, L., Song, Z., Chen, Z., Feng, J., et al. (2020) · The EMBO Journal · doi:10.15252/embj.2020105896 · Cited by 190
Impact of the COVID-19 pandemic on publication dynamics and non-COVID-19 research production
Raynaud, M., Goutaudier, V., Louis, K., Awadhi, S. A., Dubourg, Q., Truchot, A., Brousse, R., Saleh, N., Giarraputo, A., Debiais, C., et al. (2021) · BMC Medical Research Methodology · doi:10.1186/s12874-021-01404-9 · Cited by 202
Satıcı, C., Demirkol, M. A., Altunok, E. S., Gürsoy, B., Alkan, M., Kamat, S., Demirok, B., Sürmeli, C. D., Çalık, M., Çavuş, Z., et al. (2020) · International Journal of Infectious Diseases · doi:10.1016/j.ijid.2020.06.038 · Cited by 185
The effects of COVID-19 on perioperative morbidity and mortality in patients with hip fractures
Kayani, B., Onochie, E., Patil, V., Begum, F., Cuthbert, R., Ferguson, D. J. P., Bhamra, J. S., Sharma, A., Bates, P., & Haddad, F. S. (2020) · The Bone & Joint Journal · doi:10.1302/0301-620x.102b9.bjj-2020-1127.r1 · Cited by 184
Castelnuovo, A. F. D., Bonaccio, M., Costanzo, S., Gialluisi, A., Antinori, A., Berselli, N., Blandi, L., Bruno, R., Cauda, R., Guaraldi, G., et al. (2020) · Nutrition Metabolism and Cardiovascular Diseases · doi:10.1016/j.numecd.2020.07.031 · Cited by 183
Sanchez-Guijo, F. M., García‐Arranz, M., López-Parra, M., Monedero, P., Mata-Martínez, C., Santos, A., Sagredo-Meneses, V., Álvarez-Avello, J. M., Guerrero, J. E., Pérez–Calvo, C., et al. (2020) · EClinicalMedicine · doi:10.1016/j.eclinm.2020.100454 · Cited by 177
Development of a prognostic model for mortality in COVID-19 infection using machine learning
Booth, A. L., Abels, E. A., & McCaffrey, P. (2020) · Modern Pathology · doi:10.1038/s41379-020-00700-x · Cited by 187
Does “ AI ” stand for augmenting inequality in the era of covid-19 healthcare?
Leslie, D., Mazumder, A., Peppin, A., Wolters, M. K., & Hagerty, A. (2021) · BMJ · doi:10.1136/bmj.n304 · Cited by 177
Predictors of COVID-19 severity: a systematic review and meta-analysis
Mudatsir, M., Fajar, J. K., Wulandari, L., Soegiarto, G., Ilmawan, M., Purnamasari, Y., Mahdi, B. A., Jayanto, G. D., Suhendra, S., Setianingsih, Y. A., et al. (2020) · F1000Research · doi:10.12688/f1000research.26186.2 · Cited by 211
COVID-19-related medical research: a meta-research and critical appraisal
Raynaud, M., Zhang, H., Louis, K., Goutaudier, V., Wang, J., Dubourg, Q., Wei, Y., Demir, Z., Debiais, C., Aubert, O., et al. (2021) · BMC Medical Research Methodology · doi:10.1186/s12874-020-01190-w · Cited by 179
EDL-COVID: Ensemble Deep Learning for COVID-19 Case Detection From Chest X-Ray Images
Tang, S., Wang, C., Nie, J., Kumar, N., Zhang, Y., Xiong, Z., & Barnawi, A. (2021) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2021.3057683 · Cited by 175
Metadata
- Authors
- Wynants, L., Van Calster, B., Bonten, M. M., Collins, G. S., Debray, T. P., De Vos, M., Haller, M. C., Heinze, G., Moons, K. G., Riley, R. D., Schuit, E., Smits, L. J., Snell, K. I., Steyerberg, E. W., Wallisch, C., & van Smeden, M.
- Affiliations
- Maastricht University (NL)
- Utrecht University (NL)
- Krankenhaus der Elisabethinen (AT)
- Medical University of Vienna (AT)
- Keele University (GB)
- Leiden University Medical Center (NL)
- Humboldt-Universität zu Berlin (DE)
- Corresponding author
- Wynants, Laure
- ORCID
- Venue
- BMJ
- Publisher
- openRxiv
- Published
- 2020-03-27
- Volume
- 369
- Pages
- m1328-m1328
- ISSN
- 1756-1833 (ISSN-L)
- PMC
- PMC7222643
- OpenAlex
- W3014524604
- Cited by
- 3,070
- References
- 433
- Citations by year
- 2026: 191 · 2025: 185 · 2024: 261 · 2023: 402 · 2022: 619 · 2021: 880 · 2020: 531
- FWCI
- 28.74
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- BMJ (publishedVersion, cc-by)
- medRxiv (acceptedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Amsterdam UMC (VU Amsterdam) - Institutional Repository (publishedVersion, other-oa)
- University of Birmingham Research Portal (University of Birmingham) (publishedVersion)
- St George's Online Research Archive (St George's University of London) (acceptedVersion)
- Leiden Repository (Leiden University) (submittedVersion, cc-by)
- Aberdeen University Research Archive (Aberdeen University) (submittedVersion, cc-by)
- Keele Research Repository (Keele University) (submittedVersion, cc-by)
- Lirias (KU Leuven) (submittedVersion, cc-by)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Data Archiving and Networked Services (DANS) (submittedVersion, cc-by)
- Research Publications (Maastricht University) (submittedVersion, cc-by)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- HighWire Press Open Archive (submittedVersion)
- Pure Amsterdam UMC (submittedVersion, cc-by)
- Research Explorer (The University of Manchester) (submittedVersion, cc-by)
- Research Publications (Maastricht University) (submittedVersion, other-oa)
- Funders
- Cancer Research UK (27294, 16895, C49297/A27294, C49297)
- National Institute for Health and Care Research (SPCR-142, COVID-19, C49297/A27294)
- Department of Health and Social Care (COVID-19)
- European Commission (825746, HORIZON2020, COVID-19)
- ZonMw (91617050, 10430012010001, grant 91617050)
- Fonds Wetenschappelijk Onderzoek (G0B4716N)
- KU Leuven (C24/15/037)
- Vlaamse regering (C24/15/037)
- Oxford Health NHS Foundation Trust
- National Institutes of Health (COVID-19)
- NIHR Oxford Biomedical Research Centre (C49297/ A27294)
- NIHR School for Primary Care Research
- National Heart, Lung, and Blood Institute (R00HL141678)
- National Institute of General Medical Sciences (P20GM125498)
- Interreg
- Topics
- Health Sciences › Medicine › Infectious Diseases › COVID-19 Clinical Research Studies
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › COVID-19 diagnosis using AI
- Keywords
- COVID-19 pandemic, prediction model, diagnostic model, prognostic model, risk prediction, model validation, risk of bias, concordance index, diagnostic imaging, disease severity, mortality prediction, intensive care unit admission, hospitalization risk, overfitting, reporting quality, preprints, charm, PROBAST
- MeSH terms
- COVID-19, Hospitalization, Humans, Models, Theoretical, Pneumonia, Viral, Prognosis, Multivariate Analysis, Coronavirus, Coronavirus Infections, Disease Progression, Pandemics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
