The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Forecasting for COVID-19 has failed
Open accessAbstract
Epidemic forecasting has a dubious track-record, and its failures became more prominent with COVID-19. Poor data input, wrong modeling assumptions, high sensitivity of estimates, lack of incorporation of epidemiological features, poor past evidence on effects of available interventions, lack of transparency, errors, lack of determinacy, consideration of only one or a few dimensions of the problem at hand, lack of expertise in crucial disciplines, groupthink and bandwagon effects, and selective reporting are some of the causes of these failures. Nevertheless, epidemic forecasting is unlikely to be abandoned. Some (but not all) of these problems can be fixed. Careful modeling of predictive distributions rather than focusing on point estimates, considering multiple dimensions of impact, and continuously reappraising models based on their validated performance may help. If extreme values are considered, extremes should be considered for the consequences of multiple dimensions of impact so as to continuously calibrate predictive insights and decision-making. When major decisions (e.g. draconian lockdowns) are based on forecasts, the harms (in terms of health, economy, and society at large) and the asymmetry of risks need to be approached in a holistic fashion, considering the totality of the evidence.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Collaborative, T. O., Williamson, E., Walker, A. J., Bhaskaran, K., Bacon, S., Bates, C., Morton, C. E., Curtis, H. J., Mehrkar, A., Evans, D., et al. (2020) · doi:10.1101/2020.05.06.20092999 · Cited by 6,674 · cited 3×
“Based on available prognostic models (e.g. Williamson et al. (2020)), this prognostic classification should be readily attainable.”
InfluentialFactors associated with COVID-19-related death using OpenSAFELY
Williamson, E. J., Walker, A. J., Bhaskaran, K., Bacon, S., Bates, C., Morton, C. E., Curtis, H. J., Mehrkar, A., Evans, D., Inglesby, P., et al. (2020) · Nature · doi:10.1038/s41586-020-2521-4 · Cited by 6,674
Estimating the effects of non-pharmaceutical interventions on COVID-19 in Europe
Flaxman, S., Mishra, S., Gandy, A., Unwin, H. J. T., Mellan, T. A., Coupland, H., Whittaker, C., Zhu, H., Berah, T., Eaton, J. W., et al. (2020) · Nature · doi:10.1038/s41586-020-2405-7 · Cited by 3,350
Presymptomatic SARS-CoV-2 Infections and Transmission in a Skilled Nursing Facility
Arons, M. M., Hatfield, K. M., Reddy, S. C., Kimball, A., James, A., Jacobs, J. R., Taylor, J., Spicer, K., Bardossy, A. C., Oakley, L. P., et al. (2020) · New England Journal of Medicine · doi:10.1056/nejmoa2008457 · Cited by 2,052
Asymptomatic Transmission, the Achilles’ Heel of Current Strategies to Control Covid-19
Gandhi, M., Yokoe, D. S., & Havlir, D. V. (2020) · New England Journal of Medicine · doi:10.1056/nejme2009758 · Cited by 1,094
Reduced Rate of Hospital Admissions for ACS during Covid-19 Outbreak in Northern Italy
De Filippo, O., D’Ascenzo, F., Angelini, F., Bocchino, P. P., Conrotto, F., Saglietto, A., Secco, G. G., Campo, G., Gallone, G., Verardi, R., et al. (2020) · New England Journal of Medicine · doi:10.1056/nejmc2009166 · Cited by 1,020
Physical interventions to interrupt or reduce the spread of respiratory viruses
Jefferson, T., Del Mar, C. B., Dooley, L., Ferroni, E., Al-Ansary, L. A., Bawazeer, G. A., van Driel, M. L., Nair, N. S., Jones, M. A., Thorning, S., et al. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006207.pub4 · Cited by 805
Huff, D., & Geis, I. (1954) · Cited by 762
The Foot-and-Mouth Epidemic in Great Britain: Pattern of Spread and Impact of Interventions
Ferguson, N. M., Donnelly, C. A., & Anderson, R. M. (2001) · Science · doi:10.1126/science.1061020 · Cited by 671
Tools for Statistical Inference
Rukhin, A. L. (1997) · Technometrics · doi:10.1080/00401706.1997.10485139 · Cited by 659
Vijgen, L., Keyaerts, E., Moës, E., Thoelen, I., Wollants, E., Lemey, P., Vandamme, A.-M., & Van Ranst, M. (2005) · Journal of Virology · doi:10.1128/jvi.79.3.1595-1604.2005 · Cited by 594
Collateral damage: the impact on outcomes from cancer surgery of the COVID-19 pandemic
Sud, A., Jones, M. E., Broggio, J., Loveday, C., Torr, B., Garrett, A., Nicol, D. L., Jhanji, S., Boyce, S. A., Gronthoud, F., et al. (2020) · Annals of Oncology · doi:10.1016/j.annonc.2020.05.009 · Cited by 552
Five ways to ensure that models serve society: a manifesto
Saltelli, A., Bammer, G., Bruno, I., Charters, E., Di Fiore, M., Didier, E., Nelson Espeland, W., Kay, J., Lo Piano, S., Mayo, D., et al. (2020) · Nature · doi:10.1038/d41586-020-01812-9 · Cited by 486
Ferguson, N. M., Donnelly, C. A., & Anderson, R. M. (2001) · Nature · doi:10.1038/35097116 · Cited by 431
Wrong but Useful — What Covid-19 Epidemiologic Models Can and Cannot Tell Us
Holmdahl, I., & Buckee, C. (2020) · New England Journal of Medicine · doi:10.1056/nejmp2016822 · Cited by 431
IHME COVID-19 health service utilization forecasting team, & Murray, C. J. (2020) · medRxiv · doi:10.1101/2020.03.27.20043752 · Cited by 417
Coronavirus disease 2019: The harms of exaggerated information and non‐evidence‐based measures
Ioannidis, J. P. A. (2020) · European Journal of Clinical Investigation · doi:10.1111/eci.13222 · Cited by 364
Self-citationWhat Other Countries Can Learn From Italy During the COVID-19 Pandemic
Boccia, S., Ricciardi, W., & Ioannidis, J. P. A. (2020) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2020.1447 · Cited by 335
Self-citationIoannidis, J. P. A., Axfors, C., & Contopoulos-Ioannidis, D. G. (2020) · Environmental Research · doi:10.1016/j.envres.2020.109890 · Cited by 331
Self-citationCOVID-19: PCR screening of asymptomatic health-care workers at London hospital
Treibel, T. A., Manisty, C., Burton, M., McKnight, Á., Lambourne, J., Augusto, J. B., Couto-Parada, X., Cutino-Moguel, T., Noursadeghi, M., & Moon, J. C. (2020) · The Lancet · doi:10.1016/s0140-6736(20)31100-4 · Cited by 314
Jewell, N. P., Lewnard, J. A., & Jewell, B. L. (2020) · JAMA · doi:10.1001/jama.2020.6585 · Cited by 306
Tail risk of contagious diseases
Cirillo, P., & Taleb, N. N. (2020) · Nature Physics · doi:10.1038/s41567-020-0921-x · Cited by 222
ECDC Public Health Emergency Team, Danis, K., Fonteneau, L., Georges, S., Daniau, C., Bernard-Stoecklin, S., Domegan, L., O’Donnell, J., Hauge, S. H., Dequeker, S., et al. (2020) · Eurosurveillance · doi:10.2807/1560-7917.es.2020.25.22.2000956 · Cited by 213
Physical interventions to interrupt or reduce the spread of respiratory viruses
Jefferson, T., Del Mar, C., Dooley, L., Ferroni, E., Al-Ansary, L. A., Bawazeer, G. A., van Driel, M. L., Nair, S., Foxlee, R., & Rivetti, A. (2010) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006207.pub3 · Cited by 212
Saunders-Hastings, P., Crispo, J. A. G., Sikora, L., & Krewski, D. (2017) · Epidemics · doi:10.1016/j.epidem.2017.04.003 · Cited by 183
Clinical Impact of Human Coronaviruses 229E and OC43 Infection in Diverse Adult Populations
Walsh, E. E., Shin, J. H., & Falsey, A. R. (2013) · Journal of Infectious Diseases · doi:10.1093/infdis/jit393 · Cited by 182
A simple method for detecting chaos in nature
Toker, D., Sommer, F. T., & D’Esposito, M. (2020) · Communications Biology · doi:10.1038/s42003-019-0715-9 · Cited by 164
Patrick, D. M., Petric, M., Skowronski, D. M., Guasparini, R., Booth, T. F., Krajden, M., McGeer, P. L., Bastien, N., Gustafson, L., Dubord, J., et al. (2006) · Canadian Journal of Infectious Diseases and Medical Microbiology · doi:10.1155/2006/152612 · Cited by 152
Roxby, A. C., Greninger, A. L., Hatfield, K. M., Lynch, J. B., Dellit, T. H., James, A. E., Taylor, J., Page, L. C., Kimball, A. A., Arons, M. M., et al. (2020) · MMWR Morbidity and Mortality Weekly Report · doi:10.15585/mmwr.mm6914e2 · Cited by 121
Should governments continue lockdown to slow the spread of covid-19?
Melnick, E. R., & Ioannidis, J. P. A. (2020) · BMJ · doi:10.1136/bmj.m1924 · Cited by 104
Self-citationMoser, D. A., Glaus, J., Frangou, S., & Schechter, D. S. (2020) · European Psychiatry · doi:10.1192/j.eurpsy.2020.56 · Cited by 88
On single point forecasts for fat-tailed variables
Taleb, N. N., Bar-Yam, Y., & Cirillo, P. (2020) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2020.08.008 · Cited by 79
Mansour, M., Leven, E. A., Muellers, K., Stone, K., Mendu, D. R., & Wajnberg, A. (2020) · Journal of General Internal Medicine · doi:10.1007/s11606-020-05926-8 · Cited by 70
Sandri, M. T., Azzolini, E., Torri, V., Carloni, S., Pozzi, C., Salvatici, M., Tedeschi, M., Castoldi, M., Mantovani, A., & Rescigno, M. (2021) · Scientific Reports · doi:10.1038/s41598-021-91773-4 · Cited by 50
Covid-19: “Staggering number” of extra deaths in community is not explained by covid-19
Griffin, S. (2020) · BMJ · doi:10.1136/bmj.m1931 · Cited by 27
Effects of non-pharmaceutical interventions on COVID-19: A Tale of Three ModelsPreprint
Chin, V., Ioannidis, J. P. A., Tanner, M. A., & Cripps, S. (2020) · medRxiv · doi:10.1101/2020.07.22.20160341 · Cited by 13
Self-citation
Cited by
Forecasting: theory and practice
Petropoulos, F., Apiletti, D., Assimakopoulos, V., Babai, M. Z., Barrow, D. K., Ben Taieb, S., Bergmeir, C., Bessa, R. J., Bijak, J., Boylan, J. E., et al. (2022) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2021.11.001 · Cited by 921
Covasim: An agent-based model of COVID-19 dynamics and interventions
Kerr, C. C., Stuart, R. M., Mistry, D., Abeysuriya, R. G., Rosenfeld, K., Hart, G. R., Núñez, R. C., Cohen, J. A., Selvaraj, P., Hagedorn, B., et al. (2021) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1009149 · Cited by 685
Overcoming COVID-19 in China despite shortcomings of the public health system: what can we learn?
Wang, M. M., & Fleßa, S. (2021) · Health Economics Review · doi:10.1186/s13561-021-00319-x · Cited by 6
Prognostic models for COVID-19 needed updating to warrant transportability over time and space
van Klaveren, D., Zanos, T. P., Nelson, J., Levy, T. J., Park, J. G., Retel Helmrich, I. R. A., Rietjens, J. A. C., Basile, M. J., Hajizadeh, N., Lingsma, H. F., et al. (2022) · BMC Medicine · doi:10.1186/s12916-022-02651-3 · Cited by 5
Metadata
- Authors
- Ioannidis, J. P. A., Cripps, S., & Tanner, M. A.
- Venue
- International Journal of Forecasting
- Publisher
- Elsevier BV
- Published
- 2022-04
- Volume
- 38
- Issue
- 2
- Pages
- 423-438
- ISSN
- 0169-2070 (ISSN-L)
- OpenAlex
- W3080376427
- Cited by
- 430
- References
- 36
- FWCI
- 12.21
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.