The DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Predictive performance of international COVID-19 mortality forecasting models
Open accessPublished version of a preprint
Abstract
Forecasts and alternative scenarios of COVID-19 mortality have been critical inputs for pandemic response efforts, and decision-makers need information about predictive performance. We screen n = 386 public COVID-19 forecasting models, identifying n = 7 that are global in scope and provide public, date-versioned forecasts. We examine their predictive performance for mortality by weeks of extrapolation, world region, and estimation month. We additionally assess prediction of the timing of peak daily mortality. Globally, models released in October show a median absolute percent error (MAPE) of 7 to 13% at six weeks, reflecting surprisingly good performance despite the complexities of modelling human behavioural responses and government interventions. Median absolute error for peak timing increased from 8 days at one week of forecasting to 29 days at eight weeks and is similar for first and subsequent peaks. The framework and public codebase ( https://github.com/pyliu47/covidcompare ) can be used to compare predictions and evaluate predictive performance going forward.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures








Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The Reffective over time is also a crucial input into future forecasts of COVID-19.4”
From Estimating global, regional, and national daily and cumulative infections with SARS-CoV-2 through Nov 14, 2021: a statistical analysis (2022) · cites this paper as [4]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“reported deaths represent only a partial count of total death tolls from the COVID-19 pandemic”
From Estimating excess mortality due to the COVID-19 pandemic: a systematic analysis of COVID-19-related mortality, 2020–21 (2022) · cites this paper as [1]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“the reliability of reported deaths varies greatly between locations and over time”
From Estimating excess mortality due to the COVID-19 pandemic: a systematic analysis of COVID-19-related mortality, 2020–21 (2022) · cites this paper as [1]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Viner, R. M., Russell, S. J., Croker, H., Packer, J., Ward, J., Stansfield, C., Mytton, O., Bonell, C., & Booy, R. (2020) · The Lancet Child & Adolescent Health · doi:10.1016/s2352-4642(20)30095-x · Cited by 2,213
Ranney, M. L., Griffeth, V., & Jha, A. K. (2020) · New England Journal of Medicine · doi:10.1056/nejmp2006141 · Cited by 2,018
Dicker, D., Nguyen, G., Abate, D., Abate, K. H., Abay, S. M., Abbafati, C., Abbasi, N., Abbastabar, H., Abd-Allah, F., Abdela, J., et al. (2018) · The Lancet · doi:10.1016/s0140-6736(18)31891-9 · Cited by 1,209
Out-of-sample tests of forecasting accuracy: an analysis and review
Tashman, L. J. (2000) · International Journal of Forecasting · doi:10.1016/s0169-2070(00)00065-0 · Cited by 1,021
What Will Be the Economic Impact of COVID-19 in the US? Rough Estimates of Disease Scenarios
Atkeson, A. (2020) · National Bureau of Economic Research · doi:10.3386/w26867 · Cited by 888
Modeling COVID-19 scenarios for the United States
IHME COVID-19 Forecasting Team, Reiner, R. C., Barber, R. M., Collins, J. K., Zheng, P., Adolph, C., Albright, J., Antony, C. M., Aravkin, A. Y., Bachmeier, S. D., et al. (2021) · Nature Medicine · doi:10.1038/s41591-020-1132-9 · Cited by 556 · cited 1×
“This model, referred to as the IHME-MS SEIR model, is the basis for recently published work on US State level scenarios of COVID-249 19 projections in the fall and winter of 2020/2021 23 and was still in use at the time of this publication.”
MethodSelf-citationWeinberger, D. M., Chen, J., Cohen, T., Crawford, F. W., Mostashari, F., Olson, D., Pitzer, V. E., Reich, N. G., Russi, M., Simonsen, L., et al. (2020) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2020.3391 · Cited by 485
IHME COVID-19 health service utilization forecasting team, & Murray, C. J. (2020) · medRxiv · doi:10.1101/2020.03.27.20043752 · Cited by 417
Self-citationJentsch, P. C., Anand, M., & Bauch, C. T. (2021) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(21)00057-8 · Cited by 232
Local lockdowns outperform global lockdown on the far side of the COVID-19 epidemic curve
Karatayev, V. A., Anand, M., & Bauch, C. T. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2014385117 · Cited by 125
Cited by
Wang, H., Paulson, K. R., Pease, S. A., Watson, S., Comfort, H., Zheng, P., Aravkin, A. Y., Bisignano, C., Barber, R. M., Alam, T., et al. (2022) · The Lancet · doi:10.1016/s0140-6736(21)02796-3 · Cited by 2,069
Self-citationFindings from the Global Burden of Disease Study 2021
Murray, C. J. L. (2024) · The Lancet · doi:10.1016/s0140-6736(24)00769-4 · Cited by 1,057
Self-citationThe Global Burden of Disease Study at 30 years
Murray, C. J. L. (2022) · Nature Medicine · doi:10.1038/s41591-022-01990-1 · Cited by 856
Self-citationAge groups that sustain resurging COVID-19 epidemics in the United States
Monod, M., Blenkinsop, A., Xi, X., Hebert, D., Bershan, S., Tietze, S., Baguelin, M., Bradley, V. C., Chen, Y., Coupland, H., et al. (2021) · Science · doi:10.1126/science.abe8372 · Cited by 331
Barber, R. M., Sorensen, R. J. D., Pigott, D. M., Bisignano, C., Carter, A., Amlag, J. O., Collins, J. K., Abbafati, C., Adolph, C., Allorant, A., et al. (2022) · The Lancet · doi:10.1016/s0140-6736(22)00484-6 · Cited by 298
Self-citation
Metadata
- Authors
- Friedman, J., Liu, P., Troeger, C. E., Carter, A., Reiner, R. C., Barber, R. M., Collins, J., Lim, S. S., Pigott, D. M., Vos, T., Hay, S. I., Murray, C. J. L., & Gakidou, E.
- Affiliations
- University of California, Los Angeles (US)
- University of Washington (US)
- Corresponding author
- Gakidou, Emmanuela
- ORCID
- Venue
- Nature Communications
- Published
- 2021-05-10
- Volume
- 12
- Issue
- 1
- Pages
- 2609-2609
- ISSN
- 2041-1723 (ISSN-L)
- PMC
- PMC8110547
- OpenAlex
- W3163299620
- Cited by
- 118
- References
- 18
- Citations by year
- 2026: 8 · 2025: 14 · 2024: 16 · 2023: 18 · 2022: 33 · 2021: 27
- FWCI
- 9.09
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Nature Communications (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Bill and Melinda Gates Foundation (OPP1152504)
- Topics
- Physical Sciences › Mathematics › Modeling and Simulation › COVID-19 epidemiological studies
- Health Sciences › Medicine › Epidemiology › Data-Driven Disease Surveillance
- Social Sciences › Demography › Insurance, Mortality, Demography, Risk Management
- Keywords
- COVID-19 pandemic, mortality forecasting, prediction model evaluation, predictive performance, pandemic response, forecasting model comparison, state intervention
- MeSH terms
- COVID-19, SARS-CoV-2, Forecasting, Humans, Time Factors, Models, Statistical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Predictive performance of international COVID-19 mortality forecasting models. https://app.cochat.ai/papers/10.1038/s41467-021-22457-w
- The paper
- Friedman, J., Liu, P., Troeger, C. E., Carter, A., Reiner, R. C., Barber, R. M., Collins, J., Lim, S. S., Pigott, D. M., Vos, T., Hay, S. I., Murray, C. J. L., & Gakidou, E. (2021). Predictive performance of international COVID-19 mortality forecasting models. Nature Communications. https://doi.org/10.1038/s41467-021-22457-w
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.