The DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.
The RAPIDD ebola forecasting challenge: Synthesis and lessons learnt
Open accessAbstract
Infectious disease forecasting is gaining traction in the public health community; however, limited systematic comparisons of model performance exist. Here we present the results of a synthetic forecasting challenge inspired by the West African Ebola crisis in 2014-2015 and involving 16 international academic teams and US government agencies, and compare the predictive performance of 8 independent modeling approaches. Challenge participants were invited to predict 140 epidemiological targets across 5 different time points of 4 synthetic Ebola outbreaks, each involving different levels of interventions and "fog of war" in outbreak data made available for predictions. Prediction targets included 1-4 week-ahead case incidences, outbreak size, peak timing, and several natural history parameters. With respect to weekly case incidence targets, ensemble predictions based on a Bayesian average of the 8 participating models outperformed any individual model and did substantially better than a null auto-regressive model. There was no relationship between model complexity and prediction accuracy; however, the top performing models for short-term weekly incidence were reactive models with few parameters, fitted to a short and recent part of the outbreak. Individual model outputs and ensemble predictions improved with data accuracy and availability; by the second time point, just before the peak of the epidemic, estimates of final size were within 20% of the target. The 4th challenge scenario - mirroring an uncontrolled Ebola outbreak with substantial data reporting noise - was poorly predicted by all modeling teams. Overall, this synthetic forecasting challenge provided a deep understanding of model performance under controlled data and epidemiological conditions. We recommend such "peace time" forecasting challenges as key elements to improve coordination and inspire collaboration between modeling groups ahead of the next pandemic threat, and to assess model forecasting accuracy for a variety of known and hypothetical pathogens.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Mathematical models of disease transmission and control are a key tool for responding to infectious disease outbreaks and guiding policy, for example by projecting epidemic trajectories, healthcare demand, and intervention impact.
“Mathematical models of disease transmission and control are a key tool that can be deployed in response to infectious disease outbreaks and are used to guide policy, for example by projecting plausible epidemic trajectories and expected healthcare demand and assessing the potential impact of interventions (19,20).”
From Marburg virus disease outbreaks, mathematical models, and disease parameters: a systematic review (2024) · cites this paper as [20]
The Introduction (page not provided) says disease models guide public-health interventions by predicting epidemic trajectories and projecting the benefits of intervention strategies. It does not say they project expected healthcare demand; searches for healthcare and treatment needs found no supporting discussion in our copy. (LLM, conf=0.86)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Ebola virus disease in West Africa—the first 9 months of the epidemic and forward projections
WHO Ebola Response Team (2014) · New England Journal of Medicine · doi:10.1056/nejmoa1411100 · Cited by 1,674
Flagged first, then by citations
Research works printed but not linked: 31
- Ajelli, 2016, Spatiotemporal dynamics of the Ebola epidemic in Guinea and implications for vaccination and disease elimination: a computational modeling analysis, BMC Med., 14, 130, 10.1186/s12916-016-0678-3 · DOI
- Ajelli, 2017, The RAPIDD Ebola forecasting challenge: model description and synthetic data generation, Epidemics
- Asher, 2017, Forecasting Ebola with a regression transmission model, Epidemics
- Biggerstaff, 2016, Influenza Forecasting Contest Working. Results from the centers for disease control and prevention's predict the 2013–2014 Influenza Season Challenge, BMC Infect. Dis., 16, 357, 10.1186/s12879-016-1669-x · DOI
- Bogoch, 2016, Anticipating the international spread of Zika virus from Brazil, Lancet, 387, 335, 10.1016/S0140-6736(16)00080-5 · DOI
- Camacho, 2017, Real-time dynamic modelling for the design of a cluster-randomized phase 3 Ebola vaccine trial in Sierra Leone, Vaccine, 35, 544, 10.1016/j.vaccine.2016.12.019 · DOI
- Champredon, 2017, Two approaches to forecast Ebola synthetic epidemics, Epidemics
- Chowell, 2016, Mathematical models to characterize early epidemic growth, Rev. Phys. Life Rev., 10.1016/j.plrev.2016.07.005 · DOI
- Chowell, 2017, Perspectives on model forecasts of the 2014–2015 Ebola epidemic in West Africa: lessons and the way forward, BMC Med., 15, 42, 10.1186/s12916-017-0811-y · DOI
- Chretien, 2015, Mathematical modeling of the west africa ebola epidemic, Elife, 4, 10.7554/eLife.09186 · DOI
- Chretien, 2015, Advancing epidemic prediction and forecasting: a new US government initiative, Online J. Public Health Inform., 7, e13, 10.5210/ojphi.v7i1.5677 · DOI
- Funk, 2016, Real-time forecasting of infectious disease dynamics with a stochastic semi-mechanistic model, Epidemics
- Gaffey, 2017, Application of the CDC EbolaResponse modeling tool for disease predictions, Epidemics
- Gomes, 2014, Assessing the international spreading risk associated with the 2014 west african ebola outbreak, PLoS Curr., 6
- Heesterbeek, 2015, Isaac Newton Institute. Modeling infectious disease dynamics in the complex landscape of global health, Science, 6227, 4339, 10.1126/science.aaa4339 · DOI
- Lewnard, 2014, Dynamics and control of Ebola virus transmission in Montserrado, Liberia: a mathematical modelling analysis, Lancet Infect. Dis., 14, 1189, 10.1016/S1473-3099(14)70995-8 · DOI
- Lipsitch, 2011, Improving the evidence base for decision making during a pandemic: the example of 2009 influenza A/H1N1, Biosecur. Bioterror., 9, 89
- Meltzer, 2016, Modeling in real time during the ebola response, MMWR Suppl., 65, 85, 10.15585/mmwr.su6503a12 · DOI
- Merler, 2015, Spatiotemporal spread of the outbreak of Ebola virus disease in Liberia and the effectiveness of non-pharmaceutical interventions: a computational modelling analysis, Lancet Infect. Dis., 2, 204, 10.1016/S1473-3099(14)71074-6 · DOI · Paper page
- Merler, 2016, Containing ebola at the source with ring vaccination, PLoS Negl. Trop. Dis., 11, e0005093, 10.1371/journal.pntd.0005093 · DOI
- Nouvellet, 2017, A simple approach to measure transmissibility and forecast incidence, Epidemics
- Pell, 2016, Using phenomenological models for forecasting the 2015 Ebola challenge, Epidemics
- Alex Perkins, 2016, Model-based projections of Zika virus infections in childbearing women in the Americas, Nat. Microbiol., 1, 16126, 10.1038/nmicrobiol.2016.126 · DOI
- Pitzer, 2009, Demographic variability, vaccination, and the spatiotemporal dynamics of rotavirus epidemics, Science, 325, 290, 10.1126/science.1172330 · DOI
- Raftery, 2005, Using Bayesian model averaging to calibrate forecast ensembles, Month. Weather Rev., 133, 1155, 10.1175/MWR2906.1 · DOI · Paper page
- Rainisch, 2015, Estimating ebola treatment needs, United States, Emerg. Infect. Dis., 21, 1273, 10.3201/eid2107.150286 · DOI
- Shaman, 2014, Inference and forecast of the current west african ebola outbreak in Guinea, sierra leone and liberia, PLoS Curr., 6
- Tuite, 2016, The IDEA model: a single equation approach to the Ebola forecasting challenge, Epidemics
- Venkatramanan, 2017, Using data-driven agent-based models for forecasting emerging infectious diseases, Epidemics
- Viboud, 2016, A generalized-growth model to characterize the early ascending phase of infectious disease outbreaks, Epidemics, 15, 27, 10.1016/j.epidem.2016.01.002 · DOI
- Vrugt, 2007, Treatment of uncertainty using ensemble methods: comparison of sequential data assimilation and Bayesian model averaging, Water Resour. Res., 43, 10.1029/2005WR004838 · DOI · Paper page
Cited by
Sorted by citations
Marburg virus disease outbreaks, mathematical models, and disease parameters: a systematic review
Cuomo-Dannenburg, G., McCain, K., McCabe, R., Unwin, H. J. T., Doohan, P., Nash, R. K., Hicks, J. T., Charniga, K., Geismar, C., Lambert, B., et al. (2024) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(23)00515-7 · Cited by 65
“…of disease transmission and control are a key tool that can be deployed in response to infectious disease outbreaks and are used to guide policy, for example by projecting plausible epidemic trajectories and expected healthcare demand and assessing the potential impact of interventions (19,20).”
Metadata
- Authors
- Viboud, C., Sun, K., Gaffey, R., Ajelli, M., Fumanelli, L., Merler, S., Zhang, Q., Chowell, G., Simonsen, L., & Vespignani, A.
- Affiliations
- Northeastern University (US)
- National Institutes of Health (US)
- Fondazione Bruno Kessler (IT)
- ORCID
- Venue
- Epidemics
- Publisher
- Elsevier BV
- Published
- 2018-03
- Volume
- 22
- Pages
- 13-21
- ISSN
- 1755-4365 (ISSN-L)
- PMC
- PMC5927600
- OpenAlex
- W2747968860
- Cited by
- 251
- References
- 1
- Citations by year
- 2026: 18 · 2025: 15 · 2024: 30 · 2023: 27 · 2022: 42 · 2021: 46 · 2020: 40 · 2019: 20 · 2018: 8
- FWCI
- 15.37
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Epidemics (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- LSHTM Research Online (London School of Hygiene and Tropical Medicine) (acceptedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- RAPIDD Program of the Science & Technology Directorate
- Department of Homeland Security and the Fogarty International Center
- National Institutes of Health, in collaboration with the MIDAS program of the National Institute for General Medical Sciences
- Marie Curie EU
- NSF (#1518939, #1318788, #1610429)
- Models of Infectious Disease Agent Study, National Institute of General Medical Sciences (U54GM111274)
- Topics
- Health Sciences › Medicine › Infectious Diseases › Viral Infections and Outbreaks Research
- Health Sciences › Medicine › Epidemiology › Data-Driven Disease Surveillance
- Physical Sciences › Mathematics › Applied Mathematics › Mathematical and Theoretical Epidemiology and Ecology Models
- Keywords
- Ebola virus disease, infectious disease forecasting, model performance comparison, ensemble forecasting, Bayesian model averaging, epidemic peak timing, pandemic preparedness
- MeSH terms
- Bayes Theorem, Forecasting, Humans, Liberia, Reproducibility of Results, Models, Statistical, Hemorrhagic Fever, Ebola, Epidemics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
