The rationale behind the success of multi-model ensembles in seasonal forecasting - I. Basic concept
Open accessAbstract
The DEMETER multi-model ensemble system is used to investigate the rationale behind the multi-model concept. A comprehensive documentation of the differences in the single and multi-model performance in the DEMETER hindcast data set is given. Both deterministic and probabilistic diagnostics are used and a variety of analyses demonstrate the improvements achieved by using multi-model instead of single-model ensembles. In order to understand the reason behind the multi-model superiority, basic scenarios describing how the multi-model approach can improve over singlemodel skill are discussed. It is demonstrated that multi-model superiority is caused not only by error compensation but in particular by its greater consistency and reliability.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Hagedorn et al. (2005) showed with the ECMWF model that increasing single-model ensemble size raised skill, but the multimodel ensemble still had a slight advantage.
“Using the ECMWF model, Hagedorn et al. (2005) demonstrated that while increasing the ensemble size of a single model did increase skill, the multimodel ensemble retained a slight advantage over the single model ensemble.”
From Skill of Multimodel ENSO Probability Forecasts (2008) · cites this paper as Hagedorn et al. (2005)
The passage labeled Abstract in our indexed copy says the DEMETER analyses found improvements from multi-model over single-model ensembles. It does not establish the claimed ECMWF-specific comparison, that increasing the single-model ensemble size raised skill, or that the remaining multi-model advantage was slight; searches for those details returned no relevant results passage. (LLM, conf=0.74)
Experimental — accuracy not yet measuredOther studies have directly shown that multimodel ensemble skill improvement exceeds that from increasing single-model ensemble size.
“Other studies have directly shown that the skill improvement of the multimodel ensemble is greater than that achieved by increasing the ensemble size of a single model (Hagedorn et al. 2005).”
From Skill of Multimodel ENSO Probability Forecasts (2008) · cites this paper as Hagedorn et al. 2005
The passage labeled References (page unavailable) contains text describing improved skill from multi-model rather than single-model ensembles. It does not report a direct comparison with the skill gained by increasing a single model’s ensemble size, so the claim’s specific comparison is unsupported. (LLM, conf=0.72)
Experimental — accuracy not yet measuredEqually weighted multimodel combination on average improves the reliability of climate projections.
“The least surprising conclusion to be drawn is probably that equally weighted multimodel combination on average improves the reliability of climate projectionsa conclusion that is fully consistent with what is known from many verification studies in weather and seasonal forecasting (e.g., Hagedorn et al. 2005;Palmer et al. 2004;Weigel et al. 2008b).”
From Risks of Model Weighting in Multimodel Climate Projections (2010) · cites this paper as Hagedorn et al. 2005
The copy’s Abstract (indexed under References; no page given) reports improved multimodel performance in seasonal hindcasts and attributes the advantage in part to greater reliability. It does not establish that the combination was equally weighted. The sentence presents the climate-projection conclusion as the citing authors’ own; this source is cited for consistency with seasonal-forecast verification, not as a direct test of climate projections. (LLM, conf=0.82)
Experimental — accuracy not yet measuredDiffering biases among models may cancel in multimodel forecasts.
“Conceptually, three reasons for the enhanced skill of multimodel forecasts are: (i) differing biases may cancel, (ii) the ensemble size is increased so that sampling error is reduced, and (iii) while the multimodel ensemble may have less skill than the “best” model for a particular forecast, it may not be possible to identify the best model a priori (Hagedorn et al. 2005).”
From Skill of Multimodel ENSO Probability Forecasts (2008) · cites this paper as Hagedorn et al. 2005
The available passage labelled References contains a cut-off description of multimodel improvements but does not reach an explanation involving bias cancellation. Searches for differing biases, systematic errors, and ensemble size returned no further substantive passages, so our copy cannot verify this specific rationale. (LLM, conf=0.72)
Experimental — accuracy not yet measuredThe best model for a particular forecast may not be identifiable a priori.
“Conceptually, three reasons for the enhanced skill of multimodel forecasts are: (i) differing biases may cancel, (ii) the ensemble size is increased so that sampling error is reduced, and (iii) while the multimodel ensemble may have less skill than the “best” model for a particular forecast, it may not be possible to identify the best model a priori (Hagedorn et al. 2005).”
From Skill of Multimodel ENSO Probability Forecasts (2008) · cites this paper as Hagedorn et al. 2005
The available text discusses multimodel superiority and its consistency and reliability (Abstract, page not given), but does not address whether the best model for a particular forecast can be identified beforehand. Searches for “best model a priori,” model selection, and equivalent wording found no relevant passage; the indexed copy appears to contain only the abstract rather than the article’s main text. (LLM, conf=0.72)
Experimental — accuracy not yet measured
Increased ensemble size in multimodel forecasts reduces sampling error.
“Conceptually, three reasons for the enhanced skill of multimodel forecasts are: (i) differing biases may cancel, (ii) the ensemble size is increased so that sampling error is reduced, and (iii) while the multimodel ensemble may have less skill than the “best” model for a particular forecast, it may not be possible to identify the best model a priori (Hagedorn et al. 2005).”
From Skill of Multimodel ENSO Probability Forecasts (2008) · cites this paper as Hagedorn et al. 2005
The available passage labeled References describes improved performance of multimodel ensembles, but it cuts off before explaining why. Searches for sampling error and ensemble size returned no passage addressing whether a larger ensemble reduces sampling error; the decisive part of our copy may be missing. (LLM, conf=0.72)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
No references could be resolved yet.
Research works printed but not linked: 4
- Gigerenzer , G. and Todd , P. M. 1999 . Simple Heuristics That Make Us Smart . Oxford University Press , Oxford .
- Jolliffe , I. T. and Stephenson , D. B. 2003 . Forecast Verification: A Practitioner’s Guide in Atmospheric Science . Wiley , New York .
- Peng , P. , Kumar , A. , van den Dool , H. and Barnston , A. G. 2002 . An analysis of multi-model ensemble predictions for seasonal climate anomalies. J. Geophys. Res . 107 , doi: https://doi.org/10.1029/2002JDO02712 . · DOI
- (self.__next_s=self.__next_s||[]).push([0,{"children":"window.__ENV__ = {\"RECAPTCHA_SITE_KEY\":\"6Le5r8IkAAAAAHMkHq5S2Em4RKx3Xyk5my0v9HnE\",\"SERVICE_CODE\":\"su-j-tadmo\",\"FRONTEND_URL\":\"https://tellusjournal.org\",\"FRONTEND_DOMAIN\":\"tellusjournal.org\",\"OJS_API_URL\":\"https://api.account.su-j-tadmo.ubiquityjournal.website/api/journal/1\"};","id":"public-env"}])This page is for printing. If you wish to read this article, please visit its full version.The rationale behind the success of multi-model ensembles in seasonal forecasting – I. Basic conceptRenate HagedornFrancisco J. Doblas-ReyesT. N. PalmerAbstractThe DEMETER multi-model ensemble system is used to investigate the rationale behind the multi-model concept. A comprehensive documentation of the differences in the single and multi-model performance in the DEMETER hindcast data set is given. Both deterministic and probabilistic diagnostics are used and a variety of analyses demonstrate the improvements achieved by using multi-model instead of single-model ensembles. In order to understand the reason behind the multi-model superiority, basic scenarios describing how the multi-model approach can improve over singlemodel skill are discussed. It is demonstrated that multi-model superiority is caused not only by error compensation but in particular by its greater consistency and reliability.Year: 2005Volume: 57 Issue: 3Page/Article: 219–233DOI: 10.3402/tellusa.v57i3.14657Accepted on Sep 24, 2004Published on Jan 1, 2005Peer ReviewedCC BY 4.0 · DOI
Cited by
Sorted by citations
Climate-induced variations in global wildfire danger from 1979 to 2013
Jolly, W. M., Cochrane, M. A., Freeborn, P. H., Holden, Z. A., Brown, T. J., Williamson, G. J., & Bowman, D. M. J. S. (2015) · Nature Communications · doi:10.1038/ncomms8537 · Cited by 2,200 · see citing works
The use of the multi-model ensemble in probabilistic climate projections
Tebaldi, C., & Knutti, R. (2007) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2007.2076 · Cited by 1,862 · see citing works
Challenges in combining projections from multiple climate models.
Knutti, R., Furrer, R., Tebaldi, C., Cermak, J., & Meehl, G. A. (2010) · Journal of Climate · doi:10.1175/2009jcli3361.1 · Cited by 1,270 · see citing works
Decadal Prediction: Can It Be Skillful?
Meehl, G. A., Goddard, L., Murphy, J., Stouffer, R. J., Boer, G., Danabasoglu, G., Dixon, K., Giorgetta, M. A., Greene, A. M., Hawkins, E., et al. (2009) · Bulletin of the American Meteorological Society · doi:10.1175/2009bams2778.1 · Cited by 755 · see citing works
Malaria early warnings based on seasonal climate forecasts from multi-model ensembles
Thomson, M. C., Doblas-Reyes, F. J., Mason, S. J., Hagedorn, R., Connor, S. J., Phindela, T., Morse, A. P., & Palmer, T. N. (2006) · Nature · doi:10.1038/nature04503 · Cited by 470 · see citing works
Self-citationRisks of Model Weighting in Multimodel Climate Projections
Weigel, A. P., Knutti, R., Liniger, M. A., & Appenzeller, C. (2010) · Journal of Climate · doi:10.1175/2010jcli3594.1 · Cited by 386 · see citing works
The Discrete Brier and Ranked Probability Skill Scores
Weigel, A. P., Liniger, M. A., & Appenzeller, C. (2007) · Monthly Weather Review · doi:10.1175/mwr3280.1 · Cited by 254 · see citing works
Classifier calibration: a survey on how to assess and improve predicted class probabilities
Silva Filho, T., Song, H., Perello-Nieto, M., Santos-Rodriguez, R., Kull, M., & Flach, P. (2023) · Machine Learning · doi:10.1007/s10994-023-06336-7 · Cited by 228 · see citing works
de Andrade, F. M., Coelho, C. A. S., & Cavalcanti, I. F. A. (2019) · Climate Dynamics · doi:10.1007/s00382-018-4457-z · Cited by 152 · see citing works
Seasonal ensemble forecasts: Are recalibrated single models better than multimodels?
Weigel, A. P., Liniger, M. A., & Appenzeller, C. (2009) · Monthly Weather Review · doi:10.1175/2008mwr2773.1 · Cited by 66 · see citing works
Skill of Multimodel ENSO Probability Forecasts
Tippett, M. K., & Barnston, A. G. (2008) · Monthly Weather Review · doi:10.1175/2008mwr2431.1 · Cited by 64 · see citing works
Data-driven reverse engineering of signaling pathways using ensembles of dynamic models
Henriques, D., Villaverde, A. F., Rocha, M., Saez-Rodriguez, J., & Banga, J. R. (2017) · PLOS Computational Biology · doi:10.1371/journal.pcbi.1005379 · Cited by 44 · see citing works
Weigel, A. P., Liniger, M. A., & Appenzeller, C. (2007) · Monthly Weather Review · doi:10.1175/mwr3428.1 · Cited by 35 · see citing works
- Authors
- Hagedorn, R., Doblas-Reyes, F. J., & Palmer, T. N.
- Affiliations
- European Centre for Medium-Range Weather Forecasts (GB)
- Venue
- Tellus A: Dynamic Meteorology and Oceanography
- Publisher
- Stockholm University Press
- Published
- 2005-01-01
- Volume
- 57
- Issue
- 3
- Pages
- 219
- ISSN
- 0280-6495 (ISSN-L)
- 1600-0870
- OpenAlex
- W2048855290
- Cited by
- 710
- References
- 0
- Citations by year
- 2026: 20 · 2025: 30 · 2024: 40 · 2023: 37 · 2022: 41 · 2021: 44 · 2020: 36 · 2019: 47 · 2018: 36 · 2017: 53 · 2016: 31 · 2015: 38 · 2014: 34 · 2013: 31 · 2012: 36
- FWCI
- 19.62
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Tellus A Dynamic Meteorology and Oceanography (publishedVersion)
- Tellus A Dynamic Meteorology and Oceanography (publishedVersion, cc-by)
- Topics
- Physical Sciences › Environmental Science › Global and Planetary Change › Climate variability and models
- Physical Sciences › Earth and Planetary Sciences › Atmospheric Science › Meteorological Phenomena and Simulations
- Social Sciences › Decision Sciences › Management Science and Operations Research › Forecasting Techniques and Applications
- Keywords
- multi-model ensemble, seasonal forecasting, Demeter, ensemble hindcasts, error compensation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
