The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
Can multi‐model combination really enhance the prediction skill of probabilistic ensemble forecasts?
Abstract
The success of multi‐model ensemble combination has been demonstrated in many studies. Given that a multi‐model contains information from all participating models, including the less skilful ones, the question remains as to why, and under what conditions, a multi‐model can outperform the best participating single model. It is the aim of this paper to resolve this apparent paradox. The study is based on a synthetic forecast generator, allowing the generation of perfectly‐calibrated single‐model ensembles of any size and skill. Additionally, the degree of ensemble under‐dispersion (or overconfidence) can be prescribed. Multi‐model ensembles are then constructed from both weighted and unweighted averages of these single‐model ensembles. Applying this toy model, we carry out systematic model‐combination experiments. We evaluate how multi‐model performance depends on the skill and overconfidence of the participating single models. It turns out that multi‐model ensembles can indeed locally outperform a ‘best‐model’ approach, but only if the single‐model ensembles are overconfident. The reason is that multi‐model combination reduces overconfidence, i.e. ensemble spread is widened while average ensemble‐mean error is reduced. This implies a net gain in prediction skill, because probabilistic skill scores penalize overconfidence. Under these conditions, even the addition of an objectively‐poor model can improve multi‐model skill. It seems that simple ensemble inflation methods cannot yield the same skill improvement. Using seasonal near‐surface temperature forecasts from the DEMETER dataset, we show that the conclusions drawn from the toy‐model experiments hold equally in a real multi‐model ensemble prediction system. Copyright © 2008 Royal Meteorological Society
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Statistical Methods in the Atmospheric Sciences
Ramsey, P. H. (1996) · Technometrics · doi:10.1080/00401706.1996.10484553 · Cited by 7,601
Uppala, S. M., KÅllberg, P. W., Simmons, A. J., Andrae, U., Bechtold, V. D. C., Fiorino, M., Gibson, J. K., Haseler, J., Hernandez, A., Kelly, G. A., et al. (2005) · Quarterly Journal of the Royal Meteorological Society · doi:10.1256/qj.04.176 · Cited by 6,804
A Limited Memory Algorithm for Bound Constrained Optimization
Byrd, R. H., Lu, P., Nocedal, J., & Zhu, C. (1995) · SIAM Journal on Scientific Computing · doi:10.1137/0916069 · Cited by 6,221
Atmospheric Modeling, Data Assimilation and Predictability
Kalnay, E. (2002) · doi:10.1017/cbo9780511802270 · Cited by 3,391
The role of increasing temperature variability in European summer heatwaves
Schär, C., Vidale, P. L., Lüthi, D., Frei, C., Häberli, C., Liniger, M. A., & Appenzeller, C. (2004) · Nature · doi:10.1038/nature02300 · Cited by 2,781
Self-citationGordon, C., Cooper, C., Senior, C. A., Banks, H., Gregory, J. M., Johns, T. C., Mitchell, J. F. B., & Wood, R. A. (2000) · Climate Dynamics · doi:10.1007/s003820050010 · Cited by 2,578
The ECMWF Ensemble Prediction System: Methodology and validation
Molteni, F., Buizza, R., Palmer, T. N., & Petroliagis, T. (1996) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.49712252905 · Cited by 1,616
Wörle, H., Rumpf, H.-J., & Erven, J. (1994) · De Gruyter eBooks · doi:10.1515/9783486785449 · Cited by 1,465
Stochastic representation of model uncertainties in the ECMWF ensemble prediction system
Buizza, R., Milleer, M., & Palmer, T. N. (1999) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.49712556006 · Cited by 1,103
Palmer, T. N., Alessandri, A., Andersen, U., Cantelaube, P., Davey, M., Délécluse, P., Déqué, M., Díez, E., Doblas-Reyes, F. J., Feddersen, H., et al. (2004) · Bulletin of the American Meteorological Society · doi:10.1175/bams-85-6-853 · Cited by 959
Research works printed but not linked: 4
- 10.1111/j.1600-0870.2005.00103.x · doi:10.1111/j.1600-0870.2005.00103.x · Paper page
- 10.1175/1520-0442(2003)016<1684:ISPF>2.0.CO;2 · doi:10.1175/1520-0442(2003)016<1684:ispf>2.0.co;2
- 10.1175/1520-0434(1997)012<0140:ASEOTN>2.0.CO;2 · doi:10.1175/1520-0434(1997)012<0140:aseotn>2.0.co;2
- Toth Z, 2003, Forecast Verification: A Practitioner's Guide in Atmospheric Science, 137
Cited by
Challenges in combining projections from multiple climate models.
Knutti, R., Furrer, R., Tebaldi, C., Cermak, J., & Meehl, G. A. (2010) · Journal of Climate · doi:10.1175/2009jcli3361.1 · Cited by 1,270
A bitter cup: climate change profile of global production of Arabica and Robusta coffee
Bunn, C., Läderach, P., Ovalle Rivera, O., & Kirschke, D. (2015) · Climatic Change · doi:10.1007/s10584-014-1306-x · Cited by 625
Iterative near-term ecological forecasting: Needs, opportunities, and challenges
Dietze, M. C., Fox, A., Beck-Johnson, L. M., Betancourt, J. L., Hooten, M. B., Jarnevich, C. S., Keitt, T. H., Kenney, M. A., Laney, C. M., Larsen, L. G., et al. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1710231115 · Cited by 604
The end of model democracy?: An editorial comment
Knutti, R. (2010) · Climatic Change · doi:10.1007/s10584-010-9800-2 · Cited by 499
Seasonal climate predictability and forecasting: status and prospects
Doblas‐Reyes, F. J., García‐Serrano, J., Lienert, F., Biescas, A. P., & Rodrigues, L. R. L. (2013) · WIREs Climate Change · doi:10.1002/wcc.217 · Cited by 478
Risks of Model Weighting in Multimodel Climate Projections
Weigel, A. P., Knutti, R., Liniger, M. A., & Appenzeller, C. (2010) · Journal of Climate · doi:10.1175/2010jcli3594.1 · Cited by 386
Self-citationWeisheimer, A., Doblas‐Reyes, F. J., Palmer, T. N., Alessandri, A., Arribas, A., Déqué, M., Keenlyside, N., MacVean, M., Navarra, A., & Rogel, P. (2009) · Geophysical Research Letters · doi:10.1029/2009gl040896 · Cited by 285
Forecast combinations: An over 50-year review
Wang, X., Hyndman, R. J., Li, F., & Kang, Y. (2023) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2022.11.005 · Cited by 280
The Subseasonal Experiment (SubX): A Multimodel Subseasonal Prediction Experiment
Pegion, K., Kirtman, B. P., Becker, E., Collins, D. C., LaJoie, E., Burgman, R., Bell, R., DelSole, T., Min, D., Zhu, Y., et al. (2019) · Bulletin of the American Meteorological Society · doi:10.1175/bams-d-18-0270.1 · Cited by 278
Assessing the uncertainties of hydrologic model selection in climate change impact studies
Najafi, M. R., Moradkhani, H., & Jung, I. W. (2011) · Hydrological Processes · doi:10.1002/hyp.8043 · Cited by 259
Bayesian multi-model projection of climate: bias assumptions and interannual variability
Buser, C. M., Künsch, H. R., Lüthi, D., Wild, M., & Schär, C. (2009) · Climate Dynamics · doi:10.1007/s00382-009-0588-6 · Cited by 181
Seasonal ensemble forecasts: Are recalibrated single models better than multimodels?
Weigel, A. P., Liniger, M. A., & Appenzeller, C. (2009) · Monthly Weather Review · doi:10.1175/2008mwr2773.1 · Cited by 66
Self-citation
Metadata
- Authors
- Weigel, A. P., Liniger, M. A., & Appenzeller, C.
- Venue
- Quarterly Journal of the Royal Meteorological Society
- Publisher
- Wiley
- Published
- 2008-01
- Volume
- 134
- Issue
- 630
- Pages
- 241-260
- ISSN
- 0035-9009 (ISSN-L)
- 1477-870X
- OpenAlex
- W2143323261
- Cited by
- 330
- References
- 52
- FWCI
- 10.78
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.