The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
Can multi‐model combination really enhance the prediction skill of probabilistic ensemble forecasts?
Abstract
The success of multi‐model ensemble combination has been demonstrated in many studies. Given that a multi‐model contains information from all participating models, including the less skilful ones, the question remains as to why, and under what conditions, a multi‐model can outperform the best participating single model. It is the aim of this paper to resolve this apparent paradox. The study is based on a synthetic forecast generator, allowing the generation of perfectly‐calibrated single‐model ensembles of any size and skill. Additionally, the degree of ensemble under‐dispersion (or overconfidence) can be prescribed. Multi‐model ensembles are then constructed from both weighted and unweighted averages of these single‐model ensembles. Applying this toy model, we carry out systematic model‐combination experiments. We evaluate how multi‐model performance depends on the skill and overconfidence of the participating single models. It turns out that multi‐model ensembles can indeed locally outperform a ‘best‐model’ approach, but only if the single‐model ensembles are overconfident. The reason is that multi‐model combination reduces overconfidence, i.e. ensemble spread is widened while average ensemble‐mean error is reduced. This implies a net gain in prediction skill, because probabilistic skill scores penalize overconfidence. Under these conditions, even the addition of an objectively‐poor model can improve multi‐model skill. It seems that simple ensemble inflation methods cannot yield the same skill improvement. Using seasonal near‐surface temperature forecasts from the DEMETER dataset, we show that the conclusions drawn from the toy‐model experiments hold equally in a real multi‐model ensemble prediction system. Copyright © 2008 Royal Meteorological Society
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Statistical Methods in the Atmospheric Sciences
Ramsey, P. H. (1996) · Technometrics · doi:10.1080/00401706.1996.10484553 · Cited by 7,601
Uppala, S. M., KÅllberg, P. W., Simmons, A. J., Andrae, U., Bechtold, V. D. C., Fiorino, M., Gibson, J. K., Haseler, J., Hernandez, A., Kelly, G. A., et al. (2005) · Quarterly Journal of the Royal Meteorological Society · doi:10.1256/qj.04.176 · Cited by 6,804
A Limited Memory Algorithm for Bound Constrained Optimization
Byrd, R. H., Lu, P., Nocedal, J., & Zhu, C. (1995) · SIAM Journal on Scientific Computing · doi:10.1137/0916069 · Cited by 6,221
Atmospheric Modeling, Data Assimilation and Predictability
Kalnay, E. (2002) · doi:10.1017/cbo9780511802270 · Cited by 3,391
The role of increasing temperature variability in European summer heatwaves
Schär, C., Vidale, P. L., Lüthi, D., Frei, C., Häberli, C., Liniger, M. A., & Appenzeller, C. (2004) · Nature · doi:10.1038/nature02300 · Cited by 2,781
Self-citationGordon, C., Cooper, C., Senior, C. A., Banks, H., Gregory, J. M., Johns, T. C., Mitchell, J. F. B., & Wood, R. A. (2000) · Climate Dynamics · doi:10.1007/s003820050010 · Cited by 2,578
The ECMWF Ensemble Prediction System: Methodology and validation
Molteni, F., Buizza, R., Palmer, T. N., & Petroliagis, T. (1996) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.49712252905 · Cited by 1,616
Wörle, H., Rumpf, H.-J., & Erven, J. (1994) · De Gruyter eBooks · doi:10.1515/9783486785449 · Cited by 1,465
Stochastic representation of model uncertainties in the ECMWF ensemble prediction system
Buizza, R., Milleer, M., & Palmer, T. N. (1999) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.49712556006 · Cited by 1,103
Palmer, T. N., Alessandri, A., Andersen, U., Cantelaube, P., Davey, M., Délécluse, P., Déqué, M., Díez, E., Doblas-Reyes, F. J., Feddersen, H., et al. (2004) · Bulletin of the American Meteorological Society · doi:10.1175/bams-85-6-853 · Cited by 959
Improved Weather and Seasonal Climate Forecasts from Multimodel Superensemble
Krishnamurti, T. N., Kishtawal, C. M., LaRow, T. E., Bachiochi, D. R., Zhang, Z., Williford, C. E., Gadgil, S., & Surendran, S. (1999) · Science · doi:10.1126/science.285.5433.1548 · Cited by 816
The rationale behind the success of multi-model ensembles in seasonal forecasting - I. Basic concept
Hagedorn, R., Doblas-Reyes, F. J., & Palmer, T. N. (2005) · Tellus A: Dynamic Meteorology and Oceanography · doi:10.3402/tellusa.v57i3.14657 · Cited by 710
A Comparison of the ECMWF, MSC, and NCEP Global Ensemble Prediction Systems
Buizza, R., Houtekamer, P. L., Pellerin, G., Toth, Z., Zhu, Y., & Wei, M. (2005) · Monthly Weather Review · doi:10.1175/mwr2905.1 · Cited by 630
Current approaches to seasonal to interannual climate predictions
Goddard, L., Mason, S. J., Zebiak, S. E., Ropelewski, C. F., Basher, R., & Cane, M. A. (2001) · International Journal of Climatology · doi:10.1002/joc.636 · Cited by 588
A Scoring System for Probability Forecasts of Ranked Categories
Epstein, E. S. (1969) · Journal of Applied Meteorology · doi:10.1175/1520-0450(1969)008<0985:assfpf>2.0.co;2 · Cited by 563
Palmer, T. N. (2001) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.49712757202 · Cited by 466
Wang, B., Lee, J.-Y., Kang, I.-S., Shukla, J., Park, C.-K., Kumar, A., Schemm, J., Cocke, S., Kug, J.-S., Luo, J.-J., et al. (2009) · Climate Dynamics · doi:10.1007/s00382-008-0460-0 · Cited by 409
Evaluating Probabilistic Forecasts Using Information Theory
Roulston, M. S., & Smith, L. A. (2002) · Monthly Weather Review · doi:10.1175/1520-0493(2002)130<1653:epfuit>2.0.co;2 · Cited by 350
Doblas-Reyes, F. J., Hagedorn, R., & Palmer, T. N. (2005) · Tellus A: Dynamic Meteorology and Oceanography · doi:10.3402/tellusa.v57i3.14658 · Cited by 340
Operational Ensemble Prediction at the National Meteorological Center: Practical Aspects
Tracton, M. S., & Kalnay, E. (1993) · Weather and Forecasting · doi:10.1175/1520-0434(1993)008<0379:oepatn>2.0.co;2 · Cited by 327
Buizza, R. (1997) · Monthly Weather Review · doi:10.1175/1520-0493(1997)125<0099:pfsoep>2.0.co;2 · Cited by 272
Measuring forecast skill: is it real skill or is it the varying climatology?
Hamill, T. M., & Juras, J. (2006) · Quarterly Journal of the Royal Meteorological Society · doi:10.1256/qj.06.25 · Cited by 264
The Discrete Brier and Ranked Probability Skill Scores
Weigel, A. P., Liniger, M. A., & Appenzeller, C. (2007) · Monthly Weather Review · doi:10.1175/mwr3280.1 · Cited by 254
Self-citationTIGGE: Preliminary results on comparing and combining ensembles
Park, Y., Buizza, R., & Leutbecher, M. (2008) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.334 · Cited by 227
Richardson, D. S. (2001) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.49712757715 · Cited by 212
Impact of Ensemble Size on Ensemble Prediction
Buizza, R., & Palmer, T. N. (1998) · Monthly Weather Review · doi:10.1175/1520-0493(1998)126<2503:ioesoe>2.0.co;2 · Cited by 203
Multimodel Ensembling in Seasonal Climate Forecasting at IRI
Barnston, A. G., Mason, S. J., Goddard, L., DeWitt, D. G., & Zebiak, S. E. (2003) · Bulletin of the American Meteorological Society · doi:10.1175/bams-84-12-1783 · Cited by 190
A Note on the Ranked Probability Score
Murphy, A. H. (1971) · Journal of applied meteorology · doi:10.1175/1520-0450(1971)010<0155:anotrp>2.0.co;2 · Cited by 186
Improved combination of multiple atmospheric GCM ensembles for seasonal prediction.
Robertson, A. W., Lall, U., Zebiak, S. E., & Goddard, L. (2004) · Monthly Weather Review · doi:10.1175/mwr2818.1 · Cited by 154
On Using “Climatology” as a Reference Strategy in the Brier and Ranked Probability Skill Scores
Mason, S. J. (2004) · Monthly Weather Review · doi:10.1175/1520-0493(2004)132<1891:oucaar>2.0.co;2 · Cited by 146
Dynamically‐based seasonal forecasts of Atlantic tropical storm activity issued in June by EUROSIP
Vitart, F., Huddleston, M. R., Déqué, M., Peake, D., Palmer, T. N., Stockdale, T., Davey, M. K., Ineson, S., & Weisheimer, A. (2007) · Geophysical Research Letters · doi:10.1029/2007gl030740 · Cited by 116
Uncertainty and Inference for Verification Measures
Jolliffe, I. T. (2007) · Weather and Forecasting · doi:10.1175/waf989.1 · Cited by 114
Stephenson, D. B., Coelho, C. A. S., Doblas-Reyes, F. J., & Balmaseda, M. (2005) · Tellus A: Dynamic Meteorology and Oceanography · doi:10.3402/tellusa.v57i3.14664 · Cited by 111
International Geophysics Series
(2006) · Elsevier eBooks · doi:10.1016/b978-0-12-732951-2.50019-3 · Cited by 107
Müller, W. A., Appenzeller, C., Doblas-Reyes, F. J., & Liniger, M. A. (2005) · Journal of Climate · doi:10.1175/jcli3361.1 · Cited by 105
Self-citationA Synoptic Evaluation of the NCEP Ensemble
Tóth, Z., Kalnay, E., Tracton, S., Wobus, R. L., & Irwin, J. (1997) · Weather and Forecasting · Cited by 100
Seasonal Predictions, Probabilistic Verifications, and Ensemble Size
Kumar, A., Barnston, A. G., & Hoerling, M. P. (2001) · Journal of Climate · doi:10.1175/1520-0442(2001)014<1671:sppvae>2.0.co;2 · Cited by 97
On the “Ranked Probability Score”
Murphy, A. H. (1969) · Journal of applied meteorology · doi:10.1175/1520-0450(1969)008<0988:otps>2.0.co;2 · Cited by 94
Smoothing forecast ensembles with fitted probability distributions
Wilks, D. S. (2002) · Quarterly Journal of the Royal Meteorological Society · doi:10.1256/qj.01.215 · Cited by 81
Estimation of Seasonal Precipitation Tercile-Based Categorical Probabilities from Ensembles
Tippett, M. K., Barnston, A. G., & Robertson, A. W. (2007) · Journal of Climate · doi:10.1175/jcli4108.1 · Cited by 69
Seasonal ensemble forecasts: Are recalibrated single models better than multimodels?
Weigel, A. P., Liniger, M. A., & Appenzeller, C. (2009) · Monthly Weather Review · doi:10.1175/2008mwr2773.1 · Cited by 66
Self-citationWeigel, A. P., Chow, F. K., & Rotach, M. W. (2006) · Boundary-Layer Meteorology · doi:10.1007/s10546-006-9120-2 · Cited by 63
Self-citationIncreasing the horizontal resolution of ensemble forecasts at CMC
Pellerin, G., Lefaivre, L., Houtekamer, P. L., & Girard, C. (2003) · Nonlinear processes in geophysics · doi:10.5194/npg-10-463-2003 · Cited by 52
Theoretical examination of a multi‐model composite for seasonal prediction
Yoo, J. H., & Kang, I. (2005) · Geophysical Research Letters · doi:10.1029/2005gl023513 · Cited by 42
Challenges posed by and approaches to the study of seasonal-to-decadal climate variability
Schwierz, C., Appenzeller, C., Davies, H. C., Liniger, M. A., Müller, W. A., Stocker, T. F., & Yoshimori, M. (2006) · Climatic Change · doi:10.1007/s10584-006-9076-8 · Cited by 40
Self-citationAnderson, D. L. T., Stockdale, T., Balmaseda, M., Ferranti, L., Vitart, F., Doblas-Reyes, R., Hagedorn, R., Jung, T. J., Vidard, A., & Palmer, T. N. (2002) · Helmholtz-Zentrum für Polar-und Meeresforschung (Alfred-Wegener-Institut) · Cited by 38
Temperature trends in Switzerland and Europe: implications for climate normals
Scherrer, S. C., Appenzeller, C., & Liniger, M. A. (2005) · International Journal of Climatology · doi:10.1002/joc.1270 · Cited by 38
Self-citationWeigel, A. P., Liniger, M. A., & Appenzeller, C. (2007) · Monthly Weather Review · doi:10.1175/mwr3428.1 · Cited by 35
Self-citationThe Benefits of Multianalysis and Poor Man’s Ensembles
Bowler, N. E., Arribas, A., & Mylne, K. R. (2008) · Monthly Weather Review · doi:10.1175/2008mwr2381.1 · Cited by 31
(2000) · doi:10.1175/1520-0477(2000)081<2593:dsp>2.3.co;2
Rajagopalan (2002) · doi:10.1175/1520-0493(2002)130<1792:ccftra>2.0.co;2
(2003) · doi:10.1175/1520-0442(2003)016
Research works printed but not linked: 4
- 10.1111/j.1600-0870.2005.00103.x · doi:10.1111/j.1600-0870.2005.00103.x · Paper page
- 10.1175/1520-0442(2003)016<1684:ISPF>2.0.CO;2 · doi:10.1175/1520-0442(2003)016<1684:ispf>2.0.co;2
- 10.1175/1520-0434(1997)012<0140:ASEOTN>2.0.CO;2 · doi:10.1175/1520-0434(1997)012<0140:aseotn>2.0.co;2
- Toth Z, 2003, Forecast Verification: A Practitioner's Guide in Atmospheric Science, 137
Cited by
Challenges in combining projections from multiple climate models.
Knutti, R., Furrer, R., Tebaldi, C., Cermak, J., & Meehl, G. A. (2010) · Journal of Climate · doi:10.1175/2009jcli3361.1 · Cited by 1,270
A bitter cup: climate change profile of global production of Arabica and Robusta coffee
Bunn, C., Läderach, P., Ovalle Rivera, O., & Kirschke, D. (2015) · Climatic Change · doi:10.1007/s10584-014-1306-x · Cited by 625
Iterative near-term ecological forecasting: Needs, opportunities, and challenges
Dietze, M. C., Fox, A., Beck-Johnson, L. M., Betancourt, J. L., Hooten, M. B., Jarnevich, C. S., Keitt, T. H., Kenney, M. A., Laney, C. M., Larsen, L. G., et al. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1710231115 · Cited by 604
The end of model democracy?: An editorial comment
Knutti, R. (2010) · Climatic Change · doi:10.1007/s10584-010-9800-2 · Cited by 499
Seasonal climate predictability and forecasting: status and prospects
Doblas‐Reyes, F. J., García‐Serrano, J., Lienert, F., Biescas, A. P., & Rodrigues, L. R. L. (2013) · WIREs Climate Change · doi:10.1002/wcc.217 · Cited by 478
Metadata
- Authors
- Weigel, A. P., Liniger, M. A., & Appenzeller, C.
- Venue
- Quarterly Journal of the Royal Meteorological Society
- Publisher
- Wiley
- Published
- 2008-01
- Volume
- 134
- Issue
- 630
- Pages
- 241-260
- ISSN
- 0035-9009 (ISSN-L)
- 1477-870X
- OpenAlex
- W2143323261
- Cited by
- 330
- References
- 52
- FWCI
- 10.78
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.