Cited by 213Meteorological Phenomena and SimulationsForecasting Techniques and ApplicationsClimate variability and models

The DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.

Measures of skill and value of ensemble prediction systems, their interrelationship and the effect of ensemble size

Richardson, David S.·2001·Quarterly Journal of the Royal Meteorological Society

DOI 10.1002/qj.49712757715

Open this paper in CoChat — free account

Abstract

Ensemble forecasts provide probabilistic predictions for the future state of the atmosphere. Usually the probability of a given event E is determined from the fraction of ensemble members which predict the event. Hence there is a degree of sampling error inherent in the predictions. In this paper a theoretical study is made of the effect of ensemble size on forecast performance as measured by a reliability diagram and Brier (skill) score, and on users by using a simple cost‐loss decision model. The relationship between skill and value, and a generalized skill score, dependent on the distribution of users, are discussed. The Brier skill score is reduced from its potential level for all finite‐sized ensembles. The impact is most significant for small ensembles, especially when the variance of forecast probabilities is also small. The Brier score for a set of deterministic forecasts is a measure of potential predictability, assuming the forecasts are representative selections from a reliable ensemble prediction system (EPS). There is a consistent effect of finite ensemble size on the reliability diagram. Even if the underlying distribution is perfectly reliable, sampling this using only a small number of ensemble members introduces considerable unreliability. There is a consistent over‐forecasting which appears as a clockwise tilt of the reliability diagram. It is important to be aware of the expected effect of ensemble size to avoid misinterpreting results. An ensemble of ten o…

Integrity signals

Registry

Crossref · 10.1002/qj.49712757715

Cites

9 of 9 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Statistical Methods in the Atmospheric Sciences
Ramsey, P. H., & Wilks, D. S. (1996) · Technometrics · doi:10.2307/1271312 · Cited by 7,681
Skill and relative economic value of the ECMWF ensemble prediction system
Richardson, D. S. (2000) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.49712656313 · Cited by 491
Self-citation
Preface
Wilks, D. S. (1995) · International geophysics/International geophysics series · doi:10.1016/s0074-6142(06)80036-7 · Cited by 319
On the Evolution of the Southern Oscillation
Trenberth, K. E., & Shea, D. J. (1987) · Monthly Weather Review · doi:10.1175/1520-0493(1987)115 · Cited by 316
The Finley Affair: A Signal Event in the History of Forecast Verification
Murphy, A. H. (1996) · Weather and Forecasting · doi:10.1175/1520-0434(1996)011 · Cited by 246
Some Aspects of Circulation and Climate over the Eastern Equatorial Atlantic
HASTENRATH, S. L., & Lamb, P. (1977) · Monthly Weather Review · doi:10.1175/1520-0493(1977)105 · Cited by 82
On the Relationship between the Accuracy and Value of Forecasts in the Cost–Loss Ratio Situation
Murphy, A. H., & Ehrendorfer, M. (1987) · Weather and Forecasting · doi:10.1175/1520-0434(1987)002 · Cited by 69
A Note on the Utility of Probabilistic Predictions and the Probability Score in the Cost-Loss Ratio Decision Situation
Murphy, A. H. (1966) · Journal of applied meteorology · doi:10.1175/1520-0450(1966)005 · Cited by 52
Untitled work
(1995) · doi:10.1175/1520-0493(1995)123

Show all 9 references

Research works printed but not linked: 1

Talagrand O. Vautard R.andStrauss B.1997‘Evaluation of probabilistic prediction systems’. Pp.157–157in Proceedings of the ECMWF workshop on predictability 20–22 October 1997 ECMWF Shinfield Park Reading UK

Sorted by citations

Forecast verification : a practitioner's guide in atmospheric science
Jolliffe, I., & Stephenson, D. (2011) · Wiley eBooks · doi:10.1002/9781119960003 · Cited by 1,457
Ensemble flood forecasting: A review
Cloke, H., & Pappenberger, F. (2009) · Journal of Hydrology · doi:10.1016/j.jhydrol.2009.06.005 · Cited by 1,146
Calibrated Probabilistic Forecasting Using Ensemble Model Output Statistics and Minimum CRPS Estimation
Gneiting, T., Raftery, A. E., Westveld, A. H., & Goldman, T. (2005) · Monthly Weather Review · doi:10.1175/mwr2904.1 · Cited by 1,142
The North American Multimodel Ensemble: Phase-1 Seasonal-to-Interannual Prediction; Phase-2 toward Developing Intraseasonal Prediction
Kirtman, B. P., Min, D., Infanti, J. M., Kinter, J. L., Paolino, D. A., Zhang, Q., van den Dool, H., Saha, S., Mendez, M. P., Becker, E. J., et al. (2013) · Bulletin of the American Meteorological Society · doi:10.1175/bams-d-12-00050.1 · Cited by 1,032
Ensemble forecasting
Leutbecher, M., & Palmer, T. N. (2007) · Journal of Computational Physics · doi:10.1016/j.jcp.2007.02.014 · Cited by 920
Forecast Verification
Wilks, D. S. (2011) · International Geophysics · doi:10.1016/b978-0-12-385022-5.00008-7 · Cited by 727
The ROC Curve and the Area under It as Performance Measures
Marzban, C. (2004) · Weather and Forecasting · doi:10.1175/825.1 · Cited by 337
Can multi‐model combination really enhance the prediction skill of probabilistic ensemble forecasts?
Weigel, A. P., Liniger, M. A., & Appenzeller, C. (2008) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.210 · Cited by 329
Economic Value of Weather and Climate Forecasts
Katz, R. W., & Lazo, J. K. (2012) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780195398649.013.0021 · Cited by 304
The economic value of ensemble forecasts as a tool for risk assessment: From days to decades
Palmer, T. N. (2002) · Quarterly Journal of the Royal Meteorological Society · doi:10.1256/0035900021643593 · Cited by 277
Measuring forecast skill: is it real skill or is it the varying climatology?
Hamill, T., & Juras, J. (2006) · Quarterly Journal of the Royal Meteorological Society · doi:10.1256/qj.06.25 · Cited by 266
The Discrete Brier and Ranked Probability Skill Scores
Weigel, A. P., Liniger, M. A., & Appenzeller, C. (2007) · Monthly Weather Review · doi:10.1175/mwr3280.1 · Cited by 254
RECENT CHANGES IN FROST DAYS AND THE FROST-FREE SEASON IN THE UNITED STATES
Easterling, D. R. (2002) · Bulletin of the American Meteorological Society · doi:10.1175/1520-0477-83.9.1327 · Cited by 216
Self-citation
An Overview of the 2010 Hazardous Weather Testbed Experimental Forecast Program Spring Experiment
Clark, A. J., Weiss, S. J., Kain, J. S., Jirak, I. L., Coniglio, M. C., Melick, C. J., Siewert, C., Sobash, R. A., Marsh, P. T., Dean, A. R., et al. (2011) · Bulletin of the American Meteorological Society · doi:10.1175/bams-d-11-00040.1 · Cited by 243
A Comparison of Precipitation Forecast Skill between Small Convection-Allowing and Large Convection-Parameterizing Ensembles
Clark, A. J., Gallus, W. A., Xue, M., & Kong, F. (2009) · Weather and Forecasting · doi:10.1175/2009waf2222222.1 · Cited by 236
The rationale behind the success of multi-model ensembles in seasonal forecasting - II. Calibration and combination
Doblas-Reyes, F. J., Hagedorn, R., & Palmer, T. N. (2005) · Tellus A: Dynamic Meteorology and Oceanography · doi:10.3402/tellusa.v57i3.14658 · Cited by 340
On the effect of ensemble size on the discrete and continuous ranked probability scores
Ferro, C. A. T., Richardson, D., & Weigel, A. (2008) · Meteorological Applications · doi:10.1002/met.45 · Cited by 169
Self-citation
The Development of the NCEP Global Ensemble Forecast System Version 12
Zhou, X., Zhu, Y., Hou, D., Fu, B., Li, W., Guan, H., Sinsky, E., Kolczynski, W. C., Xue, X., Luo, Y., et al. (2022) · Weather and Forecasting · doi:10.1175/waf-d-21-0112.1 · Cited by 164
Skill and relative economic value of medium-range hydrological ensemble predictions
Roulin, E. (2006) · Hydrology and earth system sciences · doi:10.5194/hess-11-725-2007 · Cited by 156
Fog Prediction from a Multimodel Mesoscale Ensemble Prediction System
Zhou, B., & Du, J. (2009) · Weather and Forecasting · doi:10.1175/2009waf2222289.1 · Cited by 158
The development of seasonal climate forecasting for agricultural producers
Klemm, T., & McPherson, R. A. (2016) · Agricultural and Forest Meteorology · doi:10.1016/j.agrformet.2016.09.005 · Cited by 154
Probabilistic Precipitation Forecast Skill as a Function of Ensemble Size and Spatial Scale in a Convection-Allowing Ensemble
Clark, A. J., Kain, J. S., Stensrud, D. J., Xue, M., Kong, F., Coniglio, M. C., Thomas, K. W., Wang, Y., Brewster, K. A., Gao, J., et al. (2011) · Monthly Weather Review · doi:10.1175/2010mwr3624.1 · Cited by 140
From Climate-Change Spaghetti to Climate-Change Distributions for 21st Century California
Survey, U. S. G., & Dettinger, M. D. (2005) · San Francisco Estuary and Watershed Science · doi:10.15447/sfews.2005v3iss1art6 · Cited by 139
Ensemble Bayesian model averaging using Markov Chain Monte Carlo sampling
Vrugt, J. A., Diks, C. G. H., & Clark, M. (2008) · Environmental Fluid Mechanics · doi:10.1007/s10652-008-9106-3 · Cited by 137
Addressing model uncertainty in seasonal and annual dynamical ensemble forecasts
Doblas-Reyes, F. J., Weisheimer, A., Déqué, M., Keenlyside, N. S., McVean, M., Murphy, J. M., Rogel, P., Smith, D. M., & Palmer, T. N. (2009) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.464 · Cited by 134
Ensemble size: How suboptimal is less than infinity?
Leutbecher, M. (2019) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.3387 · Cited by 126
Verification of an Ensemble Prediction System against Observations
Candille, G., Cote, C., Houtekamer, P., & Pellerin, G. (2007) · Monthly Weather Review · doi:10.1175/mwr3414.1 · Cited by 117
Comparison of ensemble‐MOS methods in the Lorenz '96 setting
Wilks, D. S. (2006) · Meteorological Applications · doi:10.1017/s1350482706002192 · Cited by 142
Improvement of ensemble reliability with a new dressing kernel
Wang, X., & Bishop, C. (2005) · Quarterly Journal of the Royal Meteorological Society · doi:10.1256/qj.04.120 · Cited by 115
A machine learning model that outperforms conventional global subseasonal forecast models
Chen, L., Zhong, X., Li, H., Wu, J., Lu, B., Chen, D., Xie, S.-P., Wu, L., Chao, Q., Lin, C., et al. (2024) · Nature Communications · doi:10.1038/s41467-024-50714-1 · Cited by 105
Comparing Probabilistic Forecasting Systems with the Brier Score
Ferro, C. (2007) · Weather and Forecasting · doi:10.1175/waf1034.1 · Cited by 105
Extended range prediction of active-break spells of Indian summer monsoon rainfall using an ensemble prediction system in NCEP Climate Forecast System
Abhilash, S. R., Sahai, A. K., Pattnaik, S., Goswami, B. N., & Kumar, A. (2013) · International Journal of Climatology · doi:10.1002/joc.3668 · Cited by 100
Subseasonal Tropical Cyclone Genesis Prediction and MJO in the S2S Dataset
Lee, C., Camargo, S. J., Vitart, F., Sobel, A. H., & Tippett, M. K. (2018) · Weather and Forecasting · doi:10.1175/waf-d-17-0165.1 · Cited by 98
Trends in the predictive performance of raw ensemble weather forecasts
Hemri, S., Scheuerer, M., Pappenberger, F., Bogner, K., & Haiden, T. (2014) · Geophysical Research Letters · doi:10.1002/2014gl062472 · Cited by 98
A Laboratory Study of the Benefits of Including Uncertainty Information in Weather Forecasts
Roulston, M. S., Bolton, G. E., Kleit, A. N., & Sears-Collins, A. L. (2006) · Weather and Forecasting · doi:10.1175/waf887.1 · Cited by 98
Promoting the use of probabilistic weather forecasts through a dialogue between scientists, developers and end‐users
Fundel, V. J., Fleischhut, N., Herzog, S. M., Göber, M., & Hagedorn, R. (2019) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.3482 · Cited by 104
Forecast of Low Visibility and Fog from NCEP: Current Status and Efforts
Zhou, B., Du, J., Gültepe, I., & DiMego, G. (2011) · Pure and Applied Geophysics · doi:10.1007/s00024-011-0327-x · Cited by 92
Perturbation of convection-permitting NWP forecasts for flash-flood ensemble forecasting
Vincendon, B., Ducrocq, V., Nuissier, O., & Vié, B. (2011) · Natural hazards and earth system sciences · doi:10.5194/nhess-11-1529-2011 · Cited by 74
Spatial and Interannual Variability of the Reliability of Ensemble-Based Probabilistic Forecasts: Consequences for Calibration
Atger, F. (2003) · Monthly Weather Review · doi:10.1175//1520-0493(2003)131 · Cited by 85
The skill of multi-model seasonal forecasts of the wintertime North Atlantic Oscillation
Doblas-Reyes, F. J., Pavan, V., & Stephenson, D. B. (2003) · Climate Dynamics · doi:10.1007/s00382-003-0350-4 · Cited by 86
How confident are predictability estimates of the winter North Atlantic Oscillation?
Weisheimer, A., Decremer, D., MacLeod, D. A., O’Reilly, C., Stockdale, T., Johnson, S. J., & Palmer, T. N. (2018) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.3446 · Cited by 81
Smoothing forecast ensembles with fitted probability distributions
Wilks, D. S. (2002) · Quarterly Journal of the Royal Meteorological Society · doi:10.1256/qj.01.215 · Cited by 81
Benefits of increased resolution in the ECMWF ensemble system and comparison with poor‐man's ensembles
Buizza, R., Richardson, D. S., & Palmer, T. N. (2003) · Quarterly Journal of the Royal Meteorological Society · doi:10.1256/qj.02.92 · Cited by 79
Self-citation
Forecast Verification
Wilks, D. S. (2019) · Statistical Methods in the Atmospheric Sciences · doi:10.1016/b978-0-12-815823-4.00009-2 · Cited by 130
Improving the Detection and Tracking of Tropical Cyclones in Atmospheric General Circulation Models
Camargo, S. J., & Zebiak, S. E. (2002) · Weather and Forecasting · doi:10.1175/1520-0434(2002)017 · Cited by 173
Analog Probabilistic Precipitation Forecasts Using GEFS Reforecasts and Climatology-Calibrated Precipitation Analyses*
Hamill, T. M., Scheuerer, M., & Bates, G. T. (2015) · Monthly Weather Review · doi:10.1175/mwr-d-15-0004.1 · Cited by 77
The impact of horizontal resolution and ensemble size for convective‐scale probabilistic forecasts
Raynaud, L., & Bouttier, F. (2017) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.3159 · Cited by 77
The Multiensemble Approach: The NAEFS Example
Candille, G. (2008) · Monthly Weather Review · doi:10.1175/2008mwr2682.1 · Cited by 71
Estimation of Seasonal Precipitation Tercile-Based Categorical Probabilities from Ensembles
Tippett, M. K., Barnston, A. G., & Robertson, A. W. (2007) · Journal of Climate · doi:10.1175/jcli4108.1 · Cited by 69
Seasonal ensemble forecasts: Are recalibrated single models better than multimodels?
Weigel, A. P., Liniger, M. A., & Appenzeller, C. (2009) · Monthly Weather Review · doi:10.1175/2008mwr2773.1 · Cited by 66
Skill of Multimodel ENSO Probability Forecasts
Tippett, M. K., & Barnston, A. G. (2008) · Monthly Weather Review · doi:10.1175/2008mwr2431.1 · Cited by 64
A forecast quality assessment of an end-to-end probabilistic multi-model seasonal forecast system using a malaria model
Morse, A. P., Doblas-Reyes, F. J., Hoshen, M. B., Hagedorn, R., & Palmer, T. N. (2005) · Tellus A Dynamic Meteorology and Oceanography · doi:10.3402/tellusa.v57i3.14668 · Cited by 64
Reliability in ensemble data assimilation
Rodwell, M. J., Lang, S. T. K., Ingleby, N. B., Bormann, N., Hólm, E. V., Rabier, F., Richardson, D. S., & Yamaguchi, M. (2015) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.2663 · Cited by 63
Self-citation
Probabilistic Seasonal Forecasts in the North American Multimodel Ensemble: A Baseline Skill Assessment
Becker, E., & van den Dool, H. M. (2016) · Journal of Climate · doi:10.1175/jcli-d-14-00862.1 · Cited by 56
Verification of inflow into hydropower reservoirs using ensemble forecasts of the TIGGE database for large scale basins in Brazil
Fan, F. M., Schwanenberg, D., Collischonn, W., & Weerts, A. (2015) · Journal of Hydrology Regional Studies · doi:10.1016/j.ejrh.2015.05.012 · Cited by 61
Seasonal streamflow forecasts for Europe – Part 2: Sources of skill
Greuell, W., Franssen, W., & Hutjes, R. (2019) · Hydrology and earth system sciences · doi:10.5194/hess-23-371-2019 · Cited by 60
Convection-Allowing and Convection-Parameterizing Ensemble Forecasts of a Mesoscale Convective Vortex and Associated Severe Weather Environment
Clark, A. J., Gallus, W. A., Xue, M., & Kong, F. (2010) · Weather and Forecasting · doi:10.1175/2010waf2222390.1 · Cited by 59
Probabilistic Verification of Monthly Temperature Forecasts
Weigel, A. P., Baggenstos, D., Liniger, M. A., Vitart, F., & Appenzeller, C. (2008) · Monthly Weather Review · doi:10.1175/2008mwr2551.1 · Cited by 59
Bayesian model averaging using particle filtering and Gaussian mixture modeling: Theory, concepts, and simulation experiments
Rings, J., Vrugt, J. A., Schoups, G., Huisman, J. A., & Vereecken, H. (2012) · Water Resources Research · doi:10.1029/2011wr011607 · Cited by 63
The NCEP–FNMOC Combined Wave Ensemble Product: Expanding Benefits of Interagency Probabilistic Forecasts to the Oceanic Environment
Alves, J.-H., Wittmann, P. A., Sestak, M., Schauer, J., Stripling, S., Bernier, N. B., McLean, J., Chao, Y. Y., Chawla, A., Tolman, H. L., et al. (2013) · Bulletin of the American Meteorological Society · doi:10.1175/bams-d-12-00032.1 · Cited by 58
The Effect of Probabilistic Information on Threshold Forecasts
Joslyn, S. L., Pak, K., Jones, D. A., Pyles, J. A., & Hunt, E. B. (2007) · Weather and Forecasting · doi:10.1175/waf1020.1 · Cited by 54
GROUND WATER/SURFACE WATER RESPONSES TO GLOBAL CLIMATE SIMULATIONS, SANTA CLARA-CALLEGUAS BASIN, VENTURA, CALIFORNIA
Hanson, R. T., & Dettinger, M. D. (2005) · JAWRA Journal of the American Water Resources Association · doi:10.1111/j.1752-1688.2005.tb03752.x · Cited by 52
Performance targets and the Brier score
Roulston, M. (2007) · Meteorological Applications · doi:10.1002/met.21 · Cited by 56
Contributions of Mixed Physics versus Perturbed Initial/Lateral Boundary Conditions to Ensemble-Based Precipitation Forecast Skill
Clark, A. J., Gallus, W. A., & Chen, T.-C. (2008) · Monthly Weather Review · doi:10.1175/2007mwr2029.1 · Cited by 48
How does the quantification of uncertainties affect the quality and value of flood early warning systems?
Thiboult, A., Anctil, F., & Ramos, M. (2017) · Journal of Hydrology · doi:10.1016/j.jhydrol.2017.05.014 · Cited by 47
Using Varied Microphysics to Account for Uncertainty in Warm-Season QPF in a Convection-Allowing Ensemble
Duda, J. D., Wang, X., Kong, F., & Xue, M. (2014) · Monthly Weather Review · doi:10.1175/mwr-d-13-00297.1 · Cited by 46
High-resolution operational monsoon forecasts: an objective assessment
Sahai, A. K., Abhilash, S. R., Chattopadhyay, R., Borah, N., Joseph, S., Sharmila, S., & Rajeevan, M. N. (2014) · Climate Dynamics · doi:10.1007/s00382-014-2210-9 · Cited by 46
A laboratory‐based study of understanding of uncertainty in 5‐day site‐specific temperature forecasts
Roulston, M. S., & Kaplan, T. R. (2008) · Meteorological Applications · doi:10.1002/met.113 · Cited by 46
A Hybrid NWP–Analog Ensemble
Eckel, F. A., & Monache, L. D. (2015) · Monthly Weather Review · doi:10.1175/mwr-d-15-0096.1 · Cited by 44
An effective configuration of ensemble size and horizontal resolution for the NCEP GEFS
Ma, J., Zhu, Y., Wobus, R. L., & Wang, P. (2012) · Advances in Atmospheric Sciences · doi:10.1007/s00376-012-1249-y · Cited by 42
The Effect of Decision Analysis on Power System Resilience and Economic Value During a Severe Weather Event
Wu, Y., Chen, Y., Chang, H.-L., & Hong, J. (2022) · IEEE Transactions on Industry Applications · doi:10.1109/tia.2022.3145753 · Cited by 41
An alternate approach to ensemble ENSO forecast spread: Application to the 2014 forecast
Larson, S. M., & Kirtman, B. P. (2015) · Geophysical Research Letters · doi:10.1002/2015gl066173 · Cited by 40
Real-time assimilation of streamflow observations into a hydrological routing model: effects of model structures and updating methods
Mazzoleni, M., Noh, S. J., Lee, H., Liu, Y., Seo, D., Amaranto, A., Alfonso, L., & SOLOMATINE, D. P. (2018) · Hydrological Sciences Journal · doi:10.1080/02626667.2018.1430898 · Cited by 40
Quality and Value of Seasonal Precipitation Forecasts Issued by the West African Regional Climate Outlook Forum
Bliefernicht, J., Waongo, M., Salack, S., Seidel, J., Laux, P., & Kunstmann, H. G. (2018) · Journal of Applied Meteorology and Climatology · doi:10.1175/jamc-d-18-0066.1 · Cited by 39
Ensemble Methods for Meteorological Predictions
Du, J., Berner, J., Buizza, R., Charron, M., Houtekamer, P. L., Hou, D., Jankov, I., Mu, M., Wang, X., Wei, M., et al. (2018) · Springer eBooks · doi:10.1007/978-3-642-40457-3_13-1 · Cited by 39
On the Reliability of the Rank Histogram
Wilks, D. S. (2010) · Monthly Weather Review · doi:10.1175/2010mwr3446.1 · Cited by 39
What determines perceived value of seasonal climate forecasts? A theoretical analysis
Millner, A., & Washington, R. (2010) · Global Environmental Change · doi:10.1016/j.gloenvcha.2010.08.001 · Cited by 38
Evaluation of Short-Range Quantitative Precipitation Forecasts from a Time-Lagged Multimodel Ensemble
Yuan, H., Lu, C., McGinley, J. A., Schultz, P. J., Jamison, B. D., Wharton, L. S., & Anderson, C. J. (2008) · Weather and Forecasting · doi:10.1175/2008waf2007053.1 · Cited by 36
Short-Range Ensemble Forecasts of Precipitation Type
Wandishin, M. S., Baldwin, M. E., Mullen, S. L., & Cortinas, J. (2005) · Weather and Forecasting · doi:10.1175/waf871.1 · Cited by 36
Generalization of the discrete Brier and ranked probability skill scores for weighted multimodel ensemble forecasts.
Weigel, A. P., Liniger, M. A., & Appenzeller, C. (2007) · Monthly Weather Review · doi:10.1175/mwr3428.1 · Cited by 35
Model complexity versus ensemble size: allocating resources for climate prediction
Ferro, C. A. T., Jupp, T. E., Lambert, F. H., Huntingford, C., & Cox, P. M. (2012) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2011.0307 · Cited by 34
ENSO-conditioned weather resampling method for seasonal ensemble streamflow prediction
Beckers, J. V. L., Weerts, A., Tijdeman, E., & Welles, E. (2016) · Hydrology and earth system sciences · doi:10.5194/hess-20-3277-2016 · Cited by 33
Improving Week-2 Forecasts with Multimodel Reforecast Ensembles
Whitaker, J. S., Wei, X., & Vitart, F. (2006) · Monthly Weather Review · doi:10.1175/mwr3175.1 · Cited by 33
Lagged Ensembles, Forecast Configuration, and Seasonal Predictions
Chen, M., Wang, W., & Kumar, A. (2013) · Monthly Weather Review · doi:10.1175/mwr-d-12-00184.1 · Cited by 32
Bayesian Approach to Decision Making Using Ensemble Weather Forecasts
Katz, R. W., & Ehrendorfer, M. (2006) · Weather and Forecasting · doi:10.1175/waf913.1 · Cited by 32
Hydrometeorological Short-Range Ensemble Forecasts in Complex Terrain. Part II: Economic Evaluation
McCollor, D., & Stull, R. B. (2008) · Weather and Forecasting · doi:10.1175/2007waf2007064.1 · Cited by 32
The fractions skill score for ensemble forecast verification
Necker, T., Wolfgruber, L., Kugler, L., Weißmann, M., Dorninger, M., & Serafin, S. (2024) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.4824 · Cited by 30
Enforcing calibration in ensemble postprocessing
Wilks, D. S. (2017) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.3185 · Cited by 30
Statistical Forecasting
Wilks, D. S. (2011) · International geophysics/International geophysics series · doi:10.1016/b978-0-12-385022-5.00007-5 · Cited by 29
Evaluating decadal predictions of northern hemispheric cyclone frequencies
Kruschke, T., Rust, H. W., Kadow, C., Leckebusch, G. C., & Ulbrich, U. (2014) · Tellus A Dynamic Meteorology and Oceanography · doi:10.3402/tellusa.v66.22830 · Cited by 28
Predictive verification for the design of partially exchangeable multi-model ensembles
Bouallègue, Z. B., Ferro, C. A. T., Leutbecher, M., & Richardson, D. S. (2019) · Tellus A Dynamic Meteorology and Oceanography · doi:10.1080/16000870.2019.1697165 · Cited by 26
Self-citation
Assessing the Impact of L-Band Observations on Drought and Flood Risk Estimation: A Decision-Theoretic Approach in an OSSE Environment
Kumar, S. V., Harrison, K. W., Peters‐Lidard, C. D., Santanello, J. A., & Kirschbaum, D. B. (2014) · Journal of Hydrometeorology · doi:10.1175/jhm-d-13-0204.1 · Cited by 26
An assessment of real‐time extended range forecast of 2013 Indian summer monsoon
Borah, N., Sahai, A. K., Abhilash, S. R., Chattopadhyay, R., Joseph, S., Sharmila, S., & Kumar, A. (2014) · International Journal of Climatology · doi:10.1002/joc.4178 · Cited by 26
Ensemble Methods for Meteorological Predictions
Du, J., Berner, J., Buizza, R., Charron, M., Houtekamer, P. L., Hou, D., Jankov, I., Mu, M., Wang, X., Wei, M., et al. (2019) · Springer eBooks · doi:10.1007/978-3-642-39925-1_13 · Cited by 25
Huge ensembles – Part 2: Properties of a huge ensemble of hindcasts generated with spherical Fourier neural operators
Mahesh, A., Collins, W. D., Bonev, B., Brenowitz, N., Cohen, Y., Harrington, P., Kashinath, K., Kurth, T., North, J. S., O'Brien, T. A., et al. (2025) · Geoscientific model development · doi:10.5194/gmd-18-5605-2025 · Cited by 24
Predictability and economic value
Richardson, D. S. (2006) · Cambridge University Press eBooks · doi:10.1017/cbo9780511617652.026 · Cited by 24
Self-citation
Practical Aspects of Statistical Postprocessing
Hamill, T. M. (2018) · Elsevier eBooks · doi:10.1016/b978-0-12-812372-0.00007-8 · Cited by 24
The ensemble‐adjusted Ignorance Score for forecasts issued as normal distributions
Siegert, S., Ferro, C. A. T., Stephenson, D. B., & Leutbecher, M. (2018) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.3447 · Cited by 24
Calibrated Probabilistic Forecasts of Arctic Sea Ice Concentration
Dirkson, A., Merryfield, W. J., & Monahan, A. H. (2018) · Journal of Climate · doi:10.1175/jcli-d-18-0224.1 · Cited by 24
The Development of Seasonal and Inter-Annual Climate Forecasting
Harrison, M. (2005) · Climatic Change · doi:10.1007/s10584-005-5947-7 · Cited by 24

Authors: Richardson, D. S.
Affiliations: European Centre for Medium-Range Weather Forecasts (GB)
Corresponding author: Richardson, David S.
ORCID: Richardson, David S.
Venue: Quarterly Journal of the Royal Meteorological Society
Publisher: Wiley
Published: 2001-10
Volume: 127
Issue: 577
Pages: 2473-2489
ISSN: 0035-9009 (ISSN-L)
1477-870X
DOI: 10.1002/qj.49712757715
OpenAlex: W2089463042
Cited by: 212
References: 9
Citations by year: 2026: 8 · 2025: 9 · 2024: 10 · 2023: 8 · 2022: 11 · 2021: 7 · 2020: 6 · 2019: 13 · 2018: 14 · 2017: 6 · 2016: 8 · 2015: 8 · 2014: 12 · 2013: 7 · 2012: 6
FWCI: 3.05
Type: journal article
Language: en
Open access: closed
Licence: onlinelibrary.wiley.com/termsAndConditions#vor (vor, 1905 d)
Locations: Quarterly Journal of the Royal Meteorological Society (publishedVersion)
Topics: Physical Sciences › Earth and Planetary Sciences › Atmospheric Science › Meteorological Phenomena and Simulations
Social Sciences › Decision Sciences › Management Science and Operations Research › Forecasting Techniques and Applications
Physical Sciences › Environmental Science › Global and Planetary Change › Climate variability and models
Keywords: ensemble prediction systems, ensemble size, Brier skill score, reliability diagrams, forecast reliability, sampling error, forecast value, probabilistic forecasting, predictability, extreme events, user sensitivity

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Measures of skill and value of ensemble prediction systems, their interrelationship and the effect of ensemble size

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks