The DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Skill and relative economic value of the ECMWF ensemble prediction system
Abstract
The economic value of the European Centre for Medium‐Range Weather Forecasts (ECMWF) operational ensemble prediction system (EPS) is assessed relative to the value of a perfect deterministic forecast. The EPS has substantial relative value throughout the medium range. Probability forecasts derived from the EPS are of greater benefit than a deterministic forecast produced by the same model. Indeed, for many users, the probability forecasts have more value than a shorter‐range deterministic forecast. Based on the measures used here, the additional information in the EPS (reflecting the uncertainty in the initial conditions) provides a benefit to users equivalent to many years' development of the forecast model and assimilation system. The impact of ensemble size on forecast value is considered. The difference in performance between ensembles with 10 and with 50 members may appear relatively small, based on standard skill measures, yet the larger ensembles have substantial benefit to a range of users. Further increases in ensemble size may be expected to provide additional value.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The ECMWF Ensemble Prediction System: Methodology and validation
Molteni, F., Buizza, R., Palmer, T. N., & Petroliagis, T. (1996) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.49712252905 · Cited by 1,616
What Is a Good Forecast? An Essay on the Nature of Goodness in Weather Forecasting
Murphy, A. H. (1993) · Weather and Forecasting · doi:10.1175/1520-0434(1993)008<0281:wiagfa>2.0.co;2 · Cited by 833
Wilks, D. S. (1995) · International Geophysics · doi:10.1016/s0074-6142(06)80036-7 · Cited by 319
Murphy, A. H. (1977) · Monthly Weather Review · doi:10.1175/1520-0493(1977)105<0803:tvocca>2.0.co;2 · Cited by 270
Equitable Skill Scores for Categorical Forecasts
Gandin, L. S., & Murphy, A. H. (1992) · Monthly Weather Review · doi:10.1175/1520-0493(1992)120<0361:essfcf>2.0.co;2 · Cited by 243
Impact of Ensemble Size on Ensemble Prediction
Buizza, R., & Palmer, T. N. (1998) · Monthly Weather Review · doi:10.1175/1520-0493(1998)126<2503:ioesoe>2.0.co;2 · Cited by 203
Probabilistic Predictions of Precipitation Using the ECMWF Ensemble Prediction System
Buizza, R., Hollingsworth, A., Lalaurette, F., & Ghelli, A. (1999) · Weather and Forecasting · doi:10.1175/1520-0434(1999)014<0168:ppoput>2.0.co;2 · Cited by 193
A probability and decision‐model analysis of PROVOST seasonal multi‐model ensemble integrations
Palmer, T. N., Branković, Č., & Richardson, D. S. (2000) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.49712656703 · Cited by 189
Self-citationEconomic Value of Weather and Climate Forecasts
(1997) · Cambridge University Press eBooks · doi:10.1017/cbo9780511608278 · Cited by 176
Error growth and estimates of predictability from the ECMWF forecasting system
Simmons, A. J., Mureau, R., & Petroliagis, T. (1995) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.49712152711 · Cited by 163
Cited by
Atmospheric Modeling, Data Assimilation and Predictability
Kalnay, E. (2002) · doi:10.1017/cbo9780511802270 · Cited by 3,391
An Ensemble Adjustment Kalman Filter for Data Assimilation
Anderson, J. L. (2001) · Monthly Weather Review · doi:10.1175/1520-0493(2001)129<2884:aeakff>2.0.co;2 · Cited by 1,749
Decomposition of the Continuous Ranked Probability Score for Ensemble Prediction Systems
Hersbach, H. (2000) · Weather and Forecasting · doi:10.1175/1520-0434(2000)015<0559:dotcrp>2.0.co;2 · Cited by 1,531
Roberts, N. M., & Lean, H. W. (2008) · Monthly Weather Review · doi:10.1175/2007mwr2123.1 · Cited by 1,361
Ensemble flood forecasting: A review
Cloke, H. L., & Pappenberger, F. (2009) · Journal of Hydrology · doi:10.1016/j.jhydrol.2009.06.005 · Cited by 1,139
Skilful precipitation nowcasting using deep generative models of radar
Ravuri, S., Lenc, K., Willson, M., Kangin, D., Lam, R., Mirowski, P., Fitzsimons, M., Athanassiadou, M., Kashem, S., Madge, S., et al. (2021) · Nature · doi:10.1038/s41586-021-03854-z · Cited by 1,024
Palmer, T. N., Alessandri, A., Andersen, U., Cantelaube, P., Davey, M., Délécluse, P., Déqué, M., Díez, E., Doblas-Reyes, F. J., Feddersen, H., et al. (2004) · Bulletin of the American Meteorological Society · doi:10.1175/bams-85-6-853 · Cited by 959
“Brankovic, and D. S. Richardson, 2000: A probability and decision-model analysis of PROVOST seasonal multi-model ensemble integrations.”
BackgroundGneiting, T., & Katzfuss, M. (2014) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-062713-085831 · Cited by 888
Leutbecher, M., & Palmer, T. N. (2008) · Journal of Computational Physics · doi:10.1016/j.jcp.2007.02.014 · Cited by 886
Mason, S. J., & Graham, N. E. (2002) · Quarterly Journal of the Royal Meteorological Society · doi:10.1256/003590002320603584 · Cited by 868
Wilks, D. S. (2011) · International Geophysics · doi:10.1016/b978-0-12-385022-5.00008-7 · Cited by 727
The rationale behind the success of multi-model ensembles in seasonal forecasting - I. Basic concept
Hagedorn, R., Doblas-Reyes, F. J., & Palmer, T. N. (2005) · Tellus A: Dynamic Meteorology and Oceanography · doi:10.3402/tellusa.v57i3.14657 · Cited by 710
Quantifying the risk of extreme seasonal precipitation events in a changing climate
Palmer, T. N., & Räisänen, J. (2002) · Nature · doi:10.1038/415512a · Cited by 604
Fuzzy verification of high‐resolution gridded forecasts: a review and proposed framework
Ebert, E. E. (2008) · Meteorological Applications · doi:10.1002/met.25 · Cited by 480
Malaria early warnings based on seasonal climate forecasts from multi-model ensembles
Thomson, M. C., Doblas-Reyes, F. J., Mason, S. J., Hagedorn, R., Connor, S. J., Phindela, T., Morse, A. P., & Palmer, T. N. (2006) · Nature · doi:10.1038/nature04503 · Cited by 470
Probabilistic weather forecasting with machine learning
Price, I., Sanchez-Gonzalez, A., Alet, F., Andersson, T. R., El-Kadi, A., Masters, D., Ewalds, T., Stott, J., Mohamed, S., Battaglia, P., et al. (2025) · Nature · doi:10.1038/s41586-024-08252-9 · Cited by 434
“We thus use relative economic value (REV) curves 37,38 (for full details, see Supplementary Information section A.5.6) as a standard tool to characterize the potential value of a forecast over a range of different probability decision thresholds.”
InfluentialEstimation of high-dimensional prior and posterior covariance matrices in Kalman filter variants
Furrer, R., & Bengtsson, T. (2007) · Journal of Multivariate Analysis · doi:10.1016/j.jmva.2006.08.003 · Cited by 414
Wang, B., Lee, J.-Y., Kang, I.-S., Shukla, J., Park, C.-K., Kumar, A., Schemm, J., Cocke, S., Kug, J.-S., Luo, J.-J., et al. (2009) · Climate Dynamics · doi:10.1007/s00382-008-0460-0 · Cited by 409
A Comparison of Breeding and Ensemble Transform Kalman Filter Ensemble Forecast Schemes
Wang, X., & Bishop, C. H. (2003) · Journal of the Atmospheric Sciences · doi:10.1175/1520-0469(2003)060<1140:acobae>2.0.co;2 · Cited by 389
Evaluating Probabilistic Forecasts Using Information Theory
Roulston, M. S., & Smith, L. A. (2002) · Monthly Weather Review · doi:10.1175/1520-0493(2002)130<1653:epfuit>2.0.co;2 · Cited by 350
Probabilistic Weather Prediction with an Analog Ensemble
Delle Monache, L., Eckel, F. A., Rife, D. L., Nagarajan, B., & Searight, K. (2013) · Monthly Weather Review · doi:10.1175/mwr-d-12-00281.1 · Cited by 339
Forecast verification: Current status and future directions
Casati, B., Wilson, L. J., Stephenson, D. B., Nurmi, P., Ghelli, A., Pocernich, M., Damrath, U., Ebert, E. E., Brown, B. G., & Mason, S. (2008) · Meteorological Applications · doi:10.1002/met.52 · Cited by 306
“As an example, Richardson (2000) describes a method to estimate the value of the ECMWF ensemble prediction system with respect to a deterministic forecast, which depends on the user’s cost and loss ratio.”
MethodThe economic value of ensemble forecasts as a tool for risk assessment: From days to decades
Palmer, T. N. (2002) · Quarterly Journal of the Royal Meteorological Society · doi:10.1256/0035900021643593 · Cited by 276
Wei, M., Toth, Z., Wobus, R., & Zhu, Y. (2008) · Tellus A: Dynamic Meteorology and Oceanography · doi:10.1111/j.1600-0870.2007.00273.x · Cited by 270
Measuring forecast skill: is it real skill or is it the varying climatology?
Hamill, T. M., & Juras, J. (2006) · Quarterly Journal of the Royal Meteorological Society · doi:10.1256/qj.06.25 · Cited by 264
Sub‐Seasonal Forecasting With a Large Ensemble of Deep‐Learning Weather Prediction Models
Weyn, J. A., Durran, D. R., Caruana, R., & Cresswell‐Clay, N. (2021) · Journal of Advances in Modeling Earth Systems · doi:10.1029/2021ms002502 · Cited by 227
“Early evidence for the economic value of probabilistic forecasts derived from the ECMWF ensemble relative to a single deterministic forecast was provided by Richardson (2000).”
BackgroundRichardson, D. S. (2001) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.49712757715 · Cited by 212
Self-citationComparison of ensemble‐MOS methods in the Lorenz '96 setting
Wilks, D. S. (2006) · Meteorological Applications · doi:10.1017/s1350482706002192 · Cited by 142
Introduction to the special issue on “25 years of ensemble forecasting”
Buizza, R. (2019) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.3370 · Cited by 102
A global evaluation of multi‐model ensemble tropical cyclone track probability forecasts
Titley, H. A., Bowyer, R. L., & Cloke, H. L. (2020) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.3712 · Cited by 57
van Straaten, C., Whan, K., Coumou, D., van den Hurk, B., & Schmeits, M. (2022) · Monthly Weather Review · doi:10.1175/mwr-d-21-0201.1 · Cited by 50
“The potential economic value (PEV) is the value that a forecast has to a hypothetical decision-maker, compared to having no forecast available (Richardson 2000).”
BackgroundGlobal extreme heat forecasting using neural weather models
Lopez-Gomez, I., McGovern, A., Agrawal, S., & Hickey, J. (2023) · Artificial Intelligence for the Earth Systems · doi:10.1175/aies-d-22-0035.1 · Cited by 41
“Operational warning systems typically use perturbed ensembles, which have been shown to yield a higher economic value than high-resolution deterministic forecasts (Richardson 2000; Palmer 2017).”
Background
Metadata
- Authors
- Richardson, D. S.
- Venue
- Quarterly Journal of the Royal Meteorological Society
- Publisher
- Wiley
- Published
- 2000-01
- Volume
- 126
- Issue
- 563
- Pages
- 649-667
- ISSN
- 0035-9009 (ISSN-L)
- 1477-870X
- OpenAlex
- W2068732663
- Cited by
- 477
- References
- 17
- FWCI
- 14.38
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.