The DOI matches its Crossref record; 50 of the 50 references it cites were found in a registry or an index.
A verification framework for interannual-to-decadal predictions experiments
Open accessAbstract
Decadal predictions have a high profile in the climate science community and beyond, yet very little is known about their skill. Nor is there any agreed protocol for estimating their skill. This paper proposes a sound and coordinated framework for verification of decadal hindcast experiments. The framework is illustrated for decadal hindcasts tailored to meet the requirements and specifications of CMIP5 (Coupled Model Intercomparison Project phase 5). The chosen metrics address key questions about the information content in initialized decadal hindcasts. These questions are: (1) Do the initial conditions in the hindcasts lead to more accurate predictions of the climate, compared to un-initialized climate change projections? and (2) Is the prediction model’s ensemble spread an appropriate representation of forecast uncertainty on average? The first question is addressed through deterministic metrics that compare the initialized and uninitialized hindcasts. The second question is addressed through a probabilistic metric applied to the initialized hindcasts and comparing different ways to ascribe forecast uncertainty. Verification is advocated at smoothed regional scales that can illuminate broad areas of predictability, as well as at the grid scale, since many users of the decadal prediction experiments who feed the climate data into applications or decision models will use the data at grid scale, or downscale it to even higher resolution. An overall statement on skill of CMIP5 decadal hindcasts is not the aim of this paper. The results presented are only illustrative of the framework, which would enable such studies. However, broad conclusions that are beginning to emerge from the CMIP5 results include (1) Most predictability at the interannual-to-decadal scale, relative to climatological averages, comes from external forcing, particularly for temperature; (2) though moderate, additional skill is added by the initial conditions over what is imparted by external forcing alone; however, the impact of initialization may result in overall worse predictions in some regions than provided by uninitialized climate change projections; (3) limited hindcast records and the dearth of climate-quality observational data impede our ability to quantify expected skill as well as model biases; and (4) as is common to seasonal-to-interannual model predictions, the spread of the ensemble members is not necessarily a good representation of forecast uncertainty. The authors recommend that this framework be adopted to serve as a starting point to compare prediction quality across prediction systems. The framework can provide a baseline against which future improvements can be quantified. The framework also provides guidance on the use of these model predictions, which differ in fundamental ways from the climate change projections that much of the community has become familiar with, including adjustment of mean and conditional biases, and consideration of how to best approach forecast uncertainty.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
An Overview of CMIP5 and the Experiment Design
Taylor, K. E., Stouffer, R. J., & Meehl, G. A. (2012) · Bulletin of the American Meteorological Society · doi:10.1175/bams-d-11-00094.1 · Cited by 14,052
Self-citationRayner, N. A., Parker, D. E., Horton, E. B., Folland, C. K., Alexander, L. V., Rowell, D. P., Kent, E. C., & Kaplan, A. (2003) · Journal of Geophysical Research: Atmospheres · doi:10.1029/2002jd002670 · Cited by 10,991
Strictly Proper Scoring Rules, Prediction, and Estimation
Gneiting, T., & Raftery, A. E. (2007) · Journal of the American Statistical Association · doi:10.1198/016214506000001437 · Cited by 5,793
Improvements to NOAA’s Historical Merged Land–Ocean Surface Temperature Analysis (1880–2006)
Smith, T. M., Reynolds, R. W., Peterson, T. C., & Lawrimore, J. (2008) · Journal of Climate · doi:10.1175/2007jcli2100.1 · Cited by 2,923
Gordon, C., Cooper, C., Senior, C. A., Banks, H., Gregory, J. M., Johns, T. C., Mitchell, J. F. B., & Wood, R. A. (2000) · Climate Dynamics · doi:10.1007/s003820050010 · Cited by 2,578
Decadal atmosphere-ocean variations in the Pacific
Trenberth, K. E., & Hurrell, J. W. (1994) · Climate Dynamics · doi:10.1007/bf00204745 · Cited by 2,281
Uncertainty estimates in regional and global observed temperature changes: A new data set from 1850
Brohan, P., Kennedy, J. J., Harris, I., Tett, S. F. B., & Jones, P. D. (2006) · Journal of Geophysical Research: Atmospheres · doi:10.1029/2005jd006548 · Cited by 2,040
The Recent Increase in Atlantic Hurricane Activity: Causes and Implications
Goldenberg, S. B., Landsea, C. W., Mestas-Nuñez, A. M., & Gray, W. M. (2001) · Science · doi:10.1126/science.1060040 · Cited by 1,769
A signature of persistent natural thermohaline circulation cycles in observed climate.
Knight, J. R., Allan, R. J., Folland, C. K., Vellinga, M., & Mann, M. E. (2005) · Geophysical Research Letters · doi:10.1029/2005gl024233 · Cited by 1,208 · cited 1×
“Dynamical models of the climate system do produce multi-decadal scale fluctuations in the strength of their AMOC, and an associated SST pattern that closely resembles the observed pattern of SSTs, as well as other patterns of terrestrial climate during positive AMV conditions (Knight et al. 2005).”
Statistical Field Significance and its Determination by Monte Carlo Techniques
Livezey, R. E., & Chen, W. Y. (1983) · Monthly Weather Review · doi:10.1175/1520-0493(1983)111<0046:sfsaid>2.0.co;2 · Cited by 1,153
Oceanic forcing of Sahel rainfall on interannual to interdecadal time scales.
Giannini, A., Saravanan, R., & Chang, P. (2003) · Science · doi:10.1126/science.1089357 · Cited by 1,058 · cited 1×
“…produce many of the teleconnections that have been empirically identified from the observations (Zhang and Delworth 2006), such as wetter conditions over India and the Sahel during their monsoon season (Giannini et al. 2003) and increased Atlantic hurricane activity (Goldenberg et al. 2001).”
BackgroundImpact of Atlantic multidecadal oscillations on India/Sahel rainfall and Atlantic hurricanes
Zhang, R., & Delworth, T. L. (2006) · Geophysical Research Letters · doi:10.1029/2006gl026267 · Cited by 939
Self-citationThe Lincoln Declaration on Drought Indices: Universal Meteorological Drought Index Recommended
Hayes, M., Svoboda, M., Wall, N., & Widhalm, M. (2011) · Bulletin of the American Meteorological Society · doi:10.1175/2010bams3103.1 · Cited by 923
Arora, V. K., Scinocca, J. F., Boer, G. J., Christian, J. R., Denman, K. L., Flato, G. M., Kharin, V. V., Lee, W. G., & Merryfield, W. J. (2011) · Geophysical Research Letters · doi:10.1029/2010gl046270 · Cited by 885
Self-citationSkill Scores Based on the Mean Square Error and Their Relationships to the Correlation Coefficient
Murphy, A. H. (1988) · Monthly Weather Review · doi:10.1175/1520-0493(1988)116<2417:ssbotm>2.0.co;2 · Cited by 881
Improved Surface Temperature Prediction for the Coming Decade from a Global Climate Model
Smith, D. M., Cusack, S., Colman, A. W., Folland, C. K., Harris, G. R., & Murphy, J. M. (2007) · Science · doi:10.1126/science.1139540 · Cited by 756
Self-citationThe rationale behind the success of multi-model ensembles in seasonal forecasting - I. Basic concept
Hagedorn, R., Doblas-Reyes, F. J., & Palmer, T. N. (2005) · Tellus A: Dynamic Meteorology and Oceanography · doi:10.3402/tellusa.v57i3.14657 · Cited by 710
Advancing decadal-scale climate prediction in the North Atlantic sector
Keenlyside, N. S., Latif, M., Jungclaus, J., Kornblueh, L., & Roeckner, E. (2008) · Nature · doi:10.1038/nature06921 · Cited by 698
Forced and Internal Twentieth-Century SST Trends in the North Atlantic*
Ting, M., Kushnir, Y., Seager, R., & Li, C. (2009) · Journal of Climate · doi:10.1175/2008jcli2561.1 · Cited by 561
Self-citationDecadal Climate Prediction: An Update from the Trenches
Meehl, G. A., Goddard, L., Boer, G., Burgman, R., Branstator, G., Cassou, C., Corti, S., Danabasoglu, G., Doblas-Reyes, F., Hawkins, E., et al. (2014) · Bulletin of the American Meteorological Society · doi:10.1175/bams-d-12-00241.1 · Cited by 557
Self-citationDu, Y., & Xie, S. (2008) · Geophysical Research Letters · doi:10.1029/2008gl033631 · Cited by 299
Causes of the Rapid Warming of the North Atlantic Ocean in the Mid-1990s
Robson, J. I., Sutton, R. T., Lohmann, K., Smith, D. M., & Palmer, M. D. (2012) · Journal of Climate · doi:10.1175/jcli-d-11-00443.1 · Cited by 283
Self-citationCollins, M., Booth, B. B. B., Bhaskaran, B., Harris, G. R., Murphy, J. M., Sexton, D. M. H., & Webb, M. J. (2011) · Climate Dynamics · doi:10.1007/s00382-010-0808-0 · Cited by 282
Skilful multi-year predictions of Atlantic hurricane frequency
Smith, D. M., Eade, R., Dunstone, N. J., Fereday, D., Murphy, J. M., Pohlmann, H., & Scaife, A. A. (2010) · Nature Geoscience · doi:10.1038/ngeo1004 · Cited by 277
Self-citationMeehl, G. A., Hu, A., & Santer, B. D. (2009) · Journal of Climate · doi:10.1175/2008jcli2552.1 · Cited by 234
Self-citationPredictability of North Atlantic Multidecadal Climate Variability
Griffies, S. M., & Bryan, K. (1997) · Science · doi:10.1126/science.275.5297.181 · Cited by 217
Interannual to Decadal Climate Predictability in the North Atlantic: A Multimodel-Ensemble Study
Collins, M., Botzet, M., Carril, A. F., Drange, H., Jouzeau, A., Latif, M., Masina, S., Otteraa, O. H., Pohlmann, H., Sorteberg, A., et al. (2006) · Journal of Climate · doi:10.1175/jcli3654.1 · Cited by 174
Self-citationAn objective ocean temperature and salinity analysis using covariances from a global climate model
Smith, D. M., & Murphy, J. M. (2007) · Journal of Geophysical Research Atmospheres · doi:10.1029/2005jc003172 · Cited by 164
Self-citationSolomon, A., Goddard, L., Kumar, A., Carton, J., Deser, C., Fukumori, I., Greene, A. M., Hegerl, G., Kirtman, B., Kushnir, Y., et al. (2011) · Bulletin of the American Meteorological Society · doi:10.1175/2010bams2962.1 · Cited by 159
Self-citationTwo Limits of Initial-Value Decadal Predictability in a CGCM
Branstator, G., & Teng, H. (2010) · Journal of Climate · doi:10.1175/2010jcli3678.1 · Cited by 159
Verification of the First 11 Years of IRI’s Seasonal Climate Forecasts
Barnston, A. G., Li, S., Mason, S. J., DeWitt, D. G., Goddard, L., & Gong, X. (2009) · Journal of Applied Meteorology and Climatology · doi:10.1175/2009jamc2325.1 · Cited by 156
Self-citationImproved combination of multiple atmospheric GCM ensembles for seasonal prediction.
Robertson, A. W., Lall, U., Zebiak, S. E., & Goddard, L. (2004) · Monthly Weather Review · doi:10.1175/mwr2818.1 · Cited by 154
Self-citationEvaluation of the IRI'S “Net Assessment” Seasonal Climate Forecasts: 1997–2001
Goddard, L., Barnston, A. G., & Mason, S. J. (2003) · Bulletin of the American Meteorological Society · doi:10.1175/bams-84-12-1761 · Cited by 153
Self-citationCoupled Ocean–Atmosphere Forecasts in the Presence of Climate Drift
Stockdale, T. (1997) · Monthly Weather Review · Cited by 144
Self-citationEl Niño: Catastrophe or Opportunity
Goddard, L., & Dilley, M. (2005) · Journal of Climate · doi:10.1175/jcli-3277.1 · Cited by 139
Self-citationAn abrupt drop in Northern Hemisphere sea surface temperature around 1970
Thompson, D. W. J., Wallace, J. M., Kennedy, J., & Jones, P. D. (2010) · Nature · doi:10.1038/nature09394 · Cited by 138
Statistical adjustment of decadal predictions in a changing climate
Kharin, V., Boer, G. J., Merryfield, W. J., Scinocca, J., & Lee, W. ‐. (2012) · Geophysical Research Letters · doi:10.1029/2012gl052647 · Cited by 136
Self-citationAn Assessment of the CFS Real-Time Seasonal Forecasts
Wang, W., Chen, M., & Kumar, A. (2010) · Weather and Forecasting · doi:10.1175/2010waf2222345.1 · Cited by 125
Self-citationFinite samples and uncertainty estimates for skill measures for seasonal prediction.
Kumar, A. (2009) · Monthly Weather Review · doi:10.1175/2009mwr2814.1 · Cited by 119
Self-citationStephenson, D. B., Coelho, C. A. S., Doblas-Reyes, F. J., & Balmaseda, M. (2005) · Tellus A: Dynamic Meteorology and Oceanography · doi:10.3402/tellusa.v57i3.14664 · Cited by 111
Self-citationOn the Reliability and Calibration of Ensemble Forecasts
Johnson, C. J., & Bowler, N. E. (2009) · Monthly Weather Review · doi:10.1175/2009mwr2715.1 · Cited by 110
Msadek, R., Dixon, K. W., Delworth, T. L., & Hurlin, W. J. (2010) · Geophysical Research Letters · doi:10.1029/2010gl044517 · Cited by 102
Self-citationLong-range forecasting and the Global Framework for Climate Services
GRAHAM, R., Yun, W.-T., Kim, J., Kumar, A., Jones, D. A., Bettio, L., Gagnon, N., Kolli, R. K., & Smith, D. (2010) · Climate Research · doi:10.3354/cr00963 · Cited by 82
Self-citationDecadal Prediction in the Pacific Region
Meehl, G. A., Hu, A., & Tebaldi, C. (2010) · Journal of Climate · doi:10.1175/2010jcli3296.1 · Cited by 77
Self-citationSalinity Adjustments in the Presence of Temperature Data Assimilation
Troccoli, A., Balmaseda, M., Segschneider, J., Vialard, J., Anderson, D. L. T., Haines, K., Stockdale, T., Vitart, F., & Fox, A. D. (2002) · Monthly Weather Review · doi:10.1175/1520-0493(2002)130<0089:saitpo>2.0.co;2 · Cited by 76
Self-citationNeeds Assessment for Climate Information on Decadal Timescales and Longer
Vera, C. S., Barangé, M., Dube, B., Goddard, L., Griggs, D., Kobysheva, N. V., Odada, E., Parey, S., Polovina, J. J., Poveda, G., et al. (2010) · Procedia Environmental Sciences · doi:10.1016/j.proenv.2010.09.017 · Cited by 73
Self-citationHow Much Should Climate Model Output Be Smoothed in Space?
Räisänen, J., & Ylhäisi, J. S. (2010) · Journal of Climate · doi:10.1175/2010jcli3872.1 · Cited by 53
The ACPI Project, Element 1: Initializing a Coupled Climate Model from Observed Conditions
Pierce, D. W., Barnett, T. P., Tokmakian, R. T., et al. (2004) · Climatic Change · doi:10.1023/b:clim.0000013676.42672.23 · Cited by 51
Statistical–Dynamical Seasonal Forecasts of Central-Southwest Asian Winter Precipitation
Tippett, M. K., Goddard, L., & Barnston, A. G. (2005) · Journal of Climate · doi:10.1175/jcli3371.1 · Cited by 43
Self-citationTeng, H., Branstator, G. W., & Meehl, G. A. (2011) · Journal of Climate · doi:10.1175/2011jcli4207.1 · Cited by 35
Self-citation
Cited by
Kay, J. E., Deser, C., Phillips, A., Mai, A., Hannay, C., Strand, G., Arblaster, J. M., Bates, S. C., Danabasoglu, G., Edwards, J., et al. (2015) · Bulletin of the American Meteorological Society · doi:10.1175/bams-d-13-00255.1 · Cited by 2,493
Self-citationNear-term Climate Change: Projections and Predictability
(2014) · Climate Change 2013 – The Physical Science Basis · doi:10.1017/cbo9781107415324.023 · Cited by 860
Stratospheric influence on tropospheric jet streams, storm tracks and surface weather
Kidston, J., Scaife, A. A., Hardiman, S. C., Mitchell, D. M., Butchart, N., Baldwin, M. P., & Gray, L. J. (2015) · Nature Geoscience · doi:10.1038/ngeo2424 · Cited by 765
Decadal Climate Prediction: An Update from the Trenches
Meehl, G. A., Goddard, L., Boer, G., Burgman, R., Branstator, G., Cassou, C., Corti, S., Danabasoglu, G., Doblas-Reyes, F., Hawkins, E., et al. (2014) · Bulletin of the American Meteorological Society · doi:10.1175/bams-d-12-00241.1 · Cited by 557
Self-citationThe Decadal Climate Prediction Project (DCPP) contribution to CMIP6
Boer, G. J., Smith, D. M., Cassou, C., Doblas-Reyes, F., Danabasoglu, G., Kirtman, B., Kushnir, Y., Kimoto, M., Meehl, G. A., Msadek, R., et al. (2016) · Geoscientific Model Development · doi:10.5194/gmd-9-3751-2016 · Cited by 483
Self-citation
Metadata
- Authors
- Goddard, L., Kumar, A., Solomon, A., Smith, D., Boer, G., Gonzalez, P., Kharin, V., Merryfield, W., Deser, C., Mason, S. J., Kirtman, B. P., Msadek, R., Sutton, R., Hawkins, E., Fricker, T., Hegerl, G., Ferro, C. A. T., Stephenson, D. B., Meehl, G. A., Stockdale, T., Burgman, R., Greene, A. M., Kushnir, Y., Newman, M., Carton, J., Fukumori, I., & Delworth, T.
- Venue
- Climate Dynamics
- Published
- 2012-08-24
- Volume
- 40
- Issue
- 1-2
- Pages
- 245-272
- ISSN
- 0930-7575 (ISSN-L)
- 1432-0894
- OpenAlex
- W1982418161
- Cited by
- 338
- References
- 50
- FWCI
- 23.20
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.