ESD Reviews: Model dependence in multi-model climate ensembles: weighting, sub-selection and out-of-sample testing
Open accessThe DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Abstract
The rationale for using multi-model ensembles in climate change projections and impacts research is often based on the expectation that different models constitute independent estimates; therefore, a range of models allows a better characterisation of the uncertainties in the representation of the climate system than a single model. However, it is known that research groups share literature, ideas for representations of processes, parameterisations, evaluation data sets and even sections of model code. Thus, nominally different models might have similar biases because of similarities in the way they represent a subset of processes, or even be near-duplicates of others, weakening the assumption that they constitute independent estimates. If there are near-replicates of some models, then treating all models equally is likely to bias the inferences made using these ensembles. The challenge is to establish the degree to which this might be true for any given application. While this issue is recognised by many in the community, quantifying and accounting for model dependence in anything other than an ad-hoc way is challenging. Here we present a synthesis of the range of disparate attempts to define, quantify and address model dependence in multi-model climate ensembles in a common conceptual framework, and provide guidance on how users can test the efficacy of approaches that move beyond the equally weighted ensemble. In the upcoming Coupled Model Intercomparison Project phase 6 (CMIP6), several new models that are closely related to existing models are anticipated, as well as large ensembles from some models. We argue that quantitatively accounting for dependence in addition to model performance, and thoroughly testing the effectiveness of the approach used will be key to a sound interpretation of the CMIP ensembles in future scientific studies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Self-Organization and Associative Memory
Kohonen, T. (1989) · Springer Series in Information Sciences · doi:10.1007/978-3-642-88163-3 · Cited by 8,740
The potential to narrow uncertainty in regional climate predictions.
Hawkins, E., & Sutton, R. (2009) · Bulletin of the American Meteorological Society · doi:10.1175/2009bams2607.1 · Cited by 2,732
The use of the multi-model ensemble in probabilistic climate projections
Tebaldi, C., & Knutti, R. (2007) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2007.2076 · Cited by 1,862 · cited 2×
“This effect is implicit even when considering all available simulations because the broader ensembles such as CMIP are generally constructed based on an the ability of the international modelling centres to contribute, rather than a systematic sampling strategy (Tebaldi and Knutti, 2007).”
BackgroundSelf-citationPerformance metrics for climate models.
Gleckler, P. J., Taylor, K. E., & Doutriaux, C. (2008) · Journal of Geophysical Research: Atmospheres · doi:10.1029/2007jd008972 · Cited by 1,328 · cited 1×
“Gleckler et al. (2008) also show that the variability of the ensemble mean is much less than individual models or observations – and so does not represent a potentially real climate.”
BackgroundChallenges in combining projections from multiple climate models.
Knutti, R., Furrer, R., Tebaldi, C., Cermak, J., & Meehl, G. A. (2010) · Journal of Climate · doi:10.1175/2009jcli3361.1 · Cited by 1,270 · cited 2×
“Knutti et al. (2010b) noted that the ensemble mean of the CMIP3 ensemble did not appear to asymptotically converge to observations as ensemble size increased.”
BackgroundMethodSelf-citationClimate model genealogy: Generation CMIP5 and how we got there
Knutti, R., Masson, D., & Gettelman, A. (2013) · Geophysical Research Letters · doi:10.1002/grl.50256 · Cited by 918 · cited 1×
“This is the approach taken by Masson and Knutti (2011) and Knutti et al. (2013), who used hierarchical clustering of the spatiotemporal variability of surface temperature and precipitation in climate model control simulations to develop a “climate model genealogy”.”
MethodSelf-citationWeather Forecasting with Ensemble Methods
Gneiting, T., & Raftery, A. E. (2005) · Science · doi:10.1126/science.1115255 · Cited by 636 · cited 2×
“…system given uncertain initial conditions, and could be investigated using initial-condition ensembles of climate change projections (Kay et al., 2015; Deser et al., 2016) as well as in the context of numerical weather prediction ensembles (e.g. Hamill et al., 2000; Gneiting and Raftery, 2005).”
The Art and Science of Climate Model Tuning
Hourdin, F., Mauritsen, T., Gettelman, A., Golaz, J.-C., Balaji, V., Duan, Q., Folini, D., Ji, D., Klocke, D., Qian, Y., et al. (2017) · Bulletin of the American Meteorological Society · doi:10.1175/bams-d-15-00135.1 · Cited by 611
Risks of Model Weighting in Multimodel Climate Projections
Weigel, A. P., Knutti, R., Liniger, M. A., & Appenzeller, C. (2010) · Journal of Climate · doi:10.1175/2010jcli3594.1 · Cited by 386
Self-citationThe Discrete Brier and Ranked Probability Skill Scores
Weigel, A. P., Liniger, M. A., & Appenzeller, C. (2007) · Monthly Weather Review · doi:10.1175/mwr3280.1 · Cited by 254
Cited by
Taking climate model evaluation to the next level
Eyring, V., Cox, P. M., Flato, G. M., Gleckler, P. J., Abramowitz, G., Caldwell, P., Collins, W. D., Gier, B. K., Hall, A. D., Hoffman, F. M., et al. (2019) · Nature Climate Change · doi:10.1038/s41558-018-0355-y · Cited by 748
Self-citationPartitioning climate projection uncertainty with multiple large ensembles and CMIP5/6
Lehner, F., Deser, C., Maher, N., Marotzke, J., Fischer, E. M., Brunner, L., Knutti, R., & Hawkins, E. (2020) · Earth System Dynamics · doi:10.5194/esd-11-491-2020 · Cited by 651
Self-citationReduced global warming from CMIP6 projections when weighting models by performance and independence
Brunner, L., Pendergrass, A. G., Lehner, F., Merrifield, A. L., Lorenz, R., & Knutti, R. (2020) · Earth System Dynamics · doi:10.5194/esd-11-995-2020 · Cited by 362
Self-citation(2023) · Climate Change 2021 – The Physical Science Basis · doi:10.1017/9781009157896.003 · Cited by 298
Fiedler, S., Crueger, T., D’Agostino, R., Peters, K., Becker, T., Leutwyler, D., Paccini, L., Burdanowitz, J., Buehler, S. A., Cortes, A. U., et al. (2020) · Monthly Weather Review · doi:10.1175/mwr-d-19-0404.1 · Cited by 166
Metadata
- Authors
- Abramowitz, G., Herger, N., Gutmann, E., Hammerling, D., Knutti, R., Leduc, M., Lorenz, R., Pincus, R., & Schmidt, G. A.
- Venue
- Earth System Dynamics
- Publisher
- Copernicus GmbH
- Published
- 2019-02-13
- Volume
- 10
- Issue
- 1
- Pages
- 91-105
- ISSN
- 2190-4979 (ISSN-L)
- 2190-4987
- OpenAlex
- W2915014208
- Cited by
- 201
- References
- 13
- FWCI
- 10.90
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: ESD Reviews: Model dependence in multi-model climate ensembles: weighting, sub-selection and out-of-sample testing. https://app.cochat.ai/papers/10.5194/esd-10-91-2019
- The paper
- Abramowitz, G., Herger, N., Gutmann, E., Hammerling, D., Knutti, R., Leduc, M., Lorenz, R., Pincus, R., & Schmidt, G. A. (2019). ESD Reviews: Model dependence in multi-model climate ensembles: weighting, sub-selection and out-of-sample testing. Earth System Dynamics. https://doi.org/10.5194/esd-10-91-2019
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.