Analyzing precipitation projections: A comparison of different approaches to climate model evaluation
Open accessThe DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
Abstract
[1] Complexity and resolution of global climate models are steadily increasing, yet the uncertainty of their projections remains large, particularly for precipitation. Given the impacts precipitation changes have on ecosystems, there is a need to reduce projection uncertainty by assessing the performance of climate models. A common way of evaluating models is to consider global maps of errors against observations for a range of variables. However, depending on the purpose, feature-based metrics defined on a regional scale and for one variable may be more suitable to identify the most accurate models. We compare three different ways of ranking the CMIP3 climate models: errors in a broad range of climate variables, errors in global field of precipitation, and regional features of modeled precipitation in areas where pronounced future changes are expected. The same analysis is performed for temperature to identify potential differences between variables. The multimodel mean is found to outperform all single models in the global field-based rankings but performs only averagely for the feature-based ranking. Selecting the best models for each metric reduces the absolute spread in projections. If anomalies are considered, the model spread is reduced in a few regions, while the uncertainty can be increased in others. We also demonstrate that the common attribution of a lack of model agreement in precipitation projections to different model physics may be misleading. Agreement is similarly poor within different ensemble members of the same model, indicating that the lack of robust trends can be attributed partly to a low signal-to-noise ratio.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Changing Character of Precipitation
Trenberth, K. E., Dai, A., Rasmussen, R. M., & Parsons, D. B. (2003) · Bulletin of the American Meteorological Society · doi:10.1175/bams-84-9-1205 · Cited by 3,196
Performance metrics for climate models.
Gleckler, P. J., Taylor, K. E., & Doutriaux, C. (2008) · Journal of Geophysical Research: Atmospheres · doi:10.1029/2007jd008972 · Cited by 1,328 · cited 6×
“Gleckler et al. [2008] and Pincus et al. [2008] further state that all models have distinctive weaknesses in simulating specific variables.”
BackgroundMethodResultInfluentialChallenges in combining projections from multiple climate models.
Knutti, R., Furrer, R., Tebaldi, C., Cermak, J., & Meehl, G. A. (2010) · Journal of Climate · doi:10.1175/2009jcli3361.1 · Cited by 1,270 · cited 4×
“[7] Knutti et al. [2010b] showed that most statistical metrics like the root mean square error do not correlate strongly with future projections, and they suggest that feature‐based evaluations could provide additional useful information.”
BackgroundResultInfluentialSelf-citationRisks of Model Weighting in Multimodel Climate Projections
Weigel, A. P., Knutti, R., Liniger, M. A., & Appenzeller, C. (2010) · Journal of Climate · doi:10.1175/2010jcli3594.1 · Cited by 386
Self-citationSpatial Analysis to Quantify Numerical Model Bias and Dependence: How Many Climate Models Are There?
Jun, M., Knutti, R., & Nychka, D. W. (2008) · Journal of the American Statistical Association · doi:10.1198/016214507000001265 · Cited by 214
Self-citationClimate projections: Past performance no guarantee of future skill?
Reifen, C., & Toumi, R. (2009) · Geophysical Research Letters · doi:10.1029/2009gl038082 · Cited by 144 · cited 1×
“Our results tend to support previous findings showing that a good performance in the present does not guarantee skill in the future [Jun et al., 2008; Reifen and Toumi, 2009 ].”
ResultWhetton, P., Macadam, I., Bathols, J., & O'Grady, J. (2007) · Geophysical Research Letters · doi:10.1029/2007gl030025 · Cited by 104
Do weak AR4 models bias projections of future climate changes over Australia?
Perkins, S. E., & Pitman, A. J. (2009) · Climatic Change · doi:10.1007/s10584-008-9502-1 · Cited by 48
Cited by
Robustness and uncertainties in the new CMIP5 climate model projections
Knutti, R., & Sedláček, J. (2013) · Nature Climate Change · doi:10.1038/nclimate1716 · Cited by 1,569
Self-citationNear-term Climate Change: Projections and Predictability
(2014) · Climate Change 2013 – The Physical Science Basis · doi:10.1017/cbo9781107415324.023 · Cited by 860
Future change of global monsoon in the CMIP5
Lee, J.-Y., & Wang, B. (2014) · Climate Dynamics · doi:10.1007/s00382-012-1564-0 · Cited by 459
Quantifying the Relevance of Cyclones for Precipitation Extremes
Pfahl, S., & Wernli, H. (2012) · Journal of Climate · doi:10.1175/jcli-d-11-00705.1 · Cited by 395
Weaver, C. P., Lempert, R. J., Brown, C., Hall, J. A., Revell, D., & Sarewitz, D. (2013) · WIREs Climate Change · doi:10.1002/wcc.202 · Cited by 370
Metadata
- Authors
- Schaller, N., Mahlstein, I., Cermak, J., & Knutti, R.
- Venue
- Journal of Geophysical Research
- Publisher
- American Geophysical Union (AGU)
- Published
- 2011-05-27
- Volume
- 116
- Issue
- D10
- ISSN
- 0148-0227 (ISSN-L)
- OpenAlex
- W2024814568
- Cited by
- 90
- References
- 8
- FWCI
- 8.45
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Analyzing precipitation projections: A comparison of different approaches to climate model evaluation. https://app.cochat.ai/papers/10.1029/2010jd014963
- The paper
- Schaller, N., Mahlstein, I., Cermak, J., & Knutti, R. (2011). Analyzing precipitation projections: A comparison of different approaches to climate model evaluation. Journal of Geophysical Research. https://doi.org/10.1029/2010jd014963
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.