CoChat

Performance metrics for climate models.

Gleckler, P. J., Taylor, K. E., Doutriaux, C. · 2008 · Journal of Geophysical Research: Atmospheres

The DOI matches its Crossref record; 33 of the 34 references it cites were found in a registry or an index, and 1 are not checked yet.

Abstract

Objective measures of climate model performance are proposed and used to assess simulations of the 20th century, which are available from the Coupled Model Intercomparison Project (CMIP3) archive. The primary focus of this analysis is on the climatology of atmospheric fields. For each variable considered, the models are ranked according to a measure of relative error. Based on an average of the relative errors over all fields considered, some models appear to perform substantially better than others. Forming a single index of model performance, however, can be misleading in that it hides a more complex picture of the relative merits of different models. This is demonstrated by examining individual variables and showing that the relative ranking of models varies considerably from one variable to the next. A remarkable exception to this finding is that the so‐called “mean model” consistently outperforms all other models in nearly every respect. The usefulness, limitations and robustness of the metrics defined here are evaluated 1) by examining whether the information provided by each metric is correlated in any way with the others, and 2) by determining how sensitive the metrics are to such factors as observational uncertainty, spatial scale, and the domain considered (e.g., tropics versus extra‐tropics). An index that gauges the fidelity of model variability on interannual time‐scales is found to be only weakly correlated with an index of the mean climate performance. This illustrates the importance of evaluating a broad spectrum of climate processes and phenomena since accurate simulation of one aspect of climate does not guarantee accurate representation of other aspects. Once a broad suite of metrics has been developed to characterize model performance it may become possible to identify optimal subsets for various applications.

Overview

Integrity signals

Registry

Crossref · 10.1029/2007jd008972

Cites

33 of 34 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 34 references

Research works printed but not linked: 12

  1. 10.1175/1520-0477(1984)065 · doi:10.1175/1520-0477(1984)065
  2. 10.1175/1520-0477(1999)080 · doi:10.1175/1520-0477(1999)080
  3. 10.1175/1520-0477(1997)078 · doi:10.1175/1520-0477(1997)078
  4. 10.1175/1520-0442(1999)012 · doi:10.1175/1520-0442(1999)012
  5. 10.1175/1520-0477(1996)077 · doi:10.1175/1520-0477(1996)077
  6. 10.1175/1520-0477(2000)081 · doi:10.1175/1520-0477(2000)081
  7. 10.1175/1520-0477(1999)080 · doi:10.1175/1520-0477(1999)080
  8. 10.1175/1520-0442(1993)006 · doi:10.1175/1520-0442(1993)006
  9. Simmons A. J., 2000, The ERA‐40 project plan, ERA‐40 Project Report Series No. 1
  10. 10.1175/1520-0442(2004)017 · doi:10.1175/1520-0442(2004)017
  11. Taylor P.(Ed.) (2000) Final report of the Joint WCRP/SCOR Working Group on Air‐Sea Fluxes WCRP‐112.
  12. 10.1175/1520‐0477 · doi:10.1175/1520-0477

Cited by

Items 801–900 of 1,001

Metadata

Authors
Gleckler, P. J., Taylor, K. E., & Doutriaux, C.
Affiliations
Lawrence Livermore National Laboratory (US)
Venue
Journal of Geophysical Research: Atmospheres
Published
2008-03-20
Volume
113
Issue
D6
ISSN
0148-0227 (ISSN-L)
OpenAlex
W1996478169
Cited by
1,328
References
31
Citations by year
2026: 64 · 2025: 73 · 2024: 85 · 2023: 88 · 2022: 69 · 2021: 116 · 2020: 103 · 2019: 83 · 2018: 65 · 2017: 76 · 2016: 63 · 2015: 77 · 2014: 65 · 2013: 87 · 2012: 73
FWCI
41.84
Type
journal article
Language
en
Open access
closed
Locations
Journal of Geophysical Research Atmospheres (publishedVersion)
Topics
  • Physical Sciences › Environmental Science › Global and Planetary Change › Climate variability and models
  • Physical Sciences › Earth and Planetary Sciences › Atmospheric Science › Meteorological Phenomena and Simulations
  • Physical Sciences › Environmental Science › Global and Planetary Change › Climate Change and Geoengineering
Keywords
climate model evaluation, performance indicators, CMIP3, relative error, atmospheric climatology, model ranking, interannual variability, observational uncertainty, spatial scale, 20th century climate simulation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.