Quantitative performance metrics for stratospheric-resolving chemistry-climate models
Open accessThe DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
Abstract
A set of performance metrics is applied to stratospheric-resolving chemistry-climate models (CCMs) to quantify their ability to reproduce key processes relevant for stratospheric ozone. The same metrics are used to assign a quantitative measure of performance ("grade") to each model-observations comparison shown in Eyring et al. (2006). A wide range of grades is obtained, both for different diagnostics applied to a single model and for the same diagnostic applied to different models, highlighting the wide range in ability of the CCMs to simulate key processes in the stratosphere. No model scores high or low on all tests, but differences in the performance of models can be seen, especially for processes that are mainly determined by transport where several models get low grades on multiple tests. The grades are used to assign relative weights to the CCM projections of 21st century total ozone. For the diagnostics used here there are generally only small differences between weighted and unweighted multi-model mean and variances of total ozone projections. This study raises several issues with the grading and weighting of CCMs that need further examination. However, it does provide a framework and benchmarks that will enable quantification of model improvements and assignment of relative weights to the model projections.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Performance metrics for climate models.
Gleckler, P. J., Taylor, K. E., & Doutriaux, C. (2008) · Journal of Geophysical Research: Atmospheres · doi:10.1029/2007jd008972 · Cited by 1,328 · cited 2×
“A different approach was used by Reichler and Kim (2008) and Gleckler et al. (2008), who normalized the error by the “typical” model error for each quantity.”
Dynamics, stratospheric ozone, and climate change
Shepherd, T. G. (2008) · Atmosphere-Ocean · doi:10.3137/ao.460106 · Cited by 153
Cited by
Challenges in combining projections from multiple climate models.
Knutti, R., Furrer, R., Tebaldi, C., Cermak, J., & Meehl, G. A. (2010) · Journal of Climate · doi:10.1175/2009jcli3361.1 · Cited by 1,270
“Waugh and Eyring (2008) and Eyring et al. (2007) assessed the performance of stratospheric chemistry–climate models but found only small differences between weighted and unweighted projections.”
BackgroundTransport impacts on atmosphere and climate: Aviation
Lee, D. S., Pitari, G., Grewe, V., Gierens, K., Penner, J. E., Petzold, A., Prather, M. J., Schumann, U., Bais, A., & Berntsen, T. (2010) · Atmospheric Environment · doi:10.1016/j.atmosenv.2009.06.005 · Cited by 765
Taking climate model evaluation to the next level
Eyring, V., Cox, P. M., Flato, G. M., Gleckler, P. J., Abramowitz, G., Caldwell, P., Collins, W. D., Gier, B. K., Hall, A. D., Hoffman, F. M., et al. (2019) · Nature Climate Change · doi:10.1038/s41558-018-0355-y · Cited by 748
Self-citationGlobal air quality and climate
Fiore, A. M., Naik, V., Spracklen, D. V., Steiner, A., Unger, N., Prather, M., Bergmann, D., Cameron-Smith, P. J., Cionni, I., Collins, W. J., et al. (2012) · Chemical Society Reviews · doi:10.1039/c2cs35095e · Cited by 521
Self-citationA climate model projection weighting scheme accounting for performance and interdependence
Knutti, R., Sedláček, J., Sanderson, B. M., Lorenz, R., Fischer, E. M., & Eyring, V. (2017) · Geophysical Research Letters · doi:10.1002/2016gl072012 · Cited by 491
Self-citation
Metadata
- Authors
- Waugh, D. W., & Eyring, V.
- Venue
- Atmospheric Chemistry and Physics
- Publisher
- Copernicus GmbH
- Published
- 2008-09-29
- Volume
- 8
- Issue
- 18
- Pages
- 5699-5713
- ISSN
- 1680-7316 (ISSN-L)
- 1680-7324
- OpenAlex
- W2144081622
- Cited by
- 87
- References
- 2
- FWCI
- 7.18
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Quantitative performance metrics for stratospheric-resolving chemistry-climate models. https://app.cochat.ai/papers/10.5194/acp-8-5699-2008
- The paper
- Waugh, D. W., & Eyring, V. (2008). Quantitative performance metrics for stratospheric-resolving chemistry-climate models. Atmospheric Chemistry and Physics. https://doi.org/10.5194/acp-8-5699-2008
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.