ESMValTool (v1.0) – a community diagnostic and performance metrics tool for routine evaluation of Earth system models in CMIP
Open accessThe DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Abstract
A community diagnostics and performance metrics tool for the evaluation of Earth system models (ESMs) has been developed that allows for routine comparison of single or multiple models, either against predecessor versions or against observations. The priority of the effort so far has been to target specific scientific themes focusing on selected essential climate variables (ECVs), a range of known systematic biases common to ESMs, such as coupled tropical climate variability, monsoons, Southern Ocean processes, continental dry biases, and soil hydrology–climate interactions, as well as atmospheric CO2 budgets, tropospheric and stratospheric ozone, and tropospheric aerosols. The tool is being developed in such a way that additional analyses can easily be added. A set of standard namelists for each scientific topic reproduces specific sets of diagnostics or performance metrics that have demonstrated their importance in ESM evaluation in the peer-reviewed literature. The Earth System Model Evaluation Tool (ESMValTool) is a community effort open to both users and developers encouraging open exchange of diagnostic source code and evaluation results from the Coupled Model Intercomparison Project (CMIP) ensemble. This will facilitate and improve ESM evaluation beyond the state-of-the-art and aims at supporting such activities within CMIP and at individual modelling centres. Ultimately, we envisage running the ESMValTool alongside the Earth System Grid Federation (ESGF) as part of a more routine evaluation of CMIP model simulations while utilizing observations available in standard formats (obs4MIPs) or provided by the user.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The ERA‐Interim reanalysis: configuration and performance of the data assimilation system
Dee, D. P., Uppala, S. M., Simmons, A. J., Berrisford, P., Poli, P., Kobayashi, S., Andrae, U., Balmaseda, M. A., Balsamo, G., Bauer, P., et al. (2011) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.828 · Cited by 24,480
An Overview of CMIP5 and the Experiment Design
Taylor, K. E., Stouffer, R. J., & Meehl, G. A. (2012) · Bulletin of the American Meteorological Society · doi:10.1175/bams-d-11-00094.1 · Cited by 14,052
Eyring, V., Bony, S., Meehl, G. A., Senior, C. A., Stevens, B., Stouffer, R. J., & Taylor, K. E. (2016) · Geoscientific Model Development · doi:10.5194/gmd-9-1937-2016 · Cited by 11,981
Self-citationNCEP–DOE AMIP-II reanalysis (R-2)
Kanamitsu, M., Ebisuzaki, W., Woollen, J., Yang, S.-K., Hnilo, J. J., Fiorino, M., & Potter, G. L. (2002) · Bulletin of the American Meteorological Society · doi:10.1175/bams-83-11-1631 · Cited by 5,771
The Twentieth Century Reanalysis Project
Compo, G. P., Whitaker, J. S., Sardeshmukh, P. D., Matsui, N., Allan, R. J., Yin, X., Gleason, B. E., Vose, R. S., Rutledge, G., Bessemoulin, P., et al. (2011) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.776 · Cited by 3,518
Climate–Carbon Cycle Feedback Analysis: Results from the C4MIP Model Intercomparison
Friedlingstein, P., Cox, P., Betts, R., Bopp, L., von Bloh, W., Brovkin, V., Cadule, P., Doney, S., Eby, M., Fung, I., et al. (2006) · Journal of Climate · doi:10.1175/jcli3800.1 · Cited by 2,902
Self-citationPerformance metrics for climate models.
Gleckler, P. J., Taylor, K. E., & Doutriaux, C. (2008) · Journal of Geophysical Research: Atmospheres · doi:10.1029/2007jd008972 · Cited by 1,328
Self-citationSea Surface Temperature Variability: Patterns and Mechanisms
Deser, C., Alexander, M. A., Xie, S.-P., & Phillips, A. S. (2010) · Annual Review of Marine Science · doi:10.1146/annurev-marine-120408-151453 · Cited by 1,135
Self-citationToward Optimal Closure of the Earth's Top-of-Atmosphere Radiation Budget
Loeb, N. G., Wielicki, B. A., Doelling, D. R., Smith, G. L., Keyes, D. F., Kato, S., Manalo-Smith, N., & Wong, T. (2009) · Journal of Climate · doi:10.1175/2008jcli2637.1 · Cited by 960
Development cycle 2 of the Modular Earth Submodel System (MESSy2)
Jöckel, P., Kerkweg, A., Pozzer, A., Sander, R., Tost, H., Riede, H., Baumgaertner, A., Gromov, S., & Kern, B. (2010) · Geoscientific Model Development · doi:10.5194/gmd-3-717-2010 · Cited by 551
Cited by
Eyring, V., Bony, S., Meehl, G. A., Senior, C. A., Stevens, B., Stouffer, R. J., & Taylor, K. E. (2016) · Geoscientific Model Development · doi:10.5194/gmd-9-1937-2016 · Cited by 11,981
Self-citationFuture climate risk from compound events
Zscheischler, J., Westra, S., van den Hurk, B. J. J. M., Seneviratne, S. I., Ward, P. J., Pitman, A., AghaKouchak, A., Bresch, D. N., Leonard, M., Wahl, T., et al. (2018) · Nature Climate Change · doi:10.1038/s41558-018-0156-3 · Cited by 2,527
Taking climate model evaluation to the next level
Eyring, V., Cox, P. M., Flato, G. M., Gleckler, P. J., Abramowitz, G., Caldwell, P., Collins, W. D., Gier, B. K., Hall, A. D., Hoffman, F. M., et al. (2019) · Nature Climate Change · doi:10.1038/s41558-018-0355-y · Cited by 748
Self-citationBakker, D. C. E., Pfeil, B., Landa, C. S., Metzl, N., O'Brien, K. M., Olsen, A., Smith, K., Cosca, C., Harasawa, S., Jones, S. D., et al. (2016) · Earth System Science Data · doi:10.5194/essd-8-383-2016 · Cited by 681
The International Land Model Benchmarking (ILAMB) System: Design, Theory, and Implementation
Collier, N., Hoffman, F. M., Lawrence, D. M., Keppel‐Aleks, G., Koven, C. D., Riley, W. J., Mu, M., & Randerson, J. T. (2018) · Journal of Advances in Modeling Earth Systems · doi:10.1029/2018ms001354 · Cited by 447
The physics of climate variability and climate change
Ghil, M., & Lucarini, V. (2020) · Reviews of Modern Physics · doi:10.1103/revmodphys.92.035002 · Cited by 416
AerChemMIP: quantifying the effects of chemistry and aerosols in CMIP6
Collins, W. J., Lamarque, J.-F., Schulz, M., Boucher, O., Eyring, V., Hegglin, M. I., Maycock, A., Myhre, G., Prather, M., Shindell, D., et al. (2017) · Geoscientific Model Development · doi:10.5194/gmd-10-585-2017 · Cited by 380
Self-citationJöckel, P., Tost, H., Pozzer, A., Kunze, M., Kirner, O., Brenninkmeijer, C. A. M., Brinkop, S., Cai, D. S., Dyroff, C., Eckstein, J., et al. (2016) · Geoscientific Model Development · doi:10.5194/gmd-9-1153-2016 · Cited by 370
Self-citationJones, C. D., Arora, V., Friedlingstein, P., Bopp, L., Brovkin, V., Dunne, J., Graven, H., Hoffman, F., Ilyina, T., John, J. G., et al. (2016) · Geoscientific Model Development · doi:10.5194/gmd-9-2853-2016 · Cited by 365
Self-citationCMIP5 Scientific Gaps and Recommendations for CMIP6
Stouffer, R. J., Eyring, V., Meehl, G. A., Bony, S., Senior, C., Stevens, B., & Taylor, K. E. (2017) · Bulletin of the American Meteorological Society · doi:10.1175/bams-d-15-00013.1 · Cited by 328
Self-citationThe BRIDGE HadCM3 family of climate models: HadCM3@Bristol v1.0
Valdes, P. J., Armstrong, E., Badger, M. P. S., Bradshaw, C. D., Bragg, F., Crucifix, M., Davies-Barnard, T., Day, J. J., Farnsworth, A., Gordon, C., et al. (2017) · Geoscientific Model Development · doi:10.5194/gmd-10-3715-2017 · Cited by 286
Constraining human contributions to observed warming since the pre-industrial period
Gillett, N. P., Kirchmeier-Young, M., Ribes, A., Shiogama, H., Hegerl, G. C., Knutti, R., Gastineau, G., John, J. G., Li, L., Nazarenko, L., et al. (2021) · Nature Climate Change · doi:10.1038/s41558-020-00965-9 · Cited by 265
Evaluating Climate Models with the CLIVAR 2020 ENSO Metrics Package
Planton, Y. Y., Guilyardi, E., Wittenberg, A. T., Lee, J., Gleckler, P. J., Bayr, T., McGregor, S., McPhaden, M. J., Power, S., Roehrig, R., et al. (2021) · Bulletin of the American Meteorological Society · doi:10.1175/bams-d-19-0337.1 · Cited by 246
Self-citationQuantifying Progress Across Different CMIP Phases With the ESMValTool
Bock, L., Lauer, A., Schlund, M., Barreiro, M., Bellouin, N., Jones, C., Meehl, G. A., Predoi, V., Roberts, M. J., & Eyring, V. (2020) · Journal of Geophysical Research: Atmospheres · doi:10.1029/2019jd032321 · Cited by 242
Self-citationBittig, H. C., Steinhoff, T., Claustre, H., Fiedler, B., Williams, N. L., Sauzède, R., Körtzinger, A., & Gattuso, J.-P. (2018) · Frontiers in Marine Science · doi:10.3389/fmars.2018.00328 · Cited by 235
“These data collection and harmonization efforts are tremendously useful to the modeling community (e.g., Eyring et al., 2016) and the marine carbon cycle research community at large.”
MethodProjected land photosynthesis constrained by changes in the seasonal cycle of atmospheric CO2
Wenzel, S., Cox, P. M., Eyring, V., & Friedlingstein, P. (2016) · Nature · doi:10.1038/nature19772 · Cited by 202
Self-citationVisioni, D., Pitari, G., Aquila, V., Tilmes, S., Cionni, I., Di Genova, G., & Mancini, E. (2017) · Atmospheric Chemistry and Physics · doi:10.5194/acp-17-11209-2017 · Cited by 41
Self-citationPitari, G., Cionni, I., Di Genova, G., Søvde, O. A., & Lim, L. (2017) · Meteorologische Zeitschrift · doi:10.1127/metz/2016/0776 · Cited by 12
Self-citation
Metadata
- Authors
- Eyring, V., Righi, M., Lauer, A., Evaldsson, M., Wenzel, S., Jones, C., Anav, A., Andrews, O., Cionni, I., Davin, E. L., Deser, C., Ehbrecht, C., Friedlingstein, P., Gleckler, P., Gottschaldt, K.-D., Hagemann, S., Juckes, M., Kindermann, S., Krasting, J., Kunert, D., Levine, R., Loew, A., Mäkelä, J., Martin, G., Mason, E., Phillips, A. S., Read, S., Rio, C., Roehrig, R., Senftleben, D., Sterl, A., van Ulft, L. H., Walton, J., Wang, S., & Williams, K. D.
- Venue
- Geoscientific Model Development
- Publisher
- Copernicus GmbH
- Published
- 2016-05-10
- Volume
- 9
- Issue
- 5
- Pages
- 1747-1802
- ISSN
- 1991-959X (ISSN-L)
- 1991-9603
- OpenAlex
- W1957506591
- Cited by
- 191
- References
- 17
- FWCI
- 13.40
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.