Performance metrics for climate models.
The DOI matches its Crossref record; 33 of the 34 references it cites were found in a registry or an index, and 1 are not checked yet.
Abstract
Objective measures of climate model performance are proposed and used to assess simulations of the 20th century, which are available from the Coupled Model Intercomparison Project (CMIP3) archive. The primary focus of this analysis is on the climatology of atmospheric fields. For each variable considered, the models are ranked according to a measure of relative error. Based on an average of the relative errors over all fields considered, some models appear to perform substantially better than others. Forming a single index of model performance, however, can be misleading in that it hides a more complex picture of the relative merits of different models. This is demonstrated by examining individual variables and showing that the relative ranking of models varies considerably from one variable to the next. A remarkable exception to this finding is that the so‐called “mean model” consistently outperforms all other models in nearly every respect. The usefulness, limitations and robustness of the metrics defined here are evaluated 1) by examining whether the information provided by each metric is correlated in any way with the others, and 2) by determining how sensitive the metrics are to such factors as observational uncertainty, spatial scale, and the domain considered (e.g., tropics versus extra‐tropics). An index that gauges the fidelity of model variability on interannual time‐scales is found to be only weakly correlated with an index of the mean climate performance. This illustrates the importance of evaluating a broad spectrum of climate processes and phenomena since accurate simulation of one aspect of climate does not guarantee accurate representation of other aspects. Once a broad suite of metrics has been developed to characterize model performance it may become possible to identify optimal subsets for various applications.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“not even the question of which model performs best during the control simulations can be readily answered but, rather, depends strongly on the skill metric, variable, and region considered (e.g., Gleckler et al. 2008).”
From Risks of Model Weighting in Multimodel Climate Projections (2010) · cites this paper as Gleckler et al. 2008
Citations
Cites
The NCEP/NCAR 40-Year Reanalysis Project
Kalnay, E., Kanamitsu, M., Kistler, R., Collins, W., Deaven, D., Gandin, L., Iredell, M., Saha, S., White, G., Woollen, J., et al. (1996) · Bulletin of the American Meteorological Society · doi:10.1175/1520-0477(1996)077<0437:tnyrp>2.0.co;2 · Cited by 28,360
Summarizing multiple aspects of model performance in a single diagram
Taylor, K. E. (2001) · Journal of Geophysical Research: Atmospheres · doi:10.1029/2000jd900719 · Cited by 8,477
Self-citationAdler, R. F., Huffman, G. J., Chang, A., Ferraro, R., Xie, P.-P., Janowiak, J., Rudolf, B., Schneider, U., Curtis, S., Bolvin, D., et al. (2003) · Journal of Hydrometeorology · doi:10.1175/1525-7541(2003)004<1147:tvgpcp>2.0.co;2 · Cited by 3,808
Advances in Understanding Clouds from ISCCP
Rossow, W. B., & Schiffer, R. A. (1999) · Bulletin of the American Meteorological Society · doi:10.1175/1520-0477(1999)080<2261:aiucfi>2.0.co;2 · Cited by 2,256
Clouds and the Earth's Radiant Energy System (CERES): An Earth Observing System Experiment
Wielicki, B. A., Barkstrom, B. R., Harrison, E. F., Lee, R. B., Louis Smith, G., & Cooper, J. E. (1996) · Bulletin of the American Meteorological Society · doi:10.1175/1520-0477(1996)077<0853:catere>2.0.co;2 · Cited by 1,957
The Global Precipitation Climatology Project (GPCP) Combined Precipitation Dataset
Huffman, G. J., Adler, R. F., Arkin, P., Chang, A., Ferraro, R., Gruber, A., Janowiak, J., McNab, A., Rudolf, B., & Schneider, U. (1997) · Bulletin of the American Meteorological Society · doi:10.1175/1520-0477(1997)078<0005:tgpcpg>2.0.co;2 · Cited by 1,723
Quantification of modelling uncertainties in a large ensemble of climate change simulations.
Murphy, J. M., Sexton, D. M. H., Barnett, D. N., Jones, G. S., Webb, M. J., Collins, M., & Stainforth, D. A. (2004) · Nature · doi:10.1038/nature02771 · Cited by 1,655
AIRS/AMSU/HSB on the aqua mission: design, science objectives, data products, and processing systems
Aumann, H. H., Chahine, M. T., Gautier, C., Goldberg, M. D., Kalnay, E., McMillin, L. M., Revercomb, H., Rosenkranz, P. W., Smith, W. L., Staelin, D. H., et al. (2003) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2002.808356 · Cited by 1,542
Uncertainty in predictions of the climate response to rising levels of greenhouse gases
Stainforth, D. A., Aina, T., Christensen, C., Collins, M., Faull, N., Frame, D. J., Kettleborough, J. A., Knight, S., Martin, A., Murphy, J. M., et al. (2005) · Nature · doi:10.1038/nature03301 · Cited by 1,166
Research works printed but not linked: 12
- 10.1175/1520-0477(1984)065 · doi:10.1175/1520-0477(1984)065
- 10.1175/1520-0477(1999)080 · doi:10.1175/1520-0477(1999)080
- 10.1175/1520-0477(1997)078 · doi:10.1175/1520-0477(1997)078
- 10.1175/1520-0442(1999)012 · doi:10.1175/1520-0442(1999)012
- 10.1175/1520-0477(1996)077 · doi:10.1175/1520-0477(1996)077
- 10.1175/1520-0477(2000)081 · doi:10.1175/1520-0477(2000)081
- 10.1175/1520-0477(1999)080 · doi:10.1175/1520-0477(1999)080
- 10.1175/1520-0442(1993)006 · doi:10.1175/1520-0442(1993)006
- Simmons A. J., 2000, The ERA‐40 project plan, ERA‐40 Project Report Series No. 1
- 10.1175/1520-0442(2004)017 · doi:10.1175/1520-0442(2004)017
- Taylor P.(Ed.) (2000) Final report of the Joint WCRP/SCOR Working Group on Air‐Sea Fluxes WCRP‐112.
- 10.1175/1520‐0477 · doi:10.1175/1520-0477
Cited by
Items 401–500 of 1,001
How Much Should Climate Model Output Be Smoothed in Space?
Räisänen, J., & Ylhäisi, J. S. (2010) · Journal of Climate · doi:10.1175/2010jcli3872.1 · Cited by 53
“Corroborating earlier experience from model evaluation (Gleckler et al. 2008; Reichler and Kim 2008), the”
MethodAn automatic and effective parameter optimization method for model tuning
Zhang, T., Li, L., Lin, Y., Xue, W., Xie, F., Xu, H., & Huang, X. (2015) · Geoscientific Model Development · doi:10.5194/gmd-8-3579-2015 · Cited by 47
““Downhill_1_step” represents the original downhill simplex method, which is one of the most widely used local optimization algorithms and has been successfully used in the Speedy model (Severijns and Hazeleger, 2005).”
MethodCurrent and Future Estimates of Wind Energy Potential Over Saudi Arabia
Chen, W., Castruccio, S., Genton, M. G., & Crippa, P. (2018) · Journal of Geophysical Research Atmospheres · doi:10.1029/2017jd028212 · Cited by 53
Karmalkar, A. V., Thibeault, J. M., Bryan, A. M., & Seth, A. (2019) · Climatic Change · doi:10.1007/s10584-019-02411-y · Cited by 53
“Many studies focusing on global (Gleckler et al. 2008) and regional (Sheffield et al.”
BackgroundMethodExtreme temperature indices in Eurasia in a CMIP6 multi‐model ensemble: Evaluation and projection
Zhao, Y., Qian, C., Zhang, W., He, D., & Qi, Y. (2021) · International Journal of Climatology · doi:10.1002/joc.7134 · Cited by 53
“The metrics are summarized in a so-called ‘portrait’ diagram, which provides a compact illustration of model performance relative to each other for all indices (Gleckler et al., 2008).”
BackgroundMethodEvaluating the East Asian monsoon simulation in climate models
Boo, K., Martin, G., Sellar, A., Senior, C. A., & Byun, Y. (2011) · Journal of Geophysical Research Atmospheres · doi:10.1029/2010jd014737 · Cited by 52
“mance can’t be determined simply by any single factor, we consider many scalar quantities suitable for various aspects of evaluation, in a similar manner to previous studies [Reichler and Kim, 2008; Gleckler et al., 2008 ] which used a large suite of metrics to gauge model performance.”
MethodResultInfluentialEvaluations of high-resolution dynamically downscaled ensembles over the contiguous United States
Zobel, Z., Wang, J., Wuebbles, D. J., & Kotamarthi, V. R. (2017) · Climate Dynamics · doi:10.1007/s00382-017-3645-6 · Cited by 52
“To describe relative error, we employ the performance metrics developed by Gleckler et al. (2008). To begin, root-meansquare-error (RMSE) is calculated for each variable and NCA subregion for all six model runs as well as their mean and median.”
BackgroundMethodCharron, I. (2014) · Cited by 52
Sreelatha, K., & Raj, P. A. (2019) · ISH Journal of Hydraulic Engineering · doi:10.1080/09715010.2019.1634648 · Cited by 52
Clouds and Climate: Climate Science's Greatest Challenge
(2020) · Cambridge University Press eBooks · doi:10.1017/9781107447738 · Cited by 52
Regional assessment of the parameter‐dependent performance of CAM4 in simulating tropical clouds
Zhang, Y., Xie, S., Covey, C., Lucas, D. D., Gleckler, P. J., Klein, S. A., Tannahill, J., Doutriaux, C. M., & Klein, R. (2012) · Geophysical Research Letters · doi:10.1029/2012gl052184 · Cited by 52
“To systematically assess model performance, we apply standard metrics [Taylor, 2001; Gleckler et al., 2008] to the PPE.”
MethodSelf-citationPredicting pan-tropical climate change induced forest stock gains and losses – implications for REDD
Gumpenberger, M., Vohland, K., Heyder, U., Poulter, B., Macey, K., Rammig, A., Popp, A., & Cramer, W. (2010) · Environmental Research Letters · doi:10.1088/1748-9326/5/1/014013 · Cited by 51
“In terms of their ability to forecast long-term trends there are important differences between climate models, especially on a regional scale [51, 52].”
BackgroundInterpreting climate model projections of extreme weather events
Vavrus, S. J., Notaro, M., & Lorenz, D. J. (2015) · Weather and Climate Extremes · doi:10.1016/j.wace.2015.10.005 · Cited by 51
Observation and integrated Earth-system science: A roadmap for 2016–2025
Simmons, A. J., Fellous, J.-L., Ramaswamy, V., Trenberth, K. E., Asrar, G., Balmaseda, M., Burrows, J. P., Ciais, P., Drinkwater, M. R., Friedlingstein, P., et al. (2016) · Advances in Space Research · doi:10.1016/j.asr.2016.03.008 · Cited by 51
Diaconescu, E. P., Mailhot, A., Brown, R., & Chaumont, D. (2017) · Climate Dynamics · doi:10.1007/s00382-017-3736-4 · Cited by 51
Schlef, K., François, B., Robertson, A. W., & Brown, C. (2018) · Water Resources Research · doi:10.1029/2018wr023209 · Cited by 51
“, 2004), reliability ensemble averaging (Giorgi & Mearns, 2002), a variable convergence score (Johnson & Sharma, 2009), and error metrics (Gleckler et al., 2008; Pierce et al., 2009).”
BackgroundRahimi, J., Laux, P., & Khalili, A. (2020) · Theoretical and Applied Climatology · doi:10.1007/s00704-020-03190-8 · Cited by 51
A flow-weighted ensemble strategy to assess the impacts of climate change on watershed hydrology
Dong, F., Javed, A., Saber, A., Neumann, A., Arnillas, C. A., Kaltenecker, G., & Arhonditsis, G. B. (2020) · Journal of Hydrology · doi:10.1016/j.jhydrol.2020.125898 · Cited by 53
Development of a Hybrid Machine Learning Model for Asphalt Pavement Temperature Prediction
Milad, A. A., Adwan, I., Majeed, S. A., Memon, Z. A., Bilema, M., Omar, H. A., Abdolrasol, M. G. M., Usman, A., & Yusoff, N. I. M. (2021) · IEEE Access · doi:10.1109/access.2021.3129979 · Cited by 51
“Therefore, the parameter is prone to showing a good fit despite the constant offset present in the data [60]. where 𝑦 𝑜 and 𝑦 𝑝 are the observed and predicted asphalt temperature, respectively, and 𝑁 is the number of test/observation samples.”
BackgroundUsing Machine Learning to Generate a GISS ModelE Calibrated Physics Ensemble (CPE)
Elsaesser, G. S., Elsaesser, G. S., van Lier‐Walqui, M., Yang, Q., Kelley, M., Ackerman, A. S., Fridlind, A. M., Cesana, G., Schmidt, G. A., Wu, J., et al. (2025) · Journal of Advances in Modeling Earth Systems · doi:10.1029/2024ms004713 · Cited by 50
“…1 subscripts H and L are equivalent), the penalty function collapses to the numerator of the traditional root mean square error (RMSE) calculation, thus mimicking the penalty function choice presented in some other studies (e.g., Gleckler et al., 2008; Jarvinen et al., 2010; Reichler & Kim, 2008).”
Macadam, I., Pitman, A. J., Whetton, P. H., & Abramowitz, G. (2010) · Geophysical Research Letters · doi:10.1029/2010gl043877 · Cited by 48
“For example, Groves et al.’s [2008] study of water supply and demand used Tebaldi et al.’s [2005] approach to weight AOGCMs according to bias and convergence criteria, and Penman et al.”
BackgroundMethodInfluentialWATANABE, S., Hirabayashi, Y., Kotsuki, S., Hanasaki, N., Tanaka, K., Mateo, C. M. R., Kiguchi, M., Ikoma, E., Kanae, S., & Oki, T. (2014) · Hydrological Research Letters · doi:10.3178/hrl.8.33 · Cited by 50
“Gleckler et al. (2008) evaluated the reproducibility of the global distribution of 22 variables simulated by multiple CMIP3 GCMs and proposed a metric based on ranking the degree of relative error among GCMs.”
MethodLangousis, A., Mamalakis, A., Deidda, R., & Marrocu, M. (2015) · Water Resources Research · doi:10.1002/2015wr017556 · Cited by 50
“…the GCM/ RCM combination used [see Giorgi and Francisco, 2000; Pan et al., 2001; R€ais€anen, 2006; D equ e et al., 2007; Lucarini et al., 2007; Gleckler et al., 2008; Foley, 2010; Chen et al., 2011; Bastola et al., 2011; Sulis et al., 2012; Deidda et al., 2013; Hasson et al., 2014, among…”
BackgroundMethodMean fields and interannual variability in RCM simulations over Spain: the ESCENA project
Jiménez‐Guerrero, P., Montávez, J. P., Domínguez, M., Romera, R., Fita, L., Fernández, J., Cabos, W., Liguori, G., & Gaertner, Μ. A. (2013) · Climate Research · doi:10.3354/cr01165 · Cited by 50
“These re - sults are in accordance with previous works (Gleckler et al. 2008, Coppola et al. 2010, Kjellström et al. 2011).”
ResultDrier North American Monsoon in Contrast to Asian–African Monsoon under Global Warming
He, C., Li, T., & Zhou, W. (2020) · Journal of Climate · doi:10.1175/jcli-d-20-0189.1 · Cited by 51
Schnorbus, M., & Cannon, A. J. (2014) · Water Resources Research · doi:10.1002/2014wr015279 · Cited by 49
“However, if models are instead chosen based on skill in simulating historical climate [e.g., Gleckler et al., 2008; Rupp et al., 2013], as was the case with the PCIC subset [Werner, 2011], then one may view the additional model spread as simply arising from less plausible GCMs.”
MethodTwo Faces of Uncertainty: Climate Science and Water Utility Planning Methods
Barsugli, J. J., Vogel, J. M., Kaatz, L., Smith, J. B., Waage, M., & Anderson, C. (2012) · Journal of Water Resources Planning and Management · doi:10.1061/(asce)wr.1943-5452.0000188 · Cited by 49
Identification of critical sections of the Spanish transport system due to climate scenarios
Ortega, E., Martín, B., & Aparicio, Á. (2020) · Journal of Transport Geography · doi:10.1016/j.jtrangeo.2020.102691 · Cited by 49
Climate Projections Using Bayesian Model Averaging and Space–Time Dependence
Bhat, K. S., Haran, M., Terando, A. J., & Keller, K. M. (2011) · Journal of Agricultural Biological and Environmental Statistics · doi:10.1007/s13253-011-0069-3 · Cited by 49
“For a variety of resolutions of climate models, the average of a multi-model ensemble agrees more with climate observations than any single model (Gleckler et al., 2008; Knutti et al., 2010; Pierce et al., 2009).”
BackgroundProjection of Future Meteorological Droughts in Lake Urmia Basin, Iran
Ghazi, B., Dutt, S., & Haghighi, A. T. (2023) · Water · doi:10.3390/w15081558 · Cited by 48
Global climate models’ bias in surface temperature trends and variability
Davy, R., & Esau, I. (2014) · Environmental Research Letters · doi:10.1088/1748-9326/9/11/114024 · Cited by 48
“We used a statistical measure of model fidelity [15] to determine model departure from reanalysis for the historical simulations of the CMIP5 program, over the period 1979–2005.”
MethodMarchand, W., Girardin, M. P., Gauthier, S., Hartmann, H., Bouriaud, O., Babst, F., & Bergeron, Y. (2018) · Environmental Research Letters · doi:10.1088/1748-9326/aad82a · Cited by 48
“Some authors highlight the lack of a systematic assessment of data quality (Goetz et al 2005, Gleckler et al 2008, Ju and Masek 2016), which is necessary to quantify trend accuracy.”
BackgroundQuetin, G. R., Bloom, A. A., Bowman, K. W., & Konings, A. G. (2020) · Journal of Advances in Modeling Earth Systems · doi:10.1029/2019ms001889 · Cited by 48
“In the atmospheric sciences, long-term reanalyses of weather models—incorporating numerous assimilated data streams—have a strong history (Harms et al., 1992) being used as pseudo-observations to benchmark the simulated atmospheres in climate models (e.g., Gleckler et al., 2008).”
BackgroundLi, L., Li, W., & Jin, J. (2014) · Climate Dynamics · doi:10.1007/s00382-013-2031-2 · Cited by 48
“those observed (Taylor 2001) and has been widely used to evaluate climate models (e.g., AchutaRao and Sperber 2006; Gleckler et al. 2008).”
MethodThe future distribution of the savannah biome: model-based and biogeographic contingency
Moncrieff, G. R., Scheiter, S., Langan, L., Trabucco, A., & Higgins, S. I. (2016) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2015.0311 · Cited by 48
HESS Opinions "Should we apply bias correction to global and regional climate model data?"
Ehret, U., Zehe, E., Wulfmeyer, V., Warrach‐Sagi, K., & Liebert, J. (2012) · doi:10.5194/hessd-9-5355-2012 · Cited by 48
Distributions of decadal means of temperature and precipitation change under global warming
Watterson, I. G., & Whetton, P. H. (2011) · Journal of Geophysical Research Atmospheres · doi:10.1029/2010jd014502 · Cited by 47
“Variability from one of the less skilful models (as found by Gleckler et al. [2008] ) was rather different,evenafterasubstantialtrendwasremoved(FigureS4).”
BackgroundDiaconescu, E. P., Gachon, P., Laprise, R., & Scinocca, J. (2016) · ATMOSPHERE-OCEAN · doi:10.1080/07055900.2016.1185005 · Cited by 47
“…as a colour-coded table displaying values of the skill metric between each gridded observational dataset and each model and reanalysis (horizontal axis), for each variable (vertical axis), inspired by the performance “portrait diagram” proposed by Gleckler, Taylor, and Doutriaux (2008) for GCMs.”
MethodDike, V. N., Lin, Z., Fei, K., Langendijk, G. S., & Nath, D. (2022) · International Journal of Climatology · doi:10.1002/joc.7641 · Cited by 47
“Besides, the capability of the models in representing different aspects of seasonal extreme precipitation statistics in terms of how well the models match the GPCC data are illustrated using a Taylor diagram (Taylor, 2001; Gleckler et al., 2008; Baker and Taylor, 2016).”
MethodAlmazroui, M., Abid, M. A., Athar, H., Islam, M. N., & Ehsan, M. A. (2012) · International Journal of Climatology · doi:10.1002/joc.3600 · Cited by 47
“The ability of the GCMs in simulating the interannual variability of the rainfall over the Peninsula is studied by applying widely used statistical methods (Wilks, 2006; Gleckler et al ., 2008; Errasti et al ., 2011).”
MethodBurke, M., Dykema, J., Lobell, D., Miguel, E., & Satyanath, S. (2011) · National Bureau of Economic Research · doi:10.3386/w17092 · Cited by 37
Projection of climate change impacts on extreme temperature and precipitation in Central Poland
Ghazi, B., Przybylak, R., & Pospieszyńska, A. (2023) · Scientific Reports · doi:10.1038/s41598-023-46199-5 · Cited by 46
“Climate model performance can be evaluated using several criteria 54 .”
MethodBenchmarking CMIP5 models with a subset of ESA CCI Phase 2 data using the ESMValTool
Lauer, A., Eyring, V., Righi, M., Buchwitz, M., Defourny, P., Evaldsson, M., Friedlingstein, P., de Jeu, R., de Leeuw, G., Loew, A., et al. (2017) · Remote Sensing of Environment · doi:10.1016/j.rse.2017.01.007 · Cited by 46
Impacts of the Autumn Arctic Sea Ice on the Intraseasonal Reversal of the Winter Siberian High
Lü, Z., He, S., Li, F., & Wang, H. (2018) · Advances in Atmospheric Sciences · doi:10.1007/s00376-017-8089-8 · Cited by 46
Development of multi-model ensemble for projection of extreme rainfall events in Peninsular Malaysia
Noor, M., Ismail, T., Shahid, S., Nashwan, M. S., & Ullah, S. (2019) · Hydrology research · doi:10.2166/nh.2019.097 · Cited by 46
Climate changes impact on stormwater infrastructure design in Tucson Arizona
Tousi, E. G., O’Brien, W., Doulabian, S., & Toosi, A. S. (2021) · Sustainable Cities and Society · doi:10.1016/j.scs.2021.103014 · Cited by 46
The importance of hydrological uncertainty assessment methods in climate change impact studies
Honti, M., Scheidegger, A., & Stamm, C. (2014) · Hydrology and earth system sciences · doi:10.5194/hess-18-3301-2014 · Cited by 46
“We did not differentiate between individual model chains based on their skill or performance as for example Gleckler et al. (2008) did, so the future climate in our assessment was represented with a model chain ensemble with uniform weights.”
Climate change impact on Sub-Saharan Africa : An overview and analysis of scenarios and modelsPreprint
Müller, C. (2009) · RePEc: Research Papers in Economics · Cited by 46
Khosravi, Y., Ouarda, T. B. M. J., & Homayouni, S. (2025) · npj Climate and Atmospheric Science · doi:10.1038/s41612-025-01033-9 · Cited by 45
Quantitative evaluation of the seasonal variations in climate model cloud regimes
Tsushima, Y., Ringer, M. A., Webb, M. J., & Williams, K. D. (2012) · Climate Dynamics · doi:10.1007/s00382-012-1609-4 · Cited by 45
“Metrics, which provide quantitative measures of model error in various aspects of the climate system, have become more commonly used in the climate modeling community (Gleckler et al. 2008) since the IPCC AR4 (Meehl et al. 2007).”
MethodValidation of Satellite, Reanalysis and RCM Data of Monthly Rainfall in Calabria (Southern Italy)
Caroletti, G. N., Coscarelli, R., & Caloiero, T. (2019) · Remote Sensing · doi:10.3390/rs11131625 · Cited by 45
“• Mean error and standard deviation error are among the most commonly used tools in validation and error theory [68,69], and also to compare E-OBS and ENSEMBLES RCMs [25]; • The Pearson correlation coefficient is an important measure commonly used in climate science for evaluating data from independent data sources, j…”
MethodAmos, M., Young, P. J., Hosking, J. S., Lamarque, J., Abraham, N. L., Akiyoshi, H., Archibald, A. T., Bekki, S., Deushi, M., Jöckel, P., et al. (2020) · Atmospheric chemistry and physics · doi:10.5194/acp-20-9961-2020 · Cited by 45
Kim, M., Yu, D., Oh, J., Byun, Y., Boo, K., Chung, I., Park, J., Park, D. R., Min, S., & Sung, H. M. (2020) · Journal of Geophysical Research Atmospheres · doi:10.1029/2020jd032583 · Cited by 45
“…Model Intercomparison Project (CMIP) (Eyring et al., 2016; Taylor et al., 2012), global circulation models (GCMs) are collectively evaluated (e.g., Gleckler et al., 2008; Lee et al., 2019) and widely used to study the effects of past, present, and future global and regional climate change in…”
BackgroundJiang, J. H., Su, H., Wu, L., Zhai, C., & Schiro, K. A. (2021) · Earth and Space Science · doi:10.1029/2020ea001520 · Cited by 45
Balhane, S., Driouech, F., Chafki, O., Manzanas, R., Chehbouni, A., & Moufouma-Okia, W. (2021) · Climate Dynamics · doi:10.1007/s00382-021-05910-w · Cited by 45
“Consequently, some models would deserve more important weights than others in the construction of the multi-model ensemble depending on their performance for a given application (e.g., Baumberger et al. 2017; Eyring et al. 2019; Gleckler et al. 2008; Knutti et al. 2010, 2017).”
MethodDubrovský, M., Trnka, M., Holman, I., Svobodová, E., & Harrison, P. A. (2014) · Climatic Change · doi:10.1007/s10584-014-1297-7 · Cited by 45
“…was based on 11 validation statistics used in experiments made by other authors; obviously, this score, or any single validation statistic (see also Gleckler et al. 2008) could serve to identify the “best” GCM. Thirdly, the metrics used to quantify the between-GCM distance, which we based on…”
MethodResultFuture Warming Patterns Linked to Today’s Climate Variability
Dai, A. (2016) · Scientific Reports · doi:10.1038/srep19110 · Cited by 45
Hadri, A., Saidi, M. E., Khalki, E. M. E., Aachrine, B., Saouabe, T., & Elmaki, A. A. (2022) · Journal of Water and Climate Change · doi:10.2166/wcc.2022.039 · Cited by 44
Zhang, H., & Huang, G. (2012) · Climate Dynamics · doi:10.1007/s00382-012-1490-1 · Cited by 44
Quantitative evaluation of ozone and selected climate parameters in a set of EMAC simulations
Righi, M., Eyring, V., Gottschaldt, K.-D., Klinger, C., Frank, F., Jöckel, P., & Cionni, I. (2015) · Geoscientific model development · doi:10.5194/gmd-8-733-2015 · Cited by 45
“(2006), Gleckler et al. (2008), Reichler and Kim (2008), Karpechko et al. (2010) and Yokoi et al. (2011). While Taylor (2001) and Yokoi et al.”
BackgroundMethodInfluentialIntegrated Hydrological Modeling of Climate Change Impacts in a Snow‐Influenced Catchment
Cochand, F., Therrien, R., & Lemieux, J. (2018) · Ground Water · doi:10.1111/gwat.12848 · Cited by 44
Katzav, J., Thompson, E., Risbey, J. S., Stainforth, D. A., Bradley, S., & Frisch, M. (2021) · Climatic Change · doi:10.1007/s10584-021-03267-x · Cited by 44
“2, on the philosophy of probability [Hájek, 2019] to provide a general characterisation of kinds of uncertainty and how these might be misrepresented. We also use this characterisation to illustrate some of the ways in which PDFs might misrepresent our uncertainty about future weather or climate and some of the effect…”
BackgroundInfluentialRECLAMATION: Managing Water in the West
Nebraska – Kansas Area Office, & Great Plains Regional Office (2008) · Lincoln (University of Nebraska) · Cited by 44
Sun, X., Ge, F., Chen, Q., Fraedrich, K., & Li, X. (2023) · Earth s Future · doi:10.1029/2022ef003179 · Cited by 43
“The IVS is a symmetrical variability statistic used to quantify the interannual variation, with smaller values indicating better agreement with the observations (Gleckler et al., 2008).”
MethodNadege, M. N., Jiang, S. Y., Mwakipunda, G. C., Kouassi, A. K. F., Kavuba, H. P., & Roland, K. Y. H. (2023) · Geoenergy Science and Engineering · doi:10.1016/j.geoen.2023.212518 · Cited by 43
Multi-criterion model ensemble of CMIP5 surface air temperature over China
Yang, T., Tao, Y., Li, J., Zhu, Q., Su, L., He, X., & Zhang, X. (2017) · Theoretical and Applied Climatology · doi:10.1007/s00704-017-2143-4 · Cited by 43
“To comprehensively evaluate the performances of GCMs, it is recommended by many researchers (Miao et al. 2014; Duan and Phillips 2010; Giorgi and Francisco 2000a, b; Gleckler et al. 2008) that the a comprehensive evaluation on the GCMs’ outputs should include multiple statistical measures at the same time, such as roo…”
BackgroundMethodAssessment of Precipitation Simulations in Central Asia by CMIP5 Climate Models
Ta, Z., Yu, Y., Sun, L., Chen, X., Mu, G., & Yu, R. (2018) · Water · doi:10.3390/w10111516 · Cited by 43
“[6], the assessment of the performance of climate models should not only focus on”
BackgroundPadhiary, J., Patra, K. C., Dash, S. S., & Jayakumar, K. (2019) · Journal of Water and Climate Change · doi:10.2166/wcc.2019.080 · Cited by 43
Wang, G., Dommenget, D., & Frauen, C. (2014) · Climate Dynamics · doi:10.1007/s00382-014-2154-0 · Cited by 43
“Previous model evaluations focused on the mean state climate (Taylor 2001; Boer and Lambert 2001; Murphy et al. 2004; Gleckler et al. 2008), the general strength of climate variability (Boer and Lambert 2001; Gleckler et al. 2008) or on some regional aspects of climate variability (Guilyardi 2006;…”
BackgroundMethodInfluentialEffects of changes in snow pattern and the timing of runoff on NYC water supply system
Matonse, A. H., Pierson, D. C., Frei, A., Zion, M. S., Schneiderman, E. M., Anandhi, A., Mukundan, R., & Pradhanang, S. M. (2011) · Hydrological Processes · doi:10.1002/hyp.8121 · Cited by 42
“…GCM models and scenarios because different GCMs represent different physical processes and feedbacks differently resulting in high uncertainty particularly in simulated future precipitation at a regional scale (Neff et al., 2000; Annan et al., 2005; Gleckler et al., 2008; Hayhoe et al., 2007).”
BackgroundSéférian, R., Baek, S., Boucher, O., Dufresne, J., Decharme, B., Saint‐Martin, D., & Roehrig, R. (2018) · Geoscientific model development · doi:10.5194/gmd-11-321-2018 · Cited by 43
Uncertainty Quantification Using Multiple Models—Prospects and Challenges
Knutti, R., Baumberger, C., & Hadorn, G. H. (2019) · Simulation foundations, methods and applications · doi:10.1007/978-3-319-70766-2_34 · Cited by 42
An 8-model ensemble of CMIP6-derived ocean surface wave climate
Meucci, A., Young, I. R., Trenham, C., & Hemer, M. (2024) · Scientific Data · doi:10.1038/s41597-024-02932-x · Cited by 42
“Figure 4 shows the PCDMI PMP portrait plot 63 computed for each of the eight models composing the wind wave climate ensemble and for five integral parameter outputs.”
MethodReview of Downscaling Methodologies for Africa Climate Applications
Brown, C., Greene, A. M., Block, P., & Giannini, A. (2008) · doi:10.7916/d8m04c88 · Cited by 41
“• Hydrologic models add little uncertainty relative to the uncertainty associated with climate models • The choice of hydrologic model should be determined in accordance with the ability to estimate the desired variable • Hydrologic model choice affects the downscaling method selection”
BackgroundJin, C., Xiao, X., Wagle, P., Griffis, T. J., Dong, J., Wu, C., Qin, Y., & Cook, D. R. (2015) · Agricultural and Forest Meteorology · doi:10.1016/j.agrformet.2015.07.003 · Cited by 41
“…provide a statistical summary of the similarity of varibility pattern ( ), the agreement of the variability amplitudes represented by the ratio of normalized standard deviation, ̄ratio), nd the centered RMSE between the modeled results and the obserations (Gleckler et al., 2008; Taylor, 2001).”
ResultGlobal Flood Risk Modeling and Projections of Climate Change Impacts
Yamazaki, D., WATANABE, S., & Hirabayashi, Y. (2018) · Geophysical monograph · doi:10.1002/9781119217886.ch11 · Cited by 41
He, C., Zhou, T., & Li, T. (2018) · Journal of Climate · doi:10.1175/jcli-d-18-0033.1 · Cited by 41
Kamworapan, S., Thảo, P. T. B., Gheewala, S. H., Pimonsree, S., & Prueksakorn, K. (2021) · Heliyon · doi:10.1016/j.heliyon.2021.e08263 · Cited by 41
Jose, D. M., & Dwarakish, G. S. (2022) · Arabian Journal of Geosciences · doi:10.1007/s12517-021-09289-0 · Cited by 41
New climate model metrics based on object-orientated pattern matching of rainfall
Moise, A. F., & Delage, F. P. D. (2011) · Journal of Geophysical Research Atmospheres · doi:10.1029/2010jd015318 · Cited by 41
Kusunoki, S. (2016) · Climate Dynamics · doi:10.1007/s00382-016-3335-9 · Cited by 41
“This is consistent with previous findings that the MME average can be expected to outperform individual models in climate simulations (Lambert and Boer 2001; Gleckler et al. 2008; Reichler and Kim 2008).”
ResultChen, L., van den Dool, H., Becker, E. J., & Zhang, Q. (2016) · Journal of Climate · doi:10.1175/jcli-d-15-0903.1 · Cited by 41
Nooni, I. K., Ogou, F. K., Chaibou, A. A. S., Nakoty, F. M., Gnitou, G. T., & Lu, J. (2023) · Atmosphere · doi:10.3390/atmos14030607 · Cited by 40
Predictivism and old evidence: a critical look at climate model tuning
Frisch, M. (2015) · European Journal for Philosophy of Science · doi:10.1007/s13194-015-0110-4 · Cited by 40
“^ (Randall et al. 2007, 596) 4 See, e.g., (Gleckler et al. 2008, 2): Bmodels can to a certain degree be tunable to appear realistic in some respects, but as a result of compensating errors.”
BackgroundZhou, Z., Ding, Y., Fu, Q., Wang, C., Wang, Y., Cai, H., Liu, S., Huang, S., & Shi, H. (2023) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2023.165618 · Cited by 40
Sobolowski, S., & Pavelsky, T. M. (2011) · Journal of Geophysical Research Atmospheres · doi:10.1029/2011jd016430 · Cited by 39
How to incorporate climate change into modelling environmental water outcomes: a review
John, A., Nathan, R. J., Horne, A., Stewardson, M. J., & Webb, J. A. (2020) · Journal of Water and Climate Change · doi:10.2166/wcc.2020.263 · Cited by 39
Evaluation of the CMIP6 planetary albedo climatology using satellite observations
Jian, B., Li, J., Zhao, Y., He, Y., Wang, J., & Huang, J. (2020) · Climate Dynamics · doi:10.1007/s00382-020-05277-4 · Cited by 39
Methods for uncertainty assessment of climate models and model predictions over East Asia
Heo, K., Ha, K., Yun, K., Lee, S., Kim, H., & Wang, B. (2013) · International Journal of Climatology · doi:10.1002/joc.3692 · Cited by 39
“Furthermore, a multimodel ensemble usually yields better performance than any single model (Lambert and Boer, 2001; Gleckler et al ., 2008).”
MethodAnandhi, A., & Nanjundiah, R. S. (2014) · Theoretical and Applied Climatology · doi:10.1007/s00704-013-1043-5 · Cited by 39
“The performance of GCMs at the regional scale remains variable (Bollasina and Nigam 2009; Gleckler et al. 2008; Randall et al. 2007).”
BackgroundSiam, M. S., Demory, M., & Eltahir, E. A. B. (2013) · Journal of Climate · doi:10.1175/jcli-d-12-00404.1 · Cited by 39
“…that compare model outputs with observations and provide information on the performance of the model without detailing the cause of biases (e.g., Gleckler et al. 2008; Perkins et al. 2007); 2) diagnostics that provide information on the sources of model discrepancies and include a detailed…”
ResultKlocke, D., & Rodwell, M. J. (2013) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.2172 · Cited by 39
“variability (Reichler and Kim, 2008; Gleckler et al., 2008).”
BackgroundDecision Scaling (DS): Decision Support for Climate Change
Brown, C., Steinschneider, S., Ray, P. A., Wi, S., Basdekas, L., & Yates, D. N. (2019) · Springer eBooks · doi:10.1007/978-3-030-05252-2_12 · Cited by 39
Assessing the degree of hydrologic stress due to climate change
Nathan, R. J., McMahon, T. A., Peel, M., & Horne, A. (2019) · Climatic Change · doi:10.1007/s10584-019-02497-4 · Cited by 39
Pitman, A. J., & Perkins-Kirkpatrick, S. E. (2009) · Journal of Climate · doi:10.1175/2009jcli2799.1 · Cited by 39
“Gleckler et al. (2008) state that the 2-m air temperature is a diagnostic quantity, computed from complex iterative schemes based on conditions in the lowest model level and near the surface.”
BackgroundMethodModelling the Climate System: An Overview
Gramelsberger, G., & Feichter, J. (2011) · Springer eBooks · doi:10.1007/978-3-642-17700-2_2 · Cited by 38
“Interesting, for instance, is that the performance of a model to reproduce interannual variability is only weakly correlated with an index of mean climate performance (Gleckler et al. 2008).”
BackgroundAn alternative method for predicting relative humidity for climate change studies
Gunawardhana, L. N., Al-Rawas, G., & Kazama, S. (2017) · Meteorological Applications · doi:10.1002/met.1641 · Cited by 38
“Gleckler et al. (2008) compared model performances when predicting surface air temperature, precipitation and mean © 2017 Royal Meteorological Society Meteorol.”
ResultLi, C., Lu, H. M., Yang, K., Wright, J. S., Yu, L., Chen, Y., Huang, X., & Xu, S. (2017) · Atmosphere · doi:10.3390/atmos8080141 · Cited by 38
“CoLM2005 and CoLM2014 have similar values of MCPI for the global annual mean, indicating that these two versions of CoLM have similar overall performances.”
MethodInfluentialThe Optimal Multimodel Ensemble of Bias-Corrected CMIP5 Climate Models over China
Yang, X., Yu, X., Wang, Y., He, X., Pan, M., Zhang, M., Liu, Y., Ren, L., & Sheffield, J. (2020) · Journal of Hydrometeorology · doi:10.1175/jhm-d-19-0141.1 · Cited by 38
Lin, W., & Chen, H. (2020) · Atmospheric and Oceanic Science Letters · doi:10.1080/16742834.2020.1820303 · Cited by 38
Metadata
- Authors
- Gleckler, P. J., Taylor, K. E., & Doutriaux, C.
- Affiliations
- Lawrence Livermore National Laboratory (US)
- Venue
- Journal of Geophysical Research: Atmospheres
- Publisher
- American Geophysical Union (AGU)
- Published
- 2008-03-20
- Volume
- 113
- Issue
- D6
- ISSN
- 0148-0227 (ISSN-L)
- OpenAlex
- W1996478169
- Cited by
- 1,328
- References
- 31
- Citations by year
- 2026: 64 · 2025: 73 · 2024: 85 · 2023: 88 · 2022: 69 · 2021: 116 · 2020: 103 · 2019: 83 · 2018: 65 · 2017: 76 · 2016: 63 · 2015: 77 · 2014: 65 · 2013: 87 · 2012: 73
- FWCI
- 41.84
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Geophysical Research Atmospheres (publishedVersion)
- Topics
- Physical Sciences › Environmental Science › Global and Planetary Change › Climate variability and models
- Physical Sciences › Earth and Planetary Sciences › Atmospheric Science › Meteorological Phenomena and Simulations
- Physical Sciences › Environmental Science › Global and Planetary Change › Climate Change and Geoengineering
- Keywords
- climate model evaluation, performance indicators, CMIP3, relative error, atmospheric climatology, model ranking, interannual variability, observational uncertainty, spatial scale, 20th century climate simulation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.