Performance metrics for climate models.
The DOI matches its Crossref record; 33 of the 34 references it cites were found in a registry or an index, and 1 are not checked yet.
Abstract
Objective measures of climate model performance are proposed and used to assess simulations of the 20th century, which are available from the Coupled Model Intercomparison Project (CMIP3) archive. The primary focus of this analysis is on the climatology of atmospheric fields. For each variable considered, the models are ranked according to a measure of relative error. Based on an average of the relative errors over all fields considered, some models appear to perform substantially better than others. Forming a single index of model performance, however, can be misleading in that it hides a more complex picture of the relative merits of different models. This is demonstrated by examining individual variables and showing that the relative ranking of models varies considerably from one variable to the next. A remarkable exception to this finding is that the so‐called “mean model” consistently outperforms all other models in nearly every respect. The usefulness, limitations and robustness of the metrics defined here are evaluated 1) by examining whether the information provided by each metric is correlated in any way with the others, and 2) by determining how sensitive the metrics are to such factors as observational uncertainty, spatial scale, and the domain considered (e.g., tropics versus extra‐tropics). An index that gauges the fidelity of model variability on interannual time‐scales is found to be only weakly correlated with an index of the mean climate performance. This illustrates the importance of evaluating a broad spectrum of climate processes and phenomena since accurate simulation of one aspect of climate does not guarantee accurate representation of other aspects. Once a broad suite of metrics has been developed to characterize model performance it may become possible to identify optimal subsets for various applications.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“not even the question of which model performs best during the control simulations can be readily answered but, rather, depends strongly on the skill metric, variable, and region considered (e.g., Gleckler et al. 2008).”
From Risks of Model Weighting in Multimodel Climate Projections (2010) · cites this paper as Gleckler et al. 2008
Citations
Cites
The NCEP/NCAR 40-Year Reanalysis Project
Kalnay, E., Kanamitsu, M., Kistler, R., Collins, W., Deaven, D., Gandin, L., Iredell, M., Saha, S., White, G., Woollen, J., et al. (1996) · Bulletin of the American Meteorological Society · doi:10.1175/1520-0477(1996)077<0437:tnyrp>2.0.co;2 · Cited by 28,360
Summarizing multiple aspects of model performance in a single diagram
Taylor, K. E. (2001) · Journal of Geophysical Research: Atmospheres · doi:10.1029/2000jd900719 · Cited by 8,477
Self-citationAdler, R. F., Huffman, G. J., Chang, A., Ferraro, R., Xie, P.-P., Janowiak, J., Rudolf, B., Schneider, U., Curtis, S., Bolvin, D., et al. (2003) · Journal of Hydrometeorology · doi:10.1175/1525-7541(2003)004<1147:tvgpcp>2.0.co;2 · Cited by 3,808
Advances in Understanding Clouds from ISCCP
Rossow, W. B., & Schiffer, R. A. (1999) · Bulletin of the American Meteorological Society · doi:10.1175/1520-0477(1999)080<2261:aiucfi>2.0.co;2 · Cited by 2,256
Clouds and the Earth's Radiant Energy System (CERES): An Earth Observing System Experiment
Wielicki, B. A., Barkstrom, B. R., Harrison, E. F., Lee, R. B., Louis Smith, G., & Cooper, J. E. (1996) · Bulletin of the American Meteorological Society · doi:10.1175/1520-0477(1996)077<0853:catere>2.0.co;2 · Cited by 1,957
The Global Precipitation Climatology Project (GPCP) Combined Precipitation Dataset
Huffman, G. J., Adler, R. F., Arkin, P., Chang, A., Ferraro, R., Gruber, A., Janowiak, J., McNab, A., Rudolf, B., & Schneider, U. (1997) · Bulletin of the American Meteorological Society · doi:10.1175/1520-0477(1997)078<0005:tgpcpg>2.0.co;2 · Cited by 1,723
Quantification of modelling uncertainties in a large ensemble of climate change simulations.
Murphy, J. M., Sexton, D. M. H., Barnett, D. N., Jones, G. S., Webb, M. J., Collins, M., & Stainforth, D. A. (2004) · Nature · doi:10.1038/nature02771 · Cited by 1,655
AIRS/AMSU/HSB on the aqua mission: design, science objectives, data products, and processing systems
Aumann, H. H., Chahine, M. T., Gautier, C., Goldberg, M. D., Kalnay, E., McMillin, L. M., Revercomb, H., Rosenkranz, P. W., Smith, W. L., Staelin, D. H., et al. (2003) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2002.808356 · Cited by 1,542
Uncertainty in predictions of the climate response to rising levels of greenhouse gases
Stainforth, D. A., Aina, T., Christensen, C., Collins, M., Faull, N., Frame, D. J., Kettleborough, J. A., Knight, S., Martin, A., Murphy, J. M., et al. (2005) · Nature · doi:10.1038/nature03301 · Cited by 1,166
Research works printed but not linked: 12
- 10.1175/1520-0477(1984)065 · doi:10.1175/1520-0477(1984)065
- 10.1175/1520-0477(1999)080 · doi:10.1175/1520-0477(1999)080
- 10.1175/1520-0477(1997)078 · doi:10.1175/1520-0477(1997)078
- 10.1175/1520-0442(1999)012 · doi:10.1175/1520-0442(1999)012
- 10.1175/1520-0477(1996)077 · doi:10.1175/1520-0477(1996)077
- 10.1175/1520-0477(2000)081 · doi:10.1175/1520-0477(2000)081
- 10.1175/1520-0477(1999)080 · doi:10.1175/1520-0477(1999)080
- 10.1175/1520-0442(1993)006 · doi:10.1175/1520-0442(1993)006
- Simmons A. J., 2000, The ERA‐40 project plan, ERA‐40 Project Report Series No. 1
- 10.1175/1520-0442(2004)017 · doi:10.1175/1520-0442(2004)017
- Taylor P.(Ed.) (2000) Final report of the Joint WCRP/SCOR Working Group on Air‐Sea Fluxes WCRP‐112.
- 10.1175/1520‐0477 · doi:10.1175/1520-0477
Cited by
Items 901–1,000 of 1,001
Projected changes in wave storm patterns near key ports and shipping routes in the Black Sea
Çakmak, R. E., Amarouche, K., Akpınar, A., & Otay, E. N. (2025) · Ocean & Coastal Management · doi:10.1016/j.ocecoaman.2024.107537 · Cited by 9
Assessment and future projections of storm surge using CMIP6 models in the Indo-Pacific region
Saha, K. K., Kumar, P., Singh, A. K., Kamranzad, B., Young, I. R., & Rajni, R. (2025) · Ocean Modelling · doi:10.1016/j.ocemod.2025.102560 · Cited by 9
Sun, Y., Oleson, K. W., Zhao, L., Mills, G. M., He, C., Demuzere, M., Topping, D. O., Zhang, N., & Zheng, Z. (2025) · Journal of Advances in Modeling Earth Systems · doi:10.1029/2025ms004934 · Cited by 9
“…in 20 sites, we also examined the flux fluctuation by calculating the monthly mean and hourly mean heat flux with the root‐mean square error (RMSE) metric (He et al., 2008) for each site (Equation 6): We compared urban variables and metrics across simulations to evaluate model performance.”
Koelling, J., Atamanchuk, D., Wallace, D. W. R., & Karstensen, J. (2023) · Frontiers in Marine Science · doi:10.3389/fmars.2023.1202299 · Cited by 9
“We also choose a score that is based solely on the comparison with observational data, rather than a relative model score or ranking as used in other studies (Gleckler et al., 2008; Anav et al., 2013).”
BackgroundMethodResultClimate model selection via conformal clustering of spatial functional data
Villani, V., Romano, E., & Mateu, J. (2024) · Environmental and Ecological Statistics · doi:10.1007/s10651-024-00616-8 · Cited by 9
“Some studies combined several performance measures such as root mean square errors (RMSE) (Chiew et al. 2009; Gleckler and Taylor 2008; Pitman and Perkins 2008; Winter and Nychka 2009), correlation coefficients (Murphy and Epstein 1989; Murphy 1996), and average of errors (Altinsoy and Yildirim…”
Evaluating climate models to analyze drought conditions in the western region of Bangladesh
Rayhan, M., & Afroz, R. (2024) · Progress in Disaster Science · doi:10.1016/j.pdisas.2024.100356 · Cited by 9
Zheng, J., Wang, W., Wang, W., Cui, T., Chen, S., Xu, C., & Engel, B. A. (2024) · Agricultural Systems · doi:10.1016/j.agsy.2024.104087 · Cited by 9
Multifractal Analysis for Evaluating the Representation of Clouds in Global Kilometer‐Scale Models
Freischem, L. J., Weiss, P., Christensen, H. M., & Stier, P. (2024) · Geophysical Research Letters · doi:10.1029/2024gl110124 · Cited by 9
“Traditionally, climate model performance is evaluated against spatio‐temporally averaged observations (Flato et al., 2014; Gleckler et al., 2008).”
Dong, B., Ullrich, P. A., Lee, J., Gleckler, P. J., Chang, K., & O'Brien, T. A. (2025) · Geoscientific model development · doi:10.5194/gmd-18-961-2025 · Cited by 9
Self-citationThe effect of solar radiation modification on agroclimatic indices in Africa
Egbebiyi, T. S., Ajayi, V. O., Arowolo, A. V., Ogunniyi, J. A., & Ogunjo, S. T. (2025) · Environmental Research Climate · doi:10.1088/2752-5295/ade619 · Cited by 9
“The ensemble outputs were analyzed for reduced spread and improved correlation compared to individual models, demonstrating the MME’s effectiveness in minimizing biases (Gleckler et al 2008, Weigel et al 2010).”
Climate Change Impact Modelling Cascade – Benefits and Limitations for Conservation Management
Vohland, K., Rannow, S., & Stagl, J. (2013) · Advances in global change research · doi:10.1007/978-94-007-7960-0_5 · Cited by 9
Kling, C. L., Arritt, R. W., Calhoun, G., Keiser, D. A., Antle, J. M., Arnold, J. G., Carriquiry, M., Chaubey, I., Christensen, P., Ganapathysubramanian, B., et al. (2016) · American Journal of Agricultural Economics · Cited by 9
Mignot, J., Mejía, C., Sorror, C., Sylla, A., Crépon, M., & Thiria, S. (2020) · Geoscientific model development · doi:10.5194/gmd-13-2723-2020 · Cited by 9
Atmospheric River Modeling: Forecasts, Climate Simulations, and Climate Projections
Waliser, D. E., & Cordeira, J. M. (2020) · Springer eBooks · doi:10.1007/978-3-030-28906-5_6 · Cited by 9
Wickstrøm, K., Johnson, J. E., Løkse, S., Camps‐Valls, G., Mikalsen, K. Ø., Kampffmeyer, M., & Jenssen, R. (2022) · Communications in computer and information science · doi:10.1007/978-3-031-17030-0_10 · Cited by 9
Yao, Z., Xue, Z. G., He, R., Bao, X., Xie, J., & Ge, Q. (2017) · Journal of Ocean University of China · doi:10.1007/s11802-017-3012-6 · Cited by 9
Climate model projections for future seasonal rainfall cycle statistics in Northwest Costa Rica
AlMutairi, B. S., Grossmann, I., & Small, M. J. (2019) · International Journal of Climatology · doi:10.1002/joc.5993 · Cited by 9
“…bias to measure the ability of a given model to simulate the observed annual mean precipitation (Su et al., 2013) and an interannual variability index IAV to judge agreement between the modelled and observed interannual variability (Gleckler et al., 2008; Scherrer, 2011) for the period 1979–2005.”
BackgroundProjection of Potential Evapotranspiration for North Korea Based on Selected GCMs by TOPSIS
Ryu, Y., Chung, ., Seo, S. B., & Sung, J. H. (2020) · KSCE Journal of Civil Engineering · doi:10.1007/s12205-020-0283-z · Cited by 9
“…that GCMs should be selected based on their performance in simulating the climatic variable of interest for reduction of uncertainty in projection of that variable (Gleckler et al., 2008; Jiang et al., 2015; McMahon et al., 2015; Lutz et al., 2016; Ahmed et al., 2019; Sa’adi et al., 2019).”
BackgroundMethodСеменов, В. А., & Aleshina, M. A. (2022) · Water Resources · doi:10.1134/s0097807822040170 · Cited by 9
A SPATIOTEMPORAL-AWARE WEIGHTING SCHEME FOR IMPROVING CLIMATE MODEL ENSEMBLE PREDICTIONS
Fan, M., Lu, D., Rastogi, D., & Pierce, E. M. (2022) · Journal of Machine Learning for Modeling and Computing · doi:10.1615/jmachlearnmodelcomput.2022046715 · Cited by 9
Relationship of the Reproducibility of Multiple Variables among Global Climate Models
Nishii, K., Miyasaka, T., Nakamura, H., Kosaka, Y., Yokoi, S., Takayabu, Y. N., Endo, H., Ichikawa, H., Inoue, T., Oshima, K., et al. (2012) · Journal of the Meteorological Society of Japan Ser II · doi:10.2151/jmsj.2012-a04 · Cited by 9
“It should be pointed out that they do not necessarily capture every aspect of the performance of a climate model, since its reproducibility of the mean state and that of natural variability around it do not necessarily correlate positively (Gleckler et al. 2008; Santer et al. 2009).”
BackgroundMethodInfluentialBurrows, S. M., Dasgupta, A., Reehl, S., Bramer, L., Ma, P., Rasch, P. J., & Qian, Y. (2018) · Advances in Atmospheric Sciences · doi:10.1007/s00376-018-7300-x · Cited by 9
Climate change effects on watershed processes in British Columbia
Pike, R. G., Bennett, K. E., Redding, T. E., (Werner), A. T. S., Spittlehouse, D. L., Moore, R. D., Murdock, T. Q., Beckers, J., Smerdon, B., Bladon, K. D., et al. (2010) · UVic’s Research and Learning Repository (University of Victoria) · Cited by 9
Modeling meteorological drought across scales with regional and global climate indicators
Ilyas, M., Niaz, R., Cheng, H., Persio, L. D., Elwahab, M. E. A., Mehr, A. D., & Hussain, I. (2025) · Theoretical and Applied Climatology · doi:10.1007/s00704-025-05860-x · Cited by 8
“These metrics quantify each model’s accuracy, bias, and explanatory power across all six meteo - rological stations, providing a comprehensive assessment of predictive capability (Gleckler et al. 2008).”
He, M., Chen, Y., Sun, H., & Liu, J. (2023) · Remote Sensing · doi:10.3390/rs15184608 · Cited by 8
Abdulahi, S. D., Husen, S. B., & Muleta, A. H. (2024) · Water Practice & Technology · doi:10.2166/wpt.2024.134 · Cited by 8
Assessment of Annual and Seasonal Surface Air Temperature Simulations in CMIP6 Models over India
Sabarinath, A., Kesavavarthini, T., Nair, M. M., & Rajesh, A. N. (2024) · Pure and Applied Geophysics · doi:10.1007/s00024-024-03564-3 · Cited by 8
“To compare model simulations with respect to observations, statistical metrics are commonly employed (Gleckler et al., 2008).”
How well the bias-corrected CMIP6 models assess the extreme precipitation indices over India?
Vinodhkumar, B., Rao, K. K., Sayeekrishnan, H., Osuri, K. K., Athira, U. N., Desamsetti, S., Chowdary, J. S., & Kumar, T. V. L. (2024) · Modeling Earth Systems and Environment · doi:10.1007/s40808-024-02261-3 · Cited by 8
“We calculated the interannual variability skill score (IVS) (Chen et al. 2011; Gleckler et al. (2008) and Santer et al. (2009) of the model simulations relative to the observations.”
Boonman, C. C. F., Hoeks, S., Serra-Diaz, J. M., Guo, W., Enquist, B. J., Maitner, B., Merow, C., & Svenning, J. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2420059122 · Cited by 8
“S3 ) and hence our future biodiversity predictions are greatly dependent on the climate model projections and their uncertainties ( 49 – 51 ) ( SI Appendix , Fig.”
Schwarz, M., Folini, D., Yang, S., & Wild, M. (2019) · Journal of Climate · doi:10.1175/jcli-d-19-0212.1 · Cited by 8
Herrera, W. M., Bedoya, O. F., & Romero, M. R. (2020) · Revista EIA · doi:10.24050/reia.v17i34.1292 · Cited by 8
Preliminary Evaluation of Cloud Fraction Simulations by GAMIL2 Using COSP
Dong, L., Li, L., Wen-Yu, H., Yong, W., & Wang, B. (2012) · Atmospheric and Oceanic Science Letters · doi:10.1080/16742834.2012.11447002 · Cited by 8
“In this study, we assume that the impacts of uncertainties associated with the mismatching time periods of observations and simulations are small for the climatological mean state when compared with the magnitude of current model errors and the different instrument sensitivities and errors of retrieval algorithms (Gle…”
MethodResultValidation of sea ice models using an uncertainty‐based distance metric for multiple model variables
Urrego‐Blanco, J., Hunke, E., Urban, N. M., Jeffery, N., Turner, A. K., Langenbrunner, J., & Booker, J. M. (2017) · Journal of Geophysical Research Oceans · doi:10.1002/2016jc012602 · Cited by 8
“Some of these caveats in validating sea ice models and more generally climate models have already been recognized in the literature [ Gleckler et al., 2008 ; Dukhovskoy et al., 2015].”
Kaspar‐Ott, I., Hertig, E., Kaspar, S., Pollinger, F., Ring, C., Paeth, H., & Jacobeit, J. (2019) · International Journal of Climatology · doi:10.1002/joc.6045 · Cited by 8
“The varying skill of models between different regions and seasons, but nonetheless an obvious overall ranking was also recognized by former studies (Gleckler et al., 2008; Sánchez et al., 2009; Kjellström et al., 2010; Masson and Knutti, 2011; Räisänen and Ylhäisi, 2012).”
BackgroundZhang, W., Zhang, X., Li, W., Hou, N., Wei, Y., Jia, K., Yao, Y., & Cheng, J. (2019) · Remote Sensing · doi:10.3390/rs11151776 · Cited by 8
“To compare the RMSE of each GCM SSR simulation more intuitively, the average RMSEs were normalized following [46].”
MethodEnhancing the Output of Climate Models: A Weather Generator for Climate Change Impact Studies
Croce, P., Formichi, P., & Landi, F. (2021) · Atmosphere · doi:10.3390/atmos12081074 · Cited by 8
“the ability of climate model simulations to reproduce observed climate statistics [53].”
BackgroundPhysically based evaluation of climate models over the Iberian Peninsula
de Cos, C. S., Sánchez-Laulhé, J. M., Alonso, C. J., Sancho, J. M., & Camino, E. R. (2012) · Climate Dynamics · doi:10.1007/s00382-012-1619-2 · Cited by 8
“Although many different metrics of model reliability have been proposed (see, e.g., Gleckler et al. 2008) there is at present little consensus on a particular metric to discriminate ‘‘good’’ and ‘‘bad’’ models.”
BackgroundAssessment of future ENSO changes in a CMIP3 / CMIP5 multi‐model and multi‐index framework
Keupp, L., Pollinger, F., & Paeth, H. (2016) · International Journal of Climatology · doi:10.1002/joc.4928 · Cited by 8
Klassen, H., & Burton, P. J. (2014) · Applied Vegetation Science · doi:10.1111/avsc.12143 · Cited by 8
Regression‐based regionalization for bias correction of temperature and precipitation
Moghim, S., & Bras, R. L. (2019) · International Journal of Climatology · doi:10.1002/joc.6020 · Cited by 8
“The performance of the CCSM to simulate varied climate variables like temperature and precipitation is different (Gleckler et al., 2008; Sillmann et al., 2013; Abbasian et al., 2018).”
BackgroundWang, Y., He, C., & Li, T. (2019) · Climate Dynamics · doi:10.1007/s00382-019-05024-4 · Cited by 8
日本の温暖化影響・適応策評価のための気候・人口・土地利用シナリオ(第2版)
直太, 花., 潔, 高., 肱岡靖明, 博幸, 日., 飯泉, 仁., 有賀, 敏., 啓介, 松., & 信男, 三. (2014) · doi:10.11353/sesj.27.362 · Cited by 8
Svystun, T. (2015) · Lund University Publications Student Papers (Lund University) · Cited by 8
Assessing the uncertainty in climate sensitivity
Klocke, D. (2011) · Max Planck Digital Library · doi:10.17617/2.1217565 · Cited by 8
“This is generally done by comparing long-term statistics from climate model output to long-term statistics from observations (Reichler and Kim 2008; Gleckler et al. 2008).”
BackgroundPredictive analytics for hydrogen–honge oil dual fuel engine using machine learning
Sonthalia, A., Renjith, E. J., Femilda Josephin, J. S., Jerome Stanley, M., Thiyagarajan, S., Varuvel, E. G., Kiani, F., Žvirblis, T., Matijošius, J., Kilikevičius, A., et al. (2026) · International Journal of Hydrogen Energy · doi:10.1016/j.ijhydene.2026.154000 · Cited by 7
Higher-order internal modes of variability imprinted in year-to-year California streamflow changes
Duan, S., Pallotta, G., & Bonfils, C. (2024) · Communications Earth & Environment · doi:10.1038/s43247-024-01594-2 · Cited by 7
“…models to the sixth one than to the reanalysis, given that the multi-model ensemble often aligns more closely with reanalysis than any single model 24 ;and3)thepredictionisconcatenatedforCMIP6modelrealizations whereas an ensemble method is applied to the reanalysis, which may enhanceperformance…”
Shimizu, M. H., Anochi, J. A., & Santos, D. J. (2024) · Climate · doi:10.3390/cli13010004 · Cited by 7
Sardana, D., Kulkarni, S., & Agarwal, A. (2025) · Climate Dynamics · doi:10.1007/s00382-025-07646-3 · Cited by 7
Typhoon Wave Simulation Responses to Various Reanalysis Wind Fields and Computational Domain Sizes
Chen, W.-B. (2022) · Journal of Marine Science and Engineering · doi:10.3390/jmse10101360 · Cited by 7
Lu, X., Zhao, Y., Meng, L., & Zhang, R. (2022) · Atmospheric Research · doi:10.1016/j.atmosres.2022.106564 · Cited by 7
Liu, K., Xu, K., Han, T., Zhu, C., Li, N., Guo, A., & Huang, X. (2024) · Acta Oceanologica Sinica · doi:10.1007/s13131-023-2279-2 · Cited by 7
“Besides RMSE ′ , the interannual variability skill score (IVS, Gleckler et al., 2008; Yang et al., 2021) is also utilized to quantitatively evaluate the interannual variability of the CMIP6 models with respect to observations, which is defined by where STD M and STD O are the standard deviations of…”
Impact of Global Warming on the Western North Pacific Circulation Anomaly during Developing El Niño
Wang, Y., He, C., & Li, T. (2019) · Journal of Climate · doi:10.1175/jcli-d-19-0588.1 · Cited by 7
Olga, C., & Bettolli, M. L. (2011) · InTech eBooks · doi:10.5772/24163 · Cited by 7
“In order to outline and summarize the model-data comparisons, Taylor diagrams were constructed (Gleckler et al., 2008).”
MethodProbabilistic evaluation of competing climate models
Braverman, A., Chatterjee, S., Heyman, M., & Cressie, N. (2017) · Advances in statistical climatology, meteorology and oceanography · doi:10.5194/ascmo-3-93-2017 · Cited by 7
Messori, G., van Wees, D., Pausata, F. S. R., Navarro, J. C. A., Hannachi, A., & Dentener, F. J. (2018) · Tellus B · doi:10.1080/16000889.2018.1468704 · Cited by 7
“Global climate models must take into account both greenhouse gases (GHGs) and aerosols to capture the temperature trend of the last century (e.g. Nazarenko and Menon, 2005; Gleckler et al., 2008; IPCC 2013).”
BackgroundRahman, M. M., & Rob, M. A. (2019) · Earth Systems and Environment · doi:10.1007/s41748-019-00117-w · Cited by 7
Dalmonech, D., Foley, A., Anav, A., Friedlingstein, P., Friend, A. D., Kidston, M., Willeit, M., & Zaehle, S. (2014) · doi:10.5194/bgd-11-2083-2014 · Cited by 7
Adaptation of Water Sensitive Urban Design to Climate Change
Rasheed, A. (2018) · Queensland University of Technology · doi:10.5204/thesis.eprints.122960 · Cited by 7
Anebo, A. L., Woldesenbet, T. A., & Ayele, G. T. (2023) · Environmental Research Communications · doi:10.1088/2515-7620/acf02d · Cited by 6
Why we need lower-performance climate models
O’Loughlin, R. (2024) · Climatic Change · doi:10.1007/s10584-023-03661-7 · Cited by 6
“This practice is understandable for several reasons, including the fact that the unweighted multi-model ensemble mean generally outperforms any single model (Gleckler et al. (2008); see column 1 in Fig.”
InfluentialBalhane, S., Cheruy, F., Driouech, F., Rhaz, K. E., Idelkadi, A., Sima, A., Vignon, É., Drobinski, P., & Chehbouni, A. (2024) · International Journal of Climatology · doi:10.1002/joc.8405 · Cited by 6
“Given that all climate models exhibit biases (Flato et al., 2013; Gleckler et al., 2008) and impact models are commonly very sensitive to the input climatic variables and their own biases (Doblas-Reyes et al., 2021; Dosio, 2016), bias correction is usually applied at some stage in the workflow…”
BackgroundEvaluating NEX-GDDP-CMIP6 performance in complex terrain for forecasting key freezing rain factors
Zou, W., Cao, S., & Tan, W. (2024) · Theoretical and Applied Climatology · doi:10.1007/s00704-024-05159-3 · Cited by 6
“An increased SCC coupled with reduced RMSD and SDEV indicates an improved predictive skill of the climate model in capturing the observed spatial distribution of meteorological parameters(Gleckler et al. 2008; Park et al. 2016).”
Comparison of indicators to evaluate the performance of climate models
Gómez, M. J., Barboza, L. A., Hidalgo, H. G., & Alfaro, E. J. (2024) · International Journal of Climatology · doi:10.1002/joc.8619 · Cited by 6
“Despite the plethora of available statistics to accomplish this evaluation task (Raju & Kumar, 2014), there is not a standard method, because its selection will depend on the measurement purpose (Gleckler et al., 2008), which makes the exploration of new techniques still necessary (Knutti, 2010).”
Changes in human-perceived temperature extremes and associated population exposure across China
Chen, X., Jiang, D., Fan, H., Liu, Y., & Huang, C. (2024) · Atmospheric Research · doi:10.1016/j.atmosres.2024.107896 · Cited by 6
Bénard, G., Gehlen, M., & Vrac, M. (2025) · Earth System Dynamics · doi:10.5194/esd-16-1085-2025 · Cited by 6
Malla, S., & OHGUSHI, K. (2025) · Water Cycle · doi:10.1016/j.watcyc.2025.08.004 · Cited by 6
Bias-corrected climate change projections over the Upper Indus Basin using a multi-model ensemble
Bashir, J., & Romshoo, S. A. (2023) · Environmental Science and Pollution Research · doi:10.1007/s11356-023-26898-2 · Cited by 6
“Several studies (Lambert and Boer 2001; Reichler and Kim 2008; Gleckler et al. 2008; Pincus et al. 2008) have evaluated the performance of climate models for a range of climate variables on a global scale by employing statistical measures to quantify the errors.”
BackgroundEvaluation of Precipitation Simulated by the Atmospheric Global Model MRI-AGCM3.2
Kusunoki, S., Nakaegawa, T., & Mizuta, R. (2024) · Journal of the Meteorological Society of Japan Ser II · doi:10.2151/jmsj.2024-013 · Cited by 6
“This advan - tage of the MME average over individual models is consistent with previous studies such as Lambert and Boer (2001), Gleckler et al. (2008), Reichler and Kim (2008), Kusunoki and Arakawa (2015), Kusunoki (2018a) and Akinsanola et al. (2020).”
Are climate change adaptation and mitigation options eco-efficient?
Jarvis, A., Ramírez-Villegas, J., Tapasco, J., Navarro Racines, C. E., Ca, P., Zapata Caldas, E., & Fisher, M. J. (2012) · CGSPace A Repository of Agricultural Research Outputs (Consultative Group for International Agricultural Research) · Cited by 6
Samadianfard, S., & Panahi, S. (2019) · Journal of watershed management research · doi:10.29252/jwmr.9.18.157 · Cited by 6
Li, C., Xu, X., Du, H., Du, D., Filho, W. L., Wang, J., Bao, G., Ji, X., Yin, S., Bao, Y., et al. (2022) · International Journal of Climate Change Strategies and Management · doi:10.1108/ijccsm-05-2021-0050 · Cited by 6
CMIP6 projected response of the East Asian winter climate to the sea ice‐free Arctic
Song, Z., & Zhou, B. (2022) · International Journal of Climatology · doi:10.1002/joc.7931 · Cited by 6
“Following Gleckler et al. (2008), we take into full consideration the rankings of the three metrics of the Taylor diagram and the IVS score to quantitatively measure the comprehensive performance of models.”
MethodGarry, F. K., & Bernie, D. J. (2023) · International Journal of Climatology · doi:10.1002/joc.8006 · Cited by 6
“This supports wider climate modelling understanding (e.g., Gleckler et al., 2008) that the model mean typically outperforms individual model simulations.”
BackgroundBenestad, R. E., Lussana, C., & Dobler, A. (2024) · Discover Water · doi:10.1007/s43832-024-00063-3 · Cited by 10
Evaluation of ocean wave power utilizing COWCLIP 2.0 datasets: a CMIP5 model assessment
Yadav, A., Kumar, P., Bhaskaran, P. K., Hisaki, Y., & Rajni, R. (2024) · Climate Dynamics · doi:10.1007/s00382-024-07402-z · Cited by 6
“10 for all the seasons (Gleckler et al. 2008; Sillmann et al. 2013).”
What can we learn about climate feedbacks from short-term climate variations?
Koumoutsaris, S. (2013) · Tellus A Dynamic Meteorology and Oceanography · doi:10.3402/tellusa.v65i0.18887 · Cited by 6
“I focus on the CMIP3 multi-model mean, which has been shown to outperform any single model due to compensating systematic errors (Glecker et al., 2008; Pincus et al., 2008; Reichler and Kim, 2008; Knutti et al., 2010).”
BackgroundNosedal-Sanchez, A., Jackson, C. S., & Huerta, G. (2016) · Geoscientific model development · doi:10.5194/gmd-9-2407-2016 · Cited by 6
“Climate scientists are interested in developing new metrics for assessing how well climate simulations reproduce observed climate for purposes of comparing models, driving model development, and evaluating model prediction uncertainties (Gleckler et al., 2008; Reichler and Kim, 2008; Santer et al., 2009; Knutti et al.…”
BackgroundInnovations and limits in methods of forecasting global environmental change
Brook, B. W., & Blomqvist, L. (2016) · Basic and Applied Ecology · doi:10.1016/j.baae.2016.06.002 · Cited by 6
Zouganeli, E., Yazidi, A., Mello, G. B. M. E., & Lind, P. G. (2022) · Communications in computer and information science · doi:10.1007/978-3-031-17030-0 · Cited by 6
Characterizing the uncertainty of climate change projections using hierarchical models
Carvalho, C. M., & Rickershauser, J. (2018) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780198703174.013.20 · Cited by 6
“Of course some models are better than others for a given set of metrics (average temperature over North America, precipitation in the monsoon region, the frequency and intensity of the El Ni˜ no Southern Oscillation phenomenon) but for any set of ”better than average models” a different set of metrics can be found for…”
BackgroundModel consensus projections of US regional hydroclimates under greenhouse warming
Phillips, T. J., Bonfils, C., & Zhang, C. (2018) · Environmental Research Letters · doi:10.1088/1748-9326/aaf03d · Cited by 6
“By averaging a climate variable V across individual model simulations of historical climate, the resulting ensemble-mean á ñV is often found to show closer agreement with observations of V than any single model simulation (e.g. Lambert and Boer 2001, Phillips and Gleckler 2006, Gleckler et al 2008).”
MethodCalibration, Validation, and Confirmation
Frisch, M. (2019) · Simulation foundations, methods and applications · doi:10.1007/978-3-319-70766-2_41 · Cited by 6
Markus, M., Angel, J. R., Wang, K., Byard, G., McConkey, S. A., & Zaloudek, Z. (2017) · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign) · Cited by 6
Flood Risk Assessment, Management and Perceptions in a Changing World
Schlef, K. (2018) · Scholarworks (University of Massachusetts Amherst) · doi:10.7275/11932410.0 · Cited by 6
Aminoto, T., Faqih, A., Perdinan, Koesmaryono, Y., & Dasanto, B. D. (2024) · Agromet · doi:10.29244/j.agromet.38.1.19-35 · Cited by 5
“Additionally, Gleckler et al., (2008) highlighted the importance of assessing the associations between the data provided by each metric to determine their robustness.”
BackgroundJu, Y., Azad, N., Ding, W., & He, H. (2025) · Agriculture · doi:10.3390/agriculture15080826 · Cited by 5
Ge, Y., Zhang, S., Zhou, W. M., Zhao, L., & Fang, Q. (2025) · Journal of Hydrology Regional Studies · doi:10.1016/j.ejrh.2025.102712 · Cited by 5
Duan, S., Chang, L., Li, X., He, R., Li, Y., Dai, X., Luo, X., Sun, G., & Jin, Z. (2025) · Physica A Statistical Mechanics and its Applications · doi:10.1016/j.physa.2025.131226 · Cited by 5
Saxena, A., Shariq, M., Ansari, M. A., & Ansari, S. S. (2026) · Environmental Science and Pollution Research · doi:10.1007/s11356-025-37374-4 · Cited by 5
“Models closer to the reference point (representing observations) exhibit better performance, with radial distance indicating RMSE and azimuthal angle representing correlation (Gleckler et al. 2008; Hirst et al. 2018).”
An EOF-Based Emulator of Means and Covariances of Monthly Climate Fields
Geogdzhayev, G., Souza, A. N., Flierl, G. R., & Ferrari, R. (2026) · Earth System Dynamics · doi:10.5194/esd-17-235-2026 · Cited by 5
“It is also motivated by evidence that the projections of climate models are more reliable at larger scales (Gleckler et al., 2008).”
Beadling, R. L., Swaminathan, R., Beucher, R., Blockley, E., Brands, S., Hassler, B., Hegedūs, D., Hoffman, F. M., Lee, J., Lewis, J., et al. (2026) · Bulletin of the American Meteorological Society · doi:10.1175/bams-d-25-0079.1 · Cited by 5
Projected changes in extreme precipitation over eastern Asia in the 21st century
Wang, C., Cui, D., & Santisirisomboon, J. (2019) · International Journal of Climatology · doi:10.1002/joc.6422 · Cited by 5
“To compare the difference in intensity between the different model results and the observations, the RMSE was standardized according to the relative deviation defined by Gleckler et al. (2008) (Chen et al., 2014).”
MethodAmplification of compound hot-dry extremes and associated population exposure over East Africa
Ayugi, B. O., Onyutha, C., Sian, K. T. C. L. K., Zhu, H., Babousmail, H., & Chung, . (2024) · Climatic Change · doi:10.1007/s10584-024-03802-6 · Cited by 5
Özdemir, S., Yeşilyaprak, C., Aktuğ, B., Öztürk, D., Çoker, D., & Balbay, R. (2018) · Experimental Astronomy · doi:10.1007/s10686-018-9605-2 · Cited by 5
Calado, G. G., & Valverde, M. C. (2020) · Revista Brasileira de Ciências Ambientais · doi:10.5327/z2176-947820200658 · Cited by 5
Cai, F., Yang, S., Wang, Z., & Hua, W. (2021) · International Journal of Climatology · doi:10.1002/joc.7029 · Cited by 5
“As the median is deemed to be a robust estimate of the ensemble results (e.g., Gleckler et al., 2008; Gupta et al., 2013), the multi-model median (MMM) of the IAV is calculated as the ensemble average of the 12 models, and then the contribution of SST changes in multi-model evaluation is estimated…”
MethodBeaumet, J., Déqué, M., Krinner, G., Agosta, C., Alias, A., & Favier, V. (2021) · The Cryosphere · doi:10.5194/tc-15-3615-2021 · Cited by 5
The Asian–Pacific oscillation over the past millennium in PMIP3 and PMIP4
Yang, K., Hua, W., Luo, F., Ming, J., Hu, Q., Wu, X., & Fan, G. (2022) · Quaternary Science Reviews · doi:10.1016/j.quascirev.2022.107918 · Cited by 5
Predictive Study on Extreme Precipitation Trends in Henan and Their Impact on Population Exposure
Wang, Z., Wu, Y., Xi, S., & Sun, X. (2023) · Atmosphere · doi:10.3390/atmos14101484 · Cited by 5
Metadata
- Authors
- Gleckler, P. J., Taylor, K. E., & Doutriaux, C.
- Affiliations
- Lawrence Livermore National Laboratory (US)
- Venue
- Journal of Geophysical Research: Atmospheres
- Publisher
- American Geophysical Union (AGU)
- Published
- 2008-03-20
- Volume
- 113
- Issue
- D6
- ISSN
- 0148-0227 (ISSN-L)
- OpenAlex
- W1996478169
- Cited by
- 1,328
- References
- 31
- Citations by year
- 2026: 64 · 2025: 73 · 2024: 85 · 2023: 88 · 2022: 69 · 2021: 116 · 2020: 103 · 2019: 83 · 2018: 65 · 2017: 76 · 2016: 63 · 2015: 77 · 2014: 65 · 2013: 87 · 2012: 73
- FWCI
- 41.84
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Geophysical Research Atmospheres (publishedVersion)
- Topics
- Physical Sciences › Environmental Science › Global and Planetary Change › Climate variability and models
- Physical Sciences › Earth and Planetary Sciences › Atmospheric Science › Meteorological Phenomena and Simulations
- Physical Sciences › Environmental Science › Global and Planetary Change › Climate Change and Geoengineering
- Keywords
- climate model evaluation, performance indicators, CMIP3, relative error, atmospheric climatology, model ranking, interannual variability, observational uncertainty, spatial scale, 20th century climate simulation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.