Performance metrics for climate models.
The DOI matches its Crossref record; 33 of the 34 references it cites were found in a registry or an index, and 1 are not checked yet.
Abstract
Objective measures of climate model performance are proposed and used to assess simulations of the 20th century, which are available from the Coupled Model Intercomparison Project (CMIP3) archive. The primary focus of this analysis is on the climatology of atmospheric fields. For each variable considered, the models are ranked according to a measure of relative error. Based on an average of the relative errors over all fields considered, some models appear to perform substantially better than others. Forming a single index of model performance, however, can be misleading in that it hides a more complex picture of the relative merits of different models. This is demonstrated by examining individual variables and showing that the relative ranking of models varies considerably from one variable to the next. A remarkable exception to this finding is that the so‐called “mean model” consistently outperforms all other models in nearly every respect. The usefulness, limitations and robustness of the metrics defined here are evaluated 1) by examining whether the information provided by each metric is correlated in any way with the others, and 2) by determining how sensitive the metrics are to such factors as observational uncertainty, spatial scale, and the domain considered (e.g., tropics versus extra‐tropics). An index that gauges the fidelity of model variability on interannual time‐scales is found to be only weakly correlated with an index of the mean climate performance. This illustrates the importance of evaluating a broad spectrum of climate processes and phenomena since accurate simulation of one aspect of climate does not guarantee accurate representation of other aspects. Once a broad suite of metrics has been developed to characterize model performance it may become possible to identify optimal subsets for various applications.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“not even the question of which model performs best during the control simulations can be readily answered but, rather, depends strongly on the skill metric, variable, and region considered (e.g., Gleckler et al. 2008).”
From Risks of Model Weighting in Multimodel Climate Projections (2010) · cites this paper as Gleckler et al. 2008
Citations
Cites
The NCEP/NCAR 40-Year Reanalysis Project
Kalnay, E., Kanamitsu, M., Kistler, R., Collins, W., Deaven, D., Gandin, L., Iredell, M., Saha, S., White, G., Woollen, J., et al. (1996) · Bulletin of the American Meteorological Society · doi:10.1175/1520-0477(1996)077<0437:tnyrp>2.0.co;2 · Cited by 28,360
Summarizing multiple aspects of model performance in a single diagram
Taylor, K. E. (2001) · Journal of Geophysical Research: Atmospheres · doi:10.1029/2000jd900719 · Cited by 8,477
Self-citationAdler, R. F., Huffman, G. J., Chang, A., Ferraro, R., Xie, P.-P., Janowiak, J., Rudolf, B., Schneider, U., Curtis, S., Bolvin, D., et al. (2003) · Journal of Hydrometeorology · doi:10.1175/1525-7541(2003)004<1147:tvgpcp>2.0.co;2 · Cited by 3,808
Advances in Understanding Clouds from ISCCP
Rossow, W. B., & Schiffer, R. A. (1999) · Bulletin of the American Meteorological Society · doi:10.1175/1520-0477(1999)080<2261:aiucfi>2.0.co;2 · Cited by 2,256
Clouds and the Earth's Radiant Energy System (CERES): An Earth Observing System Experiment
Wielicki, B. A., Barkstrom, B. R., Harrison, E. F., Lee, R. B., Louis Smith, G., & Cooper, J. E. (1996) · Bulletin of the American Meteorological Society · doi:10.1175/1520-0477(1996)077<0853:catere>2.0.co;2 · Cited by 1,957
The Global Precipitation Climatology Project (GPCP) Combined Precipitation Dataset
Huffman, G. J., Adler, R. F., Arkin, P., Chang, A., Ferraro, R., Gruber, A., Janowiak, J., McNab, A., Rudolf, B., & Schneider, U. (1997) · Bulletin of the American Meteorological Society · doi:10.1175/1520-0477(1997)078<0005:tgpcpg>2.0.co;2 · Cited by 1,723
Quantification of modelling uncertainties in a large ensemble of climate change simulations.
Murphy, J. M., Sexton, D. M. H., Barnett, D. N., Jones, G. S., Webb, M. J., Collins, M., & Stainforth, D. A. (2004) · Nature · doi:10.1038/nature02771 · Cited by 1,655
AIRS/AMSU/HSB on the aqua mission: design, science objectives, data products, and processing systems
Aumann, H. H., Chahine, M. T., Gautier, C., Goldberg, M. D., Kalnay, E., McMillin, L. M., Revercomb, H., Rosenkranz, P. W., Smith, W. L., Staelin, D. H., et al. (2003) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2002.808356 · Cited by 1,542
Uncertainty in predictions of the climate response to rising levels of greenhouse gases
Stainforth, D. A., Aina, T., Christensen, C., Collins, M., Faull, N., Frame, D. J., Kettleborough, J. A., Knight, S., Martin, A., Murphy, J. M., et al. (2005) · Nature · doi:10.1038/nature03301 · Cited by 1,166
Research works printed but not linked: 12
- 10.1175/1520-0477(1984)065 · doi:10.1175/1520-0477(1984)065
- 10.1175/1520-0477(1999)080 · doi:10.1175/1520-0477(1999)080
- 10.1175/1520-0477(1997)078 · doi:10.1175/1520-0477(1997)078
- 10.1175/1520-0442(1999)012 · doi:10.1175/1520-0442(1999)012
- 10.1175/1520-0477(1996)077 · doi:10.1175/1520-0477(1996)077
- 10.1175/1520-0477(2000)081 · doi:10.1175/1520-0477(2000)081
- 10.1175/1520-0477(1999)080 · doi:10.1175/1520-0477(1999)080
- 10.1175/1520-0442(1993)006 · doi:10.1175/1520-0442(1993)006
- Simmons A. J., 2000, The ERA‐40 project plan, ERA‐40 Project Report Series No. 1
- 10.1175/1520-0442(2004)017 · doi:10.1175/1520-0442(2004)017
- Taylor P.(Ed.) (2000) Final report of the Joint WCRP/SCOR Working Group on Air‐Sea Fluxes WCRP‐112.
- 10.1175/1520‐0477 · doi:10.1175/1520-0477
Cited by
Items 801–900 of 1,001
Evaluation of CMIP6 models for simulations of diurnal temperature range over Africa
Babaousmail, H., Ayugi, B. O., Ojara, M., Ngoma, H., Oduro, C., Mumo, R., & Ongoma, V. (2023) · Journal of African Earth Sciences · doi:10.1016/j.jafrearsci.2023.104944 · Cited by 14
Wang, H., & Asefa, T. (2024) · International Journal of Climatology · doi:10.1002/joc.8479 · Cited by 14
“However, it excludes evaluating some aspects of climate, such as the frequency and intensity of extratropical cyclones and the frequency of extreme events that would only be evident in the sub-daily data (Gleckler et al., 2008).”
BackgroundClimate Simulations: Uncertain Projections for an Uncertain World
Hillerbrand, R. (2014) · Journal for General Philosophy of Science · doi:10.1007/s10838-014-9266-4 · Cited by 14
Keupp, L., Hertig, E., Kaspar‐Ott, I., Pollinger, F., Ring, C., Paeth, H., & Jacobeit, J. (2019) · Theoretical and Applied Climatology · doi:10.1007/s00704-019-02851-7 · Cited by 14
Local effects of climate change on row crop production and irrigation adoption
Lambert, L. H., English, B. C., Clark, C. C., Lambert, D. M., Menard, R. J., Hellwinckel, C., Smith, S. A., & Papanicolaou, A. N. (2021) · Climate Risk Management · doi:10.1016/j.crm.2021.100293 · Cited by 14
Chervenkov, H., & Slavov, K. (2022) · Atmosphere · doi:10.3390/atmos13040581 · Cited by 14
“According to the number of the GCMs, the NEX-GDDP ensemble is also significantly bigger than the GCMs-ensembles in our previous studies [10,11,31].”
BackgroundInfluentialSingh, D., Choi, Y., Dimri, R., Ghahremanloo, M., & Pouyaei, A. (2023) · Asia-Pacific Journal of Atmospheric Sciences · doi:10.1007/s13143-023-00330-8 · Cited by 14
“…systematic difference between model simulation and reference observation for ease of comprehension and is employed in numerous climate studies such as Gleckler et al. 2008 and Maraun et al. 2017 to understand deviations in model simulations compared to observations for different aspects of…”
MethodPulatov, B., Hall, K., Linderson, M., & Jönsson, A. M. (2014) · Climate Research · doi:10.3354/cr01259 · Cited by 14
“The basic idea is that some models produce overestimates while others produce underestimates, and all models are ultimately wrong (Gleckler et al. 2008).”
BackgroundThe Impact of Bias Correction and Model Selection on Passing Temperature Thresholds
Gohar, L., Lowe, J. A., & Bernie, D. J. (2017) · Journal of Geophysical Research Atmospheres · doi:10.1002/2017jd026797 · Cited by 14
“The second metric is an update of Gleckler et al. (2008) for the CMIP5 models (Flato et al., 2013).”
BackgroundMethodUse of Bayesian inference and data to improve simulations of multi-physics climate phenomena
Jackson, C. S. (2009) · Journal of Physics Conference Series · doi:10.1088/1742-6596/180/1/012029 · Cited by 14
Razavi-Termeh, S. V., Sadeghi‐Niaraki, A., Ali, F., & Choi, S.-M. (2025) · Journal of Environmental Management · doi:10.1016/j.jenvman.2025.124238 · Cited by 13
Substantial Contraction of Dense Shelf Water in the Ross Sea Under Future Climate Scenarios
Xie, C., Zhang, Z., Chen, Y., & Wang, C. (2025) · Geophysical Research Letters · doi:10.1029/2024gl112581 · Cited by 13
Developing Guidelines for working with Multi-Model Ensembles in CMIP
Katzenberger, A., Pérez‐Carrasquilla, J. S., Gemmell, K., Galytska, E., Leclerc, C., Punya, P., Roy, I., Varuolo‐Clarke, A. M., Tošić, M., & Črnivec, N. (2026) · Earth System Dynamics · doi:10.5194/esd-17-495-2026 · Cited by 13
“Model ranking has been found to be sensitive to this choice (Gleckler et al., 2008).”
Bhowmik, R. D., & Sankarasubramanian, A. (2020) · International Journal of Climatology · doi:10.1002/joc.6870 · Cited by 13
Hollaway, M. J., Henrys, P. A., Killick, R., Leeson, A. A., & Watkins, J. W. (2021) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2021.104993 · Cited by 13
Barimalala, R., James, R., Munday, C., & Reason, C. J. C. (2022) · Climate Dynamics · doi:10.1007/s00382-022-06480-1 · Cited by 13
“These methods are routinely used by modeling centers to assess new generations of models (e.g. Gleckler et al. 2008).”
Regional ensemble of CMIP6 global climate models for Sakha (Yakutia) Republic, Northern Eurasia
Тананаев, Н. И. (2024) · Polar Science · doi:10.1016/j.polar.2024.101066 · Cited by 13
Mahowald, N. M., Li, L., Vira, J., Prank, M., Hamilton, D. S., Matsui, H., Miller, R. L., Lu, L., Akyüz, E., Meidan, D., et al. (2025) · Atmospheric chemistry and physics · doi:10.5194/acp-25-4665-2025 · Cited by 14
Sapriza‐Azuri, G., Jódar, J., Carrera, J., & Gupta, H. V. (2013) · Mathematical Geosciences · doi:10.1007/s11004-013-9467-0 · Cited by 13
“…simulations of rainfall which are typically poor, SRGP takes advantage of the fact that GCMs are able to generate good simulations of other hydrometeorological variables (Gleckler et al. 2008) and, in particular, of atmospheric circulation patterns (ACP) at the synoptic scale (Randall et al. 2007).”
BackgroundPerformance of ENSEMBLES regional climate models over Central Europe using various metrics
Holtanová, E., Mikšovský, J., Kalvová, J., Pišoft, P., & Motl, M. (2011) · Theoretical and Applied Climatology · doi:10.1007/s00704-011-0542-5 · Cited by 13
“The last metric considered in this paper is the root mean square error and the relative error described by Gleckler et al. (2008).”
ResultQuantitative assessment of Southern Hemisphere ozone in chemistry-climate model simulations
Karpechko, A. Y., Gillett, N. P., Haßler, B., Rosenlof, K. H., & Rozanov, E. (2010) · Atmospheric chemistry and physics · doi:10.5194/acp-10-1385-2010 · Cited by 13
“3 Method To assess model performance we calculate a metric similar to that used by Reichler and Kim (2008) and Gleckler et al. (2008). First we calculate normalized root mean square (RMS) differencesejkl between thej -th model andk-th reference observations for the l-th diagnostic e2 jkl = 1 W ∑ i ∑ m (wim(ximjl −yimk…”
MethodInfluentialUnderstanding variability of the Southern Ocean overturning circulation in CORE-II models
Downes, S. M., Spence, P., & Hogg, A. M. (2018) · Ocean Modelling · doi:10.1016/j.ocemod.2018.01.005 · Cited by 13
“Whilst the mean across multi-model studies is often in good agreement with observations in variables such as winds, surface temperature and radiative fluxes, and precipitation (Gleckler et al., 2008), the intermodel spread is vast.”
Calle, L., Poulter, B., & Patra, P. K. (2019) · Atmospheric measurement techniques · doi:10.5194/amt-12-2611-2019 · Cited by 13
The blessing of dimensionality for the analysis of climate data
Christiansen, B. (2021) · Nonlinear processes in geophysics · doi:10.5194/npg-28-409-2021 · Cited by 13
“The annual or monthly climatologies are obvious measures for comparing models or for validating the models against observations (Gleckler et al., 2008).”
BackgroundLópez-Novoa, U., Sáenz, J., Mendiburu, A., Miguel-Alonso, J., Errasti, I., Esnaola, G., Ezcurra, A., & Ibarra‐Berastegi, G. (2014) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2014.09.019 · Cited by 13
Foley, A., Dalmonech, D., Friend, A. D., Aires, F., Archibald, A. T., Bartlein, P. J., Bopp, L., Chappellaz, J., Cox, P. M., Edwards, N. R., et al. (2013) · Biogeosciences · doi:10.5194/bg-10-8305-2013 · Cited by 13
“Gleckler et al. (2008) use Taylor diagrams to compare the performance of models in terms of both the magnitude and phase of the seasonal cycle. Reichler and Kim (2008) normalise model error variance on a grid-point basis to come up with a composite score and measure progress in model skill between generations of IPCC…”
InfluentialCoupled Climate Model Simulation of Tropical–Extratropical Cloud Bands over Southern Africa
James, R., Hart, N. C. G., Munday, C., Reason, C. J. C., & Washington, R. (2020) · Journal of Climate · doi:10.1175/jcli-d-19-0731.1 · Cited by 13
Bui, M. T., Lu, J., & Nie, L. S. (2021) · Journal of Water and Climate Change · doi:10.2166/wcc.2021.346 · Cited by 13
Radić, V., & Hock, R. (2013) · Space sciences series of ISSI · doi:10.1007/978-94-017-8789-5_15 · Cited by 13
Applicability of CMIP6 models for building climate projections for northern Europe
Ruosteenoja, K. (2021) · doi:10.35614/isbn.9789523361416 · Cited by 13
Observation and Projection of Marine Heatwaves in the Caribbean Sea from CMIP6 Models
Usta, D. F. B., Parra, R. R. T., Rodríguez‐López, L., Castillo-Alvarez, M., & Bourrel, L. (2024) · Remote Sensing · doi:10.3390/rs16132357 · Cited by 12
Dollan, I. J., Maggioni, V., Johnston, J., Coelho, G. D. A., & Kinter, J. L. (2022) · Frontiers in Climate · doi:10.3389/fclim.2022.954892 · Cited by 12
“It has been found that multi-model mean or ensemble mean can represent the climatology of atmospheric fields, given the uncertainties among climate models (Gleckler et al., 2008).”
BackgroundFeasibility of Six Metaheuristic Solutions for Estimating Induction Motor Reactance
Gör, H. (2024) · Mathematics · doi:10.3390/math12030483 · Cited by 12
Multi-model estimate of Antarctic ice-shelf basal mass budget and ocean drivers
Galton-Fenzi, B. K., Porter-Smith, R., Cook, S., Cougnon, E. A., Gwyther, D. E., Huneke, W. G. C., Rosevear, M. G., Asay‐Davis, X., Dias, F. B., Dinniman, M. S., et al. (2025) · The Cryosphere · doi:10.5194/tc-19-6507-2025 · Cited by 12
“The utilisation of a multi-model mean has emerged as a promising strategy to mitigate these limitations and enhance predictive accuracy (e.g., Tebaldi and Knutti, 2007; Gleckler et al., 2008; Smith et al., 2009; Knutti and Sedlá ˇ cek, 2013).”
Statistical uncertainty of eddy covariance CO2 fluxes inferred using a residual bootstrap approach
Wang, H.-J., Riley, W. J., & Collins, W. D. (2015) · Agricultural and Forest Meteorology · doi:10.1016/j.agrformet.2015.03.011 · Cited by 12
“Meteorol., 78 (1–2) (1996), pp. 83-105 Article Download PDF View Record in Scopus Gleckler et al., 2008 P.J. Gleckler, K.E. Taylor, C. DoutriauxPerformance metrics for climate models J. Geophys.”
BackgroundZhou, B., & Xu, Y. (2016) · Atmospheric and Oceanic Science Letters · doi:10.1080/16742834.2017.1260427 · Cited by 12
“The MME method was applied because the consensus is that it provides more robust estimates (e.g. Gleckler, Taylor, and Doutriaux 2008; Gupta et al. 2013; Zhou et al. 2014; Jiang, Tian, and Lang 2016).”
MethodSelection of Global Climate Models
Raju, K. S., & Kumar, D. N. (2017) · Springer climate · doi:10.1007/978-981-10-6110-3_2 · Cited by 12
Dong, S., & Sun, Y. (2018) · Climate Research · doi:10.3354/cr01534 · Cited by 12
“Similar to Gleckler et al. (2008), normalizing the RMSE results is an effective way to compare a model’s performance with that of another model.”
BackgroundMethodResultInfluentialGnatiuk, N. V., Radchenko, I., Davy, R., Morozov, E. G., & Bobylev, L. (2020) · Biogeosciences · doi:10.5194/bg-17-1199-2020 · Cited by 12
“Basic statistical measures were calculated, such as the root-mean-square deviation (RMSD), the correlation coefficient (r), and the standard deviation (SD; Fu et al., 2013; Gleckler et al., 2008; Kumar et al., 2015; Ruan et al., 2019).”
BackgroundMethodInfluentialPropagation of Error and the Reliability of Global Air Temperature Projections
Frank, P. (2019) · Frontiers in Earth Science · doi:10.3389/feart.2019.00223 · Cited by 12
“, 2004), or as Taylor diagrams exhibiting the spread of model realizations around observations (Covey et al., 2003; Gleckler et al., 2008; Jiang et al., 2012).”
BackgroundDetection of major weather patterns reduces number of simulations in climate impact studies
Ababaei, B., & Ullah, N. (2020) · Journal of Agronomy and Crop Science · doi:10.1111/jac.12388 · Cited by 12
Application of Extreme Learning Machine Algorithm for Drought Forecasting
Raza, M. A., Almazah, M. M. A., Ali, Z., Hussain, I., & Alduais, F. S. (2022) · Complexity · doi:10.1155/2022/4998200 · Cited by 12
“Numerous accuracy measurement criteria were developed, but each tool has inherited pros and cons, and none of the metrics is universally accepted and can be used as a threshold [84].”
Climate Models and Their Projections of Future Changes
Tebaldi, C., & Knutti, R. (2009) · Advances in global change research · doi:10.1007/978-90-481-2953-9_3 · Cited by 12
A Modelling Perspective of Future Climate Change
Abiodun, B. J., & Adedoyin, A. (2015) · Elsevier eBooks · doi:10.1016/b978-0-444-63524-2.00023-3 · Cited by 12
Evaluation of simulated recent climate change in Australia
Bhend, J., & Whetton, P. H. (2015) · Australian Meteorological and Oceanographic Journal · doi:10.1071/es15003 · Cited by 12
Myers, T. A., & Mechoso, C. R. (2020) · Geophysical Research Letters · doi:10.1029/2019gl086321 · Cited by 12
“Similarly, the average across multiple climate models also tends to have smaller biases than those found in any one model (Gleckler et al., 2008).”
BackgroundMüller, C., Elliott, J. W., & Elbehri, A. (2015) · Publication Database PIK (Potsdam Institute for Climate Impact Research (PIK)) · Cited by 12
Developing climate change scenarios for Tamil Nadu, India using MAGICC/SCENGEN
Jeganathan, A., & Ramachandran, A. (2013) · Theoretical and Applied Climatology · doi:10.1007/s00704-013-0871-7 · Cited by 12
“Indeed, the skill of a multi-model ensemble average consistently outperforms any individual model result (Gleckler et al. 2008; Pierce et al. 2009).”
BackgroundEvaluating the Appropriateness of Downscaled Climate Information for Projecting Risks of Salmonella
Guentchev, G. S., Rood, R. B., Ammann, C. M., Barsugli, J. J., Ebi, K. L., Berrocal, V. J., O’Neill, M. S., Gronlund, C. J., Vigh, J. L., Koziol, B., et al. (2016) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph13030267 · Cited by 12
“, the Tropics, or the extra-tropical zones in the Northern and Southern Hemispheres) and/or global scales [45,46].”
BackgroundKulkarni, A., Raju, P. V. S., Ashrit, R., Sagalgile, A., Singh, B. B., & Prasad, J. (2024) · Quarterly Journal of the Royal Meteorological Society · doi:10.1002/qj.4757 · Cited by 11
“Numerous studies reported on the mean climate (Gleckler et al. , 2008) and climate variability, and concluded that an MME performs better than the individual models (Pierce et al. , 2009).”
Addisuu, A. A., Tsidu, G. M., & Basupi, L. V. (2025) · Climate · doi:10.3390/cli13050093 · Cited by 11
“…of structural and parametric uncertainties [83,84], their relative performance can fluctuate based on (1) the characteristics of the reference dataset, (2) how bias correction modifies inter-model variability, and (3) the nonlinear interactions between model spread and ensemble averaging [85].”
Brajkovic, J., Fettweis, X., Noel, B. P. Y., de Vyver, H. V., Ghilain, N., Archambeau, P., Pirotton, M., & Doutreloup, S. (2025) · Journal of Hydrology Regional Studies · doi:10.1016/j.ejrh.2025.102399 · Cited by 11
A likelihood‐based comparison of temporal models for physical processes
Braverman, A., Cressie, N., & Teixeira, J. (2011) · Statistical Analysis and Data Mining The ASA Data Science Journal · doi:10.1002/sam.10113 · Cited by 11
“from a statistical perspective [3], or are based only on first”
BackgroundGeorge, J., & Athira, P. (2022) · Theoretical and Applied Climatology · doi:10.1007/s00704-022-04197-z · Cited by 11
Luo, C., Shang, S., Xie, Y., He, Z., Wei, G., Zhang, F., & Wang, L. (2023) · Water · doi:10.3390/w15081526 · Cited by 11
CMIP6 model evaluation for sea surface height responses to ENSO
Sardana, D., Kumar, P., & Rajni, R. (2023) · Climate Dynamics · doi:10.1007/s00382-023-06997-z · Cited by 11
Singh, S., Mall, R. K., Singh, P. K., Bhatla, R., & Chaubey, P. K. (2024) · Urban Climate · doi:10.1016/j.uclim.2024.101936 · Cited by 11
“For the validation of bias corrected model data, Taylor diagram constituting Correlation Coefficient (CC), Root Mean Square Deviation (RMSE) and Standard Deviations (SD) are calculated with IMD data as reference (Mann, 1945; Taylor, 2001; Gleckler et al., 2008).”
Land use change impacts on climate extremes over the historical period
Zhang, M., Gao, Y., Wang, A., Zhang, L., & Yang, K. (2024) · Climate Dynamics · doi:10.1007/s00382-024-07375-z · Cited by 11
Loikith, P. C., Waliser, D. E., Kim, J., & Ferraro, R. D. (2017) · Climate Dynamics · doi:10.1007/s00382-017-3837-0 · Cited by 11
Evaluation of GCMs historical simulations of monthly and seasonal climatology over Bolivia
Abadi, A. M., Oglesby, R. J., Rowe, C. M., & Mawalagedara, R. (2017) · Climate Dynamics · doi:10.1007/s00382-017-3952-y · Cited by 11
“2015; Grau and Aide 2008).”
Shaheen, N., jahandad, A., Goheer, M. A., & Ahmad, Q.-U.-A. (2020) · APN Science Bulletin · doi:10.30852/sb.2020.1221 · Cited by 11
Aguirre, F., Squeo, F. A., López, D., Grego, R. D., Buma, B. J., Carvajal, D. E., Jaña, R., Casassa, G., & Rozzi, R. (2021) · Anales del Instituto de la Patagonia · doi:10.22352/aip202149012 · Cited by 11
“…evaluación entre los datos meteorológicos de los modelos y las estaciones de monitoreo se utilizan diagramas de Taylor que incorporan, de forma gráfica, la desviación estándar, correlación de Pearson y el error cuadrático medio (RMSE en su sigla en inglés) (Flato et al. 2014; Gleckler et al. 2008).”
BackgroundGăinuşă-Bogdan, A., Braconnot, P., & Servonnat, J. (2015) · Journal of Geophysical Research Atmospheres · doi:10.1002/2014jd022985 · Cited by 11
Hwang, S. (2014) · Journal of Korea Water Resources Association · doi:10.3741/jkwra.2014.47.11.1039 · Cited by 11
“또한기후요소, 대상지역, 평가방법, 평 가지표에 대한모델의 성능은달리 평가될수있으므로 기 후변화연구를위한합리적인모델선정및적용을위해서 는다각적이고종합적인평가가시도되어야한다(Gleckler et al., 2008; Hwang et al., 2014).”
Contrasting Climate Ensembles: A Model-based Visualization Approach for Analyzing Extreme Events
Sisneros, R., Huang, J. P., Ostrouchov, G., Ahern, S. D., & Semeraro, B. D. (2013) · Procedia Computer Science · doi:10.1016/j.procs.2013.05.406 · Cited by 11
Curry, C. L., Tencer, B., Whan, K., Weaver, A. J., Giguère, M., & Wiebe, E. C. (2016) · ATMOSPHERE-OCEAN · doi:10.1080/07055900.2016.1158146 · Cited by 11
Decadal Climate Variability, Predictability and Prediction: Opportunities and Challenges
Hurrell, J. W., Latif, M., Visbeck, M. H., Delworth, T. L., Danabasoglu, G., Dommenget, D., Drange, H., Drinkwater, K. F., Griffies, S. M., Hazeleger, W., et al. (2010) · doi:10.5270/oceanobs09.pp.23 · Cited by 11
“Furthermore, the attraction to use metrics to select the “best” model for a climate application is problematic [113].”
BackgroundAssessing GCM Convergence for India Using the Variable Convergence Score
Ojha, R., Kumar, D. N., Sharma, A., & Mehrotra, R. (2013) · Journal of Hydrologic Engineering · doi:10.1061/(asce)he.1943-5584.0000888 · Cited by 11
“However, the simulations from different climate models vary at a local or regional scale and are highly uncertain (Molteni et al. 1996; Xu 1999; Kleeman 2002; Gleckler et al. 2008).”
BackgroundMethodInfluentialPark, C., Jeong, S.-J., Ho, C.-H., & Kim, J. (2015) · Journal of Climate · doi:10.1175/jcli-d-13-00753.1 · Cited by 11
International Land Model Benchmarking (ILAMB) Workshop Report, Technical Report DOE/SC-0186
Pacific Northwest National Lab. (PNNL), R., Hoffman, F. M., USDOE, Koven, C. D., Keppel‐Aleks, G., Lawrence, D. M., Riley, W. J., Randerson, J. T., Ahlström, A., Abramowitz, G., et al. (2016) · OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) · doi:10.2172/1413397 · Cited by 11
Self-citationShi, M., Xue, Q., Wang, Y., Liu, X., Wang, L., & Li, H. (2025) · Landscape and Urban Planning · doi:10.1016/j.landurbplan.2025.105468 · Cited by 10
Kimura, Y., Hirabayashi, Y., & Yamazaki, D. (2025) · Scientific Reports · doi:10.1038/s41598-025-16327-4 · Cited by 10
The GFDL‐CM4X Climate Model Hierarchy, Part I: Model Description and Thermal Properties
Griffies, S. M., Adcroft, A., Beadling, R. L., Bushuk, M., Chang, C., Drake, H. F., Dussin, R., Hallberg, R., Hurlin, W. J., Khatri, H., et al. (2025) · Journal of Advances in Modeling Earth Systems · doi:10.1029/2024ms004861 · Cited by 10
Kazora, J., Zhu, W., Oo, K. T., Akimana, D., Paul, K. J., Ayugi, B. O., & Gahigi, A. (2025) · Discover Atmosphere · doi:10.1007/s44292-025-00066-2 · Cited by 10
“This metric is critical for assessing whether models capture the amplitude and phase of observed variability, which is essential for reliable future projections [17].”
Cinquegrana, D., Zollo, A. L., Montesarchio, M., & Bucchignani, E. (2023) · Atmosphere · doi:10.3390/atmos14050788 · Cited by 10
Linking error measures to model questions
Jacobs, B., Tobi, H., & Hengeveld, G. (2023) · Ecological Modelling · doi:10.1016/j.ecolmodel.2023.110562 · Cited by 10
Wang, Y., He, C., Li, T., Zhang, C., & Gu, X. (2023) · Climate Dynamics · doi:10.1007/s00382-023-07013-0 · Cited by 10
“The multi-model median (MMM) is used to evaluate the climate change, for the median is more robust to the outliers (Gleckler et al. 2008).”
Tang, B., Hu, W., Duan, A., Liu, Y., Bao, W., Xin, Y., & Yang, X. (2024) · Advances in Atmospheric Sciences · doi:10.1007/s00376-023-3158-7 · Cited by 10
“The multimodel ensemble median (MMM) rather than the multimodel ensemble mean is used to guard against CMIP6 models with exceptionally large errors (outliers) unduly affecting our results (Gleckler et al., 2008).”
Mushagalusa, C. A., Fandohan, A. B., & Kakaï, R. G. (2024) · Modeling Earth Systems and Environment · doi:10.1007/s40808-024-02055-7 · Cited by 10
“Case 6: Predictive accuracy and bias of the various modelling approaches are similar, with the RGLM exhibiting a sensitivity to extreme values in terms of bias and RMSE.”
InfluentialMäättänen, A., Lameille, T., Klöck, C., Boucher, O., & Ravetta, F. (2024) · Oxford Open Climate Change · doi:10.1093/oxfclm/kgae007 · Cited by 10
Marianetti, G., Rivera, J. A., & Bettolli, M. L. (2024) · Theoretical and Applied Climatology · doi:10.1007/s00704-024-05123-1 · Cited by 10
“All the models exhibited positive trends except for the IPSL-CM6A-LR simulations, with 28 mod - els showing significant increasing trends in temperature, (2011 ), the models that best represent the interannual vari - ability of the reference data have IVS under 0.5.”
InfluentialRäisänen, P., Luomaranta, A., & Jylhä, K. (2025) · International Journal of Climatology · doi:10.1002/joc.8795 · Cited by 10
“In that report, the models were rated into three quality classes primarily based on the Model Climate Performance Index of Gleckler et al. (2008), which considers both the seasonal and spatial variations of the climatological means.”
Arowolo, A. V., Odunmorayo, M. T., Okeyode, I. A., Raji, I. A., Ibukun, B., & Tella, I. A. (2025) · Earth Systems and Environment · doi:10.1007/s41748-025-00622-1 · Cited by 10
“…evaluated for reduced spread and improved correlation compared to individual models, indicating the added value of MME in minimizing model biases (Gleckler et al. 2008; Weigel et al. 2010) All datasets were regridded to a common spatial resolution (25 km) using bilinear interpolation to ensure…”
Selecting Climate Change Scenarios Reflecting Uncertainties
Lee, J.-K., & Kim, Y.-O. (2012) · Atmosphere · doi:10.14191/atmos.2012.22.2.149 · Cited by 10
“GCM 시나리오 정확성 평가와 선정에 관련된 연구를 살펴보면, Murphy (1996), Wilks (2005), Reichler et al. (2008), Gleckler et al. (2008), Winter et al. (2009)와 Chiew et al. (2009)은 GCM 시나리오와 관측값의 평균에 대하여 (Root) Mean Square Error에 기반을 둔 지수를 개발 하거나 직접 RMSE를 이용하여 GCM 시나리오의 모 의능력을 평가하였으며, Douglass et al. (1999)은 모 형과 관측값의 평균차에…”
He, C., Wang, Y., & Li, T. (2019) · Journal of Climate · doi:10.1175/jcli-d-19-0165.1 · Cited by 10
Multimodel ensemble analysis with neural network Gaussian processes
Harris, T. A. L. V., Li, B., & Sriver, R. L. (2023) · The Annals of Applied Statistics · doi:10.1214/23-aoas1768 · Cited by 10
Sussman, F. G., SAHA, B., Bierwagen, B., Weaver, C., Cooper, W., Morefield, P. E., & Thomas, J. V. (2014) · Climate Change Economics · doi:10.1142/s2010007814500092 · Cited by 10
Analysis and Prediction of Time Series Variations of Rainfall in North-Eastern Bangladesh
Nury, A. H. (2014) · British Journal of Applied Science & Technology · doi:10.9734/bjast/2014/7722 · Cited by 10
“Weather parameter such as precipitation could be practically useful in making decisions, risk management and optimum usage of water resources [1,2,3,4].”
BackgroundA height dependent evaluation of wind and temperature over Europe in the CMIP5 Earth System Models
Devis, A., van Lipzig, N. P. M., & Demuzere, M. (2014) · Climate Research · doi:10.3354/cr01242 · Cited by 10
A spatiotemporal analysis of Indian warming target using CORDEX-SA experiment data
Prajapat, D. K., Lodha, J., & Choudhary, M. (2019) · Theoretical and Applied Climatology · doi:10.1007/s00704-019-02978-7 · Cited by 10
Kakouei, K., Domisch, S., Kiesel, J., Kail, J., & Jähnig, S. C. (2020) · Scientific Reports · doi:10.1038/s41598-020-59107-y · Cited by 10
Chaudhary, S., & Dhanya, C. T. (2020) · Atmospheric Research · doi:10.1016/j.atmosres.2020.105119 · Cited by 10
Assessing and predicting soil carbon density in China using CMIP5 earth system models
Qiu, L., Yu, M., Wu, Y., Yao, Y., Wang, Z., Shi, Z., & Guan, Y. (2021) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2021.149247 · Cited by 10
Addressing Climate Change in Long-Term Water Resources Planning and Management
Brekke, L. D., White, K. D., Olsen, J., & Williams, D. G. (2011) · Cited by 10
The Role of Climate ServicesClimate services in AdaptingAdaptation to Climate Variability and Change
Bowyer, P. K., Brasseur, G. P., & Jacob, D. (2014) · Springer eBooks · doi:10.1007/978-3-642-40455-9_29-1 · Cited by 10
“This issue is related to model skill, and the development of model quality metrics, and is an increasingly important research area (Glecklers et al. 2008; Knutti et al. 2010).”
BackgroundErfanian, A., Wang, G., Fomenko, L., & Yu, M. (2017) · Geophysical Research Letters · doi:10.1002/2017gl073053 · Cited by 10
“Properly evaluating the models and designing optimal ensembles pose yet another challenge [Gleckler et al., 2008; Knutti et al., 2010; Schaller et al., 2011; Pennell and Reichler, 2011], as a model may perform well for one variable but not another.”
BackgroundThe implications of climate policy for avoided impacts on water scarcity
Gosling, S. N., Arnell, N. W., & Lowe, J. A. (2011) · Procedia Environmental Sciences · doi:10.1016/j.proenv.2011.05.012 · Cited by 10
“This is a reasonable assumption at present because of the challenge in defining appropriate measures of relative performance [14], but this does require further investigation.”
BackgroundManaging climate change scenarios for societal impact studies
Carbone, G. J. (2014) · Physical Geography · doi:10.1080/02723646.2013.869714 · Cited by 10
“Multi-model ensemble averages often outperform individual models for the historical period (Gleckler et al., 2008; Pierce, Barnett, Santer, & Gleckler, 2009; Reichler & Kim, 2008), as they overcome the limitations of any single model.”
MethodUncertainty Quantification in Multi-Model Ensembles
Sanderson, B. M. (2018) · Oxford University Press eBooks · doi:10.1093/acrefore/9780190228620.013.707 · Cited by 10
Development of projected depth-duration frequency curves (2050–89) for south Florida
Irizarry-Ortiz, M. M., Stamm, J. F., Maran, C., & Obeysekera, J. T. B. (2022) · Scientific investigations report · doi:10.3133/sir20225093 · Cited by 10
Metadata
- Authors
- Gleckler, P. J., Taylor, K. E., & Doutriaux, C.
- Affiliations
- Lawrence Livermore National Laboratory (US)
- Venue
- Journal of Geophysical Research: Atmospheres
- Publisher
- American Geophysical Union (AGU)
- Published
- 2008-03-20
- Volume
- 113
- Issue
- D6
- ISSN
- 0148-0227 (ISSN-L)
- OpenAlex
- W1996478169
- Cited by
- 1,328
- References
- 31
- Citations by year
- 2026: 64 · 2025: 73 · 2024: 85 · 2023: 88 · 2022: 69 · 2021: 116 · 2020: 103 · 2019: 83 · 2018: 65 · 2017: 76 · 2016: 63 · 2015: 77 · 2014: 65 · 2013: 87 · 2012: 73
- FWCI
- 41.84
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Geophysical Research Atmospheres (publishedVersion)
- Topics
- Physical Sciences › Environmental Science › Global and Planetary Change › Climate variability and models
- Physical Sciences › Earth and Planetary Sciences › Atmospheric Science › Meteorological Phenomena and Simulations
- Physical Sciences › Environmental Science › Global and Planetary Change › Climate Change and Geoengineering
- Keywords
- climate model evaluation, performance indicators, CMIP3, relative error, atmospheric climatology, model ranking, interannual variability, observational uncertainty, spatial scale, 20th century climate simulation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.