Statistical significance of climate sensitivity predictors obtained by data mining
Open accessAbstract
Several recent efforts to estimate Earth's equilibrium climate sensitivity (ECS) focus on identifying quantities in the current climate which are skillful predictors of ECS yet can be constrained by observations. This study automates the search for observable predictors using data from phase 5 of the Coupled Model Intercomparison Project. The primary focus of this paper is assessing statistical significance of the resulting predictive relationships. Failure to account for dependence between models, variables, locations, and seasons is shown to yield misleading results. A new technique for testing the field significance of data‐mined correlations which avoids these problems is presented. Using this new approach, all 41,741 relationships we tested were found to be explainable by chance. This leads us to conclude that data mining is best used to identify potential relationships which are then validated or discarded using physically based hypothesis testing.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Data science models have had limited applicability in scientific problems involving complex physical phenomena, and black-box application of data science has met with limited success in scientific domains.
“Data science models, although successful in a number of commercial domains, have had limited applicability in scientific problems involving complex physical phenomena.”
From Theory-Guided Data Science: A New Paradigm for Scientific Discovery from Data (2017) · cites this paper as [14]
Any high statistics observed in the relationship mining study were found to be explainable by chance.
“Thus, any high statistics observed were found to be explainable by chance”
From A Big Data Guide to Understanding Climate Change: The Case for Theory-Guided Data Science (2014) · cites this paper as [34]
In relationship mining, scientists attempted to automatically identify climate variables that are good predictors of simulated increases in global temperatures if CO2 levels in the atmosphere were to double (equilibrium climate sensitivity).
“In a relationship mining application, 34 scientists attempted to automatically identify the various climate variables that are currently observable and are good predictors of simulated increases in global temperatures if CO 2 levels in the atmosphere were to double (known as equilibrium climate sensitivity).”
From A Big Data Guide to Understanding Climate Change: The Case for Theory-Guided Data Science (2014) · cites this paper as [34]
The researchers set up a search space comprised of 41,741 vectors representing data from various spatiotemporal scales.
“To identify such variables, the researchers set up a search space comprised of 41,741 vectors representing data from various spatiotemporal scales.”
From A Big Data Guide to Understanding Climate Change: The Case for Theory-Guided Data Science (2014) · cites this paper as [34]
The authors showed that although many candidate vectors were identified as skilled predictors using traditional significance testing methods, all of the potential 41,741 relationships were not statistically significant once they accounted for dependence between models, variables, locations, and seasons.
“The authors showed that although many of the candidate vectors were identified as skilled predictors using traditional significance testing methods, all of the potential 41,741 relationships were not statistically significant once they accounted for dependence between models, variables, locations, and seasons.”
From A Big Data Guide to Understanding Climate Change: The Case for Theory-Guided Data Science (2014) · cites this paper as [34]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Radiative forcing and climate response
Hansen, J., Sato, M., & Ruedy, R. (1997) · Journal of Geophysical Research: Atmospheres · doi:10.1029/96jd03436 · Cited by 2,241 · see citing works
Climate model genealogy: Generation CMIP5 and how we got there
Knutti, R., Masson, D., & Gettelman, A. (2013) · Geophysical Research Letters · doi:10.1002/grl.50256 · Cited by 918 · see citing works
Forcing, feedbacks and climate sensitivity in CMIP5 coupled atmosphere‐ocean climate models
Andrews, T., Gregory, J. M., Webb, M. J., & Taylor, K. E. (2012) · Geophysical Research Letters · doi:10.1029/2012gl051607 · Cited by 781 · see citing works
Spread in model climate sensitivity traced to atmospheric convective mixing
Sherwood, S. C., Bony, S., & Dufresne, J.-L. (2014) · Nature · doi:10.1038/nature12829 · Cited by 762 · see citing works
Sensitivity of tropical carbon to climate change constrained by carbon dioxide variability
Cox, P. M., Pearson, D., Booth, B. B. B., Friedlingstein, P., Huntingford, C., JONES, C. D., & Luke, C. M. (2013) · Nature · doi:10.1038/nature11882 · Cited by 742 · see citing works
The Role of Gravity Wave Induced Drag and Diffusion in the Momentum Budget of the Mesosphere
Holton, J. R. (1982) · Journal of the Atmospheric Sciences · doi:10.1175/1520-0469(1982)039 · Cited by 643 · see citing works
On “Field Significance” and the False Discovery Rate
Wilks, D. S. (2006) · Journal of Applied Meteorology and Climatology · doi:10.1175/jam2404.1 · Cited by 545 · see citing works
Knutti, R. (2010) · Climatic Change · doi:10.1007/s10584-010-9800-2 · Cited by 505 · see citing works
The equilibrium sensitivity of the Earth's temperature to radiation changes
Knutti, R., & Hegerl, G. C. (2008) · Nature Geoscience · doi:10.1038/ngeo337 · Cited by 491 · see citing works
Using the current seasonal cycle to constrain snow albedo feedback in future climate change
Hall, A., & Qu, X. (2006) · Geophysical Research Letters · doi:10.1029/2005gl025127 · Cited by 469 · see citing works
Simulation of Present-Day and Twenty-First-Century Energy Budgets of the Southern Oceans
Trenberth, K. E., & Fasullo, J. (2009) · Journal of Climate · doi:10.1175/2009jcli3152.1 · Cited by 456 · see citing works
Robust direct effect of carbon dioxide on tropical circulation and regional precipitation
Bony, S., Bellon, G., Klocke, D., Sherwood, S. C., Fermepin, S., & Denvil, S. (2013) · Nature Geoscience · doi:10.1038/ngeo1799 · Cited by 399 · see citing works
On the Effective Number of Climate Models
Pennell, C., & Reichler, T. J. (2010) · Journal of Climate · doi:10.1175/2010jcli3814.1 · Cited by 237 · see citing works
On the spread of changes in marine low cloud cover in climate model simulations of the 21st century
Qu, X., Hall, A. D., Klein, S. A., & Caldwell, P. (2013) · Climate Dynamics · doi:10.1007/s00382-013-1945-z · Cited by 208 · see citing works
Self-citationConstraining Climate Sensitivity from the Seasonal Cycle in Surface Temperature
Knutti, R., Meehl, G. A., Allen, M. R., & Stainforth, D. A. (2006) · Journal of Climate · doi:10.1175/jcli3865.1 · Cited by 183 · see citing works
A Less Cloudy Future: The Role of Subtropical Subsidence in Climate Sensitivity
Fasullo, J., & Trenberth, K. E. (2012) · Science · doi:10.1126/science.1227465 · Cited by 166 · see citing works
Deriving global climate sensitivity from palaeoclimate reconstructions
Hoffert, M. I., & Covey, C. (1992) · Nature · doi:10.1038/360573a0 · Cited by 164 · see citing works
Klocke, D., Pincus, R. L., & Quaas, J. (2011) · Journal of Climate · doi:10.1175/2011jcli4193.1 · Cited by 142 · see citing works
Effect of climate sensitivity on the response to volcanic forcing
Wigley, T. M. L., Ammann, C. M., Santer, B. D., & Raper, S. C. B. (2005) · Journal of Geophysical Research Atmospheres · doi:10.1029/2004jd005557 · Cited by 137 · see citing works
Self-citationComparison of Measurements of Cloud Droplets and Cloud Nuclei
Twomey, S. A., & Warner, J. (1967) · Journal of the Atmospheric Sciences · doi:10.1175/1520-0469(1967)024 · Cited by 125 · see citing works
Improved constraints on 21st‐century warming derived using 160 years of temperature observations
Gillett, N. P., Arora, V. K., Flato, G. M., et al. (2011) · Geophysical Research Letters · doi:10.1029/2011gl050226 · Cited by 92 · see citing works
The seasonal cycle in coupled ocean-atmosphere general circulation models
Covey, C., Abe‐Ouchi, A., Boer, G. J., Boville, B. A., Ulrich Cubasch, B., Fairhead, L., Flato, G. M., Gordon, H. B., Guilyardi, É., Jiang, X., et al. (2000) · Climate Dynamics · doi:10.1007/s003820000081 · Cited by 68 · see citing works
Constraints on Climate Sensitivity from Radiation Patterns in Climate Models
Huber, M., Mahlstein, I., Wild, M., Fasullo, J., & Knutti, R. (2010) · Journal of Climate · doi:10.1175/2010jcli3403.1 · Cited by 50 · see citing works
Masson, D., & Knutti, R. (2012) · Journal of Climate · doi:10.1175/jcli-d-11-00540.1 · Cited by 43 · see citing works
Volodin, E. M. (2008) · Izvestiya Atmospheric and Oceanic Physics · doi:10.1134/s0001433808030043 · Cited by 43 · see citing works
Field Significance of Regression Patterns
DelSole, T., & Yang, X. (2011) · Journal of Climate · doi:10.1175/2011jcli4105.1 · Cited by 33 · see citing works
Emanuel (1983) · doi:10.1175/1520-0493(1983)111
Flagged first, then by citations
Research works printed but not linked: 2
- Collins, M., et al. (2013), Long-term climate change: Projections, commitments and irreversibility, in Climate Change 2013: The Physical Science Basis. Contribution of Working Group I to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change, edited by T. F. Stocker et al., pp. 1029-1136, Cambridge Univ. Press, Cambridge, U. K., and New York.
- Flato, G., et al. (2013), Evaluation of climate models, in Climate Change 2013: The Physical Science Basis. Contribution of Working Group I to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change, edited by T. F. Stocker et al., pp. 741-866, Cambridge Univ. Press, Cambridge, U. K., and New York.
Cited by
Sorted by citations
Deep learning and process understanding for data-driven Earth system science
Reichstein, M., Camps-Valls, G., Stevens, B., Jung, M., Denzler, J., Carvalhais, N., & Prabhat (2019) · Nature · doi:10.1038/s41586-019-0912-1 · Cited by 5,673 · see citing works
Theory-Guided Data Science: A New Paradigm for Scientific Discovery from Data
Karpatne, A., Atluri, G., Faghmous, J. H., Steinbach, M., Banerjee, A., Ganguly, A., Shekhar, S., Samatova, N., & Kumar, V. (2017) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2017.2720168 · Cited by 1,752 · see citing works
An Assessment of Earth's Climate Sensitivity Using Multiple Lines of Evidence
Sherwood, S. C., Webb, M. J., Annan, J. D., Armour, K. C., Forster, P. M., Hargreaves, J. C., Hegerl, G., Klein, S. A., Marvel, K. D., Rohling, E. J., et al. (2020) · Reviews of Geophysics · doi:10.1029/2019rg000678 · Cited by 1,254 · see citing works
Self-citationMachine learning and artificial intelligence to aid climate change research and preparedness
Huntingford, C., Jeffers, E. S., Bonsall, M. B., Christensen, H. M., Lees, T., & Yang, H. (2019) · Environmental Research Letters · doi:10.1088/1748-9326/ab4e55 · Cited by 469 · see citing works
A Big Data Guide to Understanding Climate Change: The Case for Theory-Guided Data Science
Faghmous, J. H., & Kumar, V. (2014) · Big Data · doi:10.1089/big.2014.0026 · Cited by 223 · see citing works
Emergent constraints on equilibrium climate sensitivity in CMIP5: do they hold for CMIP6?
Schlund, M., Lauer, A., Gentine, P., Sherwood, S. C., & Eyring, V. (2020) · Earth System Dynamics · doi:10.5194/esd-11-1233-2020 · Cited by 178 · see citing works
Theory-Guided Data Science for Climate Change
Faghmous, J. H., Banerjee, A., Shekhar, S., Steinbach, M., Kumar, V., Ganguly, A. R., & Samatova, N. (2014) · Computer · doi:10.1109/mc.2014.335 · Cited by 42 · see citing works
- Authors
- Caldwell, P. M., Bretherton, C. S., Zelinka, M. D., Klein, S. A., Santer, B. D., & Sanderson, B. M.
- Affiliations
- Lawrence Livermore National Laboratory (US)
- University of Washington (US)
- NSF National Center for Atmospheric Research (US)
- Corresponding author
- Caldwell, Peter M.
- ORCID
- Venue
- Geophysical Research Letters
- Publisher
- American Geophysical Union (AGU)
- Published
- 2014-03-07
- Volume
- 41
- Issue
- 5
- Pages
- 1803-1808
- ISSN
- 0094-8276 (ISSN-L)
- 1944-8007
- OpenAlex
- W2130694448
- Cited by
- 153
- References
- 27
- Citations by year
- 2026: 6 · 2025: 13 · 2024: 11 · 2023: 8 · 2022: 22 · 2021: 21 · 2020: 12 · 2019: 17 · 2018: 7 · 2017: 14 · 2016: 6 · 2015: 12 · 2014: 4
- FWCI
- 9.92
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Geophysical Research Letters (publishedVersion)
- UEA Digital Repository (University of East Anglia) (submittedVersion)
- Funders
- National Science Foundation (DE-AC52-07-NA27344)
- U.S. Department of Energy (DEAC5207NA27344, AC52‐07NA27344, DEFC0297ER62402, FC02-97ER62402)
- National Aeronautics and Space Administration (DE-AC52-07NA27344, NNX09AH73G)
- National Center for Atmospheric Research
- Office of Science (DEFC0297ER62402, DE-AC52-07NA27344)
- Biological and Environmental Research (DE-FC02-97ER62402, DE-AC52-07NA27344)
- Lawrence Livermore National Laboratory (DE‐AC5207NA27344, AC52-07NA27344)
- Topics
- Physical Sciences › Environmental Science › Global and Planetary Change › Climate variability and models
- Physical Sciences › Earth and Planetary Sciences › Atmospheric Science › Meteorological Phenomena and Simulations
- Physical Sciences › Environmental Science › Global and Planetary Change › Climate Change and Geoengineering
- Keywords
- equilibrium climate sensitivity, data mining, statistical significance testing, CMIP5, model dependence, observational constraints
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
