The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
What do we mean by sensitivity analysis? The need for comprehensive characterization of “global” sensitivity in Earth and Environmental systems models
Abstract
Sensitivity analysis is an essential paradigm in Earth and Environmental Systems modeling. However, the term “sensitivity” has a clear definition, based in partial derivatives, only when specified locally around a particular point (e.g., optimal solution) in the problem space. Accordingly, no unique definition exists for “global sensitivity” across the problem space, when considering one or more model responses to different factors such as model parameters or forcings. A variety of approaches have been proposed for global sensitivity analysis, based on different philosophies and theories, and each of these formally characterizes a different “intuitive” understanding of sensitivity. These approaches focus on different properties of the model response at a fundamental level and may therefore lead to different (even conflicting) conclusions about the underlying sensitivities. Here we revisit the theoretical basis for sensitivity analysis, summarize and critically evaluate existing approaches in the literature, and demonstrate their flaws and shortcomings through conceptual examples. We also demonstrate the difficulty involved in interpreting “global” interaction effects, which may undermine the value of existing interpretive approaches. With this background, we identify several important properties of response surfaces that are associated with the understanding and interpretation of sensitivities in the context of Earth and Environmental System models. Finally, we highlight the need for a new, comprehensive framework for sensitivity analysis that effectively characterizes all of the important sensitivity‐related properties of model response surfaces.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Response Surface Methodology: Process and Product Optimization Using Designed Experiments
Gunst, R. F., Myers, R. H., & Montgomery, D. C. (1996) · Technometrics · doi:10.2307/1270613 · Cited by 9,530
Design and Analysis of Computer Experiments
Sacks, J., Welch, W. J., Mitchell, T. J., & Wynn, H. P. (1989) · Statistical Science · doi:10.1214/ss/1177012413 · Cited by 6,919
Gupta, H. V., Kling, H., Yilmaz, K. K., & Martinez, G. F. (2009) · Journal of Hydrology · doi:10.1016/j.jhydrol.2009.08.003 · Cited by 6,758
Self-citationGlobal Sensitivity Analysis. The Primer
Saltelli, A., Ratto, M., Andres, T., Campolongo, F., Cariboni, J., Gatelli, D., Saisana, M., & Tarantola, S. (2007) · doi:10.1002/9780470725184 · Cited by 6,430
Global sensitivity indices for nonlinear mathematical models and their monte carlo estimates
Sobol′, I. M. (2001) · Mathematics and Computers in Simulation · doi:10.1016/s0378-4754(00)00270-6 · Cited by 6,389
Fisher, R. A. (1936) · BMJ · doi:10.1136/bmj.1.3923.554-a · Cited by 4,980
Factorial Sampling Plans for Preliminary Computational Experiments
Morris, M. D. (1991) · Technometrics · doi:10.1080/00401706.1991.10484804 · Cited by 4,943
Effective and efficient global optimization for conceptual rainfall‐runoff models
Duan, Q., Sorooshian, S., & Gupta, V. (1992) · Water Resources Research · doi:10.1029/91wr02985 · Cited by 3,310
Response Surface Methodology: Process and Product Optimization Using Designed Experiments
Gunst, R. F. (1996) · Technometrics · doi:10.1080/00401706.1996.10484509 · Cited by 3,255
Sensitivity Estimates for Nonlinear Mathematical Models
Sobol, I. M. (1990) · Medical Entomology and Zoology · Cited by 2,806
A Quantitative Model-Independent Method for Global Sensitivity Analysis of Model Output
Saltelli, A., Tarantola, S., & Chan, K. P.-S. (1999) · Technometrics · doi:10.1080/00401706.1999.10485594 · Cited by 2,275
Nonlinear Parameter Estimation
Chambers, J. M., & Bard, Y. (1975) · Technometrics · doi:10.2307/1268363 · Cited by 2,158
Nonlinear Parameter Estimation
Brubaker, T. A., & O'Keefe, K. R. (1979) · Analytical Chemistry · doi:10.1021/ac50049a796 · Cited by 2,134
Importance measures in global sensitivity analysis of nonlinear models
Homma, T., & Saltelli, A. (1996) · Reliability Engineering & System Safety · doi:10.1016/0951-8320(96)00002-6 · Cited by 2,011
An effective screening design for sensitivity analysis of large models
Campolongo, F., Cariboni, J., & Saltelli, A. (2007) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2006.10.004 · Cited by 2,011
A review of techniques for parameter sensitivity analysis of environmental models
Hamby, D. M. (1994) · Environmental Monitoring and Assessment · doi:10.1007/bf00547132 · Cited by 1,952
Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models
McCulloch, A. (2005) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2005.358_16.x · Cited by 1,842
Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models
Saltelli, A., Tarantola, S., Campolongo, F., & Ratto, M. (2004) · Cited by 1,797
Gupta, H. V., Sorooshian, S., & Yapo, P. O. (1998) · Water Resources Research · doi:10.1029/97wr03495 · Cited by 1,426
Self-citationA global sensitivity analysis tool for the parameters of multi-variable catchment models
van Griensven, A., Meixner, T., Grunwald, S., Bishop, T., Diluzio, M., & Srinivasan, R. (2006) · Journal of Hydrology · doi:10.1016/j.jhydrol.2005.09.008 · Cited by 1,223
Vrugt, J. A., Gupta, H. V., Bouten, W., & Sorooshian, S. (2003) · Water Resources Research · doi:10.1029/2002wr001642 · Cited by 1,196
Self-citationStudy of the sensitivity of coupled reaction systems to uncertainties in rate coefficients. I Theory
Cukier, R. I., Fortuin, C. M., Shuler, K. E., Petschek, A. G., & Schaibly, J. H. (1973) · The Journal of Chemical Physics · doi:10.1063/1.1680571 · Cited by 1,046
Design and Analysis of Experiments
Shuster, J. J. (2007) · Methods in Molecular Biology™ · doi:10.1007/978-1-59745-530-5_12 · Cited by 1,037
Review of surrogate modeling in water resources
Razavi, S., Tolson, B. A., & Burn, D. H. (2012) · Water Resources Research · doi:10.1029/2011wr011527 · Cited by 983
Self-citationHow to avoid a perfunctory sensitivity analysis
Saltelli, A., & Annoni, P. (2010) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2010.04.012 · Cited by 974
Kuczera, G., & Parent, E. (1998) · Journal of Hydrology · doi:10.1016/s0022-1694(98)00198-x · Cited by 777
Sobol’ sensitivity analysis of a complex environmental model
Nossent, J., Elsen, P., & Bauwens, W. (2011) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2011.08.010 · Cited by 738
Reconciling theory with observations: elements of a diagnostic approach to model evaluation
Gupta, H. V., Wagener, T., & Liu, Y. (2008) · Hydrological Processes · doi:10.1002/hyp.6989 · Cited by 644
Self-citationSpear, R. (1980) · Water Research · doi:10.1016/0043-1354(80)90040-8 · Cited by 604
Pursuing the method of multiple working hypotheses for hydrological modeling
Clark, M. P., Kavetski, D., & Fenicia, F. (2011) · Water Resources Research · doi:10.1029/2010wr009827 · Cited by 594
Sensitivity Analysis in Chemical Kinetics
Rabitz, H., Kramer, M., & Dacol, D. (1983) · Annual Review of Physical Chemistry · doi:10.1146/annurev.pc.34.100183.002223 · Cited by 536
Muleta, M. K., & Nicklow, J. W. (2005) · Journal of Hydrology · doi:10.1016/j.jhydrol.2004.09.005 · Cited by 524
The Use of the Rank Transform in Regression
Iman, R. L., & Conover, W. J. (1979) · Technometrics · doi:10.1080/00401706.1979.10489820 · Cited by 514
Derivative based global sensitivity measures and their link with global sensitivity indices
Sobol’, I. M., & Kucherenko, S. (2009) · Mathematics and Computers in Simulation · doi:10.1016/j.matcom.2009.01.023 · Cited by 445
An alternative way to compute Fourier amplitude sensitivity test (FAST)
Saltelli, A., & Bolado, R. (1998) · Computational Statistics & Data Analysis · doi:10.1016/s0167-9473(97)00043-1 · Cited by 367
An overview of the design and analysis of simulation experiments for sensitivity analysis
Kleijnen, J. P. C. (2005) · European Journal of Operational Research · doi:10.1016/j.ejor.2004.02.005 · Cited by 320
Multi-method global sensitivity analysis of flood inundation models
Pappenberger, F., Beven, K. J., Ratto, M., & Matgen, P. (2008) · Advances in Water Resources · doi:10.1016/j.advwatres.2007.04.009 · Cited by 276
Relative Entropy Based Method for Probabilistic Sensitivity Analysis in Engineering Design
Liu, H., Chen, W., & Sudjianto, A. (2006) · Journal of Mechanical Design · doi:10.1115/1.2159025 · Cited by 270
Sensitivity and uncertainty analysis in spatial modelling based on GIS
Crosetto, M., Tarantola, S., & Saltelli, A. (2000) · Agriculture, Ecosystems & Environment · doi:10.1016/s0167-8809(00)00169-9 · Cited by 253
Stuart, A., & Edwards, A. W. F. (1973) · Applied Statistics · doi:10.2307/2346929 · Cited by 199
Sensitivity analysis in model calibration: GSA-GLUE approach
Ratto, M., Tarantola, S., & Saltelli, A. (2001) · Computer Physics Communications · doi:10.1016/s0010-4655(01)00159-x · Cited by 188
Shin, M.-J., Guillaume, J. H. A., Croke, B. F. W., & Jakeman, A. J. (2013) · Journal of Hydrology · doi:10.1016/j.jhydrol.2013.08.047 · Cited by 184
Kavetski, D., & Clark, M. P. (2010) · Water Resources Research · doi:10.1029/2009wr008896 · Cited by 176
Parameter uncertainty and interaction in complex environmental models
Spear, R. C., Grieb, T. M., & Shang, N. (1994) · Water Resources Research · doi:10.1029/94wr01732 · Cited by 174
Distributed Evaluation of Local Sensitivity Analysis (DELSA), with application to hydrologic models
Rakovec, O., Hill, M. C., Clark, M. P., Weerts, A. H., Teuling, A. J., & Uijlenhoet, R. (2014) · Water Resources Research · doi:10.1002/2013wr014063 · Cited by 171
A fully multiple‐criteria implementation of the Sobol′ method for parameter sensitivity analysis
Rosolem, R., Gupta, H. V., Shuttleworth, W. J., Zeng, X., & Goncalves, L. (2012) · Journal of Geophysical Research Atmospheres · doi:10.1029/2011jd016355 · Cited by 155
Self-citationRuano, M. V., Ribes, J., Seco, A., & Ferrer, J. (2012) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2012.03.008 · Cited by 155
Chu, M. L., Muñoz‐Carpena, R., Kiker, G. A., et al. (2011) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2010.12.003 · Cited by 149
Numerical assessment of metamodelling strategies in computationally intensive optimization
Razavi, S., Tolson, B. A., & Burn, D. H. (2011) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2011.09.010 · Cited by 145
Self-citationSensitivity analysis for the reduction of complex metabolism models
Degenring, D., Froemel, C., Dikta, G., & Takors, R. (2004) · Journal of Process Control · doi:10.1016/j.jprocont.2003.12.008 · Cited by 135
Kleijnen, J. P. C. (1995) · System Dynamics Review · doi:10.1002/sdr.4260110403 · Cited by 108
Bastidas, L. A., Hogue, T. S., Sorooshian, S., Gupta, H. V., & Shuttleworth, W. J. (2006) · Journal of Geophysical Research Atmospheres · doi:10.1029/2005jd006377 · Cited by 94
Self-citation
Cited by
The Future of Sensitivity Analysis: An essential discipline for systems modeling and policy support
Razavi, S., Jakeman, A., Saltelli, A., Prieur, C., Iooss, B., Borgonovo, E., Plischke, E., Lo Piano, S., Iwanaga, T., Becker, W., et al. (2021) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2020.104954 · Cited by 628
Self-citationToward SALib 2.0: Advancing the accessibility and interpretability of global sensitivity analyses
Iwanaga, T., Usher, W., & Herman, J. (2022) · Socio-Environmental Systems Modelling · doi:10.18174/sesmo.18155 · Cited by 367
Introductory overview: Optimization using evolutionary algorithms and other metaheuristics
Maier, H. R., Razavi, S., Kapelan, Z., Matott, L. S., Kasprzyk, J., & Tolson, B. A. (2019) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2018.11.018 · Cited by 272
Self-citationSheikholeslami, R., & Razavi, S. (2017) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2017.03.010 · Cited by 255
Self-citationThe sensitivity of simulated streamflow to individual hydrologic processes across North America
Mai, J., Craig, J. R., Tolson, B. A., & Arsenault, R. (2022) · Nature Communications · doi:10.1038/s41467-022-28010-7 · Cited by 63
Metadata
- Authors
- Razavi, S., & Gupta, H. V.
- Venue
- Water Resources Research
- Publisher
- American Geophysical Union (AGU)
- Published
- 2015-05
- Volume
- 51
- Issue
- 5
- Pages
- 3070-3092
- ISSN
- 0043-1397 (ISSN-L)
- 1944-7973
- OpenAlex
- W2052960370
- Cited by
- 372
- References
- 52
- FWCI
- 22.43
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.