The DOI matches its Crossref record; 63 of the 63 references it cites were found in a registry or an index.
A comprehensive evaluation of various sensitivity analysis methods: A case study with a hydrological model
Open accessAbstract
Sensitivity analysis (SA) is a commonly used approach for identifying important parameters that dominate model behaviors. We use a newly developed software package, a Problem Solving environment for Uncertainty Analysis and Design Exploration (PSUADE), to evaluate the effectiveness and efficiency of ten widely used SA methods, including seven qualitative and three quantitative ones. All SA methods are tested using a variety of sampling techniques to screen out the most sensitive (i.e., important) parameters from the insensitive ones. The Sacramento Soil Moisture Accounting (SAC-SMA) model, which has thirteen tunable parameters, is used for illustration. The South Branch Potomac River basin near Springfield, West Virginia in the U.S. is chosen as the study area. The key findings from this study are: (1) For qualitative SA methods, Correlation Analysis (CA), Regression Analysis (RA), and Gaussian Process (GP) screening methods are shown to be not effective in this example. Morris One-At-a-Time (MOAT) screening is the most efficient, needing only 280 samples to identify the most important parameters, but it is the least robust method. Multivariate Adaptive Regression Splines (MARS), Delta Test (DT) and Sum-Of-Trees (SOT) screening methods need about 400–600 samples for the same purpose. Monte Carlo (MC), Orthogonal Array (OA) and Orthogonal Array based Latin Hypercube (OALH) are appropriate sampling techniques for them; (2) For quantitative SA methods, at least 2777 samples are needed for Fourier Amplitude Sensitivity Test (FAST) to identity parameter main effect. McKay method needs about 360 samples to evaluate the main effect, more than 1000 samples to assess the two-way interaction effect. OALH and LPτ (LPTAU) sampling techniques are more appropriate for McKay method. For the Sobol' method, the minimum samples needed are 1050 to compute the first-order and total sensitivity indices correctly. These comparisons show that qualitative SA methods are more efficient but less accurate and robust than quantitative ones.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Classification and Regression Trees.
Gordon, A. D., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1984) · Biometrics · doi:10.2307/2530946 · Cited by 23,522
Classification and Regression Trees.
Ryzin, J. V., Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (1986) · Journal of the American Statistical Association · doi:10.2307/2288003 · Cited by 20,825
Classification and regression trees
Praagman, J. (1985) · European Journal of Operational Research · doi:10.1016/0377-2217(85)90321-2 · Cited by 10,053
The proof and measurement of association between two things
Spearman, C. (1904) · The American Journal of Psychology · doi:10.2307/1412159 · Cited by 8,039
Multivariate Adaptive Regression Splines
Friedman, J. H. (1991) · The Annals of Statistics · doi:10.1214/aos/1176347963 · Cited by 8,000
McKay, M. D., Beckman, R. J., & Conover, W. J. (2000) · Technometrics · doi:10.2307/1271432 · Cited by 7,415
Design and Analysis of Computer Experiments
Sacks, J., Welch, W. J., Mitchell, T. J., & Wynn, H. P. (1989) · Statistical Science · doi:10.1214/ss/1177012413 · Cited by 6,919
Metropolis, N., & Ulam, S. (1949) · Journal of the American Statistical Association · doi:10.1080/01621459.1949.10483310 · Cited by 6,474
Global Sensitivity Analysis. The Primer
Saltelli, A., Ratto, M., Andres, T., Campolongo, F., Cariboni, J., Gatelli, D., Saisana, M., & Tarantola, S. (2007) · doi:10.1002/9780470725184 · Cited by 6,430
Global sensitivity indices for nonlinear mathematical models and their monte carlo estimates
Sobol′, I. M. (2001) · Mathematics and Computers in Simulation · doi:10.1016/s0378-4754(00)00270-6 · Cited by 6,389
Factorial Sampling Plans for Preliminary Computational Experiments
Morris, M. D. (1991) · Technometrics · doi:10.1080/00401706.1991.10484804 · Cited by 4,943
The future of distributed models: model calibration and uncertainty prediction
Beven, K., & Binley, A. (1992) · Hydrological Processes · doi:10.1002/hyp.3360060305 · Cited by 4,527
On the Experimental Attainment of Optimum Conditions
Box, G. E. P., & Wilson, K. B. (1951) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1951.tb00067.x · Cited by 4,464
Some New Three Level Designs for the Study of Quantitative Variables
Box, G. E. P., & Behnken, D. W. (1960) · Technometrics · doi:10.1080/00401706.1960.10489912 · Cited by 4,177
The design of optimum multifactorial experiments
PLACKETT, R. L., & BURMAN, J. P. (1946) · Biometrika · doi:10.1093/biomet/33.4.305 · Cited by 3,906
McKay, M. D., Beckman, R. J., & Conover, W. J. (1979) · Technometrics · doi:10.2307/1268522 · Cited by 3,757
On the Experimental Attainment of Optimum Conditions
Box, G. E. P., & Wilson, K. B. (1992) · Springer Series in Statistics · doi:10.1007/978-1-4612-4380-9_23 · Cited by 3,448
Effective and efficient global optimization for conceptual rainfall‐runoff models
Duan, Q., Sorooshian, S., & Gupta, V. (1992) · Water Resources Research · doi:10.1029/91wr02985 · Cited by 3,310
Self-citationComputer Models of Watershed Hydrology
Singh, V. P. (1995) · Medical Entomology and Zoology · Cited by 3,273
Sensitivity Estimates for Nonlinear Mathematical Models
Sobol, I. M. (1990) · Medical Entomology and Zoology · Cited by 2,806
A manifesto for the equifinality thesis
Beven, K. (2006) · Journal of Hydrology · doi:10.1016/j.jhydrol.2005.07.007 · Cited by 2,581
A Quantitative Model-Independent Method for Global Sensitivity Analysis of Model Output
Saltelli, A., Tarantola, S., & Chan, K. P.-S. (1999) · Technometrics · doi:10.1080/00401706.1999.10485594 · Cited by 2,275
Making best use of model evaluations to compute sensitivity indices
Saltelli, A. (2002) · Computer Physics Communications · doi:10.1016/s0010-4655(02)00280-1 · Cited by 2,228
Rainfall‐Runoff Modelling: The Primer
Beven, K. (2012) · Wiley eBooks · doi:10.1002/9781119951001 · Cited by 2,051
Importance measures in global sensitivity analysis of nonlinear models
Homma, T., & Saltelli, A. (1996) · Reliability Engineering & System Safety · doi:10.1016/0951-8320(96)00002-6 · Cited by 2,011
An effective screening design for sensitivity analysis of large models
Campolongo, F., Cariboni, J., & Saltelli, A. (2007) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2006.10.004 · Cited by 2,011
Scale issues in hydrological modelling: A review
Blöschl, G., & Sivapalan, M. (1995) · Hydrological Processes · doi:10.1002/hyp.3360090305 · Cited by 1,837
The proof and measurement of association between two things
Spearman, C. (2010) · International Journal of Epidemiology · doi:10.1093/ije/dyq191 · Cited by 1,681
BART: Bayesian additive regression trees
Chipman, H. A., George, E. I., & McCulloch, R. E. (2010) · The Annals of Applied Statistics · doi:10.1214/09-aoas285 · Cited by 1,542
Optimal use of the SCE-UA global optimization method for calibrating watershed models
Duan, Q., Sorooshian, S., & Gupta, V. K. (1994) · Journal of Hydrology · doi:10.1016/0022-1694(94)90057-4 · Cited by 1,508
Self-citationRegression Towards Mediocrity in Hereditary Stature.
Galton, F. (1886) · The Journal of the Anthropological Institute of Great Britain and Ireland · doi:10.2307/2841583 · Cited by 1,454
Gupta, H. V., Sorooshian, S., & Yapo, P. O. (1998) · Water Resources Research · doi:10.1029/97wr03495 · Cited by 1,426
A global sensitivity analysis tool for the parameters of multi-variable catchment models
van Griensven, A., Meixner, T., Grunwald, S., Bishop, T., Diluzio, M., & Srinivasan, R. (2006) · Journal of Hydrology · doi:10.1016/j.jhydrol.2005.09.008 · Cited by 1,223
How much complexity is warranted in a rainfall‐runoff model?
Jakeman, A. J., & Hornberger, G. M. (1993) · Water Resources Research · doi:10.1029/93wr00877 · Cited by 1,216
Karypis, G., & Kumar, V. (1997) · University of Minnesota Digital Conservancy (University of Minnesota) · Cited by 1,215
Study of the sensitivity of coupled reaction systems to uncertainties in rate coefficients. I Theory
Cukier, R. I., Fortuin, C. M., Shuler, K. E., Petschek, A. G., & Schaibly, J. H. (1973) · The Journal of Chemical Physics · doi:10.1063/1.1680571 · Cited by 1,046
Renard, B., Kavetski, D., Kuczera, G., Thyer, M., & Franks, S. W. (2010) · Water Resources Research · doi:10.1029/2009wr008328 · Cited by 910
BART: Bayesian Additive Regression Trees
Chipman, H., George, E. I., & McCulloch, R. E. (2006) · ScholarlyCommons (University of Pennsylvania) · Cited by 889
Sensitivity analysis for non-linear mathematical models
Sobol, I. M. (1993) · Medical Entomology and Zoology · Cited by 878
Orthogonal Array-Based Latin Hypercubes
Tang, B. (1993) · Journal of the American Statistical Association · doi:10.1080/01621459.1993.10476423 · Cited by 858
Choosing the Sample Size of a Computer Experiment: A Practical Guide
Loeppky, J. L., Sacks, J., & Welch, W. J. (2009) · Technometrics · doi:10.1198/tech.2009.08040 · Cited by 758
Duan, Q., Schaake, J., Andréassian, V., Franks, S., Goteti, G., Gupta, H. V., Gusev, Y. M., Habets, F., Hall, A., Hay, L., et al. (2006) · Journal of Hydrology · doi:10.1016/j.jhydrol.2005.07.031 · Cited by 701
Self-citationBoyle, D. P., Gupta, H. V., & Sorooshian, S. (2000) · Water Resources Research · doi:10.1029/2000wr900207 · Cited by 696
Pursuing the method of multiple working hypotheses for hydrological modeling
Clark, M. P., Kavetski, D., & Fenicia, F. (2011) · Water Resources Research · doi:10.1029/2010wr009827 · Cited by 594
AN APPROACH TO THE PRELIMINARY ANALYSIS OF ENVIRONMENTAL SYSTEMS
Hornberger, G. M., & Spear, R. C. (1981) · Cited by 566
The 2k—pFractional Factorial Designs
Box, G. E. P., & Hunter, J. S. (1961) · Technometrics · doi:10.1080/00401706.1961.10489951 · Cited by 437
Tang, Y., Reed, P., Wagener, T., & van Werkhoven, K. (2007) · Hydrology and Earth System Sciences · doi:10.5194/hess-11-793-2007 · Cited by 422
The 2 k-p Fractional Factorial Designs Part I
Box, G. E. P., & Hunter, J. S. (2000) · Technometrics · doi:10.2307/1271430 · Cited by 414
The 2 k-p Fractional Factorial Designs Part I
Box, G. E. P., & Hunter, J. S. (1961) · Technometrics · doi:10.2307/1266725 · Cited by 328
Sorooshian, S., & Gupta, V. K. (1983) · Water Resources Research · doi:10.1029/wr019i001p00260 · Cited by 266
Bayesian Additive Regression Trees
Chipman, H., George, E. I., & McCulloch, R. E. (2006) · Cited by 254
Characterization of watershed model behavior across a hydroclimatic gradient
van Werkhoven, K., Wagener, T., Reed, P., & Tang, Y. (2008) · Water Resources Research · doi:10.1029/2007wr006271 · Cited by 217
State Dependent Parameter metamodelling and sensitivity analysis
Ratto, M., Pagano, A., & Young, P. (2007) · Computer Physics Communications · doi:10.1016/j.cpc.2007.07.011 · Cited by 217
Kavetski, D., & Clark, M. P. (2010) · Water Resources Research · doi:10.1029/2009wr008896 · Cited by 176
Borgonovo, E., Castaings, W., & Tarantola, S. (2012) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2011.06.006 · Cited by 159
Efficient implementation of gaussian processes
Gibbs, M. (1997) · Cited by 151
Gómez‐Gutiérrez, Á., Schnabel, S., & Contador, J. F. L. (2009) · Ecological Modelling · doi:10.1016/j.ecolmodel.2009.06.020 · Cited by 113
Computer experiments: a review
Levy, S., & Steinberg, D. M. (2010) · AStA Advances in Statistical Analysis · doi:10.1007/s10182-010-0147-9 · Cited by 102
Crawford, S. L. (2006) · Circulation · doi:10.1161/circulationaha.105.586495 · Cited by 82
Multilevel calibration strategy for complex hydrologic simulation models
Brazil, L. E. (1988) · Cited by 68
Exploring parameter sensitivities of the land surface using a locally coupled land‐atmosphere model
Liu, Y., Gupta, H. V., Sorooshian, S., Bastidas, L. A., & Shuttleworth, W. J. (2004) · Journal of Geophysical Research Atmospheres · doi:10.1029/2004jd004730 · Cited by 62
Variance-based sensitivity analysis of model outputs using surrogate models
Shahsavani, D., & Grimvall, A. B. (2011) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2011.01.002 · Cited by 54
A Practical Global Sensitivity Analysis Methodology for Multi-Physics Applications
Tong, C., & Graziani, F. R. (2008) · Lecture notes in computational science and engineering · doi:10.1007/978-3-540-77362-7_12 · Cited by 19
Self-citation
Cited by
Song, X., Zhang, J., Zhan, C., Xuan, Y., Ye, M., & Xu, C. (2015) · Journal of Hydrology · doi:10.1016/j.jhydrol.2015.02.013 · Cited by 654
A Matlab toolbox for Global Sensitivity Analysis
Pianosi, F., Sarrazin, F., & Wagener, T. (2015) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2015.04.009 · Cited by 653
The Future of Sensitivity Analysis: An essential discipline for systems modeling and policy support
Razavi, S., Jakeman, A., Saltelli, A., Prieur, C., Iooss, B., Borgonovo, E., Plischke, E., Lo Piano, S., Iwanaga, T., Becker, W., et al. (2021) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2020.104954 · Cited by 628
Self-citationSheikholeslami, R., & Razavi, S. (2017) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2017.03.010 · Cited by 255
Sensitivity analysis methods in the biomedical sciences
Qian, G., & Mahdi, A. (2020) · Mathematical Biosciences · doi:10.1016/j.mbs.2020.108306 · Cited by 169
Metadata
- Authors
- Gan, Y., Duan, Q., Gong, W., Tong, C., Sun, Y., Chu, W., Ye, A., Miao, C., & Di, Z.
- Venue
- Environmental Modelling & Software
- Publisher
- Elsevier BV
- Published
- 2014-01
- Volume
- 51
- Pages
- 269-285
- ISSN
- 1364-8152 (ISSN-L)
- OpenAlex
- W1995711095
- Cited by
- 360
- References
- 63
- FWCI
- 10.41
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.