The DOI matches its Crossref record; 64 of the 64 references it cites were found in a registry or an index.
Convergence and uncertainty analyses in Monte-Carlo based sensitivity analysis
Abstract
Sensitivity analysis plays an important role in model development, calibration, uncertainty analysis, scenario analysis, and, hence, decision making. With the availability of different sensitivity analysis techniques, selecting an appropriate technique, monitoring the convergence and estimating the uncertainty of the sensitivity indices are very crucial for environmental modelling, especially for distributed models due to their high non-linearity, non-monotonicity, highly correlated parameters, and intensive computational requirements. It would be useful to identify whether some techniques outperform others with respect to computational requirements, reliability, and other criteria. This paper proposes two methods to monitor the convergence and estimate the uncertainty of sensitivity analysis techniques. One is based on the central limit theorem and the other on the bootstrap technique. These two methods are implemented to assess five different sensitivity analysis techniques applied to an environmental model. These techniques are: the Sobol' method, the Morris method, Linear Regression (LR), Regionalized Sensitivity Analysis (RSA), and non-parametric smoothing. The results show that: (i) the Sobol' method is very robust in quantifying sensitivities and ranking parameters despite a large number of model evaluations; (ii) the Morris method is efficient to rank out unimportant parameters at a medium cost; (iii) the non-parametric smoothing is reliable and robust in quantifying the main effects and low-order interactions while requiring a small number of model evaluations; finally (iv) the other two techniques, that is, LR and RSA, should be used with care.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
River flow forecasting through conceptual models part I — A discussion of principles
Nash, J. E., & Sutcliffe, J. V. (1970) · Journal of Hydrology · doi:10.1016/0022-1694(70)90255-6 · Cited by 23,674
(1973) · CERN Document Server (European Organization for Nuclear Research) · Cited by 12,895
On Estimation of a Probability Density Function and Mode
Parzen, E. (1962) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177704472 · Cited by 10,585
Response Surface Methodology: Process and Product Optimization Using Designed Experiments
Gunst, R. F., Myers, R. H., & Montgomery, D. C. (1996) · Technometrics · doi:10.2307/1270613 · Cited by 9,530
Design and Analysis of Computer Experiments
Sacks, J., Welch, W. J., Mitchell, T. J., & Wynn, H. P. (1989) · Statistical Science · doi:10.1214/ss/1177012413 · Cited by 6,919
Global Sensitivity Analysis. The Primer
Saltelli, A., Ratto, M., Andres, T., Campolongo, F., Cariboni, J., Gatelli, D., Saisana, M., & Tarantola, S. (2007) · doi:10.1002/9780470725184 · Cited by 6,430
Factorial Sampling Plans for Preliminary Computational Experiments
Morris, M. D. (1991) · Technometrics · doi:10.1080/00401706.1991.10484804 · Cited by 4,943
Spline Models for Observational Data.
Muller, H.-G., & Wahba, G. (1991) · Journal of the American Statistical Association · doi:10.2307/2290434 · Cited by 4,928
The future of distributed models: model calibration and uncertainty prediction
Beven, K., & Binley, A. (1992) · Hydrological Processes · doi:10.1002/hyp.3360060305 · Cited by 4,527
Pattern Classification and Scene Analysis
Thompson, M., Duda, R. O., & Hart, P. E. (1974) · Leonardo · doi:10.2307/1573081 · Cited by 4,495
Sensitivity Estimates for Nonlinear Mathematical Models
Sobol, I. M. (1990) · Medical Entomology and Zoology · Cited by 2,806
A Quantitative Model-Independent Method for Global Sensitivity Analysis of Model Output
Saltelli, A., Tarantola, S., & Chan, K. P.-S. (1999) · Technometrics · doi:10.1080/00401706.1999.10485594 · Cited by 2,275
Making best use of model evaluations to compute sensitivity indices
Saltelli, A. (2002) · Computer Physics Communications · doi:10.1016/s0010-4655(02)00280-1 · Cited by 2,228
Modelling hydrology and water quality in the pre-alpine/alpine Thur watershed using SWAT
Abbaspour, K. C., Yang, J., Maximov, I., Siber, R., Bogner, K., Mieleitner, J., Zobrist, J., & Srinivasan, R. (2007) · Journal of Hydrology · doi:10.1016/j.jhydrol.2006.09.014 · Cited by 2,020
Self-citationAn effective screening design for sensitivity analysis of large models
Campolongo, F., Cariboni, J., & Saltelli, A. (2007) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2006.10.004 · Cited by 2,011
Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models
McCulloch, A. (2005) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2005.358_16.x · Cited by 1,842
Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models
Saltelli, A., Tarantola, S., Campolongo, F., & Ratto, M. (2004) · Cited by 1,797
A global sensitivity analysis tool for the parameters of multi-variable catchment models
van Griensven, A., Meixner, T., Grunwald, S., Bishop, T., Diluzio, M., & Srinivasan, R. (2006) · Journal of Hydrology · doi:10.1016/j.jhydrol.2005.09.008 · Cited by 1,223
Vrugt, J. A., Gupta, H. V., Bouten, W., & Sorooshian, S. (2003) · Water Resources Research · doi:10.1029/2002wr001642 · Cited by 1,196
All of Statistics: A Concise Course in Statistical Inference
Wasserman, L. (2004) · Springer Texts in Statistics · doi:10.1007/978-0-387-21736-9 · Cited by 1,090
Probabilistic Sensitivity Analysis of Complex Models: A Bayesian Approach
Oakley, J. E., & O'Hagan, A. (2004) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2004.05304.x · Cited by 1,085
Study of the sensitivity of coupled reaction systems to uncertainties in rate coefficients. I Theory
Cukier, R. I., Fortuin, C. M., Shuler, K. E., Petschek, A. G., & Schaibly, J. H. (1973) · The Journal of Chemical Physics · doi:10.1063/1.1680571 · Cited by 1,046
Identification and Review of Sensitivity Analysis Methods
Christopher Frey, H., & Patil, S. R. (2002) · Risk Analysis · doi:10.1111/0272-4332.00039 · Cited by 1,036
How to avoid a perfunctory sensitivity analysis
Saltelli, A., & Annoni, P. (2010) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2010.04.012 · Cited by 974
Multi-objective global optimization for hydrologic models
Yapo, P. O., Gupta, H. V., & Sorooshian, S. (1998) · Journal of Hydrology · doi:10.1016/s0022-1694(97)00107-8 · Cited by 908
Gu, C. (2013) · Springer Series in Statistics · doi:10.1007/978-1-4614-5369-7 · Cited by 867
Uncertainties in risk analysis: Six levels of treatment
Paté-Cornell, M. E. (1996) · Reliability Engineering & System Safety · doi:10.1016/s0951-8320(96)00067-1 · Cited by 811
Helton, J. C. (1993) · Reliability Engineering & System Safety · doi:10.1016/0951-8320(93)90097-i · Cited by 735
Hoffman, F. O., & Hammonds, J. S. (1994) · Risk Analysis · doi:10.1111/j.1539-6924.1994.tb00281.x · Cited by 700
Boyle, D. P., Gupta, H. V., & Sorooshian, S. (2000) · Water Resources Research · doi:10.1029/2000wr900207 · Cited by 696
Nonlinear sensitivity analysis of multiparameter model systems
Cukier, R. I., Levine, H. B., & Shuler, K. E. (1978) · Journal of Computational Physics · doi:10.1016/0021-9991(78)90097-9 · Cited by 694
All Of Statistic A Concise Course In Statistical Inference
Waserman, L. (2003) · Cited by 652
An Investigation of Uncertainty and Sensitivity Analysis Techniques for Computer Models
Iman, R. L., & Helton, J. C. (1988) · Risk Analysis · doi:10.1111/j.1539-6924.1988.tb01155.x · Cited by 608
Spear, R. (1980) · Water Research · doi:10.1016/0043-1354(80)90040-8 · Cited by 604
The probability-distributed principle and runoff production at point and basin scales
MOORE, R. J. (1985) · Hydrological Sciences Journal · doi:10.1080/02626668509490989 · Cited by 590
Vrugt, J. A., Diks, C. G. H., Gupta, H. V., Bouten, W., & Verstraten, J. M. (2005) · Water Resources Research · doi:10.1029/2004wr003059 · Cited by 582
Bayesian analysis of input uncertainty in hydrological modeling: 1. Theory
Kavetski, D., Kuczera, G., & Franks, S. W. (2006) · Water Resources Research · doi:10.1029/2005wr004368 · Cited by 580
Sensitivity measures,anova-like Techniques and the use of bootstrap
Archer, G. E. B., Saltelli, A., & Sobol, I. M. (1997) · Journal of Statistical Computation and Simulation · doi:10.1080/00949659708811825 · Cited by 545
Tang, Y., Reed, P., Wagener, T., & van Werkhoven, K. (2007) · Hydrology and Earth System Sciences · doi:10.5194/hess-11-793-2007 · Cited by 422
About the use of rank transformation in sensitivity analysis of model output
Saltelli, A., & Sobol', I. M. (1995) · Reliability Engineering & System Safety · doi:10.1016/0951-8320(95)00099-2 · Cited by 340
McRae, G. J., Tilden, J. W., & Seinfeld, J. H. (1982) · Computers & Chemical Engineering · doi:10.1016/0098-1354(82)80003-3 · Cited by 320
Helton, J. C., Davis, F. J., & Johnson, J. D. (2005) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2004.09.006 · Cited by 278
Multi-method global sensitivity analysis of flood inundation models
Pappenberger, F., Beven, K. J., Ratto, M., & Matgen, P. (2008) · Advances in Water Resources · doi:10.1016/j.advwatres.2007.04.009 · Cited by 276
Practical Approaches To Construct RS-HDMR Component Functions
Li, G., Wang, S.-W., & Rabitz, H. (2002) · The Journal of Physical Chemistry A · doi:10.1021/jp014567t · Cited by 275
Helton, J. C., & Burmaster, D. E. (1996) · Reliability Engineering & System Safety · doi:10.1016/s0951-8320(96)00066-x · Cited by 258
Yang, J., Reichert, P., Abbaspour, K. C., & Yang, H. (2007) · Journal of Hydrology · doi:10.1016/j.jhydrol.2007.04.006 · Cited by 233
Self-citationState Dependent Parameter metamodelling and sensitivity analysis
Ratto, M., Pagano, A., & Young, P. (2007) · Computer Physics Communications · doi:10.1016/j.cpc.2007.07.011 · Cited by 217
Sensitivity analysis: Could better methods be used?
Saltelli, A. (1999) · Journal of Geophysical Research: Atmospheres · doi:10.1029/1998jd100042 · Cited by 208
Li, G., Hu, J., Wang, S.-W., Georgopoulos, P. G., Schoendorf, J., & Rabitz, H. (2006) · The Journal of Physical Chemistry A · doi:10.1021/jp054148m · Cited by 180
Vrugt, J. A., Bouten, W., Gupta, H. V., & Sorooshian, S. (2002) · Water Resources Research · doi:10.1029/2001wr001118 · Cited by 168
A Measure of Top – Down Correlation
Iman, R. L., & Conover, W. J. (1987) · Technometrics · doi:10.1080/00401706.1987.10488244 · Cited by 166
Varella, H., Guérif, M., & Buis, S. (2009) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2009.09.012 · Cited by 143
Confalonieri, R., Bellocchi, G., Tarantola, S., Acutis, M., Donatelli, M., & Genovese, G. (2009) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2009.10.005 · Cited by 107
Uncertainty, sensitivity analysis and the role of data based mechanistic modeling in hydrology
Ratto, M., Young, P. C., Romanowicz, R. J., Pappenberger, F., Saltelli, A., & Pagano, A. J. (2007) · Hydrology and earth system sciences · doi:10.5194/hess-11-1249-2007 · Cited by 106
The Repeatability of Uncertainty and Sensitivity Analyses for Complex Probabilistic Risk Assessments
Iman, R. L., & Helton, J. C. (1991) · Risk Analysis · doi:10.1111/j.1539-6924.1991.tb00649.x · Cited by 101
Tarantola, S., Giglioli, N., Jesinghaus, J., & Saltelli, A. (2002) · Stochastic Environmental Research and Risk Assessment · doi:10.1007/s00477-001-0085-x · Cited by 89
Helton, J. C., Martell, M., & Tierney, M. S. (2000) · Reliability Engineering & System Safety · doi:10.1016/s0951-8320(00)00032-6 · Cited by 67
Multiple predictor smoothing methods for sensitivity analysis: Example results
Storlie, C. B., & Helton, J. C. (2007) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2006.10.013 · Cited by 60
(2011) · Monographs on statistics and applied probability · doi:10.1201/b10954-5 · Cited by 56
Management Option Rank Equivalence (MORE) – A new method of sensitivity analysis for decision-making
Ravalico, J. K., Dandy, G. C., & Maier, H. R. (2009) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2009.06.012 · Cited by 49
Draft Guidance on the Development, Evaluation, and Application of Regulatory Environmental Models
Pascual, P., Stiber, N. A., & Sunderland, E. M. (2003) · Cited by 38
.), Helton, J. C., & Sallaberry, C. J.-M. (2007) · OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) · doi:10.2172/910196 · Cited by 27
Sensitivity analysis in fuzzy systems: Integration of SimLab and DANA
Foscarini, F., Bellocchi, G., Confalonieri, R., Cristian, S., & den Eede, G. V. (2010) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2010.03.024 · Cited by 20
Helton, J. C., & Sallaberry, C. J.-M. (2008) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2008.06.018 · Cited by 20
Cited by
Sensitivity analysis of environmental models: A systematic review with practical workflow
Pianosi, F., Beven, K., Freer, J., Hall, J. W., Rougier, J., Stephenson, D. B., & Wagener, T. (2016) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2016.02.008 · Cited by 1,704
Characterising performance of environmental models
Bennett, N. D., Croke, B. F. W., Guariso, G., Guillaume, J. H. A., Hamilton, S. H., Jakeman, A. J., Marsili-Libelli, S., Newham, L. T. H., Norton, J. P., Perrin, C., et al. (2013) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2012.09.011 · Cited by 1,536
Sobol’ sensitivity analysis of a complex environmental model
Nossent, J., Elsen, P., & Bauwens, W. (2011) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2011.08.010 · Cited by 738
Song, X., Zhang, J., Zhan, C., Xuan, Y., Ye, M., & Xu, C. (2015) · Journal of Hydrology · doi:10.1016/j.jhydrol.2015.02.013 · Cited by 654
A Matlab toolbox for Global Sensitivity Analysis
Pianosi, F., Sarrazin, F., & Wagener, T. (2015) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2015.04.009 · Cited by 653
Metadata
- Authors
- Yang, J.
- Venue
- Environmental Modelling & Software
- Publisher
- Elsevier BV
- Published
- 2011-04
- Volume
- 26
- Issue
- 4
- Pages
- 444-457
- ISSN
- 1364-8152 (ISSN-L)
- OpenAlex
- W2041202986
- Cited by
- 326
- References
- 64
- FWCI
- 17.20
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.