The DOI matches its Crossref record; 65 of the 65 references it cites were found in a registry or an index.
Global Sensitivity Analysis of environmental models: Convergence and validation
Open accessAbstract
We address two critical choices in Global Sensitivity Analysis (GSA): the choice of the sample size and of the threshold for the identification of insensitive input factors. Guidance to assist users with those two choices is still insufficient. We aim at filling this gap. Firstly, we define criteria to quantify the convergence of sensitivity indices, of ranking and of screening, based on a bootstrap approach. Secondly, we investigate the screening threshold with a quantitative validation procedure for screening results. We apply the proposed methodologies to three hydrological models with varying complexity utilizing three widely-used GSA methods (RSA, Morris, Sobol’). We demonstrate that convergence of screening and ranking can be reached before sensitivity estimates stabilize. Convergence dynamics appear to be case-dependent, which suggests that “fit-for-all” rules for sample sizes should not be used. Other modellers can easily adopt our criteria and procedures for a wide range of GSA methods and cases.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1994) · doi:10.1201/9780429246593 · Cited by 40,500
River flow forecasting through conceptual models part I — A discussion of principles
Nash, J. E., & Sutcliffe, J. V. (1970) · Journal of Hydrology · doi:10.1016/0022-1694(70)90255-6 · Cited by 23,674
System Identification: Theory for the User
Lennart Ljung, A. (1987) · Cited by 17,690
System identification—Theory for the user
Sinha, N. K. (1989) · Automatica · doi:10.1016/0005-1098(89)90019-8 · Cited by 9,184
The proof and measurement of association between two things
Spearman, C. (1904) · The American Journal of Psychology · doi:10.2307/1412159 · Cited by 8,039
LARGE AREA HYDROLOGIC MODELING AND ASSESSMENT PART I: MODEL DEVELOPMENT 1
Arnold, J. G., Srinivasan, R., Muttiah, R. S., & Williams, J. R. (1998) · JAWRA Journal of the American Water Resources Association · doi:10.1111/j.1752-1688.1998.tb05961.x · Cited by 7,945
Factorial Sampling Plans for Preliminary Computational Experiments
Morris, M. D. (1991) · Technometrics · doi:10.1080/00401706.1991.10484804 · Cited by 4,943
Soil and Water Assessment Tool Theoretical Documentation Version 2009
Neitsch, S. L., Arnold, J. G., Kiniry, J. R., & Williams, J. T. (2011) · OakTrust (Texas A&M University Libraries) · Cited by 3,980
Computer Models of Watershed Hydrology
Singh, V. P. (1995) · Medical Entomology and Zoology · Cited by 3,273
Making best use of model evaluations to compute sensitivity indices
Saltelli, A. (2002) · Computer Physics Communications · doi:10.1016/s0010-4655(02)00280-1 · Cited by 2,228
An effective screening design for sensitivity analysis of large models
Campolongo, F., Cariboni, J., & Saltelli, A. (2007) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2006.10.004 · Cited by 2,011
Sensitivity analysis of environmental models: A systematic review with practical workflow
Pianosi, F., Beven, K., Freer, J., Hall, J. W., Rougier, J., Stephenson, D. B., & Wagener, T. (2016) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2016.02.008 · Cited by 1,704
Self-citationThe proof and measurement of association between two things
Spearman, C. (2010) · International Journal of Epidemiology · doi:10.1093/ije/dyq191 · Cited by 1,681
Sulla determinazione empirica di una legge di distribuzione
Kolmogorov, A. (1933) · Medical Entomology and Zoology · Cited by 1,247
An Introduction to the Bootstrap.
Kimber, A., Efron, B., & Tibshirani, R. J. (1994) · The Statistician · doi:10.2307/2348146 · Cited by 1,245
A comprehensive surface-groundwater flow model
Arnold, J. G., Allen, P. M., & Bernhardt, G. (1993) · Journal of Hydrology · doi:10.1016/0022-1694(93)90004-s · Cited by 939
Wagner, H. M. (1995) · Operations Research · doi:10.1287/opre.43.6.948 · Cited by 932
Freer, J., Beven, K., & Ambroise, B. (1996) · Water Resources Research · doi:10.1029/95wr03723 · Cited by 763
Sobol’ sensitivity analysis of a complex environmental model
Nossent, J., Elsen, P., & Bauwens, W. (2011) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2011.08.010 · Cited by 738
Duan, Q., Schaake, J., Andréassian, V., Franks, S., Goteti, G., Gupta, H. V., Gusev, Y. M., Habets, F., Hall, A., Hay, L., et al. (2006) · Journal of Hydrology · doi:10.1016/j.jhydrol.2005.07.031 · Cited by 701
Self-citationSong, X., Zhang, J., Zhan, C., Xuan, Y., Ye, M., & Xu, C. (2015) · Journal of Hydrology · doi:10.1016/j.jhydrol.2015.02.013 · Cited by 654
A Matlab toolbox for Global Sensitivity Analysis
Pianosi, F., Sarrazin, F., & Wagener, T. (2015) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2015.04.009 · Cited by 653
Self-citationSpear, R. (1980) · Water Research · doi:10.1016/0043-1354(80)90040-8 · Cited by 604
A framework for development and application of hydrological models
Wagener, T., Boyle, D. P., Lees, M. J., Wheater, H. S., Gupta, H. V., & Sorooshian, S. (2001) · Hydrology and Earth System Sciences · doi:10.5194/hess-5-13-2001 · Cited by 560
Self-citationSensitivity measures,anova-like Techniques and the use of bootstrap
Archer, G. E. B., Saltelli, A., & Sobol, I. M. (1997) · Journal of Statistical Computation and Simulation · doi:10.1080/00949659708811825 · Cited by 545
Pianosi, F., & Wagener, T. (2015) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2015.01.004 · Cited by 509
Self-citationSaltelli, A., & Tarantola, S. (2002) · Journal of the American Statistical Association · doi:10.1198/016214502388618447 · Cited by 447
Zhou, X., Lin, H. C., & Lin, H. C. (2008) · Encyclopedia of GIS · doi:10.1007/978-0-387-35973-1_538 · Cited by 435
From screening to quantitative sensitivity analysis. A unified approach
Campolongo, F., Saltelli, A., & Cariboni, J. (2011) · Computer Physics Communications · doi:10.1016/j.cpc.2010.12.039 · Cited by 431
Tang, Y., Reed, P., Wagener, T., & van Werkhoven, K. (2007) · Hydrology and Earth System Sciences · doi:10.5194/hess-11-793-2007 · Cited by 422
Self-citationContributions to the Theory of Rank Order Statistics-the Two-Sample Case
Savage, I. R. (1956) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177728170 · Cited by 343
Convergence and uncertainty analyses in Monte-Carlo based sensitivity analysis
Yang, J. (2011) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2010.10.007 · Cited by 326
Contributions to the Theory of Rank Order Statistics--The One-Sample Case
Savage, I. R. (1959) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177706084 · Cited by 297
Sensitivity analysis of an environmental model: an application of different analysis methods
Campolongo, F., & Saltelli, A. (1997) · Reliability Engineering & System Safety · doi:10.1016/s0951-8320(97)00021-5 · Cited by 292
Sorooshian, S., Gupta, V. K., & Fulton, J. L. (1983) · Water Resources Research · doi:10.1029/wr019i001p00251 · Cited by 291
Estimation of Parameter Uncertainty in the HBV Model
Seibert, J. (1997) · Hydrology research · doi:10.2166/nh.1998.15 · Cited by 269
Herman, J. D., Kollat, J. B., Reed, P. M., & Wagener, T. (2013) · Hydrology and Earth System Sciences · doi:10.5194/hess-17-2893-2013 · Cited by 237
Self-citationCharacterization of watershed model behavior across a hydroclimatic gradient
van Werkhoven, K., Wagener, T., Reed, P., & Tang, Y. (2008) · Water Resources Research · doi:10.1029/2007wr006271 · Cited by 217
Self-citationGlobal sensitivity analysis of yield output from the water productivity model
Vanuytrecht, E., Raes, D., & Willems, P. (2014) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2013.10.017 · Cited by 206
Hydrologic uncertainties in climate change from IPCC AR4 GCM simulations of the Chungju Basin, Korea
Bae, D., Jung, I., & Lettenmaier, D. P. (2011) · Journal of Hydrology · doi:10.1016/j.jhydrol.2011.02.012 · Cited by 196
Kannan, N., White, S. M., Worrall, F., & Whelan, M. J. (2007) · Journal of Hydrology · doi:10.1016/j.jhydrol.2006.08.001 · Cited by 195
Wang, J., Li, X., Lu, L., & Fang, F. (2013) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2013.06.007 · Cited by 189
An introduction to sensitivity assessment of simulation models
Norton, J. (2015) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2015.03.020 · Cited by 186
Cross-validation and bootstrapping are unreliable in small sample classification
Isaksson, A., Wallman, M., Göransson, H., & Gustafsson, M. G. (2008) · Pattern Recognition Letters · doi:10.1016/j.patrec.2008.06.018 · Cited by 186
Shin, M.-J., Guillaume, J. H. A., Croke, B. F. W., & Jakeman, A. J. (2013) · Journal of Hydrology · doi:10.1016/j.jhydrol.2013.08.047 · Cited by 184
WATER QUALITY MODELING OF ALTERNATIVE AGRICULTURAL SCENARIOS IN THE U.S. CORN BELT1
Vaché, K. B., Eilers, J. M., & Santelmann, M. V. (2002) · JAWRA Journal of the American Water Resources Association · doi:10.1111/j.1752-1688.2002.tb00996.x · Cited by 181
Sensitivity analyses of a distributed catchment model to verify the model structure
Sieber, A., & Uhlenbrook, S. (2005) · Journal of Hydrology · doi:10.1016/j.jhydrol.2005.01.004 · Cited by 169
Vrugt, J. A., Bouten, W., Gupta, H. V., & Sorooshian, S. (2002) · Water Resources Research · doi:10.1029/2001wr001118 · Cited by 168
Sin, G., Gernaey, K. V., Neumann, M. B., van Loosdrecht, M. C. M., & Gujer, W. (2011) · Water Research · doi:10.1016/j.watres.2010.08.025 · Cited by 167
A Measure of Top – Down Correlation
Iman, R. L., & Conover, W. J. (1987) · Technometrics · doi:10.1080/00401706.1987.10488244 · Cited by 166
Hartmann, A., Wagener, T., Rimmer, A., Lange, J., Brielmann, H., & Weiler, M. (2013) · Water Resources Research · doi:10.1002/wrcr.20229 · Cited by 129
Self-citationCosenza, A., Mannina, G., Vanrolleghem, P. A., & Neumann, M. B. (2013) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2013.07.009 · Cited by 121
Vanrolleghem, P. A., Mannina, G., Cosenza, A., & Neumann, M. B. (2015) · Journal of Hydrology · doi:10.1016/j.jhydrol.2014.12.056 · Cited by 108
Confalonieri, R., Bellocchi, G., Tarantola, S., Acutis, M., Donatelli, M., & Genovese, G. (2009) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2009.10.005 · Cited by 107
Identifiability of transient storage model parameters along a mountain stream
Kelleher, C. A., Wagener, T., McGlynn, B. L., Ward, A. S., Gooseff, M. N., & Payn, R. A. (2013) · Water Resources Research · doi:10.1002/wrcr.20413 · Cited by 103
Self-citationSingh, R., Wagener, T., Crane, R. G., MANN, M. E., & Ning, L. (2014) · Water Resources Research · doi:10.1002/2013wr014988 · Cited by 95
Self-citationSediment balances in the Blue Nile River Basin
Ali, Y. S. A., Crosato, A., Mohamed, Y. A., Abdalla, S. H., & Wright, N. (2014) · International Journal of Sediment Research · doi:10.1016/s1001-6279(14)60047-0 · Cited by 93
Assessment of the different sources of uncertainty in a SWAT model of the River Senne (Belgium)
Leta, O. T., Nossent, J., Vélez, C. E., Shrestha, N. K., van Griensven, A., & Bauwens, W. (2015) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2015.02.010 · Cited by 91
On the uniform asymptotic validity of subsampling and the bootstrap
Romano, J. P., & Shaikh, A. M. (2010) · Cited by 87
Nossent, J., & Bauwens, W. (2012) · Water Science & Technology · doi:10.2166/wst.2012.884 · Cited by 67
Pesticide transport simulation in a tropical catchment by SWAT
Bannwarth, M., Sangchan, W., Hugenschmidt, C., Lamers, M., Ingwersen, J., Ziegler, A. D., & Streck, T. (2014) · Environmental Pollution · doi:10.1016/j.envpol.2014.04.011 · Cited by 63
On the uniform asymptotic validity of subsampling and the bootstrap
Romano, J. P., & Shaikh, A. M. (2012) · The Annals of Statistics · doi:10.1214/12-aos1051 · Cited by 54
On Two Classes of Weighted Rank Correlation Measures Deriving from the Spearman’s ρ
Dancelli, L., Manisera, M., & Vezzoli, M. (2013) · Studies in classification, data analysis, and knowledge organization · doi:10.1007/978-3-319-00032-9_13 · Cited by 34
Statistical Models for Data Analysis
Giudici, P., Ingrassia, S., & Vichi, M. (2013) · Studies in classification, data analysis, and knowledge organization · doi:10.1007/978-3-319-00032-9 · Cited by 26
(1933) · Medical Entomology and Zoology
Cited by
Flood inundation modelling: A review of methods, recent advances and uncertainty analysis
Teng, J., Jakeman, A. J., Vaze, J., Croke, B. F. W., Dutta, D., & Kim, S. (2017) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2017.01.006 · Cited by 1,456
Metadata
- Authors
- Sarrazin, F., Pianosi, F., & Wagener, T.
- Venue
- Environmental Modelling & Software
- Publisher
- Elsevier BV
- Published
- 2016-05
- Volume
- 79
- Pages
- 135-152
- ISSN
- 1364-8152 (ISSN-L)
- OpenAlex
- W2341416175
- Cited by
- 391
- References
- 65
- FWCI
- 30.25
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.