The DOI matches its Crossref record; 320 of the 320 references it cites were found in a registry or an index.
The Future of Sensitivity Analysis: An essential discipline for systems modeling and policy support
Open accessAbstract
Sensitivity analysis (SA) is en route to becoming an integral part of mathematical modeling. The tremendous potential benefits of SA are, however, yet to be fully realized, both for advancing mechanistic and data-driven modeling of human and natural systems, and in support of decision making. In this perspective paper, a multidisciplinary group of researchers and practitioners revisit the current status of SA, and outline research challenges in regard to both theoretical frameworks and their applications to solve real-world problems. Six areas are discussed that warrant further attention, including (1) structuring and standardizing SA as a discipline, (2) realizing the untapped potential of SA for systems modeling, (3) addressing the computational burden of SA, (4) progressing SA in the context of machine learning, (5) clarifying the relationship and role of SA to uncertainty quantification, and (6) evolving the use of SA in support of decision making. An outlook for the future of SA is provided that underlines how SA must underpin a wide variety of activities to better serve science and society.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
(2008) · Journal of Machine Learning Research · Cited by 33,051
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
Multilayer feedforward networks are universal approximators
Hornik, K., Stinchcombe, M., & White, H. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90020-8 · Cited by 21,728
The FAIR Guiding Principles for scientific data management and stewardship
Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J.-W., da Silva Santos, L. B., Bourne, P. E., et al. (2016) · Scientific Data · doi:10.1038/sdata.2016.18 · Cited by 18,423
Advances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,886
Gaussian Processes for Machine Learning
Rasmussen, C. E., & Williams, C. K. I. (2005) · doi:10.7551/mitpress/3206.001.0001 · Cited by 11,705
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561
From local explanations to global understanding with explainable AI for trees
Lundberg, S. M., Erion, G., Chen, H., DeGrave, A., Prutkin, J. M., Nair, B., Katz, R., Himmelfarb, J., Bansal, N., & Lee, S.-I. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-019-0138-9 · Cited by 10,551
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Multilayer feedforward networks are universal approximators
HornikK., StinchcombeM., & WhiteH. (1989) · Neural Networks · doi:10.5555/70405.70408 · Cited by 9,081
Multivariate Adaptive Regression Splines
Friedman, J. H. (1991) · The Annals of Statistics · doi:10.1214/aos/1176347963 · Cited by 8,000
17. A Value for n-Person Games
Shapley, L. S. (1953) · Contributions to the Theory of Games (AM-28), Volume II · doi:10.1515/9781400881970-018 · Cited by 6,950
Design and Analysis of Computer Experiments
Sacks, J., Welch, W. J., Mitchell, T. J., & Wynn, H. P. (1989) · Statistical Science · doi:10.1214/ss/1177012413 · Cited by 6,919
“Why Should I Trust You?”: Explaining the Predictions of Any Classifier
Ribeiro, M., Singh, S., & Guestrin, C. (2016) · Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations · doi:10.18653/v1/n16-3020 · Cited by 6,682
Global Sensitivity Analysis. The Primer
Saltelli, A., Ratto, M., Andres, T., Campolongo, F., Cariboni, J., Gatelli, D., Saisana, M., & Tarantola, S. (2007) · doi:10.1002/9780470725184 · Cited by 6,430
Self-citationGlobal sensitivity indices for nonlinear mathematical models and their monte carlo estimates
Sobol′, I. M. (2001) · Mathematics and Computers in Simulation · doi:10.1016/s0378-4754(00)00270-6 · Cited by 6,389
Sensitivity analysis in observational research: Introducing the E-value
VanderWeele, T. J., & Ding, P. (2017) · Annals of Internal Medicine · doi:10.7326/m16-2607 · Cited by 6,350
The ASA Statement on p -Values: Context, Process, and Purpose
Wasserstein, R. L., & Lazar, N. A. (2016) · The American Statistician · doi:10.1080/00031305.2016.1154108 · Cited by 6,306 · cited 1×
“…’ (Saltelli and Funtowicz, 2017; Saltelli, 2018) where misuse or abuse of statistics (Stark and Saltelli, 2018; Gigerenzer and Marewski, 2015; Leek et al., 2017; Singh Chawla, 2017; Wasserstein and Lazar, 2016; Gigerenzer, 2018) is often cited as the root cause of the crisis (Ioannidis, 2005).”
Deep learning and process understanding for data-driven Earth system science
Reichstein, M., Camps-Valls, G., Stevens, B., Jung, M., Denzler, J., Carvalhais, N., & Prabhat (2019) · Nature · doi:10.1038/s41586-019-0912-1 · Cited by 5,673
Gaussian Processes in Machine Learning
Rasmussen, C. E. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-28650-9_4 · Cited by 5,397
Fisher, R. A. (1936) · BMJ · doi:10.1136/bmj.1.3923.554-a · Cited by 4,980
Factorial Sampling Plans for Preliminary Computational Experiments
Morris, M. D. (1991) · Technometrics · doi:10.1080/00401706.1991.10484804 · Cited by 4,943
The Wiener--Askey Polynomial Chaos for Stochastic Differential Equations
Xiu, D., & Karniadakis, G. E. (2002) · SIAM Journal on Scientific Computing · doi:10.1137/s1064827501387826 · Cited by 4,807
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 1×
“…’ (Saltelli and Funtowicz, 2017; Saltelli, 2018) where misuse or abuse of statistics (Stark and Saltelli, 2018; Gigerenzer and Marewski, 2015; Leek et al., 2017; Singh Chawla, 2017; Wasserstein and Lazar, 2016; Gigerenzer, 2018) is often cited as the root cause of the crisis (Ioannidis, 2005).”
Science for the post-normal age
Funtowicz, S. O., & Ravetz, J. R. (1993) · Futures · doi:10.1016/0016-3287(93)90022-l · Cited by 4,255
Predicting the future - big data, machine learning, and clinical medicine
Obermeyer, Z., & Emanuel, E. J. (2016) · New England Journal of Medicine · doi:10.1056/nejmp1606181 · Cited by 3,652
A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play
Silver, D., Hubert, T., Schrittwieser, J., Antonoglou, I., Lai, M., Guez, A., Lanctot, M., Sifre, L., Kumaran, D., Graepel, T., et al. (2018) · Science · doi:10.1126/science.aar6404 · Cited by 3,562 · cited 1×
“Machine Learning (ML) has achieved unprecedented performance in complex tasks typically performed by humans, such as image classification (Krizhevsky et al., 2017), natural language processing (Yang et al., 2018) and gaming (Silver et al., 2018).”
Better Bootstrap Confidence Intervals
Efron, B. (1987) · Journal of the American Statistical Association · doi:10.1080/01621459.1987.10478410 · Cited by 3,364
On the distribution of points in a cube and the approximate evaluation of integrals
Sobol', I. M. (1967) · USSR Computational Mathematics and Mathematical Physics · doi:10.1016/0041-5553(67)90144-9 · Cited by 2,975
Global sensitivity analysis using polynomial chaos expansions
Sudret, B. (2008) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2007.04.002 · Cited by 2,539
Explaining prediction models and individual predictions with feature contributions
Štrumbelj, E., & Kononenko, I. (2014) · Knowledge and Information Systems · doi:10.1007/s10115-013-0679-x · Cited by 2,286
Making best use of model evaluations to compute sensitivity indices
Saltelli, A. (2002) · Computer Physics Communications · doi:10.1016/s0010-4655(02)00280-1 · Cited by 2,228
Self-citationAvoidable waste in the production and reporting of research evidence
Chalmers, I., & Glasziou, P. (2009) · The Lancet · doi:10.1016/s0140-6736(09)60329-9 · Cited by 2,208
The Design and Analysis of Computer Experiments
Santner, T. J., Williams, B. J., & Notz, W. I. (2003) · Springer Series in Statistics · doi:10.1007/978-1-4757-3799-8 · Cited by 2,170
A Taxonomy of Global Optimization Methods Based on Response Surfaces
Jones, D. R. (2001) · Journal of Global Optimization · doi:10.1023/a:1012771025575 · Cited by 2,159
Burgess, S., Bowden, J., Fall, T., Ingelsson, E., & Thompson, S. G. (2017) · Epidemiology · doi:10.1097/ede.0000000000000559 · Cited by 2,017
Importance measures in global sensitivity analysis of nonlinear models
Homma, T., & Saltelli, A. (1996) · Reliability Engineering & System Safety · doi:10.1016/0951-8320(96)00002-6 · Cited by 2,011
Self-citationAn effective screening design for sensitivity analysis of large models
Campolongo, F., Cariboni, J., & Saltelli, A. (2007) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2006.10.004 · Cited by 2,011
Self-citationA review of techniques for parameter sensitivity analysis of environmental models
Hamby, D. M. (1994) · Environmental Monitoring and Assessment · doi:10.1007/bf00547132 · Cited by 1,952
International encyclopedia of the social & behavioral sciences
(2006) · Choice Reviews Online · doi:10.5860/choice.43sup-0452 · Cited by 1,947
A Class of Statistics with Asymptotically Normal Distribution
Hoeffding, W. (1948) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177730196 · Cited by 1,928
Sensitivity Analysis of k-Fold Cross Validation in Prediction Error Estimation
Rodriguez, J. D., Perez, A., & Lozano, J. A. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.187 · Cited by 1,890 · cited 1×
“Such mathematical models can be data-driven (also called statistical), directly mapping inputs to outputs (Engelbrecht et al., 1995; Rodriguez et al., 2010), or mechanistic (also called process-based), solving a set of differential or other mathematical equations governing the (possibly)…”
A Class of Statistics with Asymptotically Normal Distribution
Hoeffding, W. (1992) · Springer Series in Statistics · doi:10.1007/978-1-4612-0919-5_20 · Cited by 1,851
Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning)
Rasmussen, C. E., & Williams, C. K. I. (2005) · The MIT Press eBooks · Cited by 1,829
Sensitivity analysis of environmental models: A systematic review with practical workflow
Pianosi, F., Beven, K., Freer, J., Hall, J. W., Rougier, J., Stephenson, D. B., & Wagener, T. (2016) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2016.02.008 · Cited by 1,704
Landscape Ecology in Theory and Practice: Pattern and Process
Turner, M. G., & Gardner, R. H. (2015) · Thammasat University Digital Collections · doi:10.1007/978-1-4939-2794-4 · Cited by 1,614
Uncertainty and Quality in Science for Policy
Funtowicz, S. O., & Ravetz, J. R. (1990) · doi:10.1007/978-94-009-0621-1 · Cited by 1,575
Measuring Statistical Dependence with Hilbert-Schmidt Norms
Gretton, A., Bousquet, O., Smola, A., & Schölkopf, B. (2005) · Lecture Notes in Computer Science · doi:10.1007/11564089_7 · Cited by 1,560
SALib: An open-source Python library for Sensitivity Analysis
Herman, J., & Usher, W. (2017) · The Journal of Open Source Software · doi:10.21105/joss.00097 · Cited by 1,524
Adaptive Computation and Machine Learning
(2012) · Boosting · doi:10.7551/mitpress/8291.003.0026 · Cited by 1,402
Explainable AI: Interpreting, Explaining and Visualizing Deep Learning
(2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-28954-6 · Cited by 1,377
Shaping the Next One Hundred Years: New Methods for Quantitative, Long-Term Policy Analysis
Lempert, R., Popper, S., & Bankes, S. (2003) · RAND Corporation eBooks · doi:10.7249/mr1626 · Cited by 1,244
How much complexity is warranted in a rainfall‐runoff model?
Jakeman, A. J., & Hornberger, G. M. (1993) · Water Resources Research · doi:10.1029/93wr00877 · Cited by 1,216
Self-citationSensitivity analysis: A review of recent advances
Borgonovo, E., & Plischke, E. (2016) · European Journal of Operational Research · doi:10.1016/j.ejor.2015.06.032 · Cited by 1,167
Self-citationDesign and Modeling for Computer Experiments
Fang, K.-T., Li, R., & Sudjianto, A. (2005) · Chapman & Hall/CRC eBooks · doi:10.1201/9781420034899 · Cited by 1,138
Ten iterative steps in development and evaluation of environmental models
Jakeman, A. J., Letcher, R. A., & Norton, J. P. (2006) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2006.01.004 · Cited by 1,111
Self-citationInterpretable Machine Learning
Molnar, C. (2020) · Cited by 1,104
Probabilistic Sensitivity Analysis of Complex Models: A Bayesian Approach
Oakley, J. E., & O'Hagan, A. (2004) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2004.05304.x · Cited by 1,085
Heaton, J. (2018) · Genetic Programming and Evolvable Machines · doi:10.1007/s10710-017-9314-z · Cited by 1,054
Study of the sensitivity of coupled reaction systems to uncertainties in rate coefficients. I Theory
Cukier, R. I., Fortuin, C. M., Shuler, K. E., Petschek, A. G., & Schaibly, J. H. (1973) · The Journal of Chemical Physics · doi:10.1063/1.1680571 · Cited by 1,046
Explainable machine learning for scientific insights and discoveries
Roscher, R., Bohn, B., Duarte, M. F., & Garcke, J. (2020) · IEEE Access · doi:10.1109/access.2020.2976199 · Cited by 1,040 · cited 1×
“These two terms, usually used interchangeably in the literature, refer to the ability of a model developer to make sense of why the model functions the way it does and to explain that to a user (Rudin, 2019; Samek and Müller, 2019; Roscher et al., 2020).”
Linear Programming and Economic Analysis
Henderson, J. S., & Dorfman, R. (1958) · Southern Economic Journal · doi:10.2307/1055268 · Cited by 1,033
A new uncertainty importance measure
Borgonovo, E. (2007) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2006.04.015 · Cited by 1,026
Self-citationQuickCheck: a lightweight tool for random testing of Haskell programs
Claessen, K., & Hughes, J. (2011) · ACM SIGPLAN Notices · doi:10.1145/1988042.1988046 · Cited by 1,011
A Review on Global Sensitivity Analysis Methods
Iooss, B., & Lemaître, P. (2015) · Operations Research/Computer Science Interfaces Series · doi:10.1007/978-1-4899-7547-8_5 · Cited by 994
Self-citationReview of surrogate modeling in water resources
Razavi, S., Tolson, B. A., & Burn, D. H. (2012) · Water Resources Research · doi:10.1029/2011wr011527 · Cited by 983
Self-citationHow to avoid a perfunctory sensitivity analysis
Saltelli, A., & Annoni, P. (2010) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2010.04.012 · Cited by 974
Self-citationInterpretable machine learning
Parliamentary Office of Science and Technology, & Christie, L. (2020) · doi:10.58248/pn633 · Cited by 952
(2020) · doi:10.1109/sc41405.2020.00024 · Cited by 928
UQLab: A Framework for Uncertainty Quantification in Matlab
Marelli, S., & Sudret, B. (2014) · Vulnerability, Uncertainty, and Risk · doi:10.1061/9780784413609.257 · Cited by 899
Maier, H. R., Jain, A., Dandy, G. C., & Sudheer, K. P. (2010) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2010.02.003 · Cited by 871
Self-citationThe natural selection of bad science
Smaldino, P. E., & McElreath, R. (2016) · Royal Society Open Science · doi:10.1098/rsos.160384 · Cited by 860 · cited 1×
“Current non-reproducible science is ecologically fit to the existing science governance arrangements (Smaldino and McElreath, 2016), including its ‘publish or perish ’ culture (Banobi et al., 2011), and is resistent to reform (Chalmers and Glasziou, 2009; Edwards and Roy, 2017).”
Saltelli, A., Aleksankina, K., Becker, W., Fennell, P., Ferretti, F., Holst, N., Li, S., & Wu, Q. (2019) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2019.01.012 · Cited by 813
Self-citationAn investigation of the false discovery rate and the misinterpretation of p -values
Colquhoun, D. (2014) · Royal Society Open Science · doi:10.1098/rsos.140216 · Cited by 792
Dimension Reduction by Local Principal Component Analysis
Kambhatla, N., & Leen, T. K. (1997) · Neural Computation · doi:10.1162/neco.1997.9.7.1493 · Cited by 717
Application of neural networks to modelling nonlinear relationships in ecology
Lek, S., Delacoste, M., Baran, P., Dimopoulos, I., Lauga, J., & Aulagnier, S. (1996) · Ecological Modelling · doi:10.1016/0304-3800(95)00142-5 · Cited by 716
Open Science now: A systematic literature review for an integrated definition
Vicente-Saez, R., & Martinez-Fuentes, C. (2018) · Journal of Business Research · doi:10.1016/j.jbusres.2017.12.043 · Cited by 707
Kidwell, M. C., Lazarević, L. B., Baranski, E., Hardwicke, T. E., Piechowski, S., Falkenberg, L.-S., Kennett, C., Slowik, A., Sonnleitner, C., Hess-Holden, C., et al. (2016) · PLOS Biology · doi:10.1371/journal.pbio.1002456 · Cited by 700
Nonlinear sensitivity analysis of multiparameter model systems
Cukier, R. I., Levine, H. B., & Shuler, K. E. (1978) · Journal of Computational Physics · doi:10.1016/0021-9991(78)90097-9 · Cited by 694
Edwards, M. A., & Roy, S. (2017) · Environmental Engineering Science · doi:10.1089/ees.2016.0223 · Cited by 688
Maier, H. R., Kapelan, Z., Kasprzyk, J., Kollat, J., Matott, L. S., Cunha, M. C., Dandy, G. C., Gibbs, M. S., Keedwell, E., Marchi, A., et al. (2014) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2014.09.013 · Cited by 657
Self-citationA Matlab toolbox for Global Sensitivity Analysis
Pianosi, F., Sarrazin, F., & Wagener, T. (2015) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2015.04.009 · Cited by 653
Analysis of variance designs for model output
Jansen, M. J. W. (1999) · Computer Physics Communications · doi:10.1016/s0010-4655(98)00154-4 · Cited by 650
Denning, P. J., Comer, D. E., Gries, D., Mulder, M. C., Tucker, A., Turner, A. J., & Young, P. R. (1989) · Computer · doi:10.1109/2.19833 · Cited by 635
Sensitivity Anaysis as an Ingredient of Modeling
Campolongo, F., Saltelli, A., & Tarantola, S. (2000) · Statistical Science · doi:10.1214/ss/1009213004 · Cited by 606
Self-citationSpear, R. (1980) · Water Research · doi:10.1016/0043-1354(80)90040-8 · Cited by 604
Decision Making under Deep Uncertainty: From Theory to Practice
(2019) · Springer eBooks · doi:10.1007/978-3-030-05252-2 · Cited by 599
Wagener, T., McIntyre, N., Lees, M. J., Wheater, H. S., & Gupta, H. V. (2003) · Hydrological Processes · doi:10.1002/hyp.1135 · Cited by 591
Self-citationHandbook of Uncertainty Quantification
(2016) · Springer eBooks · doi:10.1007/978-3-319-11259-6 · Cited by 581
Towards Explainable Artificial Intelligence
Samek, W., & Müller, K.-R. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-28954-6_1 · Cited by 580
Deep learning for lung cancer prognostication: A retrospective multi-cohort radiomics study
Hosny, A., Parmar, C., Coroller, T. P., Grossmann, P., Zeleznik, R., Kumar, A., Bussink, J., Gillies, R. J., Mak, R. H., & Aerts, H. J. W. L. (2018) · PLOS Medicine · doi:10.1371/journal.pmed.1002711 · Cited by 579
An Analysis for Unreplicated Fractional Factorials
Box, G. E. P., & Meyer, R. D. (1986) · Technometrics · doi:10.1080/00401706.1986.10488093 · Cited by 576
AN APPROACH TO THE PRELIMINARY ANALYSIS OF ENVIRONMENTAL SYSTEMS
Hornberger, G. M., & Spear, R. C. (1981) · Cited by 566
Journal of Open Source Software
(2016) · The Journal of Open Source Software · doi:10.21105/joss · Cited by 552
Variable importance analysis: A comprehensive review
Wei, P., Lu, Z., & Song, J. (2015) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2015.05.018 · Cited by 525
Pianosi, F., & Wagener, T. (2015) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2015.01.004 · Cited by 509
Maier, H. R., Guillaume, J. H. A., van Delden, H., Riddell, G. A., Haasnoot, M., & Kwakkel, J. H. (2016) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2016.03.014 · Cited by 490
Self-citationFive ways to ensure that models serve society: a manifesto
Saltelli, A., Bammer, G., Bruno, I., Charters, E., Di Fiore, M., Didier, E., Nelson Espeland, W., Kay, J., Lo Piano, S., Mayo, D., et al. (2020) · Nature · doi:10.1038/d41586-020-01812-9 · Cited by 486
Self-citationDevelopment of a Coupled Land Surface and Groundwater Model
Maxwell, R. M., & Miller, N. L. (2005) · Journal of Hydrometeorology · doi:10.1175/jhm422.1 · Cited by 471
Saltelli, A., & Tarantola, S. (2002) · Journal of the American Statistical Association · doi:10.1198/016214502388618447 · Cited by 447
Self-citationDerivative based global sensitivity measures and their link with global sensitivity indices
Sobol’, I. M., & Kucherenko, S. (2009) · Mathematics and Computers in Simulation · doi:10.1016/j.matcom.2009.01.023 · Cited by 445
Self-citationFrom screening to quantitative sensitivity analysis. A unified approach
Campolongo, F., Saltelli, A., & Cariboni, J. (2011) · Computer Physics Communications · doi:10.1016/j.cpc.2010.12.039 · Cited by 431
Self-citationGlobal sensitivity measures from given data
Plischke, E., Borgonovo, E., & Smith, C. L. (2013) · European Journal of Operational Research · doi:10.1016/j.ejor.2012.11.047 · Cited by 429
Self-citationVan Der Sluijs, J. P., Craye, M., Funtowicz, S., Kloprogge, P., Ravetz, J., & Risbey, J. (2005) · Risk Analysis · doi:10.1111/j.1539-6924.2005.00604.x · Cited by 426
Tang, Y., Reed, P., Wagener, T., & van Werkhoven, K. (2007) · Hydrology and Earth System Sciences · doi:10.5194/hess-11-793-2007 · Cited by 422
Dilettantism in hydrology: Transition or destiny?
Klemeš, V. (1986) · Water Resources Research · doi:10.1029/wr022i09sp0177s · Cited by 407
Design of computer experiments: space filling and beyond
Pronzato, L., & Müller, W. G. (2011) · Statistics and Computing · doi:10.1007/s11222-011-9242-3 · Cited by 396
Estimating the approximation error when fixing unessential factors in global sensitivity analysis
Sobol’, I. M., Tarantola, S., Gatelli, D., Kucherenko, S. S., & Mauntz, W. (2007) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2006.07.001 · Cited by 394
Self-citationGlobal Sensitivity Analysis of environmental models: Convergence and validation
Sarrazin, F., Pianosi, F., & Wagener, T. (2016) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2016.02.005 · Cited by 391
A formal framework for scenario development in support of environmental decision-making
Mahmoud, M., Liu, Y., Hartmann, H., Stewart, S., Wagener, T., Semmens, D., Stewart, R., Gupta, H., Dominguez, D., Dominguez, F., et al. (2009) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2008.11.010 · Cited by 378
Self-citationRazavi, S., & Gupta, H. V. (2015) · Water Resources Research · doi:10.1002/2014wr016527 · Cited by 372
Self-citationHow Should Robustness Be Defined for Water Systems Planning under Change?
Herman, J. D., Reed, P. M., Zeff, H. B., & Characklis, G. W. (2015) · Journal of Water Resources Planning and Management · doi:10.1061/(asce)wr.1943-5452.0000509 · Cited by 371
An empirical analysis of journal policy effectiveness for computational reproducibility
Stodden, V., Seiler, J., & Ma, Z. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1708290115 · Cited by 369
Prediction: science, decision making, and the future of nature
(2001) · Choice Reviews Online · doi:10.5860/choice.38-2705 · Cited by 363
Gan, Y., Duan, Q., Gong, W., Tong, C., Sun, Y., Chu, W., Ye, A., Miao, C., & Di, Z. (2014) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2013.09.031 · Cited by 360
Self-citationEstimation of global sensitivity indices for models with dependent variables
Kucherenko, S., Tarantola, S., & Annoni, P. (2012) · Computer Physics Communications · doi:10.1016/j.cpc.2011.12.020 · Cited by 347
Self-citationScenario analysis in environmental impact assessment: Improving explorations of the future
Duinker, P. N., & Greig, L. A. (2007) · Environmental Impact Assessment Review · doi:10.1016/j.eiar.2006.11.001 · Cited by 331
Wu, W., Dandy, G. C., & Maier, H. R. (2014) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2013.12.016 · Cited by 330
Self-citationCalculations of Sobol indices for the Gaussian process metamodel
Marrel, A., Iooss, B., Laurent, B., & Roustant, O. (2009) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2008.07.008 · Cited by 328
Self-citationThe Complexities of Agent-Based Modeling Output Analysis
Lee, J.-S., Filatova, T., Ligmann-Zielinska, A., Hassani-Mahmooei, B., Stonedahl, F., Lorscheid, I., Voinov, A., Polhill, G., Sun, Z., & Parker, D. C. (2015) · Journal of Artificial Societies and Social Simulation · doi:10.18564/jasss.2897 · Cited by 324
Guo, L., Meng, Z., Sun, Y., & Wang, L. (2016) · Energy Conversion and Management · doi:10.1016/j.enconman.2015.11.041 · Cited by 297
Kernel techniques: From machine learning to meshless methods
Schaback, R., & Wendland, H. (2006) · Acta Numerica · doi:10.1017/s0962492906270016 · Cited by 289
Legacy, Rather Than Adequacy, Drives the Selection of Hydrological Models
Addor, N., & Melsen, L. A. (2019) · Water Resources Research · doi:10.1029/2018wr022958 · Cited by 286
Useless arithmetic: why environmental scientists can't predict the future
(2007) · Choice Reviews Online · doi:10.5860/choice.44-6814 · Cited by 284
How do scientists develop and use scientific software?
Hannay, J. E., MacLeod, C., Singer, J., Langtangen, H. P., Pfahl, D., & Wilson, G. (2009) · 2009 ICSE Workshop on Software Engineering for Computational Science and Engineering · doi:10.1109/secse.2009.5069155 · Cited by 275
Introductory overview: Optimization using evolutionary algorithms and other metaheuristics
Maier, H. R., Razavi, S., Kapelan, Z., Matott, L. S., Kasprzyk, J., & Tolson, B. A. (2019) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2018.11.018 · Cited by 272
Self-citationRelative Entropy Based Method for Probabilistic Sensitivity Analysis in Engineering Design
Liu, H., Chen, W., & Sudjianto, A. (2006) · Journal of Mechanical Design · doi:10.1115/1.2159025 · Cited by 270
Uncertainty and sensitivity analysis for models with correlated parameters
Xu, C., & Gertner, G. Z. (2008) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2007.06.003 · Cited by 270
Use of some sensitivity criteria for choosing networks with good generalization ability
Dimopoulos, Y., Bourret, P., & Lek, S. (1995) · Neural Processing Letters · doi:10.1007/bf02309007 · Cited by 267
Sobol' Indices and Shapley Value
Owen, A. B. (2014) · SIAM/ASA Journal on Uncertainty Quantification · doi:10.1137/130936233 · Cited by 266
Statistical Rituals: The Replication Delusion and How We Got There
Gigerenzer, G. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245918771329 · Cited by 263
A hybrid regularized inversion methodology for highly parameterized environmental models
Tonkin, M. J., & Doherty, J. (2005) · Water Resources Research · doi:10.1029/2005wr003995 · Cited by 255
Sheikholeslami, R., & Razavi, S. (2017) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2017.03.010 · Cited by 255
Self-citationMcPhail, C., Maier, H. R., Kwakkel, J. H., Giuliani, M., Castelletti, A., & Westra, S. (2018) · Earth's Future · doi:10.1002/2017ef000649 · Cited by 251
Self-citationAsymptotic normality and efficiency of two Sobol index estimators
Janon, A., Klein, T., Lagnoux, A., Nodet, M., & Prieur, C. (2014) · ESAIM: Probability and Statistics · doi:10.1051/ps/2013040 · Cited by 245
Self-citationTrends in sensitivity analysis practice in the last decade
Ferretti, F., Saltelli, A., & Tarantola, S. (2016) · Science of The Total Environment · doi:10.1016/j.scitotenv.2016.02.133 · Cited by 241
Self-citationActive Subspaces: Emerging Ideas for Dimension Reduction in Parameter Studies
Constantine, P. G. (2015) · CERN Document Server (European Organization for Nuclear Research) · Cited by 241
Searching for important factors in simulation models with many factors: Sequential bifurcation
Bettonvil, B., & Kleijnen, J. P. C. (1997) · European Journal of Operational Research · doi:10.1016/s0377-2217(96)00156-7 · Cited by 239
Hutton, C., Wagener, T., Freer, J., Han, D., Duffy, C., & Arheimer, B. (2016) · Water Resources Research · doi:10.1002/2016wr019285 · Cited by 233
Surrogate Science: The Idol of a Universal Method for Scientific Inference
Gigerenzer, G., & Marewski, J. N. (2015) · Journal of Management · doi:10.1177/0149206314547522 · Cited by 215
The future shocks: On discontinuity and scenario development
van Notten, P. W. F., Sleegers, A. M., & van Asselt, M. B. A. (2005) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2003.12.003 · Cited by 213
Construction and Comparison of High-Dimensional Sobol' Generators
Sobol, I. M., Asotsky, D., Kreinin, A., & Kucherenko, S. S. (2011) · Wilmott · doi:10.1002/wilm.10056 · Cited by 208
Self-citationAn evaluation framework for input variable selection algorithms for environmental data-driven models
Galelli, S., Humphrey, G. B., Maier, H. R., Castelletti, A., Dandy, G. C., & Gibbs, M. S. (2014) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2014.08.015 · Cited by 207
Self-citationEight grand challenges in socio-environmental systems modeling
Elsawah, S., Filatova, T., Jakeman, A. J., Kettner, A. J., Zellner, M. L., Athanasiadis, I. N., Hamilton, S. H., Axtell, R. L., Brown, D. G., Gilligan, J. M., et al. (2020) · Socio-Environmental Systems Modelling · doi:10.18174/sesmo.2020a16226 · Cited by 205
A new variable importance measure for random forests with missing data
Hapfelmeier, A., Hothorn, T., Ulm, K., & Strobl, C. (2014) · Statistics and Computing · doi:10.1007/s11222-012-9349-1 · Cited by 205
A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Theory
Razavi, S., & Gupta, H. V. (2016) · Water Resources Research · doi:10.1002/2015wr017558 · Cited by 204
Self-citationGuillaume, J. H. A., Jakeman, J. D., Marsili-Libelli, S., Asher, M., Brunner, P., Croke, B., Hill, M. C., Jakeman, A. J., Keesman, K. J., Razavi, S., et al. (2019) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2019.07.007 · Cited by 199
Self-citationGlobal sensitivity analysis with dependence measures
Da Veiga, S. (2015) · Journal of Statistical Computation and Simulation · doi:10.1080/00949655.2014.945932 · Cited by 195
Khorashadi Zadeh, F., Nossent, J., Sarrazin, F., Pianosi, F., van Griensven, A., Wagener, T., & Bauwens, W. (2017) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2017.02.001 · Cited by 191
Adams, B., Ebeida, M., Eldred, M., Jakeman, J., Swiler, L., Stephens, J., Vigil, D., Wildey, T., Bohnhoff, W., Eddy, J., et al. (2014) · doi:10.2172/1177048 · Cited by 188
Self-citationWhat is science’s crisis really about?
Saltelli, A., & Funtowicz, S. (2017) · Futures · doi:10.1016/j.futures.2017.05.010 · Cited by 187
Self-citationAn introduction to sensitivity assessment of simulation models
Norton, J. (2015) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2015.03.020 · Cited by 186
Shin, M.-J., Guillaume, J. H. A., Croke, B. F. W., & Jakeman, A. J. (2013) · Journal of Hydrology · doi:10.1016/j.jhydrol.2013.08.047 · Cited by 184
Self-citationKucherenko, S. S., Albrecht, D., & Saltelli, A. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.02350 · Cited by 180
Self-citationSensitivity Analysis and Performance Extrapolation for Computer Simulation Models
Rubinstein, R. Y. (1989) · Operations Research · doi:10.1287/opre.37.1.72 · Cited by 179
Kavetski, D., & Clark, M. P. (2010) · Water Resources Research · doi:10.1029/2009wr008896 · Cited by 176
Parameter uncertainty and interaction in complex environmental models
Spear, R. C., Grieb, T. M., & Shang, N. (1994) · Water Resources Research · doi:10.1029/94wr01732 · Cited by 174
Are we unnecessarily constraining the agility of complex process-based models?
Mendoza, P. A., Clark, M. P., Barlage, M., Rajagopalan, B., Samaniego, L., Abramowitz, G., & Gupta, H. (2015) · Water Resources Research · doi:10.1002/2014wr015820 · Cited by 174
Self-citationDistributed Evaluation of Local Sensitivity Analysis (DELSA), with application to hydrologic models
Rakovec, O., Hill, M. C., Clark, M. P., Weerts, A. H., Teuling, A. J., & Uijlenhoet, R. (2014) · Water Resources Research · doi:10.1002/2013wr014063 · Cited by 171
On Shapley Value for Measuring Importance of Dependent Inputs
Owen, A. B., & Prieur, C. (2017) · SIAM/ASA Journal on Uncertainty Quantification · doi:10.1137/16m1097717 · Cited by 169
Self-citationMcCarthy, E. (2005) · CFA Institute Magazine · doi:10.2469/cfm.v16.n4.2942 · Cited by 163
A new framework for comprehensive, robust, and efficient global sensitivity analysis: 2. Application
Razavi, S., & Gupta, H. V. (2015) · Water Resources Research · doi:10.1002/2015wr017559 · Cited by 163
Self-citationDerivative-based global sensitivity measures: General links with Sobol’ indices and numerical tests
Lamboni, M., Iooss, B., Popelin, A.-L., & Gamboa, F. (2013) · Mathematics and Computers in Simulation · doi:10.1016/j.matcom.2013.02.002 · Cited by 162
Self-citationDistribution-based sensitivity analysis from a generic input-output sample
Pianosi, F., & Wagener, T. (2018) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2018.07.019 · Cited by 162
Herman, J. D., Reed, P. M., & Wagener, T. (2013) · Water Resources Research · doi:10.1002/wrcr.20124 · Cited by 161
Gamboa, F., Hal, H. I., & Chastaing, G. (2015) · Cited by 159
Dimopoulos, I., Chronopoulos, J., Chronopoulou-Sereli, A., & Lek, S. (1999) · Ecological Modelling · doi:10.1016/s0304-3800(99)00099-x · Cited by 158
Elsawah, S., Hamilton, S. H., Jakeman, A. J., Rothman, D. S., Schweizer, V. J., Trutnevyte, E., Carlsen, H., Drakes, C., Frame, B., Fu, B., et al. (2020) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2020.138393 · Cited by 156
Self-citationZhang, P. (2019) · Applied Soft Computing · doi:10.1016/j.asoc.2019.105859 · Cited by 155
Estimating Quantile Sensitivities
Hong, L. J. (2008) · Operations Research · doi:10.1287/opre.1080.0531 · Cited by 153
The identification of model effective dimensions using global sensitivity analysis
Kucherenko, S. S., Feil, B., Shah, N., & Mauntz, W. (2010) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2010.11.003 · Cited by 147
Self-citationGramacy, R. B., & Taddy, M. A. (2010) · Journal of Statistical Software · doi:10.18637/jss.v033.i06 · Cited by 144
Identifying wildland fire ignition factors through sensitivity analysis of a neural network
Vasilakos, C., Kalabokidis, K. D., Hatzopoulos, J. N., & Matsinos, I. (2008) · Natural Hazards · doi:10.1007/s11069-008-9326-3 · Cited by 143
Simulating Sensitivities of Conditional Value at Risk
Hong, L. J., & Liu, G. (2008) · Management Science · doi:10.1287/mnsc.1080.0901 · Cited by 143
An effective algorithm for computing global sensitivity indices (EASI)
Plischke, E. (2009) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2009.11.005 · Cited by 143
Self-citationA New Formulation for Feedforward Neural Networks
Razavi, S., & Tolson, B. A. (2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2011.2163169 · Cited by 134
Self-citationQuestioning the Ecological Footprint
Galli, A., Giampietro, M., Goldfinger, S., Lazarus, E., Lin, D. H., Saltelli, A., Wackernagel, M., & Müller, F. (2016) · Ecological Indicators · doi:10.1016/j.ecolind.2016.04.014 · Cited by 133
Self-citationLocal Polynomial Estimation for Sensitivity Analysis on Models With Correlated Inputs
da Veiga, S., Wahl, F., & Gamboa, F. (2009) · Technometrics · doi:10.1198/tech.2009.08124 · Cited by 132
4 OPTIMAL LOW-RANK APPROXIMATIONS OF BAYESIAN LINEAR INVERSE PROBLEMSPreprint
(n.d.) · Cited by 131
Sensitivity of projected long-term CO2 emissions across the Shared Socioeconomic Pathways
Marangoni, G., Tavoni, M., Bosetti, V., Borgonovo, E., Capros, P., Fricko, O., Gernaat, D., Guivarch, C., Havlík, P., Huppmann, D., et al. (2017) · Nature Climate Change · doi:10.1038/nclimate3199 · Cited by 126
Self-citationThe Value of Information and Stochastic Programming
Avriel, M., & Williams, A. C. (1970) · Operations Research · doi:10.1287/opre.18.5.947 · Cited by 124
Certain trends in uncertainty and sensitivity analysis: An overview of software tools and techniques
Douglas-Smith, D., Iwanaga, T., Croke, B., & Jakeman, A. J. (2019) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2019.104588 · Cited by 123
Self-citationSensitivity analysis in economic evaluation: A review of published studies
Briggs, A. H., & Sculpher, M. J. (1995) · Health Economics · doi:10.1002/hec.4730040502 · Cited by 122
A Common Rationale for Global Sensitivity Measures and Their Estimation
Borgonovo, E., Hazen, G. B., & Plischke, E. (2016) · Risk Analysis · doi:10.1111/risa.12555 · Cited by 122
Self-citationGlobal Stock Market Prediction Based on Stock Chart Images Using Deep Q-Network
Lee, J., Kim, R., Koh, Y., & Kang, J. (2019) · IEEE Access · doi:10.1109/access.2019.2953542 · Cited by 121
Feltham, G. A. (1968) · The Accounting Review · doi:10.2308/tar-4482105 · Cited by 117
Shaping the Next One Hundred Years: New Methods for Quantitative, Long-Term Policy Analysis
Linstone, H. A. (2003) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2003.09.006 · Cited by 116
VARS-TOOL: A toolbox for comprehensive, efficient, and robust sensitivity and uncertainty analysis
Razavi, S., Sheikholeslami, R., Gupta, H. V., & Haghnegahdar, A. (2018) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2018.10.005 · Cited by 115
Self-citationManaging Structural Uncertainty in Health Economic Decision Models: A Discrepancy Approach
Strong, M., Oakley, J. E., & Chilcott, J. B. (2011) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.1111/j.1467-9876.2011.01014.x · Cited by 112
Wagener, T., & Pianosi, F. (2019) · Earth-Science Reviews · doi:10.1016/j.earscirev.2019.04.006 · Cited by 110
A Bayesian Approach for Global Sensitivity Analysis of (Multifidelity) Computer Codes
Gratiet, L. L., Cannamela, C., & Iooss, B. (2014) · SIAM/ASA Journal on Uncertainty Quantification · doi:10.1137/130926869 · Cited by 108
Self-citationKleijnen, J. P. C. (1995) · System Dynamics Review · doi:10.1002/sdr.4260110403 · Cited by 108
A short comment on statistical versus mathematical modelling
Saltelli, A. (2019) · Nature Communications · doi:10.1038/s41467-019-11865-8 · Cited by 108
Self-citationOn the General Theory of Skew Correlation and Non-Linear Regression
Pearson, K. (2010) · Medical Entomology and Zoology · Cited by 106
Cited by
Toward SALib 2.0: Advancing the accessibility and interpretability of global sensitivity analyses
Iwanaga, T., Usher, W., & Herman, J. (2022) · Socio-Environmental Systems Modelling · doi:10.18174/sesmo.18155 · Cited by 367
Self-citationLafferty, D. C., & Sriver, R. L. (2023) · npj Climate and Atmospheric Science · doi:10.1038/s41612-023-00486-0 · Cited by 139
Sivanesan, U., Wu, K., McInnes, M. D. F., Dhindsa, K., Salehi, F., & van der Pol, C. B. (2023) · Canadian Association of Radiologists Journal · doi:10.1177/08465371221134056 · Cited by 19
Time-integrated radiation risk metrics and interpopulation variability of survival
Ulanowski, A., Ban, N., Ozasa, K., Rühm, W., Semones, E., Shavers, M., & Vaillant, L. (2024) · Zeitschrift für Medizinische Physik · doi:10.1016/j.zemedi.2023.08.002 · Cited by 4
Metadata
- Authors
- Razavi, S., Jakeman, A., Saltelli, A., Prieur, C., Iooss, B., Borgonovo, E., Plischke, E., Lo Piano, S., Iwanaga, T., Becker, W., Tarantola, S., Guillaume, J. H. A., Jakeman, J., Gupta, H., Melillo, N., Rabitti, G., Chabridon, V., Duan, Q., Sun, X., Smith, S., Sheikholeslami, R., Hosseini, N., Asadzadeh, M., Puy, A., Kucherenko, S., & Maier, H. R.
- Venue
- Environmental Modelling & Software
- Publisher
- Elsevier BV
- Published
- 2021-03
- Volume
- 137
- Pages
- 104954
- ISSN
- 1364-8152 (ISSN-L)
- OpenAlex
- W3112134271
- Cited by
- 628
- References
- 320
- FWCI
- 38.58
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.