Combining Quantitative and Qualitative Measures of Uncertainty in Model‐Based Environmental Assessment: The NUSAP System
Open accessAbstract
This article discusses recent experiences with the Numeral Unit Spread Assessment Pedigree (NUSAP) system for multidimensional uncertainty assessment, based on four case studies that vary in complexity. We show that the NUSAP method is applicable not only to relatively simple calculation schemes but also to complex models in a meaningful way and that NUSAP is useful to assess not only parameter uncertainty but also (model) assumptions. A diagnostic diagram can be used to synthesize results of quantitative analysis of parameter sensitivity and qualitative review (pedigree analysis) of parameter strength. It provides an analytic tool to prioritize uncertainties according to quantitative and qualitative insights in the limitations of available knowledge. We show that extension of the pedigree scheme to include societal dimensions of uncertainty, such as problem framing and value‐laden assumptions, further promotes reflexivity and collective learning. When used in a deliberative setting, NUSAP pedigree assessment has the potential to foster a deeper social debate and a negotiated management of complex environmental problems.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Tools of post-normal-science inspiration for knowledge and modelling quality assessment could be adapted to the analysis of algorithms, such as the NUSAP notation system and sensitivity auditing.
“Tools of post-normal-science inspiration for knowledge and modelling quality assessment could be adapted to the analysis of algorithms, such as the NUSAP (Numeral Unit Spread Assessment Pedigree) notation system for the management and communication of uncertainty (Funtowicz and Ravetz, 1990;Van Der Sluijs, 2005) and sensitivity auditing (Saltelli and Funtowicz, 2014), respectively.”
From Ethical principles in machine learning and artificial intelligence: cases from the field and possible ways forward (2020) · cites this paper as Van Der Sluijs, 2005
Cites
Toulmin, S. E. (2003) · doi:10.1017/cbo9780511840005 · Cited by 7,101 · see citing works
Factorial Sampling Plans for Preliminary Computational Experiments
Morris, M. D. (1991) · Technometrics · doi:10.1080/00401706.1991.10484804 · Cited by 4,943 · see citing works
Science for the post-normal age
Funtowicz, S. O., & Ravetz, J. R. (1993) · Futures · doi:10.1016/0016-3287(93)90022-l · Cited by 4,285 · see citing works
Self-citationVerification, Validation, and Confirmation of Numerical Models in the Earth Sciences
Орескес, Н., Shrader‐Frechette, K., & Belitz, K. (1994) · Science · doi:10.1126/science.263.5147.641 · Cited by 3,026 · see citing works
Factorial Sampling Plans for Preliminary Computational Experiments
Morris, M. D. (1991) · Technometrics · doi:10.2307/1269043 · Cited by 2,648 · see citing works
Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models
McCulloch, A. (2005) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2005.358_16.x · Cited by 1,873 · see citing works
Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models
Saltelli, A., Tarantola, S., Campolongo, F., & Ratto, M. (2004) · Cited by 1,797 · see citing works
Uncertainty and Quality in Science for Policy
Funtowicz, S. O., & Ravetz, J. R. (1990) · doi:10.1007/978-94-009-0621-1 · Cited by 1,575 · see citing works
Self-citationSensitivity Analysis in Practice
Saltelli, A., Tarantola, S., Campolongo, F., & Ratto, M. (2002) · Wiley eBooks · doi:10.1002/0470870958 · Cited by 526 · see citing works
On the Relative Importance of Input Factors in Mathematical Models
Saltelli, A., & Tarantola, S. (2002) · Journal of the American Statistical Association · doi:10.1198/016214502388618447 · Cited by 448 · see citing works
A way out of the credibility crisis of models used in integrated environmental assessment
van der Sluijs, J. P. (2002) · Futures · doi:10.1016/s0016-3287(01)00051-9 · Cited by 154 · see citing works
Technology Assessment through interaction. A guide
Grin, J., van de Graaf, H., & Hoppe, R. (1997) · UvA-DARE (University of Amsterdam) · Cited by 104 · see citing works
Technical analysis of IIASA energy scenarios
Keepin, B., & Wynne, B. (1984) · Nature · doi:10.1038/312691a0 · Cited by 84 · see citing works
LONG-TERM CHANGE IN VILLAGE-SCALE ECOSYSTEMS IN CHINA USING LANDSCAPE AND STATISTICAL METHODS
Ellis, E. C., Li, R. G., Yang, L., & Cheng, X. (2000) · Ecological Applications · doi:10.1890/1051-0761(2000)010[1057:ltcivs]2.0.co;2 · Cited by 56 · see citing works
A reflexive approach to dealing with uncertainties in environmental health risk science and policy
Craye, M., Funtowicz, S. O., & van der Sluijs, J. P. (2005) · International Journal of Risk Assessment and Management · doi:10.1504/ijram.2005.007169 · Cited by 54 · see citing works
Self-citationUncertainty assessment of the IMAGE/TIMER B1 CO2 emissions scenario, using the NUSAP method
van der Sluijs, J. P., Potting, J., Risbey, J. S., van, V. D., de, V. B., Beusen, A. H. W., Sc, Q., Funtowicz, S. O., Heuberger, P. S. C., Kloprogge, P., et al. (2002) · Cited by 38 · see citing works
Self-citationUncertainty Assessment of Voc Emissions from Paint in the Netherlands Using the Nusap System
van der Sluijs, J. P., Risbey, J. S., & Ravetz, J. R. (2005) · Environmental Monitoring and Assessment · doi:10.1007/s10661-005-3697-7 · Cited by 33 · see citing works
Self-citationUncertainty assessment of NOx, SO2 and NH3 emissions in the Netherlands
van Westenenk, L., van Gijlswijk, R., Coenen, P., Pulles, T., & van der Sluijs, J. P. (2004) · Cited by 17 · see citing works
A protocol for assessment of uncertainty and strength of emissions data
Risbey, J. S., van der Sluijs, J. P., & Ravetz, J. R. (2006) · Utrecht University Repository (Utrecht University) · Cited by 11 · see citing works
Self-citationKeune, H., Goorden, L., Mertens, R., & Loots, I. (2003) · Cited by 1 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Forest disturbances under climate change
Seidl, R., Thom, D., Kautz, M., Martin-Benito, D., Peltoniemi, M., Vacchiano, G., Wild, J., Ascoli, D., Petr, M., Honkaniemi, J., et al. (2017) · Nature Climate Change · doi:10.1038/nclimate3303 · Cited by 2,878 · see citing works
Risk assessment and risk management: Review of recent advances on their foundation
Aven, T. (2016) · European Journal of Operational Research · doi:10.1016/j.ejor.2015.12.023 · Cited by 1,713 · see citing works
“…and Pedigree) ( Funtowicz & Ravetz, 1990, 1993; Kloprogge, van der Sluijs, & Pe-tersen, 2005, 2011; Laes, Meskens, & van der Sluijs, 2011; van der Sluijs et al., 2005a, 2005b ), originally designed for the purpose of analysis and diagnosis of uncertainty in science for policy by performing a…”
BackgroundCharacterising performance of environmental models
Bennett, N. D., Croke, B. F. W., Guariso, G., Guillaume, J. H. A., Hamilton, S. H., Jakeman, A. J., Marsili-Libelli, S., Newham, L. T. H., Norton, J. P., Perrin, C., et al. (2013) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2012.09.011 · Cited by 1,536 · see citing works
Flood inundation modelling: A review of methods, recent advances and uncertainty analysis
Teng, J., Jakeman, A. J., Vaze, J., Croke, B., Dutta, D., & Kim, S. (2017) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2017.01.006 · Cited by 1,473 · see citing works
Uncertainty in the environmental modelling process – A framework and guidance
Refsgaard, J. C., van der Sluijs, J. P., Højberg, A. L., & Vanrolleghem, P. A. (2007) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2007.02.004 · Cited by 1,200 · see citing works
Ten iterative steps in development and evaluation of environmental models
Jakeman, A. J., Letcher, R., & Norton, J. P. (2006) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2006.01.004 · Cited by 1,113 · see citing works
Re-thinking new knowledge production: A literature review and a research agenda
Hessels, L. K., & van Lente, H. (2008) · Research Policy · doi:10.1016/j.respol.2008.01.008 · Cited by 779 · see citing works
Gridded global datasets for Gross Domestic Product and Human Development Index over 1990–2015
Kummu, M., Taka, M., & Guillaume, J. H. A. (2018) · Scientific Data · doi:10.1038/sdata.2018.4 · Cited by 737 · see citing works
Use (and abuse) of expert elicitation in support of decision making for public policy
Morgan, M. G. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1319946111 · Cited by 705 · see citing works
Authority, E. G. F. T. C. O. S. R. E. F. S. (2010) · EFSA Journal · doi:10.2903/j.efsa.2010.1637 · Cited by 693 · see citing works
Opinion of the Scientific Committee related to Uncertainties in Dietary Exposure Assessment
(EFSA), E. F. S. A. (2007) · EFSA Journal · doi:10.2903/j.efsa.2007.438 · Cited by 676 · see citing works
The Future of Sensitivity Analysis: An essential discipline for systems modeling and policy support
Razavi, S., Jakeman, A. J., Saltelli, A., Prieur, C., Iooss, B., Borgonovo, E., Plischke, E., Piano, S. L., Iwanaga, T., Becker, W., et al. (2020) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2020.104954 · Cited by 651 · see citing works
Democracy, sustainability and dialogic accounting technologies: Taking pluralism seriously
Brown, J. A. (2008) · Critical Perspectives on Accounting · doi:10.1016/j.cpa.2008.08.002 · Cited by 568 · see citing works
A framework for dealing with uncertainty due to model structure error
Refsgaard, J. C., van der Sluijs, J. P., Brown, J., & van der Keur, P. (2006) · Advances in Water Resources · doi:10.1016/j.advwatres.2005.11.013 · Cited by 536 · see citing works
Ascough, J. C., Maier, H. R., Ravalico, J. K., & Strudley, M. W. (2008) · Ecological Modelling · doi:10.1016/j.ecolmodel.2008.07.015 · Cited by 520 · see citing works
The role of expert opinion in environmental modelling
Krueger, T., Page, T., Hubacek, K., Smith, L., & Hiscock, K. M. (2012) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2012.01.011 · Cited by 480 · see citing works
Climate change: What do we know about the IPCC?
Hulme, M., & Mahony, M. (2010) · Progress in Physical Geography Earth and Environment · doi:10.1177/0309133310373719 · Cited by 401 · see citing works
Brugnach, M., Dewulf, A., Pahl-Wostl, C., & Taillieu, T. (2008) · Ecology and Society · doi:10.5751/es-02616-130230 · Cited by 397 · see citing works
Accounting technologies and sustainability assessment models
Bebbington, J., Brown, J. A., & Frame, B. (2006) · Ecological Economics · doi:10.1016/j.ecolecon.2006.10.021 · Cited by 365 · see citing works
Storyline approach to the construction of regional climate change information
Shepherd, T. G. (2019) · Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rspa.2019.0013 · Cited by 331 · see citing works
Augusiak, J., den Brink, P. J. V., & Grimm, V. (2013) · Ecological Modelling · doi:10.1016/j.ecolmodel.2013.11.009 · Cited by 297 · see citing works
A review of the effects of uncertainty in public science communication
Gustafson, A., & Rice, R. E. (2020) · Public Understanding of Science · doi:10.1177/0963662520942122 · Cited by 291 · see citing works
Deconstructing adaptive management: criteria for applications to environmental management.
Gregory, R. S., Ohlson, D., & Arvai, J. (2006) · Ecological Applications · doi:10.1890/1051-0761(2006)016[2411:damcfa]2.0.co;2 · Cited by 311 · see citing works
New foundations for ecological economics
Spash, C. L. (2012) · Ecological Economics · doi:10.1016/j.ecolecon.2012.02.004 · Cited by 291 · see citing works
The use of expert elicitation in environmental health impact assessment: a seven step procedure
Knol, A. B., Slottje, P., van der Sluijs, J. P., & Lebret, E. (2010) · Environmental Health · doi:10.1186/1476-069x-9-19 · Cited by 278 · see citing works
A review of catchment-scale water quality and erosion models and a synthesis of future prospects
Fu, B., Merritt, W., Croke, B., Weber, T. R., & Jakeman, A. J. (2018) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2018.12.008 · Cited by 273 · see citing works
What is wrong with evidence based policy, and how can it be improved?
Saltelli, A., & Giampietro, M. (2017) · Futures · doi:10.1016/j.futures.2016.11.012 · Cited by 264 · see citing works
Failing, L., Gregory, R., & Harstone, M. (2007) · Ecological Economics · doi:10.1016/j.ecolecon.2007.03.010 · Cited by 247 · see citing works
The principles and methods behind EFSA's Guidance on Uncertainty Analysis in Scientific Assessment
Benford, D., Halldorsson, T., Jeger, M., Knutsen, H., More, S., Naegeli, H., Noteborn, H., Ockleford, C., Ricci, A., Rychen, G., et al. (2018) · EFSA journal. European Food Safety Authority · doi:10.2903/j.efsa.2018.5122 · Cited by 229 · see citing works
Lo Piano, S. (2020) · Humanities and Social Sciences Communications · doi:10.1057/s41599-020-0501-9 · Cited by 229 · see citing works
A framework for uncertainty analysis in flood risk management decisions
Hall, J. W., & SOLOMATINE, D. P. (2008) · International Journal of River Basin Management · doi:10.1080/15715124.2008.9635339 · Cited by 225 · see citing works
Young, J., McQueen, N., Charalambous, C., Foteinis, S., Hawrot, O., Ojeda, M., Pilorgé, H., Andresen, J., Psarras, P., Renforth, P., et al. (2023) · One Earth · doi:10.1016/j.oneear.2023.06.004 · Cited by 222 · see citing works
Eight grand challenges in socio-environmental systems modeling
Elsawah, S., Filatova, T., Jakeman, A. J., Kettner, A. J., Zellner, M. L., Athanasiadis, I. N., Hamilton, S. H., Axtell, R. L., Brown, D. G., Gilligan, J. M., et al. (2020) · Socio-Environmental Systems Modelling · doi:10.18174/sesmo.2020a16226 · Cited by 205 · see citing works
Wardekker, A., van der Sluijs, J. P., Janssen, P. H., Kloprogge, P., & Petersen, A. C. (2008) · Environmental Science & Policy · doi:10.1016/j.envsci.2008.05.005 · Cited by 189 · see citing works
Self-citationModelling net-zero emissions energy systems requires a change in approach
Pye, S., Broad, O., Bataille, C., Brockway, P. E., Daly, H., Freeman, R., Gambhir, A., Geden, O., Rogan, F., Sanghvi, S., et al. (2020) · Climate Policy · doi:10.1080/14693062.2020.1824891 · Cited by 180 · see citing works
Introduction to Sensitivity Analysis
Saltelli, A., Ratto, M., Andres, T., Campolongo, F., Cariboni, J., Gatelli, D., Saisana, M., & Tarantola, S. (2007) · doi:10.1002/9780470725184.ch1 · Cited by 178 · see citing works
Navigating uncertainty in environmental composite indicators
Burgass, M. J., Halpern, B. S., Nicholson, E., & Milner-Gulland, E. J. (2017) · Ecological Indicators · doi:10.1016/j.ecolind.2016.12.034 · Cited by 176 · see citing works
Expert assessment concludes negative emissions scenarios may not deliver
Vaughan, N. E., & Gough, C. (2016) · Environmental Research Letters · doi:10.1088/1748-9326/11/9/095003 · Cited by 190 · see citing works
Brokerage at the science–policy interface: from conceptual framework to practical guidance
Gluckman, P. D., Bardsley, A., & Kaiser, M. (2021) · Humanities and Social Sciences Communications · doi:10.1057/s41599-021-00756-3 · Cited by 184 · see citing works
Uncertainty and climate change adaptation : a scoping study
Dessai, S., & van der Sluijs, J. P. (2007) · Utrecht University Repository (Utrecht University) · Cited by 168 · see citing works
Strand, R. (2017) · doi:10.4324/9781315679747.ch28 · Cited by 163 · see citing works
Identification and classification of uncertainties in the application of environmental models
Warmink, J. J., Janssen, J., Booij, M. J., & Krol, M. S. (2010) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2010.04.011 · Cited by 154 · see citing works
The added value of participatory modelling in fisheries management – what has been learnt?
Röckmann, C., Ulrich, C., Dreyer, M., Bell, E., Borodzicz, E. P., Haapasaari, P., Hauge, K. H., Howell, D., Mäntyniemi, S., Miller, D., et al. (2012) · Marine Policy · doi:10.1016/j.marpol.2012.02.027 · Cited by 151 · see citing works
A methodology for the design and development of integrated models for policy support
van Delden, H., Seppelt, R., White, R. W., & Jakeman, A. J. (2010) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2010.03.021 · Cited by 150 · see citing works
Why Endocrine Disrupting Chemicals (EDCs) Challenge Traditional Risk Assessment and How to Respond
Futran Fuhrman, V., Tal, A., & Arnon, S. (2015) · Journal of Hazardous Materials · doi:10.1016/j.jhazmat.2014.12.012 · Cited by 152 · see citing works
Evaluating integrated assessment models of global climate change
Schwanitz, V. (2013) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2013.09.005 · Cited by 134 · see citing works
Magliocca, N. R., Ellis, E. C., Allington, G. R. H., de Bremond, A., Dell’Angelo, J., Mertz, O., Messerli, P., Meyfroidt, P., Seppelt, R., & Verburg, P. H. (2018) · Global Environmental Change · doi:10.1016/j.gloenvcha.2018.03.003 · Cited by 145 · see citing works
Buizer, M., Arts, B. J. M., & Kok, K. I. (2011) · Ecology and Society · doi:10.5751/es-03908-160121 · Cited by 140 · see citing works
A guide to good practice in modeling semantics for authors and referees
Beven, K. J., & Young, P. C. (2013) · Water Resources Research · doi:10.1002/wrcr.20393 · Cited by 138 · see citing works
Best practices and recent advances in CCS cost engineering and economic analysis
van der Spek, M. W., Roussanaly, S., & Rubin, E. S. (2019) · International journal of greenhouse gas control · doi:10.1016/j.ijggc.2019.02.006 · Cited by 139 · see citing works
Koetz, T., Farrell, K. N., & Bridgewater, P. B. (2011) · International Environmental Agreements Politics Law and Economics · doi:10.1007/s10784-011-9152-z · Cited by 138 · see citing works
Swart, R. J., Bernstein, L., Ha‐Duong, M., & Petersen, A. C. (2008) · Climatic Change · doi:10.1007/s10584-008-9444-7 · Cited by 138 · see citing works
Badham, J., Elsawah, S., Guillaume, J. H. A., Hamilton, S. H., Hunt, R. J., Jakeman, A. J., Pierce, S. A., Snow, V. O., Babbar‐Sebens, M., Fu, B., et al. (2019) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2019.02.013 · Cited by 135 · see citing works
Evaluating process-based integrated assessment models of climate change mitigation
Wilson, C., Céline Guivarch, A., Kriegler, E., van Ruijven, B. J., van Vuuren, D. P., Krey, V., Schwanitz, V. J., & Thompson, E. (2021) · Climatic Change · doi:10.1007/s10584-021-03099-9 · Cited by 138 · see citing works
Aven, T. (2017) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2017.05.006 · Cited by 132 · see citing works
A method for the analysis of assumptions in model-based environmental assessments
Kloprogge, P., van der Sluijs, J. P., & Petersen, A. C. (2009) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2009.06.009 · Cited by 130 · see citing works
Self-citationExploring the quality of evidence for complex and contested policy decisions
van der Sluijs, J. P., Petersen, A. C., Janssen, P. H., Risbey, J. S., & Ravetz, J. R. (2008) · Environmental Research Letters · doi:10.1088/1748-9326/3/2/024008 · Cited by 130 · see citing works
Self-citationIntroduction to Sensitivity Analysis
Iooss, B., & Saltelli, A. (2017) · Springer eBooks · doi:10.1007/978-3-319-12385-1_31 · Cited by 125 · see citing works
Where Now for Post-Normal Science?: A Critical Review of its Development, Definitions, and Uses
Turnpenny, J. R., Jones, M., & Lorenzoni, I. (2010) · Science Technology & Human Values · doi:10.1177/0162243910385789 · Cited by 122 · see citing works
Climate change vulnerability of sustainable forest management in the Eastern Alps
Seidl, R., Rammer, W., & Lexer, M. J. (2010) · Climatic Change · doi:10.1007/s10584-010-9899-1 · Cited by 117 · see citing works
Quantitative Storytelling in the Making of a Composite Indicator
Kuc-Czarnecka, M. E., Piano, S. L., & Saltelli, A. (2020) · Social Indicators Research · doi:10.1007/s11205-020-02276-0 · Cited by 117 · see citing works
van der Spek, M. W., Fout, T., García, M., Kuncheekanna, V. N., Matuszewski, M., McCoy, S., Morgan, J. C., Nazir, S. M., Ramírez, A., Roussanaly, S., et al. (2020) · International journal of greenhouse gas control · doi:10.1016/j.ijggc.2020.103113 · Cited by 117 · see citing works
Frame-based guide to situated decision-making on climate change
de Boer, J., Wardekker, A., & van der Sluijs, J. P. (2010) · Global Environmental Change · doi:10.1016/j.gloenvcha.2010.03.003 · Cited by 116 · see citing works
Röckmann, C., van Leeuwen, J., Goldsborough, D., Kraan, M., & Piet, G. J. (2014) · Marine Policy · doi:10.1016/j.marpol.2014.10.019 · Cited by 114 · see citing works
Post-Normal Science in Practice at the Netherlands Environmental Assessment Agency
Petersen, A. C., Cath, A., Hage, M. D. S. C., Kunseler, E., & van der Sluijs, J. P. (2010) · Science Technology & Human Values · doi:10.1177/0162243910385797 · Cited by 113 · see citing works
Social Ecological Economics: Understanding the Past to See the Future
Spash, C. L. (2011) · American Journal of Economics and Sociology · doi:10.1111/j.1536-7150.2011.00777.x · Cited by 112 · see citing works
Environmental risk analysis for nanomaterials: Review and evaluation of frameworks
Grieger, K., Linkov, I., Hansen, S. F., & Baun, A. (2011) · Nanotoxicology · doi:10.3109/17435390.2011.569095 · Cited by 111 · see citing works
Beven, K. J., Almeida, S., Aspinall, W. P., Bates, P., Blazkova, S. D., Borgomeo, E., Freer, J. E., Goda, K., Hall, J. W., Phillips, J. C., et al. (2018) · Natural hazards and earth system sciences · doi:10.5194/nhess-18-2741-2018 · Cited by 110 · see citing works
Perceptions and communication strategies for the many uncertainties relevant for climate policy
Patt, A., & Weber, E. U. (2013) · Wiley Interdisciplinary Reviews Climate Change · doi:10.1002/wcc.259 · Cited by 108 · see citing works
Dewulf, A. R. P. J., & Biesbroek, R. G. (2018) · Policy and Society · doi:10.1080/14494035.2018.1504484 · Cited by 108 · see citing works
A protocol for horizontal averaging of unit process data—including estimates for uncertainty
Henriksson, P. J. G., Guinée, J. B., Heijungs, R., de Koning, A., & Green, D. M. (2013) · The International Journal of Life Cycle Assessment · doi:10.1007/s11367-013-0647-4 · Cited by 103 · see citing works
Estimating the effect of entrepreneur education on graduates’ intention to be entrepreneurs
-, N. B. M., Lim, H. E., Yusof, N., & Soon, J. (2015) · Education + Training · doi:10.1108/et-03-2014-0030 · Cited by 103 · see citing works
How some types of risk assessments can support resilience analysis and management
Aven, T. (2017) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2017.07.005 · Cited by 102 · see citing works
Liu, H.-Y., Kobernus, M., Broday, D. M., & Bartoňová, A. (2014) · Environmental Health · doi:10.1186/1476-069x-13-107 · Cited by 101 · see citing works
Spash, C. L., & Aslaksen, I. (2015) · Journal of Environmental Management · doi:10.1016/j.jenvman.2015.04.049 · Cited by 125 · see citing works
Pandemic Preparedness and the Influenza Risk Assessment Tool (IRAT)
Cox, N. J., Trock, S. C., & Burke, S. A. (2014) · Current topics in microbiology and immunology · doi:10.1007/82_2014_419 · Cited by 98 · see citing works
The epistemic, ethical, and political dimensions of uncertainty in integrated assessment modeling
Beck, M., & Krüger, T. (2016) · Wiley Interdisciplinary Reviews Climate Change · doi:10.1002/wcc.415 · Cited by 107 · see citing works
A framework for characterising and evaluating the effectiveness of environmental modelling
Hamilton, S. H., Fu, B., Guillaume, J. H. A., Badham, J., Elsawah, S., Gober, P. A., Hunt, R. J., Iwanaga, T., Jakeman, A. J., Ames, D. P., et al. (2019) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2019.04.008 · Cited by 99 · see citing works
Comparison of scale and scaling issues in integrated land-use models for policy support
van Delden, H., van Vliet, J., Rutledge, D. T., & Kirkby, M. J. (2011) · Agriculture Ecosystems & Environment · doi:10.1016/j.agee.2011.03.005 · Cited by 94 · see citing works
Automated sub-zoning of water distribution systems
Perelman, L., Allen, M. P., Preis, A., Iqbal, M., & Whittle, A. J. (2014) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2014.11.025 · Cited by 87 · see citing works
Uncertainty and precaution in environmental management: Insights from the UPEM conference
van der Sluijs, J. P. (2006) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2005.12.020 · Cited by 84 · see citing works
Expressions of likelihood and confidence in the IPCC uncertainty assessment process
Risbey, J. S., & Kandlikar, M. (2007) · Climatic Change · doi:10.1007/s10584-007-9315-7 · Cited by 83 · see citing works
Self-citationGlynn, P. D., Voinov, A. A., Shapiro, C. D., & White, P. A. (2017) · Earth's Future · doi:10.1002/2016ef000487 · Cited by 98 · see citing works
Hamilton, S. H., Pollino, C., Stratford, D., Fu, B., & Jakeman, A. J. (2021) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2021.105278 · Cited by 81 · see citing works
Dealing with uncertainties in environmental burden of disease assessment
Knol, A. B., Petersen, A. C., van der Sluijs, J. P., & Lebret, E. (2009) · Environmental Health · doi:10.1186/1476-069x-8-21 · Cited by 80 · see citing works
Guidelines for Environmental Risk Assessment and Management
Leaves, G. (2011) · Cited by 80 · see citing works
Ran, J., MacGillivray, B. H., Gong, Y., & Hales, T. C. (2019) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2019.134486 · Cited by 80 · see citing works
Ramírez, A., de Keizer, C., van der Sluijs, J. P., Olivier, J. G. J., & Brandes, L. (2008) · Atmospheric Environment · doi:10.1016/j.atmosenv.2008.07.059 · Cited by 79 · see citing works
Assessing model-based and conflict-based uncertainty
Patt, A. (2006) · Global Environmental Change · doi:10.1016/j.gloenvcha.2006.10.002 · Cited by 76 · see citing works
Costabile, F., & Allegrini, I. (2007) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2007.03.001 · Cited by 76 · see citing works
Parsons, S., Abeln, F., McManus, M. C., & Chuck, C. J. (2018) · Journal of Chemical Technology & Biotechnology · doi:10.1002/jctb.5811 · Cited by 74 · see citing works
A high-resolution integrated hydrology–hydrodynamic model of the Barataria Basin system
Inoue, M., PARK, D., Justić, D., & WISEMANJR, W. (2008) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2008.02.011 · Cited by 72 · see citing works
Agent-based modelling of socio-ecological systems: Models, projects and ontologies
Gotts, N., Voorn, G., Polhill, J. G., de Jong, E., Edmonds, B., Hofstede, G., & Meyer, R. (2019) · Ecological Complexity : An International Journal on Biocomplexity in the Environment and Theoretical Ecology · doi:10.1016/j.ecocom.2018.07.007 · Cited by 56 · see citing works
Little, J. C., Hester, E., Elsawah, S., Filz, G., Sandu, A., Carey, C., Iwanaga, T., & Jakeman, A. (2019) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2018.11.011 · Cited by 72 · see citing works
van der Spek, M. W., Fernández, E. S., Eldrup, N. H., Skagestad, R., Ramírez, A., & Faaij, A. P. C. (2016) · International journal of greenhouse gas control · doi:10.1016/j.ijggc.2016.11.021 · Cited by 71 · see citing works
Barriers to investment in utility-scale variable renewable electricity (VRE) generation projects
Hu, J., Harmsen, R., Crijns-Graus, W. H. J., & Worrell, E. (2018) · Renewable Energy · doi:10.1016/j.renene.2018.01.092 · Cited by 71 · see citing works
Rozema, J. G., Bond, A., Cashmore, M., & Chilvers, J. (2011) · Environmental Impact Assessment Review · doi:10.1016/j.eiar.2011.11.003 · Cited by 69 · see citing works
Approaches to displaying information to assist decisions under uncertainty
Kreye, M. E., Goh, Y. M., Newnes, L., & Goodwin, P. (2011) · Omega · doi:10.1016/j.omega.2011.05.010 · Cited by 68 · see citing works
Wolfslehner, B., & Seidl, R. (2009) · Environmental Management · doi:10.1007/s00267-009-9414-5 · Cited by 66 · see citing works
Perceptual models of uncertainty for socio-hydrological systems: a flood risk change example
Westerberg, I. K., Baldassarre, G. D., Beven, K. J., Coxon, G., & Krueger, T. (2017) · Hydrological Sciences Journal · doi:10.1080/02626667.2017.1356926 · Cited by 61 · see citing works
- Authors
- Van Der Sluijs, J. P., Craye, M., Funtowicz, S., Kloprogge, P., Ravetz, J., & Risbey, J.
- Affiliations
- Joint Research Centre (IT)
- Utrecht University (NL)
- European Commission (BE)
- Monash University (AU)
- Venue
- Risk Analysis
- Publisher
- Wiley
- Published
- 2005-04
- Volume
- 25
- Issue
- 2
- Pages
- 481-492
- ISSN
- 0272-4332 (ISSN-L)
- 1539-6924
- OpenAlex
- W2119214313
- Cited by
- 426
- References
- 19
- Citations by year
- 2026: 13 · 2025: 4 · 2024: 15 · 2023: 38 · 2022: 11 · 2021: 19 · 2020: 22 · 2019: 21 · 2018: 26 · 2017: 35 · 2016: 15 · 2015: 25 · 2014: 21 · 2013: 23 · 2012: 23
- FWCI
- 65.98
- Type
- journal article
- Language
- en
- Open access
- green
- dspace.library.uu.nl
- Licence
- Locations
- Risk Analysis (publishedVersion)
- PubMed (publishedVersion)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- Topics
- Physical Sciences › Environmental Science › Management, Monitoring, Policy and Law › Environmental and Social Impact Assessments
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Probabilistic and Robust Engineering Design
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Risk and Safety Analysis
- Keywords
- uncertainty analysis, multidimensional uncertainty, pedigree analysis, parameter sensitivity, model assumptions, diagnostic diagrams, problem framing, deliberative decision-making, collective learning, environmental assessment, model-based evaluation
- MeSH terms
- Vehicle Emissions, Computer Simulation, Environment, Environmental Monitoring, Environmental Pollutants, Models, Theoretical, Public Health, Software, Systems Theory, Models, Statistical, Risk Assessment, Uncertainty
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
