The DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Combining Quantitative and Qualitative Measures of Uncertainty in Model‐Based Environmental Assessment: The NUSAP System
Open accessAbstract
This article discusses recent experiences with the Numeral Unit Spread Assessment Pedigree (NUSAP) system for multidimensional uncertainty assessment, based on four case studies that vary in complexity. We show that the NUSAP method is applicable not only to relatively simple calculation schemes but also to complex models in a meaningful way and that NUSAP is useful to assess not only parameter uncertainty but also (model) assumptions. A diagnostic diagram can be used to synthesize results of quantitative analysis of parameter sensitivity and qualitative review (pedigree analysis) of parameter strength. It provides an analytic tool to prioritize uncertainties according to quantitative and qualitative insights in the limitations of available knowledge. We show that extension of the pedigree scheme to include societal dimensions of uncertainty, such as problem framing and value‐laden assumptions, further promotes reflexivity and collective learning. When used in a deliberative setting, NUSAP pedigree assessment has the potential to foster a deeper social debate and a negotiated management of complex environmental problems.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Tools of post-normal-science inspiration for knowledge and modelling quality assessment could be adapted to the analysis of algorithms, such as the NUSAP (Numeral Unit Spread Assessment Pedigree) notation system for the management and communication of uncertainty (Funtowicz and Ravetz, 1990;Van Der Sluijs, 2005) and sensitivity auditing (Saltelli and Funtowicz, 2014), respectively.”
From Ethical principles in machine learning and artificial intelligence: cases from the field and possible ways forward (2020) · cites this paper as Van Der Sluijs, 2005
Citations
Cites
Toulmin, S. E. (2003) · doi:10.1017/cbo9780511840005 · Cited by 7,101
Factorial Sampling Plans for Preliminary Computational Experiments
Morris, M. D. (1991) · Technometrics · doi:10.1080/00401706.1991.10484804 · Cited by 4,943
Science for the post-normal age
Funtowicz, S., & Ravetz, J. R. (1993) · Futures · doi:10.1016/0016-3287(93)90022-l · Cited by 4,639
Self-citationVerification, Validation, and Confirmation of Numerical Models in the Earth Sciences
Орескес, Н., Shrader‐Frechette, K., & Belitz, K. (1994) · Science · doi:10.1126/science.263.5147.641 · Cited by 3,026
Factorial Sampling Plans for Preliminary Computational Experiments
Morris, M. D. (1991) · Technometrics · doi:10.2307/1269043 · Cited by 2,648
Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models
McCulloch, A. (2005) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2005.358_16.x · Cited by 1,866
Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models
Saltelli, A., Tarantola, S., Campolongo, F., & Ratto, M. (2004) · Cited by 1,797
Uncertainty and Quality in Science for Policy
Funtowicz, S. O., & Ravetz, J. R. (1990) · doi:10.1007/978-94-009-0621-1 · Cited by 1,575
Self-citationSensitivity Analysis in Practice
Saltelli, A., Tarantola, S., Campolongo, F., & Ratto, M. (2002) · Wiley eBooks · doi:10.1002/0470870958 · Cited by 526
On the Relative Importance of Input Factors in Mathematical Models
Saltelli, A., & Tarantola, S. (2002) · Journal of the American Statistical Association · doi:10.1198/016214502388618447 · Cited by 448
A way out of the credibility crisis of models used in integrated environmental assessment
van der Sluijs, J. P. (2002) · Futures · doi:10.1016/s0016-3287(01)00051-9 · Cited by 154
Technology Assessment through interaction. A guide
Grin, J., van de Graaf, H., & Hoppe, R. (1997) · UvA-DARE (University of Amsterdam) · Cited by 104
Technical analysis of IIASA energy scenarios
Keepin, B., & Wynne, B. (1984) · Nature · doi:10.1038/312691a0 · Cited by 84
LONG-TERM CHANGE IN VILLAGE-SCALE ECOSYSTEMS IN CHINA USING LANDSCAPE AND STATISTICAL METHODS
Ellis, E. C., Li, R. G., Yang, L., & Cheng, X. (2000) · Ecological Applications · doi:10.1890/1051-0761(2000)010[1057:ltcivs]2.0.co;2 · Cited by 56
A reflexive approach to dealing with uncertainties in environmental health risk science and policy
Craye, M., Funtowicz, S. O., & van der Sluijs, J. P. (2005) · International Journal of Risk Assessment and Management · doi:10.1504/ijram.2005.007169 · Cited by 54
Self-citationUncertainty assessment of the IMAGE/TIMER B1 CO2 emissions scenario, using the NUSAP method
van der Sluijs, J. P., Potting, J., Risbey, J. S., van, V. D., de, V. B., Beusen, A. H. W., Sc, Q., Funtowicz, S. O., Heuberger, P. S. C., Kloprogge, P., et al. (2002) · Cited by 38
Self-citationUncertainty Assessment of Voc Emissions from Paint in the Netherlands Using the Nusap System
van der Sluijs, J. P., Risbey, J. S., & Ravetz, J. R. (2005) · Environmental Monitoring and Assessment · doi:10.1007/s10661-005-3697-7 · Cited by 33
Self-citationUncertainty assessment of NOx, SO2 and NH3 emissions in the Netherlands
van Westenenk, L., van Gijlswijk, R., Coenen, P., Pulles, T., & van der Sluijs, J. P. (2004) · Cited by 17
A protocol for assessment of uncertainty and strength of emissions data
Risbey, J. S., van der Sluijs, J. P., & Ravetz, J. R. (2006) · Utrecht University Repository (Utrecht University) · Cited by 11
Self-citationKeune, H., Goorden, L., Mertens, R., & Loots, I. (2003) · Cited by 1
Research works printed but not linked: 11
- Craye M., 2001, Milieu en gezondheid in Vlaanderen: naar en adequate dialoog tussen overheid, bevolking en wetenschap
- Van der Sluijs J. P., 1997, Anchoring amid uncertainty; On the management of uncertainties in risk assessment of anthropogenic climate change
- Van der Sluijs J. P. Risbey J. S. &Ravetz J. R.(in press).Uncertainty assessment of VOC emissions from paint in the Netherlands.Environmental Monitoring and Assessment.
- EEA, 2003, EMEP/CORINAIR Emission Inventory Guidebook
- Kloprogge P., 2005, A Method for the Analysis of Assumptions in Assessments Applied to Two Indicators in the Fifth Dutch Environmental Outlook
- Aelvoet W., 1998, Risico op gezondheidsschade bij kinderen van de Neerlandwijk te Wilrijk
- Schoeters G., 1998, Studie van gezondheidsaspecten en gezondheidsrisico's ten gevolge van milieuverontreiniging in de Neerlandwijk te Wilrijk
- Koppen G., 2001, Biomonitoringsonderzoek 50‐65j vrouwen en 21‐40j mannen: evaluatie van de relatie milieu‐gezondheid op basis van meting van biomerkers
- Staessen J., 2001, Koepeltekst biomonitoring bij adolescenten: een gevoelige nieuwe methode voor het inschatten van verontreiniging van het leefmilieu en geassocieerde gezondheidsrisico's
- Van de Graaf H., 2000, Beleid en Politiek
- Von Schomberg R., 1997, Argumentatie in de context van een wetenschappelijke controverse
Cited by
Sorted by citations
Forest disturbances under climate change
Seidl, R., Thom, D., Kautz, M., Martin-Benito, D., Peltoniemi, M., Vacchiano, G., Wild, J., Ascoli, D., Petr, M., Honkaniemi, J., et al. (2017) · Nature Climate Change · doi:10.1038/nclimate3303 · Cited by 2,878
Risk assessment and risk management: Review of recent advances on their foundation
Aven, T. (2016) · European Journal of Operational Research · doi:10.1016/j.ejor.2015.12.023 · Cited by 1,713
“…and Pedigree) ( Funtowicz & Ravetz, 1990, 1993; Kloprogge, van der Sluijs, & Pe-tersen, 2005, 2011; Laes, Meskens, & van der Sluijs, 2011; van der Sluijs et al., 2005a, 2005b ), originally designed for the purpose of analysis and diagnosis of uncertainty in science for policy by performing a…”
BackgroundCharacterising performance of environmental models
Bennett, N. D., Croke, B. F. W., Guariso, G., Guillaume, J. H. A., Hamilton, S. H., Jakeman, A. J., Marsili-Libelli, S., Newham, L. T. H., Norton, J. P., Perrin, C., et al. (2013) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2012.09.011 · Cited by 1,536
Flood inundation modelling: A review of methods, recent advances and uncertainty analysis
Teng, J., Jakeman, A. J., Vaze, J., Croke, B., Dutta, D., & Kim, S. (2017) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2017.01.006 · Cited by 1,473
Uncertainty in the environmental modelling process – A framework and guidance
Refsgaard, J. C., van der Sluijs, J. P., Højberg, A. L., & Vanrolleghem, P. A. (2007) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2007.02.004 · Cited by 1,200
Metadata
- Authors
- Van Der Sluijs, J. P., Craye, M., Funtowicz, S., Kloprogge, P., Ravetz, J., & Risbey, J.
- Affiliations
- Joint Research Centre (IT)
- Utrecht University (NL)
- European Commission (BE)
- Monash University (AU)
- Venue
- Risk Analysis
- Publisher
- Wiley
- Published
- 2005-04
- Volume
- 25
- Issue
- 2
- Pages
- 481-492
- ISSN
- 0272-4332 (ISSN-L)
- 1539-6924
- OpenAlex
- W2119214313
- Cited by
- 426
- References
- 19
- Citations by year
- 2026: 13 · 2025: 4 · 2024: 15 · 2023: 38 · 2022: 11 · 2021: 19 · 2020: 22 · 2019: 21 · 2018: 26 · 2017: 35 · 2016: 15 · 2015: 25 · 2014: 21 · 2013: 23 · 2012: 23
- FWCI
- 65.98
- Type
- journal article
- Language
- en
- Open access
- green
- dspace.library.uu.nl
- Licence
- Locations
- Risk Analysis (publishedVersion)
- PubMed (publishedVersion)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- Topics
- Physical Sciences › Environmental Science › Management, Monitoring, Policy and Law › Environmental and Social Impact Assessments
- Physical Sciences › Engineering › Computational Mechanics › Probabilistic and Robust Engineering Design
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Risk and Safety Analysis
- Keywords
- uncertainty analysis, multidimensional uncertainty, pedigree analysis, parameter sensitivity, model assumptions, diagnostic diagrams, problem framing, deliberative decision-making, collective learning, environmental assessment, model-based evaluation
- MeSH terms
- Vehicle Emissions, Computer Simulation, Environment, Environmental Monitoring, Environmental Pollutants, Models, Theoretical, Public Health, Software, Systems Theory, Models, Statistical, Risk Assessment, Uncertainty
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
