The DOI matches its DataCite record; 1 of the 1 references it cites were found in a registry or an index.
Problems with Risk Matrices Using Ordinal Scales
Open accessAbstract
In this paper, we discuss various problems in the usage and definition of risk matrices. We give an overview of the general process of risk assessment with risk matrices and ordinal scales. Furthermore, we explain the fallacies in each phase of this process and give hints on which decisions may lead to more problems than others and how to avoid them. Among those 24 discussed problems are ordinal scales, semi-quantitative arithmetics, range compression, risk inversion, ambiguity, and neglection of uncertainty. Finally, we make a case for avoiding risk matrices altogether and instead propose using fully quantitative risk assessment methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Risk matrices are widely recognised tools by academics and practitioners in various sectors to assess and rank risks according to likelihood of occurrence and impact on project objectives.
“Risk matrices are widely recognised tools by academics and practitioners in various sectors to assess and rank risks according to their likelihood of occurrence and impact on project objectives.”
From Beyond probability-impact matrices in project risk management: A quantitative methodology for risk prioritisation (2024) · cites this paper as Krisper, 2021
The Introduction describes risk matrices as established tools for assessing and ranking risks across many domains and industries. Phase 1 identifies impact and probability (likelihood of occurrence) as common assessment factors. Our copy does not specifically define impact in terms of project objectives, so that qualification is not supported. (LLM, conf=0.82)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
On the Theory of Scales of Measurement
Stevens, S. S. (1946) · Science · doi:10.1126/science.103.2684.677 · Cited by 4,761 · cited 3×
“According to Stevens [22], ordinal scales only allow for ordering or ranking the items.”
BackgroundInfluential
Flagged first, then by citations
Research works printed but not linked: 26
- A. L. Cox, D. Babayev, and W. Huber, "Some Limitations of Qualitative Risk Rating Systems," Risk Analysis, vol. 25, no. 3, pp. 651-662, 2005.
- A. L. Cox and D. A. Popken, "Some Limitations of Aggregate Exposure Metrics," Risk Analysis, vol. 27, no. 2, pp. 439-445, 2007.
- A. L. Cox, "What's Wrong with Risk Matrices?" Risk Analysis, vol. 28, no. 2, pp. 497-512, Apr. 2008. [Online]. Available: http://doi.wiley.com/10.1111/j.1539-6924.2008.01030.x · DOI · Paper page
- L. A. Cox, Risk analysis of complex and uncertain systems, ser. International series in operations research & management science. New York: Springer, 2009, no. 129, oCLC: ocn276223899.
- L. A. T. Cox, "Confronting Deep Uncertainties in Risk Analysis," Risk Analysis, vol. 32, no. 10, pp. 1607-1629, Oct. 2012. [Online]. Available: http://doi.wiley.com/10.1111/j.1539-6924.2012.01792.x · DOI
- D. W. Hubbard, How to measure anything: finding the value of intan- gibles in business, 3rd ed. Hoboken, New Jersey: John Wiley & Sons, Inc, 2014, 00000.
- D. W. Hubbard and R. Seiersen, How to measure anything in cyberse- curity risk. Hoboken: Wiley, 2016, 00000.
- P. Artzner, F. Delbaen, J.-M. Eber, and D. Heath, "Coherent Measures of Risk," Mathematical Finance, vol. 9, no. 3, pp. 203-228, Jul. 1999. [Online]. Available: http://doi.wiley.com/10.1111/1467-9965.00068 · DOI
- D. Kahneman and A. Tversky, "Subjective probability: A judgement of representativeness," Cognitive psychology, vol. 3, no. 3, pp. 430-454, 1972. [Online]. Available: http://datacolada.org/wp-content/ uploads/2014/08/Kahneman-Tversky-1972.pdf
- J. Dobaj, C. Schmittner, M. Krisper, and G. Macher, "Towards Unified Quantitative Integrated Security and Safety Risk Assessment," Under review, p. 14, 2019.
- M. Krisper, J. Dobaj, and G. Macher, "RISKEE: A Risk-Tree Based Method for Assessing Risk in Cyber Security," Under Review, p. 12, 2019.
- J.-L. Martin and D. Wu, "Pedestrian fatality and impact speed squared: Cloglog modeling from French national data," Traffic Injury Prevention, pp. -, Jan. 2017. [Online]. Available: https: //hal.archives-ouvertes.fr/hal-01557978
- C. Jurewicz, A. Sobhani, J. Woolley, J. Dutschke, and B. Corben, "Exploration of Vehicle Impact Speed -Injury Severity Relationships for Application in Safer Road Design," Transportation Research Procedia, vol. 14, pp. 4247-4256, 2016. [Online]. Available: https://linkinghub.elsevier.com/retrieve/pii/S2352146516304021
- L. Aarts and I. van Schagen, "Driving speed and the risk of road crashes: A review," Accident Analysis & Prevention, vol. 38, no. 2, pp. 215-224, Mar. 2006. [Online]. Available: https://linkinghub.elsevier.com/retrieve/pii/S0001457505001247
- T. DeMarco, Controlling Software Projects: Management, Measurement, and Estimates: Management, Measurement and Estimation. Englewood Cliffs, N.J: Pearson Education, Nov. 1982.
- T. R. Knapp, "Treating Ordinal Scales as Interval Scales: An Attempt To Resolve the Controversy," Nursing research, vol. 39, no. 2, pp. 121-123, 1990.
- E. H. Ho, D. V. Budescu, M. K. Dhami, and D. R. Mandel, "Improving the communication of uncertainty in climate science and intelligence analysis," Behavioral Science & Policy, vol. 1, no. 2, pp. 43-55, 2015.
- E. D. Smith, W. T. Siefert, and D. Drain, "Risk matrix input data biases," Systems Engineering, vol. 12, no. 4, pp. 344-360, Sep. 2009. [Online]. Available: http://doi.wiley.com/10.1002/sys.20126 · DOI
- D. W. Hubbard, The failure of risk management: why it's broken and how to fix it. Hoboken, N.J: Wiley, 2009, oCLC: ocn268790760.
- G. Moors, N. D. Kieruj, and J. K. Vermunt, "The Effect of Labeling and Numbering of Response Scales on the Likelihood of Response Bias," Sociological Methodology, vol. 44, no. 1, pp. 369-399, Aug. 2014. [Online]. Available: http://journals.sagepub.com/ doi/10.1177/0081175013516114 · DOI
- N. K. Malhotra, Basic Marketing Research, 4th ed. Boston: Pearson, Jul. 2011.
- J. Dawes, "Do Data Characteristics Change According to the Number of Scale Points Used? An Experiment Using 5-Point, 7-Point and 10-Point Scales," International Journal of Market Research, vol. 50, no. 1, pp. 61-104, Jan. 2008. [Online]. Available: https://doi.org/10.1177/147078530805000106 · DOI
- P. Thomas, R. B. Bratvold, and E. Bickel, "The Risk of Using Risk Matrices," in SPE Annual Technical Conference and Exhibition. New Orleans, Louisiana, USA: Society of Petroleum Engineers, 2013. [Online]. Available: http://www.onepetro.org/doi/10.2118/166269-MS · DOI
- P. D. Windschitl and E. U. Weber, "The Interpretation of "Likely" Depends on the Context, but "70%" Is 70%-Right? The Influence of Associative Processes on Perceived Certainty," Journal of Experimental Psychology: Learning, Memory, and Cognition, vol. 20, no. 6, p. 20, 1999.
- T. Gilovich, D. Griffin, and D. Kahneman, Heuristics and Biases: The Psychology of Intuitive Judgment. Cambridge, U.K. ; New York: Cambridge University Press, Jul. 2002.
- R. A. Howard and A. E. Abbas, Foundations of Decision Analysis, 01st ed. Boston: Pearson, Jan. 2015.
Cited by
Sorted by citations
Acebes, F., González-Varona, J. M., López-Paredes, A., & Pajares, J. (2024) · Humanities and Social Sciences Communications · doi:10.1057/s41599-024-03180-5 · Cited by 74
Metadata
- Authors
- Krisper, M.
- Affiliations
- Graz University of Technology (AT)
- Corresponding author
- Krisper, Michael
- ORCID
- Krisper, Michael
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2021
- arXiv
- 2103.05440
- OpenAlex
- W3135049945
- Cited by
- 26
- References
- 1
- Citations by year
- 2026: 9 · 2025: 7 · 2024: 6 · 2023: 3 · 2022: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- RePEc: Research Papers in Economics (submittedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Risk and Safety Analysis
- Social Sciences › Decision Sciences › Management Science and Operations Research › Leadership, Behavior, and Decision-Making Studies
- Social Sciences › Decision Sciences › Management Science and Operations Research › Multi-Criteria Decision Making
- Keywords
- risk matrix, ordinal scale, risk assessment, semi-quantitative risk analysis, range compression, ambiguity, uncertainty, quantitative risk assessment, decision-making
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
