The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Risk and Uncertainty Communication
Abstract
This review briefly examines the vast range of techniques used to communicate risk assessments arising from statistical analysis. After discussing essential psychological and sociological issues, I focus on individual health risks and relevant research on communicating numbers, verbal expressions, graphics, and conveying deeper uncertainty. I then consider practice in a selection of diverse case studies, including gambling, the benefits and risks of pharmaceuticals, weather forecasting, natural hazards, climate change, environmental exposures, security and intelligence, industrial reliability, and catastrophic national and global risks. There are some tentative final conclusions, but the primary message is to acknowledge expert guidance, be clear about objectives, and work closely with intended audiences.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“research shows that humans in general suffer from a variety of cognitive biases, some of which hinder our understanding of uncertainty [60,107,115].”
From Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty (2021) · cites this paper as [115]
Citations
Cites
GRADE guidelines: 3. Rating the quality of evidence
Balshem, H., Helfand, M., Schünemann, H. J., Oxman, A. D., Kunz, R., Brozek, J., Vist, G. E., Falck-Ytter, Y., Meerpohl, J., & Norris, S. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.015 · Cited by 8,285
“Risk, Ambiguity, and the Savage Axioms.”
Ellsberg, D. (1961) · The Quarterly Journal of Economics · doi:10.2307/1884324 · Cited by 7,891
Visualizing Uncertainty About the Future
Spiegelhalter, D., Pearson, M., & Short, I. (2011) · Science · doi:10.1126/science.1191181 · Cited by 682 · cited 1×
“…many systematic and nonsystematic reviews of the use of graphics in risk communication, primarily in the health fields (Ancker et al. 2006, Visschers et al. 2009, Spiegelhalter et al. 2011, Hildon et al. 2012, Stephens et al. 2012, Trevena et al. 2013, Zipkin et al. 2014, Hallgreen et al. 2016).”
BackgroundSelf-citationThe role of social and decision sciences in communicating uncertain climate risks
Pidgeon, N., & Fischhoff, B. (2011) · Nature Climate Change · doi:10.1038/nclimate1080 · Cited by 680
The interpretation of IPCC probabilistic statements around the world
Budescu, D. V., Por, H.-H., Broomell, S. B., & Smithson, M. (2014) · Nature Climate Change · doi:10.1038/nclimate2194 · Cited by 291
Assessing and presenting summaries of evidence in Cochrane Reviews
Langendam, M. W., Akl, E. A., Dahm, P., Glasziou, P., Guyatt, G., & Schünemann, H. J. (2013) · Systematic Reviews · doi:10.1186/2046-4053-2-81 · Cited by 282
Research works printed but not linked: 124
- Akl EA, 2011, Cochrane Database Syst. Rev., 3, CD006776
- 10.1197/jamia.M2115 · DOI
- 10.1177/0272989X10369006 · DOI
- ASQ (Am. Soc. for Qual.), 2004, Failure Mode Effects Analysis (FMEA)
- Bammer G, 2008, Uncertainty and Risk: Multidisciplinary Perspectives
- 10.1016/j.envint.2015.12.031 · DOI
- 10.1016/j.envhaz.2007.08.004 · DOI
- Bouder F, 2007, The Tolerability of Risk: A New Framework for Risk Management
- 10.1175/BAMS-88-5-651 · DOI
- Cancer Research UK, 2015, Lifetime Risk of Cancer
- Carlyle T, 1871, Oliver Cromwell's Letters and Speeches: With Elucidations
- Channel 4, 2011, Channel 4 Television Documentary
- COMEAP (Comm. Med. Eff. Air Pollut.)., 2009, Long-term exposure to air pollution: effect on mortality
- 10.1177/0272989X07306783 · DOI
- Cvetkovich G, 2013, Social Trust and the Management of Risk, 10.4324/9781315071350 · DOI
- 10.1175/BAMS-D-11-00150.1 · DOI
- Dep. Homel. Secur., 2011, The Strategic National Risk Assessment in Support of PPD 8: A Comprehensive Risk-Based Approach Toward a Secure and Resilient Nation
- 10.1515/9781400835287 · DOI
- 10.1111/risa.12358 · DOI
- Douglas M, 1983, Risk and Culture: An Essay on the Selection of Technological and Environmental Dangers, 10.1525/9780520907393 · DOI · Paper page
- 10.2903/j.efsa.2015.3978 · DOI
- EMA (Eur. Med. Agency)., 2014, Benefit-Risk Methodology Project. Update on Work Package 5: Effects Table Pilot (Phase I)
- EMA (Eur. Med. Agency)., 2016, Section 4.8: Undesirable Effects
- EPA (U.S. Environ. Prot. Agency)., 2011, Vocabulary catalog: Integrated Risk Information System (IRIS) glossary
- 10.1177/0272989X05278931 · DOI
- 10.1093/jnci/djr318 · DOI
- 10.1177/0272989X06297101 · DOI
- 10.1111/j.1539-6924.1995.tb00308.x · DOI · Paper page
- Fischhoff B, 2006, Horizons, 10, 3
- Fischhoff B, 2011, Communicating Risks and Benefits: An Evidence-Based User's Guide
- 10.1073/pnas.1317504111 · DOI · Paper page
- Friedman JA, 2015, Intell. Natl. Secur., 30, 1, 10.1080/02684527.2014.972613 · DOI
- 10.1037/a0024850 · DOI
- 10.1001/archinternmed.2009.481 · DOI · Paper page
- Galesic M, 2009, Health Psychol. Off. J. Div. Health Psychol. Am. Psychol. Assoc., 28, 210
- 10.1111/tops.12172 · DOI
- 10.1177/0272989X10369000 · DOI
- 10.1136/bmj.g7091 · DOI
- 10.1136/bmj.e245 · DOI
- 10.1111/j.1539-6924.2005.00608.x · DOI
- 10.1038/477264a · DOI
- 10.1002/pds.3880 · DOI
- 10.1177/0272989X10371830 · DOI · Paper page
- 10.3399/bjgp14X677824 · DOI
- 10.1093/intqhc/mzr072 · DOI
- 10.1002/pds.3958 · DOI
- IPCC (Intergov. Panel Clim. Change)., 2013, Climate Change 2013: The Physical Science Basis. Contribution of Working Group I to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change
- ISO (Int. Stan. Organ.), 2009, ISO 31000—Risk Management
- 10.1109/IEEM.2012.6838079 · DOI
- 10.1016/S0304-3894(02)00283-2 · DOI
- 10.1002/met.121 · DOI
- 10.22323/2.14030404 · DOI
- Kahneman D, 2006, Judgment under Uncertainty: Heuristics and Biases
- 10.1111/j.1539-6924.2012.01844.x · DOI
- 10.1111/j.1539-6924.2009.01261.x · DOI
- 10.1136/qshc.2003.009076 · DOI
- 10.1111/hex.12344 · DOI
- Knight F, 1921, Risk, Uncertainty and Profit
- 10.1111/risa.12531 · DOI
- 10.1111/j.1539-6924.2010.01511.x · DOI
- 10.1177/0002716296545001009 · DOI
- 10.1111/j.1539-6924.2007.00940.x · DOI
- 10.1111/j.1539-6924.2011.01722.x · DOI
- 10.1177/2047487313518479 · DOI
- Marchio J, 2014, Stud. Intell., 58, 31
- 10.1002/9780470061596 · DOI
- Moore RA, 2014, Cochrane Database Syst. Rev., 3, CD007938
- Morgan G, 2009, Best Practice Approaches for Characterizing, Communicating, and Incorporating Scientific Uncertainty in Climate Decision Making
- 10.1017/CBO9780511814679 · DOI
- 10.1016/j.envint.2016.01.004 · DOI
- 10.1175/2008WAF2007088.1 · DOI
- NRC (Natl. Res. Counc.), 1989, Improving Risk Communication.
- 10.1136/bmj.327.7417.745 · DOI
- 10.1002/hyp.9253 · DOI
- 10.1196/annals.1399.001 · DOI
- 10.1037/a0016978 · DOI
- 10.1177/0272989X10391672 · DOI
- 10.1002/pd.4606 · DOI
- 10.1002/pds.3859 · DOI
- 10.1177/0272989X12451057 · DOI
- 10.1111/bjop.12081 · DOI
- 10.1111/risa.12453 · DOI
- Ropeik D, 2010, How Risky Is It, Really? Why Our Fears Don't Always Match the Facts
- Sandberg A, Bostrom N. 2008. Global catastrophic risks survey. Tech. Rep. 2008–1, Future Humanity Inst., Univ. Oxford
- 10.1111/j.1539-6924.1993.tb01321.x · DOI
- 10.1111/j.1539-6924.1994.tb00026.x · DOI
- 10.1080/01621459.1951.10500768 · DOI
- Savage LJ, 1972, The Foundations of Statistics
- 10.1111/risa.12169 · DOI
- 10.1073/pnas.1214646110 · DOI
- 10.1177/0272989X13514776 · DOI
- 10.1111/j.1539-6924.2011.01773.x · DOI
- 10.1126/science.3563507 · DOI · Paper page
- Slovic P, 2000, The Perception of Risk
- 10.1111/j.1467-8721.2006.00461.x · DOI · Paper page
- 10.1186/1472-6947-8-31 · DOI
- 10.1136/bmj.324.7332.261 · DOI
- 10.1136/bmj.e8223 · DOI
- 10.1111/1471-0528.12663 · DOI
- 10.1098/rsta.2011.0163 · DOI
- 10.1002/wcc.187 · DOI
- 10.1038/sj.embor.7400953 · DOI
- 10.1016/S0749-5978(03)00003-7 · DOI
- 10.1213/ANE.0b013e3181e8570a · DOI
- Taleb N, 2007, The Black Swan: the Impact of the Highly Improbable
- 10.1111/j.1365-2753.2005.00596.x · DOI
- 10.1186/1472-6947-13-S2-S7 · DOI
- UK Cabinet Off., 2010, Fact Sheet2: National Security Risk Assessment
- UK Cabinet Off., 2011, Communicating Risk Guidance
- UK Cabinet Off., 2012, National Risk Register for Civil Emergencies—2012 Update
- UK Cabinet Off., 2015, National Risk Register for Civil Emergencies—2015 Edition
- 10.1177/1075547015614970 · DOI
- 10.1146/annurev-statistics-062713-085841 · DOI
- 10.1111/j.1539-6924.2008.01137.x · DOI
- 10.1007/978-3-642-12233-0_18 · DOI
- World Econ. Forum., 2016, The Global Risks Report 2016, 11th Edition
- 10.1177/016224399101600108 · DOI · Paper page
- 10.1002/(SICI)1099-0720(199712)11:6<495::AID-ACP481>3.0.CO;2-J · DOI · Paper page
- 10.1177/0272989X13516198 · DOI
- 10.2196/jmir.1665 · DOI
- 10.1177/0272989X10364244 · DOI
- 10.1016/j.pec.2008.05.010 · DOI
- 10.2196/jmir.2030 · DOI
- 10.7326/M14-0295 · DOI
Cited by
Sorted by citations
Using Behavioral Science to help fight the Coronavirus
Lunn, P., Belton, C. A., Lavin, C., McGowan, F., Timmons, S., & Robertson, D. A. (2020) · Journal of Behavioral Public Administration · doi:10.30636/jbpa.31.147 · Cited by 316
Presenting machine learning model information to clinical end users with model facts labels
Sendak, M. P., Gao, M., Brajer, N., & Balu, S. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0253-3 · Cited by 279
Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty
Bhatt, U., Antorán, J., Zhang, Y., Liao, Q. V., Sattigeri, P., Fogliato, R., Melançon, G., Krishnan, R., Stanley, J., Tickoo, O., et al. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462571 · Cited by 300
Towards operational predictions of the near-term climate
Kushnir, Y., Scaife, A. A., Arritt, R., Balsamo, G., Boer, G., Doblas-Reyes, F., Hawkins, E., Kimoto, M., Kolli, R., Kumar, A., et al. (2019) · Nature Climate Change · doi:10.1038/s41558-018-0359-7 · Cited by 168
Guidance on Communication of Uncertainty in Scientific Assessments
Hart, A., Maxim, L., Siegrist, M., von Goetz, N., da Cruz, C., Merten, C., Mosbach-Schulz, O., Lahaniatis, M., Smith, A., & Hardy, A. (2019) · EFSA journal. European Food Safety Authority · doi:10.2903/j.efsa.2019.5520 · Cited by 134
“• Online versions of communication documents could provide links between probability terms and optional graphic displays related to the sources of uncertainty and to uncertainty scales (Spiegelhalter et al., 2011; Spiegelhalter, 2017).”
BackgroundMethodInfluentialEvaluating language models for mathematics through interactions
Collins, K. M., Jiang, A. Q., Frieder, S., Wong, L., Zilka, M., Bhatt, U., Lukasiewicz, T., Wu, Y., Tenenbaum, J. B., Hart, W., et al. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2318124121 · Cited by 108
Reynolds, J., Archer, S., Pilling, M., Kenny, M. J., Hollands, G., & Marteau, T. M. (2019) · Social Science & Medicine (1967) · doi:10.1016/j.socscimed.2019.112395 · Cited by 111
“The extensive literature on the communication of risks, harms and benefits to individuals provides some guidance for communicating similar types of information, albeit about harms and benefits of a policy for a population or group (Brick et al., 2018; Spiegelhalter, 2017).”
BackgroundReynolds, J., Stautz, K., Pilling, M., van der Linden, S., & Marteau, T. (2020) · Royal Society Open Science · doi:10.1098/rsos.190522 · Cited by 109
“Further research should investigate optimal communication methods from the perspectives of comprehension, accuracy and successful belief change [8,48].”
BackgroundCurrent Challenges When Using Numbers in Patient Decision Aids: Advanced Concepts
Trevena, L., Bonner, C., Okan, Y., Peters, E., Gaissmaier, W., Han, P., Ozanne, E., Timmermans, D., & Zikmund-Fisher, B. (2021) · Medical decision making · doi:10.1177/0272989x21996342 · Cited by 95
“(30, 53) This could be achieved by building on recent efforts to develop a unifying framework for understanding decision making with visualizations.”
BackgroundClinical utility and implementation of polygenic risk scores for predicting cardiovascular disease
Schunkert, H., Angelantonio, E. D., Inouye, M., Patel, R. S., Ripatti, S., Widén, E., Sanderson, S. C., Kaski, J. P., McEvoy, J. W., Vardas, P. E., et al. (2024) · European Heart Journal · doi:10.1093/eurheartj/ehae649 · Cited by 107
Kloecker, D. E., Davies, M. J., Khunti, K., & Zaccardi, F. (2020) · Annals of Internal Medicine · doi:10.7326/m19-3286 · Cited by 93
“For example, large HRs may translate to negligible absolute clinical differences or, conversely, small differences to large absolute benefits when the risk for the outcome is high (2).”
BackgroundPadilla, L., Powell, M., Kay, M., & Hullman, J. R. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.579267 · Cited by 82
“The second category of uncertainty is comprised of those uncertainties that cannot be quantified directly, e.g., ontological (Spiegelhalter, 2017), ambiguity or true uncertainty (Knight, 2012), and second-order uncertainty (Bach et al., 2009).”
BackgroundAgerbo, E., Trabjerg, B. B., Børglum, A. D., Schork, A. J., Vilhjálmsson, B. J., Pedersen, C. B., Hakulinen, C. A., Albiñana, C., HOUGAARD, D. M., Grove, J., et al. (2021) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2020.4172 · Cited by 79
Glycaemic control is still central in the hierarchy of priorities in type 2 diabetes management
Khunti, K., Zaccardi, F., Amod, A., Aroda, V. R., Montoya, P. A., Colagiuri, S., Mohan, V., & Chan, J. C. N. (2024) · Diabetologia · doi:10.1007/s00125-024-06254-w · Cited by 78
Renn, O., Baghramian, M., Capaccioli, M., de Rijcke, S., Drotner, K., Dubertret, L., Irwin, A., Luty, T., Makarow, M., Moberg, C., et al. (2019) · CBS Research Portal (Copenhagen Business School) · doi:10.26356/masos · Cited by 84
“It can also be done qualitatively, e.g. by accompanying the subjective probability with a qualitative expression of the assessors’ degree of confidence in their assessment of that probability (Spiegelhalter, 2017).”
BackgroundInfluentialCommunicating uncertainty using words and numbers
Dhami, M. K., & Mandel, D. R. (2022) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2022.03.002 · Cited by 69
“Recent reviews focus on applied domains [87,89–93] or specific readerships [94], while others consider the context of”
Anderson, M., Pitchforth, E., Asaria, M., Brayne, C., Casadei, B., Charlesworth, A., Coulter, A., Franklin, B. D., Donaldson, C., Drummond, M., et al. (2021) · The Lancet · doi:10.1016/s0140-6736(21)00232-4 · Cited by 116
Chen, T., Dai, M., Xia, S., & Zhou, Y. (2021) · Health Communication · doi:10.1080/10410236.2021.1876814 · Cited by 76
“Inspired by the results of Tversky and Kahneman’s (1981) study, Rothman and Salovey (1997) suggested that gain frames may work well when designing messages that advocate preven-”
BackgroundMethodInfluentialSocial perception of systemic risks
Schweizer, P., Goble, R., & Renn, O. (2022) · Risk Analysis · doi:10.1111/risa.13831 · Cited by 63
“Scientists are required to mention and characterize uncertainty, explicitly stress the stochastic nature of what we know, and point out the various ambiguities in the interpretation of complex relationships (Spiegelhalter, 2017).”
BackgroundBostrom, A., Demuth, J. L., Wirz, C. D., Cains, M. G., Schumacher, A., Madlambayan, D., Bansal, A. S., Bearth, A., Chase, R. J., Crosman, K. M., et al. (2023) · Risk Analysis · doi:10.1111/risa.14245 · Cited by 81
“Risk communication research (e.g., MacEachren et al., 2012; Padilla et al., 2018, 2023; Spiegelhalter, 2017; van der Bles et al., 2019, 2020) shows, however, that the effects of visualizations and other communications of uncertainty on trust are likely to depend on the quality and context of…”
BackgroundReynolds, J., Pilling, M., & Marteau, T. M. (2018) · Social Science & Medicine (1967) · doi:10.1016/j.socscimed.2018.09.037 · Cited by 57
Webb, E., Meads, D. M., Eskytė, I., King, N. V., Dracup, N., Chataway, J., Ford, H. L., Marti, J., Pavitt, S. H., Schmierer, K., et al. (2018) · Patient · doi:10.1007/s40271-017-0296-y · Cited by 49
“It is generally difficult to appropriately communicate probabilities (see Spiegelhalter [50] and Apter et al.”
BackgroundRisk communication in tables versus text: a registered report randomized trial on ‘fact boxes'
Brick, C., McDowell, M., & Freeman, A. L. J. (2020) · Royal Society Open Science · doi:10.1098/rsos.190876 · Cited by 63
A user-centred design framework for disaster risk visualisation
Twomlow, A. E., Grainger, S., Cieslik, K., Paul, J. D., & Buytaert, W. (2022) · International Journal of Disaster Risk Reduction · doi:10.1016/j.ijdrr.2022.103067 · Cited by 62
Comparative Relevance of Physical Fitness and Adiposity on Life Expectancy
Zaccardi, F., Davies, M. J., Khunti, K., & Yates, T. E. (2019) · Mayo Clinic Proceedings · doi:10.1016/j.mayocp.2018.10.029 · Cited by 45
The communication of uncertainty in health: A scoping review
Kalke, K., Studd, H., & Scherr, C. L. (2021) · Patient Education and Counseling · doi:10.1016/j.pec.2021.01.034 · Cited by 52
Machine Learning Explainability for External Stakeholders
Bhatt, U., Andrus, M., Weller, A., & Xiang, A. (2020) · arXiv · doi:10.48550/arxiv.2007.05408 · Cited by 41
“Luckily, once uncertainty is accurately measured, there is a plethora of work on conveying model confidence (more generally, on communicating statistics) (Spiegelhalter, 2017; Hullman, 2019).”
BackgroundHart, A., Embleton, N., Bradburn, M., Connolly, D., Mandefield, L., Mooney, C., & Griffiths, P. (2019) · The Lancet Child & Adolescent Health · doi:10.1016/s2352-4642(19)30349-9 · Cited by 41
Tervonen, T., Veldwijk, J., Payne, K., Ng, X.-Y., Levitan, B., Lackey, L., Marsh, K., Thokala, P., Pignatti, F., Donnelly, A. M., et al. (2023) · Value in Health · doi:10.1016/j.jval.2022.12.006 · Cited by 39
Castro, S., Quinan, P. S., Hosseinpour, H., & Padilla, L. M. K. (2021) · IEEE Transactions on Visualization and Computer Graphics · doi:10.31234/osf.io/wpz8b · Cited by 38
“Although the term uncertainty has varying meanings across domains (for an expanded discussion, see [85]), in this paper, we focus on quantified uncertainty, such as in distributions.”
BackgroundNormalizing Flows for Interventional Density Estimation
Melnychuk, V., Frauen, D., & Feuerriegel, S. (2022) · International Conference on Machine Learning · doi:10.48550/arxiv.2209.06203 · Cited by 36
“However, making decisions based on averaged causal quantities can be misleading and, in some applications, even dangerous (Spiegelhalter, 2017; van der Bles et al., 2019).”
BackgroundKeogh, R. H., Seaman, S., Barrett, J. K., Taylor-Robinson, D., & Szczesniak, R. (2018) · Epidemiology · doi:10.1097/ede.0000000000000920 · Cited by 35
Impact of COVID-19 forecast visualizations on pandemic risk perceptions
Padilla, L., Hosseinpour, H., Fygenson, R., Howell, J. L., Chunara, R., & Bertini, E. (2022) · Scientific Reports · doi:10.1038/s41598-022-05353-1 · Cited by 44
“Participants answered the four questions below about (1) themselves, (2) an average 22-year-old in their state who is generally in good health, and (3) an average 78-year-old in their state who is generally in good health (referred to as target type).”
BackgroundInfluentialDasgupta, A., Arnal, L., Emerton, R., Harrigan, S., Matthews, G., Muhammad, A., O'regan, K. L., Perez-Ciria, T., Valdez, E., van Osnabrugge, B., et al. (2023) · Journal of Flood Risk Management · doi:10.1111/jfr3.12880 · Cited by 33
“Effective communication of forecast uncertainty is needed in order to translate the technical improvements of the Hydrologic Ensemble Forecasting Systems (HEFS) into practical benefits ([ES-K; WP-K]; Spiegelhalter, 2017).”
Climate uncertainty and policy making—what do policy makers want to know?
Meah, N. (2019) · Regional Environmental Change · doi:10.1007/s10113-019-01492-w · Cited by 44
“Perhaps the most intuitively accessible articulation of uncertainty is that given by Donald Rumsfeld—which has received a surprising amount of academic comment and analysis (e.g. Spiegelhalter 2017)—where he says: there are known knowns; there are things we know we know.”
BackgroundKliskey, A., Williams, P., Trammell, E., Cronan, D., Griffith, D., Alessa, L., Lammers, R., de Haro-Martí, M., & Oxarango-Ingram, J. (2023) · Frontiers in Environmental Science · doi:10.3389/fenvs.2023.1007105 · Cited by 32
“Scientists often struggle with communicating likelihood of occurrence (confidence intervals) to stakeholders (Spiegelhalter, 2017).”
BackgroundAm I Private and If So, how Many?
Franzén, D., von Voigt, S. N., Sörries, P., Tschorsch, F., & Müller-Birn, C. (2022) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3548606.3560693 · Cited by 23
“, [60, 61]) recommend presenting the probabilities of both events together.”
BackgroundCritical volcanology? Thinking holistically about risk and uncertainty
Donovan, A. R. (2019) · Bulletin of Volcanology · doi:10.1007/s00445-019-1279-8 · Cited by 39
“…also considerable differences in how uncertainty is managed or even identified between disciplines—dealing with uncertainty about a medical prognosis on an individual scale is different from managing hurricanes (Spiegelhalter 2008, 2017; Joslyn and Savelli 2010; Stephens et al. 2012; Beven 2016).”
BackgroundDynamic predictive probabilities to monitor rapid cystic fibrosis disease progression
Szczesniak, R. D., Su, W., Brokamp, C., Keogh, R. H., Pestian, J. P., Seid, M., Diggle, P. J., & Clancy, J. P. (2020) · Statistics in Medicine · doi:10.1002/sim.8443 · Cited by 30
Holmes, J., Angus, C., Meier, P., Buykx, P., & Brennan, A. (2018) · Addiction · doi:10.1111/add.14381 · Cited by 30
Adam, S., Urman, A., Arlt, D., Gil-López, T., Makhortykh, M., & Maier, M. (2022) · Communication Research · doi:10.1177/00936502221127484 · Cited by 29
“Media trust shapes the audience selection of news (Tsfati & Cappella, 2003) as well as the potential of information to have an influence: trusted sources affect our opinions and behaviors whereas non-trusted sources are less likely to do so (e.g., Ladd, 2012; Spiegelhalter, 2017).”
BackgroundA Probabilistic Grammar of Graphics
Pu, X., & Kay, M. (2020) · International Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376466 · Cited by 29
“In addition to the single probability distribution PGoG handles, the taxonomies cover more complex data structures and semantics, such as the epistemic v.s. aleatory distinction [48] and errors [52].”
BackgroundDiverse, Global and Amortised Counterfactual Explanations for Uncertainty Estimates
Ley, D., Bhatt, U., & Weller, A. (2021) · AAAI Conference on Artificial Intelligence · doi:10.48550/arxiv.2112.02646 · Cited by 27
“As these systems are deployed in high stakes settings, well-calibrated uncertainty estimates are in high demand (Spiegelhalter 2017).”
BackgroundVromans, R., Pauws, S. C., Bol, N., van de Poll-Franse, L. V., & Krahmer, E. (2020) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-020-01296-7 · Cited by 25
Economics for an uncertain world
Demartino, G. F., Grabel, I., & Scoones, I. (2024) · World Development · doi:10.1016/j.worlddev.2023.106426 · Cited by 24
Reflections on 50 years of MCDM: Issues and future research needs
French, S. (2023) · EURO Journal on Decision Processes · doi:10.1016/j.ejdp.2023.100030 · Cited by 29
Oxman, A. D., Glenton, C., Flottorp, S. A., Lewin, S. A., Rosenbaum, S., & Fretheim, A. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-036348 · Cited by 35
“Risk and uncertainty communication 53 Explores the major issues in communicating risk assessments arising from statistical analysis and concludes with a set of recommendations.”
Vitruvian plot: a visualisation tool for multiple outcomes in network meta-analysis
Ostinelli, E., Efthimiou, O., Naci, H., Furukawa, T., Leucht, S., Salanti, G., Wainwright, L., Zangani, C., Crescenzo, F. D., Smith, K. A., et al. (2022) · Evidence-Based Mental Health · doi:10.1136/ebmental-2022-300457 · Cited by 23
Winners and losers: communicating the potential impacts of policies
Brick, C., Freeman, A. L. J., Wooding, S., Skylark, W. J., Marteau, T. M., & Spiegelhalter, D. J. (2018) · Humanities and Social Sciences Communications · doi:10.1057/s41599-018-0121-9 · Cited by 30
“For individual decisions about personal health or finances, much is known about how to present potential harms and benefits to effectively inform rather than persuade (e.g., Fischhoff et al., 2011; Spiegelhalter, 2017).”
BackgroundSelf-citationEffective sample size: A measure of individual uncertainty in predictions
Thomassen, D., le Cessie, S., van Houwelingen, H. V., & Steyerberg, E. (2023) · Statistics in Medicine · doi:10.1002/sim.10018 · Cited by 22
“It has also been suggested to communicate numbers only when they are estimated with sufficient certainty [8].”
BackgroundBlack Swan Events and Intelligent Automation for Routine Safety Surveillance
Kjoersvik, O., & Bate, A. (2022) · Drug Safety · doi:10.1007/s40264-022-01169-0 · Cited by 24
“Humans are often irrational when considering risk and probabilities [18].”
BackgroundFish harvesting advice under climate change: A risk-equivalent empirical approach
Duplisea, D., Roux, M., Hunter, K., & Rice, J. (2020) · bioRxiv · doi:10.1371/journal.pone.0239503 · Cited by 22
Haug, S., Schnell, R., & Weber, K. (2021) · Das Gesundheitswesen · doi:10.1055/a-1538-6069 · Cited by 21
“Um Wahrschein-lichkeiten unter 1 % zu veranschaulichen bieten sich sogenannte „Icon Arrays“ als einfache, verständliche, grafische Darstellungen an [25, 26].”
BackgroundLindell, M., Bostrom, A., Goltz, J., & Prater, C. S. (2021) · International Journal of Disaster Risk Reduction · doi:10.1016/j.ijdrr.2021.102361 · Cited by 21
Cristina, N., Alan, R., Kanungo, D., Sarah, D., Thomas, G., Christian, K., Bastian, K., Konrad, J., Charlene, L., Olivier, V., et al. (2025) · Trends in Food Science & Technology · doi:10.1016/j.tifs.2025.104987 · Cited by 20
Woof, V. G., Howell, A., McWilliams, L., Evans, D., & French, D. (2022) · British Journal of Cancer · doi:10.1038/s41416-022-01944-x · Cited by 20
“Numerical risk information and the use of risk communication strategies produce little effect in comparison to the impact that personal experiences and affective responses can have on subjective risk appraisals [8].”
BackgroundRipberger, J., Bell, A., Fox, A., Forney, A., Livingston, W., Gaddie, C., Silva, C. L., & JENKINS-SMITHa, H. (2022) · Weather, Climate, and Society · doi:10.1175/wcas-d-21-0034.1 · Cited by 20
Anticipatory Ethics and the Role of UncertaintyPreprint
Nanayakkara, P., Diakopoulos, N., & Hullman, J. (2020) · arXiv.org · doi:10.48550/arxiv.2011.13170 · Cited by 20
“Ontological uncertainty is the most philosophical in nature, describing “uncertainty about the entire modeling process as a description of reality” [25].”
Xu, Z., Song, T., & Lee, Y.-C. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2025.103455 · Cited by 18
Doyle, E., Thompson, J., Hill, S., Williams, M., Paton, D., Harrison, S., Bostrom, A., & Becker, J. (2023) · International Journal of Disaster Risk Reduction · doi:10.1016/j.ijdrr.2023.103948 · Cited by 18
Learning Personalized Decision Support Policies
Bhatt, U., Chen, V., Collins, K. M., Kamalaruban, P., Kallina, E., Weller, A., & Talwalkar, A. (2023) · AAAI Conference on Artificial Intelligence · doi:10.48550/arxiv.2304.06701 · Cited by 18
Agent-based modelling for Urban Analytics: State of the art and challenges
Malleson, N., Birkin, M., Birks, D., Ge, J.-Q., Heppenstall, A., Manley, E., McCulloch, J., & Ternes, P. (2022) · AI Communications · doi:10.3233/aic-220114 · Cited by 18
“Predictions are often highly uncertain and it is crucial that these uncertainties are communicated properly [75,84], but this uncertainty in outcomes can be challenging for practitioners.”
Keage, H., & Loetscher, T. (2018) · PLoS ONE · doi:10.1371/journal.pone.0207356 · Cited by 18
Chan, H. W., Russell, A. M., & Smith, M. Y. (2018) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.4614 · Cited by 18
Zaccardi, F., Rowlands, A. V., Dempsey, P. C., Razieh, C., Henson, J. J., Goldney, J., Maylor, B. D., Bhattacharjee, A., Chudasama, Y., Edwardson, C. L., et al. (2024) · Journal of sport and health science/Journal of Sport and Health Science · doi:10.1016/j.jshs.2024.100970 · Cited by 24
“16,17 A previous analysis investigated the association of self-reported MVPA and objectively measured total PA with life expectancy, showing high vs. low categories of PA were associated with a 3 (cid:1) 5 year difference in life expectancy at 45 years of age.”
Hakulinen, C. A., Komulainen, K., Suokas, K., Pirkola, S. P., Pulkki-Råbäck, L., Lumme, S., Elovainio, M. J., & Böckerman, P. (2023) · Journal of Epidemiology & Community Health · doi:10.1136/jech-2022-219674 · Cited by 45
Cristea, F., Weishaar, H., Geurts, B., Delamou, A., Tan, M., Legido-Quigley, H., Aminu, K., Marí-Saéz, A., Rocha, C., Camara, B., et al. (2022) · Globalization and Health · doi:10.1186/s12992-022-00857-x · Cited by 17
“Previous examples include epistemic uncertainty (about facts, numbers, and science) [20] or aleatory uncertainty (inevitable unpredictability of the future due to unforeseeable factors, commonly used in statistical modeling of risk) [21].”
BackgroundHart, A., Vollmer, B., Howe, D., Boxall, S., Foulds, N., de Lacy, P., Vasudevan, C., Griffiths, P., & Piercy, H. (2021) · Developmental Medicine & Child Neurology · doi:10.1111/dmcn.15022 · Cited by 17
“001 1/100 000 Spiegelhalter et al.(33) Probability intervals and terminology used by the inter-governmental panel for climate change Virtually certain 99–100 Extremely likely 95–100 Very likely 90–100 Likely 66–100 More likely than not 50–100 About as likely as not 33–66 Unlikely 0–33 Very unlikely 0–10 Exceptionally…”
BackgroundInfluentialMultiple Hazard Uncertainty Visualization Challenges and Paths Forward
Padilla, L. M. K., Dryhurst, S., Hosseinpour, H., & Kruczkiewicz, A. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.579207 · Cited by 17
Wiedemann, P., Boerner, F., & Freudenstein, F. (2021) · PLoS ONE · doi:10.1371/journal.pone.0253762 · Cited by 17
Okan, Y., Stone, E. R., Parillo, J., de Bruin, W. B., & Parker, A. M. (2020) · Risk Analysis · doi:10.1111/risa.13431 · Cited by 17
“The question of how best to communicate risk has received a lot of attention (Spiegelhalter, 2017; Trevena et al., 2013; Zipkin et al., 2014).”
BackgroundEnhancing central bank communications with behavioural insightsPreprint
Bholat, D., Broughton, N., Parker, A., Termeer, J., & Walczak, E. J. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3233695 · Cited by 17
“…have found that making information relevant to individual circumstances can increase engagement (Garner 2005; Behavioural Insights Team 2012), and that expressing financial costs in pound values instead of percentages can improve comprehension (Gigerenzer and Edwards 2003; Spiegelhalter 2017).”
A review of uncertainty visualization errors: Working memory as an explanatory theory
Padilla, L. M. K., Castro, S., & Hosseinpour, H. (2021) · The Psychology of learning and motivation/The psychology of learning and motivation · doi:10.1016/bs.plm.2021.03.001 · Cited by 17
Risk Assessment of Chemical Mixtures in Foods: A Comprehensive Methodological and Regulatory Review
Combarros, R. G., García, M. G., Blanco-Díaz, G. D., Bravo, K. S., Moya, J. L. R., & López-Sánchez, J. I. (2026) · Foods · doi:10.3390/foods15020244 · Cited by 15
“…clear narratives, contribution plots and intuitive analogies to explain mixture metrics and uncertainty while maintaining scientific accuracy, thereby supporting informed, transparent decision-making on the management of chemical “cocktails” in food and other exposure contexts [87,89,90,109].”
InfluentialSanderson, S. C., & Inouye, M. (2025) · Nature Human Behaviour · doi:10.1038/s41562-025-02200-x · Cited by 15
Farmer, G. D., Pearson, M., Skylark, W., Freeman, A., & Spiegelhalter, D. (2021) · Cancer Medicine · doi:10.1002/cam4.4072 · Cited by 15
“We also did not foresee that, although recommended in the risk communication literature, 13,28,29 icon arrays would be deemed to be distressingly morbid when human-shaped.”
BackgroundSelf-citationAmericans’ Views of Fusion Energy: Implications for Sustainable Public Support
Gupta, K., Jenkins-Smith, H., Ripberger, J., Silva, C. L., Fox, A., & Livingston, W. (2024) · Fusion science and technology · doi:10.1080/15361055.2024.2328457 · Cited by 14
“15] While nuclear fission continues to provide the largest share of clean energy for the U.S. grid, the enduring legacy of nuclear accidents and elevated risk perceptions have made it difficult to site nuclear facilities.”
Multiple models for outbreak decision support in the face of uncertainty
Shea, K., Borchering, R., Probert, W., Howerton, E., Bogich, T., Li, S.-L., van Panhuis, W. V., Viboud, C., Aguás, R., Belov, A., et al. (2023) · Proceedings of the National Academy of Sciences of the United States of America · doi:10.1073/pnas.2207537120 · Cited by 14
“The ensemble provided valuable risk quantification for decision making (20) since all models provided projections as probability distributions.”
Communication of Statistics and Evidence in Times of Crisis
Schneider, C., Kerr, J. R., Dryhurst, S., & Aston, J. (2023) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-040722-052011 · Cited by 14
“One clear way to avoid the effects of such framing is to provide the statistic from both sides (Gigerenzer, 2014; Spiegelhalter, 2017).”
BackgroundMethodInfluentialCommunicating personalised risks from COVID-19: guidelines from an empirical studyPreprint
Freeman, A., Kerr, J. R., Recchia, G., Schneider, C., Lawrence, A. C. E., Finikarides, L., Luoni, G., Dryhurst, S., & Spiegelhalter, D. (2020) · medRxiv · doi:10.1101/2020.10.05.20206961 · Cited by 14
“Numbers on their own do not convey a perception of quantity – they need to be given context, particularly when the numbers are small [29].”
BackgroundSelf-citationInterpretable Machine Learning for Privacy-Preserving Pervasive Systems
Baron, B., & Musolesi, M. (2017) · IEEE pervasive computing · doi:10.1109/mprv.2019.2918540 · Cited by 14
“Indeed, risk itself is difficult to communicate (see for instance [18]).”
BackgroundData-driven recommendations for enhancing real-time natural hazard warnings
Saunders, K., Forbes, O., Hopf, J. K., Patterson, C. R., Vollert, S. A., Brown, K., Browning, R., Canizares, M., Cottrell, R. S., Li, L., et al. (2025) · One Earth · doi:10.1016/j.oneear.2025.101274 · Cited by 16
Runge, M. C., Shea, K., Howerton, E., Yan, K.-T., Hochheiser, H., Rosenstrom, E. T., Probert, W., Borchering, R., Marathe, M., Lewis, B., et al. (2024) · Epidemics · doi:10.1016/j.epidem.2024.100775 · Cited by 13
“Key challenges include communication of the nuances of scenario projections to lay audiences (as different from forecasts), and visualization of uncertainty (Kamal et al., 2021; Hullman et al., 2019; Hägele et al., 2022; Spiegelhalter, 2017).”
Rosen, J., Agrawal, N., Flum, D., & Liao, J. M. (2021) · Annals of Surgery · doi:10.1097/sla.0000000000005255 · Cited by 13
Communicating personalized risks from COVID-19: guidelines from an empirical study
Freeman, A. L. J., Kerr, J. R., Recchia, G. L., Schneider, C. R., Lawrence, A. C. E., Finikarides, L., Luoni, G., Dryhurst, S., & Spiegelhalter, D. J. (2021) · Royal Society Open Science · doi:10.1098/rsos.201721 · Cited by 37
“Numbers on their own do not convey a perception of quantity—they need to be given context, particularly when the numbers are small [29].”
BackgroundSelf-citationSelf-reported walking pace, polygenic risk scores and risk of coronary artery disease in UK biobank.
Zaccardi, F., Timmins, I., Goldney, J., Dudbridge, F., Dempsey, P., Davies, M., Khunti, K., & Yates, T. (2022) · NMCD. Nutrition Metabolism and Cardiovascular Diseases · doi:10.1016/j.numecd.2022.08.021 · Cited by 12
Simple methods for improving the communication of uncertainty in species’ temporal trends
Pescott, O., Stroh, P. A., Humphrey, T., & Walker, K. (2022) · Ecological Indicators · doi:10.1016/j.ecolind.2022.109117 · Cited by 12
Threlkeld, R., Ashiku, L., Canfield, C., Shank, D. B., Schnitzler, M., Lentine, K., Axelrod, D., Battineni, A. C. R., Randall, H., & Dagli, C. (2021) · Current Transplantation Reports · doi:10.1007/s40472-021-00351-0 · Cited by 12
McDowell, M., & Kause, A. (2021) · Risk Analysis · doi:10.1111/risa.13739 · Cited by 12
“communicating uncertainty is that many of the techniques that have been developed have not been empirically tested or compared for communicating different types of uncertainty (Spiegelhalter, 2017; van der Bles et al., 2019).”
BackgroundMethodInfluentialSchäfer, U., Sipos, L., & Müller-Birn, C. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757451 · Cited by 10
“") with a fixed denominator of 100 [28, 148, 164].”
Dhungana, A., Doyle, E. H., McDonald, G. W., & Prasanna, R. (2025) · International Journal of Disaster Risk Reduction · doi:10.1016/j.ijdrr.2025.105260 · Cited by 11
Self-reported walking pace and 10-year cause-specific mortality: A UK biobank investigation
Goldney, J., Dempsey, P. C., Henson, J. J., Rowlands, A. V., Bhattacharjee, A., Chudasama, Y., Razieh, C., Laukkanen, J. A., Davies, M. J., Khunti, K., et al. (2023) · Progress in Cardiovascular Diseases · doi:10.1016/j.pcad.2023.09.003 · Cited by 13
McCartney, M., Sullivan, F., & Heneghan, C. (2020) · BMJ evidence-based medicine · doi:10.1136/bmjebm-2020-111541 · Cited by 11
Berryessa, C. M., & Caplan, J. M. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.02222 · Cited by 13
“As individuals have their own perspectives on risk communication, risk information in relation to crime and public safety can often take on different forms (Spiegelhalter, 2017; Lundgren and McMakin, 2018).”
BackgroundThorneloe, R., Hall, L., Walter, F., Side, L., Lloyd, K., & Smith, S. (2019) · Cancer Prevention Research · doi:10.1158/1940-6207.capr-19-0424 · Cited by 11
“Although these aids frame numerical information using evidence-based approaches (31), they are lengthy documents that contain complex terminology.”
McCrorie, A., Chen, J.-W., Weller, R. R., McGlade, K., Donnelly, C., & Shiri, R. (2018) · Cogent Medicine · doi:10.1080/2331205x.2018.1483591 · Cited by 11
“Icon array was chosen as the infographic to test due to its ability to unambiguously represent part-to-whole relationships while also negating the effects of denominator neglect (Spiegelhalter, 2017; Spiegelhalter, Pearson, & Short, 2011).”
Dynamic calibration with approximate Bayesian computation for a microsimulation of disease spread
Asher, M., Lomax, N., Morrissey, K., Spooner, F., & Malleson, N. (2023) · Scientific Reports · doi:10.1038/s41598-023-35580-z · Cited by 10
Effects of flood hazard visualization format on house purchasing decisions
Dobson, B., Miles-Wilson, J., Gilchrist, I., Leslie, D. S., & Wagener, T. (2018) · Urban Water Journal · doi:10.1080/1573062x.2018.1537370 · Cited by 10
“Gestalt principles) that facilitate the perception and interpretation of visual scenes (e.g. Pinker 1990; Carpenter and Shah, 1998; Kelleher and Wagener 2011; Spiegelhalter 2017), but which have been largely overlooked or underused by risk communicators.”
BackgroundSarma, A., Hedayati, M., & Kay, M. (2025) · International Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713725 · Cited by 9
“…is the quantifiable uncertainty arising due to statistical variability, 2 which we will refer to as probabilistic uncertainty ; the other is the “un-certainty about the modeling process as a description of reality” [57] which, following Ferson and Siegrist [12], we will refer to as incertitude .”
Takoutsing, B., Heuvelink, G., Aynekulu, E., & Shepherd, K. (2024) · Precision Agriculture · doi:10.1007/s11119-024-10200-6 · Cited by 9
“…in critical situations such as the COVID-19 pandemic (McCabe et al., 2021 ), it becomes evident that scientists often do not convey uncertainty through appropriate communication channels or present in a format easily digestible by farmers for integration into decision-making (Spiegelhalter, 2017 ).”
Metadata
- Authors
- Spiegelhalter, D.
- Affiliations
- University of Cambridge (GB)
- Corresponding author
- Spiegelhalter, David
- ORCID
- Spiegelhalter, David
- Venue
- Annual Review of Statistics and Its Application
- Publisher
- Annual Reviews
- Published
- 2017-03-07
- Volume
- 4
- Issue
- 1
- Pages
- 31-60
- ISSN
- 2326-8298 (ISSN-L)
- 2326-831X
- OpenAlex
- W2529483981
- Cited by
- 248
- References
- 0
- Citations by year
- 2026: 33 · 2025: 35 · 2024: 32 · 2023: 31 · 2022: 31 · 2021: 29 · 2020: 26 · 2019: 15 · 2018: 16
- FWCI
- 4.77
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Annual Review of Statistics and Its Application (publishedVersion)
- Topics
- Social Sciences › Sociology and Political Science › Risk Perception and Management
- Health Sciences › Health Professions › General Health Professions › Patient-Provider Communication in Healthcare
- Social Sciences › Sociology and Political Science › Climate Change Communication and Perception
- Keywords
- risk communication, uncertainty communication, risk assessment, health risks, gambling, weather forecasting, natural hazards, climate change, environmental exposure, production reliability, catastrophic risk, expert guidance, audience understanding
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review