Assessing what to address in science communication
Open accessThe DOI matches its Crossref record; 97 of the 97 references it cites were found in a registry or an index.
Abstract
As members of a democratic society, individuals face complex decisions about whether to support climate change mitigation, vaccinations, genetically modified food, nanotechnology, geoengineering, and so on. To inform people’s decisions and public debate, scientific experts at government agencies, nongovernmental organizations, and other organizations aim to provide understandable and scientifically accurate communication materials. Such communications aim to improve people’s understanding of the decision-relevant issues, and if needed, promote behavior change. Unfortunately, existing communications sometimes fail when scientific experts lack information about what people need to know to make more informed decisions or what wording people use to describe relevant concepts. We provide an introduction for scientific experts about how to use mental models research with intended audience members to inform their communication efforts. Specifically, we describe how to conduct interviews to characterize people’s decision-relevant beliefs or mental models of the topic under consideration, identify gaps and misconceptions in their knowledge, and reveal their preferred wording. We also describe methods for designing follow-up surveys with larger samples to examine the prevalence of beliefs as well as the relationships of beliefs with behaviors. Finally, we discuss how findings from these interviews and surveys can be used to design communications that effectively address gaps and misconceptions in people’s mental models in wording that they understand. We present applications to different scientific domains, showing that this approach leads to communications that improve recipients’ understanding and ability to make informed decisions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Development and validation of brief measures of positive and negative affect: The PANAS scales.
Watson, D., Clark, L. A., & Tellegen, A. (1988) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.54.6.1063 · Cited by 32,433
How Many Interviews Are Enough?: An Experiment with Data Saturation and Variability
Guest, G., Bunce, A., & Johnson, L. (2006) · Field Methods · doi:10.1177/1525822x05279903 · Cited by 20,808
The role of deliberate practice in the acquisition of expert performance
Ericsson, K. A., Krampe, R. T., & Tesch-Römer, C. (1993) · Psychological Review · doi:10.1037/0033-295x.100.3.363 · Cited by 9,564 · cited 3×
“quality: Non-attitude reduction or an invitation to satisfice? Public Opin Q 66(3):”
BackgroundInfluentialThe Content Analysis Guidebook
Neuendorf, K. A. (2017) · doi:10.4135/9781071802878 · Cited by 9,286
Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit
Cohen, J. (1968) · Psychological Bulletin · doi:10.1037/h0026256 · Cited by 8,736
(2017) · The SAGE Encyclopedia of Communication Research Methods · doi:10.4135/9781483381411.n168 · Cited by 8,276
Internet, Mail, and Mixed-Mode Surveys: The Tailored Design Method
Dillman, D. A., Smyth, J. D., & Christian, L. M. (2008) · Cited by 7,000
(2008) · Cited by 6,727
Dunlap, R. E., Van Liere, K. D., Mertig, A. G., & Jones, R. E. (2000) · Journal of Social Issues · doi:10.1111/0022-4537.00176 · Cited by 5,678
Shute, V. J. (2008) · Review of Educational Research · doi:10.3102/0034654307313795 · Cited by 4,631
Interviews – Learning the Craft of Qualitative Research Interviewing
Vaivio, J. (2012) · European Accounting Review · doi:10.1080/09638180.2012.675165 · Cited by 4,535
Answering the Call for a Standard Reliability Measure for Coding Data
Hayes, A. F., & Krippendorff, K. (2007) · Communication Methods and Measures · doi:10.1080/19312450709336664 · Cited by 4,225 · cited 1×
“Measures of reliability typically reflect interrater agreement after correcting for chance agreement (48, 49).”
BackgroundKincaid, J. P., Fishburne, Jr., Robert P., R., Richard L., C., & Brad, S. (1975) · doi:10.21236/ada006655 · Cited by 3,497 · cited 1×
“The Flesch–Kincaid readability measure (72) computes the grade level of education needed to read a text: a formula takes into account the number of words per sentence and the number of syllables per word (73, 74).”
BackgroundInternational Encyclopedia of the Social Sciences.
Duncan, O. D., & Sills, D. L. (1968) · American Sociological Review · doi:10.2307/2092889 · Cited by 3,486
A domain‐specific risk‐attitude scale: measuring risk perceptions and risk behaviors
Weber, E. U., Blais, A., & Betz, N. E. (2002) · Journal of Behavioral Decision Making · doi:10.1002/bdm.414 · Cited by 2,936
Self-reports: How the questions shape the answers.
Schwarz, N. (1999) · American Psychologist · doi:10.1037/0003-066x.54.2.93 · Cited by 2,846 · cited 1×
“Because the question format may affect people’s answers (69), using multiple question formats allows for the computation of correlations between differentially elicited responses, thus revealing the consistency of the underlying preferences.”
BackgroundInternet, mail, and mixed-mode surveys: The tailored design method *
Wetzel, A. P. (2010) · Journal of Continuing Education in the Health Professions · doi:10.1002/chp.20083 · Cited by 2,556
The Paradox of Choice: Why More Is Less
Schwartz, B. N. (2004) · Works - Scholarship, Research, & Creative Expression (Swarthmore College) · Cited by 2,512
Analyzing Qualitative Data: Systematic Approaches
Bernard, H. R., & Ryan, G. W. (2009) · Cited by 2,373
Fina, A. D. (2019) · The Routledge Handbook of Linguistic Ethnography · doi:10.4324/9781315675824-12 · Cited by 2,119
International encyclopedia of the social and behavioral sciences
(2015) · Choice Reviews Online · doi:10.5860/choice.192822 · Cited by 1,831
Handbook of Interview Research
Gubrium, J., & Holstein, J. (2001) · Sage eBooks · doi:10.4135/9781412973588 · Cited by 1,647
International Encyclopedia of the Social and Behavioral Sciences
Plomin, R. J. (2001) · Cited by 1,567
Handbook of Interview Research: Context and Method
Gubrium, J. F., & Holstein, J. A. (2001) · Medical Entomology and Zoology · Cited by 1,515
General Performance on a Numeracy Scale among Highly Educated Samples
Lipkus, I. M., Samsa, G., & Rimer, B. K. (2001) · Medical Decision Making · doi:10.1177/0272989x0102100105 · Cited by 1,497
Handbook of Interview Research. Context and Method
Guillemette, F. (2003) · Loisir et Société / Society and Leisure · doi:10.1080/07053436.2003.10707615 · Cited by 1,459
Dedre Gentner, C. (1983) · Cited by 1,329
Love, sex, and power: Considering women's realities in HIV prevention.
Amaro, H. (1995) · American Psychologist · doi:10.1037//0003-066x.50.6.437 · Cited by 971
Risk communication: a mental models approach
(2002) · Choice Reviews Online · doi:10.5860/choice.39-6394 · Cited by 969
Individual differences in adult decision-making competence.
Bruine de Bruin, W., Parker, A. M., & Fischhoff, B. (2007) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.92.5.938 · Cited by 906 · cited 2×
“The judgment and decisionmaking literature offers validated individual differences measures of decision-making competence (60), numeracy or the ability to use numerical information in decisions (61, 62), and graph literacy or the ability to understand graphical communications (63).”
BackgroundAmerican Risk Perceptions: Is Climate Change Dangerous?
Leiserowitz, A. A. (2005) · Risk Analysis · doi:10.1111/j.1540-6261.2005.00690.x · Cited by 888
How we know--and sometimes misjudge--what others know: Imputing one's own knowledge to others.
Nickerson, R. S. (1999) · Psychological Bulletin · doi:10.1037//0033-2909.125.6.737 · Cited by 836
Reliability in Coding Open-Ended Data: Lessons Learned from HIV Behavioral Research
Hruschka, D. J., Schwartz, D., St.John, D. C., Picone-Decaro, E., Jenkins, R. A., & Carey, J. W. (2004) · Field Methods · doi:10.1177/1525822x04266540 · Cited by 829
Survey Questions: Handcrafting the Standardized Questionnaire
Quarm, D., Converse, J. M., & Presser, S. (1988) · Teaching Sociology · doi:10.2307/1317544 · Cited by 766
Measuring Risk Literacy: The Berlin Numeracy Test
Cokely, E. T., Galesic, M., Schulz, E., Ghazal, S., & Garcia-Retamero, R. (2012) · Judgment and Decision Making · doi:10.1017/s1930297500001819 · Cited by 745
Kincaid, J. P., Robert, P. F. L., Richard, L. R., & Brad, S. (1975) · Cited by 739
Public perceptions of energy consumption and savings
Attari, S. Z., DeKay, M. L., Davidson, C. I., & Bruine de Bruin, W. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1001509107 · Cited by 690
Self-citationCommon-sense models of illness: The example of hypertension.
Meyer, D., Leventhal, H., & Gutmann, M. (1985) · Health Psychology · doi:10.1037//0278-6133.4.2.115 · Cited by 641
Advanced social psychology: (540002004-001)
(1993) · PsycEXTRA Dataset · doi:10.1037/e540002004-001 · Cited by 623
What Do People Know About Global Climate Change? 1. Mental Models
Bostrom, A., Morgan, M. G., Fischhoff, B., & Read, D. (1994) · Risk Analysis · doi:10.1111/j.1539-6924.1994.tb00065.x · Cited by 618
Self-citationConverse, J., & Presser, S. (1986) · Sage eBooks · doi:10.4135/9781412986045 · Cited by 586
The Role of the Affect and Availability Heuristics in Risk Communication
Keller, C., Siegrist, M., & Gutscher, H. (2006) · Risk Analysis · doi:10.1111/j.1539-6924.2006.00773.x · Cited by 551 · cited 1×
“For example, engineers talk about a 100-y flood rather than a flood that has a 1% chance of happening each year without realizing that people interpret the former as more likely and more predictable than the latter (11, 12).”
BackgroundCalibration and probability judgements: Conceptual and methodological issues
Keren, G. (1991) · Acta Psychologica · doi:10.1016/0001-6918(91)90036-y · Cited by 549
The Impact of “No Opinion” Response Options on Data Quality
Krosnick, J. A., Holbrook, A. L., Berent, M. K., Carson, R. T., Hanemann, W. M., Kopp, R. J., Mitchell, R. C., Presser, S., Ruud, P. A., Smith, V. K., et al. (2001) · Public Opinion Quarterly · doi:10.1086/341394 · Cited by 546
Cognition and Communication: Judgmental Biases, Research Methods, and the Logic of Conversation
Schwarz, N. (2014) · Medical Entomology and Zoology · Cited by 537
The Short List: The Most Effective Actions U.S. Households Can Take to Curb Climate Change
Gardner, G. T., & Stern, P. C. (2008) · Environment: Science and Policy for Sustainable Development · doi:10.3200/envt.50.5.12-25 · Cited by 529
Friedman, D. B., & Hoffman-Goetz, L. (2006) · Health Education & Behavior · doi:10.1177/1090198105277329 · Cited by 517
Interview Mode Effects in Surveys of Drug and Alcohol Use: A Field Experiment
Aquilino, W. S. (1994) · Public Opinion Quarterly · doi:10.1086/269419 · Cited by 458
Bloor, D. (1993) · Social Studies of Science · doi:10.1177/030631293023004005 · Cited by 446
Context Effects in Social and Psychological Research
(1992) · Springer eBooks · doi:10.1007/978-1-4612-2848-6 · Cited by 429
Kirby, D. B. (2008) · Sexuality Research and Social Policy · doi:10.1525/srsp.2008.5.3.18 · Cited by 410
Communicating Risks and Benefits: An Evidence Based User's Guide
Fischhoff, B., Brewer, N. T., & Down, J. S. (2012) · Cited by 410
The Positive and Negative Consequences of Multiple-Choice Testing.
Roediger, H. L., & Marsh, E. J. (2005) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/0278-7393.31.5.1155 · Cited by 383
Characterizing Mental Models of Hazardous Processes: A Methodology and an Application to Radon
Bostrom, A., Fischhoff, B., & Morgan, M. G. (1992) · Journal of Social Issues · doi:10.1111/j.1540-4560.1992.tb01946.x · Cited by 358
Self-citationAledort, J. E., Lurie, N., Wasserman, J., & Bozzette, S. A. (2007) · BMC Public Health · doi:10.1186/1471-2458-7-208 · Cited by 328
Two Theories of Home Heat Control*
Kempton, W. (1986) · Cognitive Science · doi:10.1207/s15516709cog1001_3 · Cited by 327
Giere, R. N. (2010) · University of Minnesota Press eBooks · doi:10.5749/9780816683963 · Cited by 317
Lay Foibles and Expert Fables in Judgments about Risk
Fischhoff, B., Slovic, P., & Lichtenstein, S. (1982) · The American Statistician · doi:10.1080/00031305.1982.10482845 · Cited by 266
Parents’ vaccination comprehension and decisions
Downs, J. S., de Bruin, W. B., & Fischhoff, B. (2008) · Vaccine · doi:10.1016/j.vaccine.2008.01.011 · Cited by 245
The False Consensus Effect and Overconfidence: Flaws in Judgment or Flaws in How We Study Judgment?
Dawes, R. M., & Mulford, M. (1996) · Organizational Behavior and Human Decision Processes · doi:10.1006/obhd.1996.0020 · Cited by 239
HALPERIN, D. T. (1999) · AIDS Patient Care and STDs · doi:10.1089/apc.1999.13.717 · Cited by 233
Kaplowitz, M. D., & Hoehn, J. P. (2001) · Ecological Economics · doi:10.1016/s0921-8009(00)00226-3 · Cited by 232
Krishnamurti, T., Schwartz, D., Davis, A., Fischhoff, B., de Bruin, W. B., Lave, L., & Wang, J. (2012) · Energy Policy · doi:10.1016/j.enpol.2011.11.047 · Cited by 230
BRUINE de BRUIN, W., VANDERKLAAUW, W., DOWNS, J. S., FISCHHOFF, B., TOPA, G., & ARMANTIER, O. (2010) · Journal of Consumer Affairs · doi:10.1111/j.1745-6606.2010.01174.x · Cited by 228
Self-citationEffects of telephone versus face‐to‐face interview modes on reports of alcohol consumption
Greenfield, T. K., Midanik, L. T., & Rogers, J. D. (2000) · Addiction · doi:10.1046/j.1360-0443.2000.95227714.x · Cited by 222
Downs, J. S., Murray, P. J., Bruine de Bruin, W., Penrose, J., Palmgren, C., & Fischhoff, B. (2004) · Social Science & Medicine · doi:10.1016/j.socscimed.2004.01.032 · Cited by 196
Self-citationInitial Public Perceptions of Deep Geological and Oceanic Disposal of Carbon Dioxide
Palmgren, C. R., Morgan, M. G., Bruine de Bruin, W., & Keith, D. W. (2004) · Environmental Science & Technology · doi:10.1021/es040400c · Cited by 189
Self-citationSchuster, M. A., Bell, R. M., & Kanouse, D. E. (1996) · American Journal of Public Health · doi:10.2105/ajph.86.11.1570 · Cited by 181
Causal thinking and support for climate change policies: International survey findings
Bostrom, A., O’Connor, R. E., Böhm, G., Hanss, D., Bodi, O., Ekström, F., Halder, P., Jeschke, S., Mack, B., Qu, M., et al. (2012) · Global Environmental Change · doi:10.1016/j.gloenvcha.2011.09.012 · Cited by 177
Self-citationDangerous Climate Change: The Role for Risk Research
Lorenzoni, I., Pidgeon, N. F., & O'Connor, R. E. (2005) · Risk Analysis · doi:10.1111/j.1539-6924.2005.00686.x · Cited by 168
Why (Cancer) Risk Communication Can Be Hard
Fischhoff, B. (1999) · JNCI Monographs · doi:10.1093/oxfordjournals.jncimonographs.a024213 · Cited by 165
Efficient and effective? The 100-year flood in the communication and perception of flood risk
BELL, H., & TOBIN, G. (2007) · Environmental Hazards · doi:10.1016/j.envhaz.2007.08.004 · Cited by 161
Attali, Y., & Bar‐Hillel, M. (2003) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.2003.tb01099.x · Cited by 147
Keren, G. B. (2011) · Psychology Press eBooks · doi:10.4324/9780203854167 · Cited by 144
Individual Differences in Graph Literacy: Overcoming Denominator Neglect in Risk Comprehension
Okan, Y., Garcia‐Retamero, R., Cokely, E. T., & Maldonado, A. (2012) · Journal of Behavioral Decision Making · doi:10.1002/bdm.751 · Cited by 140
Mode of administration and interviewer effects in self-reported symptoms of anxiety and depression
Moum, T. (1998) · Social Indicators Research · doi:10.1023/a:1006958100504 · Cited by 136
Toward an Understanding of (Em)Power(Ment) for HIV/AIDS Prevention with Adolescent Women
Gutiérrez, L., Oh, H. J., & Gillmore, M. R. (2000) · Sex Roles · doi:10.1023/a:1007047306063 · Cited by 128
The effect of question wording on consumers’ reported inflation expectations
de Bruin, W. B., van der Klaauw, W., Topa, G., Downs, J. S., Fischhoff, B., & Armantier, O. (2012) · Journal of Economic Psychology · doi:10.1016/j.joep.2012.02.001 · Cited by 120
Informed Public Preferences for Electricity Portfolios with CCS and Other Low‐Carbon Technologies
Fleishman, L. A., De Bruin, W. B., & Morgan, M. G. (2010) · Risk Analysis · doi:10.1111/j.1539-6924.2010.01436.x · Cited by 120
Expectations of Inflation: The Biasing Effect of Thoughts about Specific Prices
de Bruin, W. B., van der Klaauw, W., & Topa, G. (2011) · Journal of Economic Psychology · doi:10.1016/j.joep.2011.07.002 · Cited by 114
Kastenholz, H. G. (2008) · Human and Ecological Risk Assessment: An International Journal · doi:10.1080/10807030802012477 · Cited by 112
(2007) · Psychology Press eBooks · doi:10.4324/9780203810057 · Cited by 111
Willis, H. H., DeKay, M. L., Morgan, M. G., Florig, H. K., & Fischbeck, P. S. (2004) · Risk Analysis · doi:10.1111/j.0272-4332.2004.00438.x · Cited by 101
Lay Foibles and Expert Fables in Judgments about Risk
Fischhoff, B., Slovic, P., & Lichtenstein, S. (1982) · The American Statistician · doi:10.2307/2683835 · Cited by 100
Issues regarding data on race and ethnicity: the Census Bureau experience.
McKenney, N. R., & Bennett, C. E. (1994) · Europe PMC (PubMed Central) · Cited by 99
Cognitive Mapping Tools: Review and Risk Management Needs
Wood, M. D., Bostrom, A., Bridges, T. S., & Linkov, I. (2012) · Risk Analysis · doi:10.1111/j.1539-6924.2011.01767.x · Cited by 97
Self-citationEricsson, K. A., & Fox, M. C. (2011) · Psychological Bulletin · doi:10.1037/a0022388 · Cited by 72
Analyzing disaster risks and plans: An avian flu example
Fischhoff, B., de Bruin, W. B., Güvenç, Ü., Caruso, D., & Brilliant, L. (2006) · Journal of Risk and Uncertainty · doi:10.1007/s11166-006-0175-8 · Cited by 64
The Impact of Item Readability on the Endorsement of the Midpoint Response in Surveys
Velez, P. M., & Ashworth, S. D. (2007) · Survey research methods · doi:10.18148/srm/2007.v1i2.76 · Cited by 60
Bostrom, A., & Lashof, D. A. (2007) · Cambridge University Press eBooks · doi:10.1017/cbo9780511535871.004 · Cited by 56
Self-citationExpert judgments of pandemic influenza risks
de Bruin, W. B., Fischhoff, B., Brilliant, L., & Caruso, D. (2006) · Global Public Health · doi:10.1080/17441690600673940 · Cited by 50
Trust in Cooperative Risk Management: Uncertainty and Scepticism in the Public Mind
Siegrist, M., Earle, T. C., & Gutscher, H. (2007) · Cited by 49
Learning from Tests: The Case of True-False Examinations
Toppino, T. C., & Ann Brochin, H. (1989) · The Journal of Educational Research · doi:10.1080/00220671.1989.10885940 · Cited by 41
Crosby, R. A., DiClemente, R. J., Wingood, G. M., Rose, E., & Lang, D. L. (2003) · Sexually Transmitted Diseases · doi:10.1097/00007435-200301000-00012 · Cited by 27
de Bruin, W. B., & Fischhoff, B. (2000) · AIDS Education and Prevention · doi:10.1521/aeap.2000.12.3.187 · Cited by 21
de Bruin, W. B., Downs, J. S., Fischhoff, B., & Palmgren, C. (2007) · Journal of HIV/AIDS Prevention in Children & Youth · doi:10.1300/j499v08n01_03 · Cited by 20
Public Perceptions of Hurricane Modification
Klima, K., de Bruin, W. B., Morgan, M. G., & Grossmann, I. (2011) · Risk Analysis · doi:10.1111/j.1539-6924.2011.01717.x · Cited by 12
Cited by
Moser, S. C. (2016) · WIREs Climate Change · doi:10.1002/wcc.403 · Cited by 581
The sciences of science communication
Fischhoff, B. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1213273110 · Cited by 362
Unravelling complexity in seagrass systems for management: Australia as a microcosm
Kilminster, K., McMahon, K., Waycott, M., Kendrick, G. A., Scanes, P., McKenzie, L., O'Brien, K. R., Lyons, M., Ferguson, A., Maxwell, P., et al. (2015) · Science of The Total Environment · doi:10.1016/j.scitotenv.2015.04.061 · Cited by 355
Communicating scientific uncertainty
Fischhoff, B., & Davis, A. L. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1317504111 · Cited by 312
The science of sharing and the sharing of science
Milkman, K. L., & Berger, J. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1317511111 · Cited by 154
Metadata
- Authors
- Bruine de Bruin, W., & Bostrom, A.
- Venue
- Proceedings of the National Academy of Sciences
- Published
- 2013-08-13
- Volume
- 110
- Issue
- supplement_3
- Pages
- 14062-14068
- ISSN
- 0027-8424 (ISSN-L)
- 1091-6490
- OpenAlex
- W1995726594
- Cited by
- 290
- References
- 97
- FWCI
- 53.58
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.