Cited by 187Data Visualization and AnalyticsUsability and User Interface DesignComputer Graphics and Visualization Techniques

The DOI matches its Crossref record; 90 of the 93 references it cites were found in a registry or an index.

In Pursuit of Error: A Survey of Uncertainty Visualization Evaluation

Hullman, Jessica, Qiao, Xiaoli, Correll, Michael, Kale, Alex, Kay, Matthew·2019·IEEE Transactions on Visualization and Computer Graphics

Abstract

Understanding and accounting for uncertainty is critical to effectively reasoning about visualized data. However, evaluating the impact of an uncertainty visualization is complex due to the difficulties that people have interpreting uncertainty and the challenge of defining correct behavior with uncertainty information. Currently, evaluators of uncertainty visualization must rely on general purpose visualization evaluation frameworks which can be ill-equipped to provide guidance with the unique difficulties of assessing judgments under uncertainty. To help evaluators navigate these complexities, we present a taxonomy for characterizing decisions made in designing an evaluation of an uncertainty visualization. Our taxonomy differentiates six levels of decisions that comprise an uncertainty visualization evaluation: the behavioral targets of the study, expected effects from an uncertainty visualization, evaluation goals, measures, elicitation techniques, and analysis approaches. Applying our taxonomy to 86 user studies of uncertainty visualizations, we find that existing evaluation practice, particularly in visualization research, focuses on Performance and Satisfaction-based measures that assume more predictable and statistically-driven judgment behavior than is suggested by research on human judgment and decision making. We reflect on common themes in evaluation practice concerning the interpretation and semantics of uncertainty, the use of confidence reporting, and a bias to…

Overview

Integrity signals

Registry

Crossref · 10.1109/tvcg.2018.2864889

Cites

90 of 90 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 93 references

Research works printed but not linked: 9

  1. 2006, Completing the forecast Characterizing and communicating uncertainty for better decisions using weather and climate forecasts
  2. potter, 0, Uncertainty visualization and data references
  3. quinan, 2015, Towards ecological validity in evaluating uncertainty, Proceedings of Workshop on Visualization for Decision Making Under Uncertainty (VIS'15)
  4. harrower, 2003, Representing uncertainty Does it help people make better decisions
  5. hastie, 1982, Concepts in judgment and decision research-anderson bf deane dh hammond kr mcclelland gh shanteau jc
  6. 10.3758/s13423-012-0247-5 · DOI · Paper page
  7. 1997, Guide to the Expression of Uncertainty in Measurement
  8. deitrick, 2012, Evaluating implicit visualization of uncertainty for public policy decision, Proceedings AutoCarto
  9. 2017, Edelman 2017 edelman trust barometer

Cited by

Sorted by citations

Items 101–187 of 187

Metadata

Authors
Hullman, J., Qiao, X., Correll, M., Kale, A., & Kay, M.
Affiliations
  • Northwestern University (US)
  • Tableau Software (United States) (US)
  • University of Washington (US)
  • University of Michigan (US)
Venue
IEEE Transactions on Visualization and Computer Graphics
Published
2019-01
Volume
25
Issue
1
Pages
903-913
ISSN
  • 1077-2626 (ISSN-L)
  • 1941-0506
  • 2160-9306
OpenAlex
W2891628559
Cited by
187
References
93
Citations by year
2026: 18 · 2025: 20 · 2024: 32 · 2023: 22 · 2022: 23 · 2021: 37 · 2020: 23 · 2019: 11
FWCI
9.01
Type
journal article
Language
en
Open access
closed
Locations
  • IEEE Transactions on Visualization and Computer Graphics (publishedVersion)
  • PubMed (publishedVersion)
Topics
  • Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Data Visualization and Analytics
  • Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
  • Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Computer Graphics and Visualization Techniques
Keywords
uncertainty visualization, visualization evaluation, evaluation taxonomy, decision quality, performance indicators, user satisfaction, elicitation methods, evaluation goals

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review