The DOI matches its Crossref record; 78 of the 78 references it cites were found in a registry or an index.
The Role of Uncertainty, Awareness, and Trust in Visual Analytics
Open accessAbstract
Visual analytics supports humans in generating knowledge from large and often complex datasets. Evidence is collected, collated and cross-linked with our existing knowledge. In the process, a myriad of analytical and visualisation techniques are employed to generate a visual representation of the data. These often introduce their own uncertainties, in addition to the ones inherent in the data, and these propagated and compounded uncertainties can result in impaired decision making. The user's confidence or trust in the results depends on the extent of user's awareness of the underlying uncertainties generated on the system side. This paper unpacks the uncertainties that propagate through visual analytics systems, illustrates how human's perceptual and cognitive biases influence the user's awareness of such uncertainties, and how this affects the user's trust building. The knowledge generation model for visual analytics is used to provide a terminology and framework to discuss the consequences of these aspects in knowledge construction and though examples, machine uncertainty is compared to human trust measures with provenance. Furthermore, guidelines for the design of uncertainty-aware systems are presented that can aid the user in better decision making.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2011) · Cited by 14,317
Norman, D. (2016) · Vahlen eBooks · doi:10.15358/9783800648108 · Cited by 6,374
Subjective probability: A judgment of representativeness
Kahneman, D., & Tversky, A. (1972) · Cognitive Psychology · doi:10.1016/0010-0285(72)90016-3 · Cited by 4,851
(1996) · Cited by 4,738
The eyes have it: a task by data type taxonomy for information visualizations
Shneiderman, B. (2002) · Proceedings 1996 IEEE Symposium on Visual Languages · doi:10.1109/vl.1996.545307 · Cited by 4,535
Bayesian calibration of computer models
Kennedy, M. C., & O'Hagan, A. (2001) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00294 · Cited by 4,222
Readings in Information Visualization: Using Vision to Think
Card, S. K., Mackinlay, J. D., & Shneiderman, B. (1999) · Cited by 3,935
(2013) · Information Visualization · doi:10.1016/b978-0-12-381464-7.00018-1 · Cited by 3,742
Information credibility on twitter
Castillo, C., Mendoza, M., & Poblete, B. (2011) · Proceedings of the 20th international conference on World wide web · doi:10.1145/1963405.1963500 · Cited by 2,593
Barber, B. (1983) · DigitalGeorgetown (Georgetown University Library) · Cited by 2,495
Direct Manipulation: A Step Beyond Programming Languages
Shneiderman (1983) · Computer · doi:10.1109/mc.1983.1654471 · Cited by 1,939
Improvements on Cross-Validation: The 632+ Bootstrap Method
Efron, B., & Tibshirani, R. (1997) · Journal of the American Statistical Association · doi:10.1080/01621459.1997.10474007 · Cited by 1,636
The role of trust in automation reliance
Dzindolet, M. T., Peterson, S. A., Pomranky, R. A., Pierce, L. G., & Beck, H. P. (2003) · International Journal of Human-Computer Studies · doi:10.1016/s1071-5819(03)00038-7 · Cited by 1,276
Power to the People: The Role of Humans in Interactive Machine Learning
Amershi, S., Cakmak, M., Knox, W. B., & Kulesza, T. (2014) · AI Magazine · doi:10.1609/aimag.v35i4.2513 · Cited by 1,084
Model uncertainty in cross‐country growth regressions
Fernández, C., Ley, E., & Steel, M. F. J. (2001) · Journal of Applied Econometrics · doi:10.1002/jae.623 · Cited by 891
Trust between humans and machines, and the design of decision aids
Muir, B. M. (1987) · International Journal of Man-Machine Studies · doi:10.1016/s0020-7373(87)80013-5 · Cited by 762
The credibility of volunteered geographic information
Flanagin, A. J., & Metzger, M. J. (2008) · GeoJournal · doi:10.1007/s10708-008-9188-y · Cited by 751
A Multi-Level Typology of Abstract Visualization Tasks
Brehmer, M., & Munzner, T. (2013) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2013.124 · Cited by 747
Cullen, A. C., & Frey, H. C. (1999) · Cited by 612
Approaches to uncertainty visualization
Pang, A. T., Wittenbrink, C. M., & Lodha, S. K. (1997) · The Visual Computer · doi:10.1007/s003710050111 · Cited by 544
Visualizing Geospatial Information Uncertainty: What We Know and What We Need to Know
MacEachren, A. M., Robinson, A., Hopper, S., Gardner, S., Murray, R., Gahegan, M., & Hetzler, E. (2005) · Cartography and Geographic Information Science · doi:10.1559/1523040054738936 · Cited by 540
Uncertainty-Based Information: Elements of Generalized Information Theory
Klir, G. J., Wierman, M. J., & Kacprzyk, J. (1998) · Medical Entomology and Zoology · Cited by 494
Klir, G. J., & Wierman, M. J. (1999) · Studies in fuzziness and soft computing · doi:10.1007/978-3-7908-1869-7 · Cited by 488
Zhou, L., Burgoon, J. K., Nunamaker, J. F., & Twitchell, D. (2004) · Group Decision and Negotiation · doi:10.1023/b:grup.0000011944.62889.6f · Cited by 440
Storytelling: The Next Step for Visualization
Kosara, R., & Mackinlay, J. (2013) · Computer · doi:10.1109/mc.2013.36 · Cited by 437
Human factors in visualization research
Tory, M., & Moller, T. (2004) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2004.1260759 · Cited by 420
Knowledge Generation Model for Visual Analytics
Sacha, D., Stoffel, A., Stoffel, F., Kwon, B. C., Ellis, G., & Keim, D. A. (2014) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2014.2346481 · Cited by 387
Self-citationVisualizing Uncertain Information
MacEachren, A. M. (1992) · Cartographic Perspectives · doi:10.14714/cp13.1000 · Cited by 371
Cognitive biases and decision support systems development: a design science approach
Arnott, D. (2006) · Information Systems Journal · doi:10.1111/j.1365-2575.2006.00208.x · Cited by 354
Kandel, S., Heer, J., Plaisant, C., Kennedy, J., van Ham, F., Riche, N. H., Weaver, C., Lee, B., Brodbeck, D., & Buono, P. (2011) · Information Visualization · doi:10.1177/1473871611415994 · Cited by 332
A comparative study of uncertainty propagation methods for black-box-type problems
Lee, S. H., & Chen, W. (2009) · Structural and Multidisciplinary Optimization · doi:10.1007/s00158-008-0234-7 · Cited by 330
Approximated and User Steerable tSNE for Progressive Visual Analytics
Pezzotti, N., Lelieveldt, B. P. F., Maaten, L. V. D., Hollt, T., Eisemann, E., & Vilanova, A. (2017) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2016.2570755 · Cited by 308
Disruption and recovery of computing tasks: field study, analysis, and directions
Iqbal, S. T., & Horvitz, E. (2007) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1240624.1240730 · Cited by 300
Visual Semiotics & Uncertainty Visualization: An Empirical Study
MacEachren, A. M., Roth, R. E., O'Brien, J., Li, B., Swingley, D., & Gahegan, M. (2012) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2012.279 · Cited by 278
Manzey, D., Reichenbach, J., & Onnasch, L. (2012) · Journal of Cognitive Engineering and Decision Making · doi:10.1177/1555343411433844 · Cited by 277
Mental Models, Visual Reasoning and Interaction in Information Visualization: A Top-down Perspective
Liu, Z., & Stasko, J. T. (2010) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2010.177 · Cited by 255
Interpretation and trust: designing model-driven visualizations for text analysis
Chuang, J., Ramage, D., Manning, C., & Heer, J. (2012) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2207738 · Cited by 244
iPCA: An Interactive System for PCA‐based Visual Analytics
Jeong, D. H., Ziemkiewicz, C., Fisher, B., Ribarsky, W., & Chang, R. (2009) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2009.01475.x · Cited by 220
A Review of Uncertainty in Data Visualization
Brodlie, K., Allendes Osorio, R., & Lopes, A. (2012) · Expanding the Frontiers of Visual Analytics and Visualization · doi:10.1007/978-1-4471-2804-5_6 · Cited by 212
The Sandbox for analysis: concepts and methods
Wright, W., Schroh, D., Proulx, P., Skaburskis, A., & Cort, B. (2006) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1124772.1124890 · Cited by 211
Selecting good views of high‐dimensional data using class consistency
Sips, M., Neubert, B., Lewis, J. P., & Hanrahan, P. (2009) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2009.01467.x · Cited by 206
The human is the loop: new directions for visual analytics
Endert, A., Hossain, M. S., Ramakrishnan, N., North, C. L., Fiaux, P., & Andrews, C. (2014) · Journal of Intelligent Information Systems · doi:10.1007/s10844-014-0304-9 · Cited by 185
A typology for visualizing uncertainty
Thomson, J., Hetzler, E., MacEachren, A., Gahegan, M., & Pavel, M. (2005) · SPIE Proceedings · doi:10.1117/12.587254 · Cited by 175
Semantic Interaction for Sensemaking: Inferring Analytical Reasoning for Model Steering
Endert, A., Fiaux, P., & North, C. L. (2012) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2012.260 · Cited by 164
Automation bias: Empirical results assessing influencing factors
Goddard, K., Roudsari, A., & Wyatt, J. C. (2014) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2014.01.001 · Cited by 163
A survey of data provenance techniques
Simmhan, Y. L., Plale, B., & Gannon, D. B. (2005) · Cited by 160
Finding Waldo: Learning about Users from their Interactions
Brown, E. T., Ottley, A., Zhao, H., Lin, Q., Souvenir, R., Endert, A., & Chang, R. (2014) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2014.2346575 · Cited by 152
Building and Applying a Human Cognition Model for Visual Analytics
Green, T. M., Ribarsky, W., & Fisher, B. (2009) · Information Visualization · doi:10.1057/ivs.2008.28 · Cited by 142
Klemmer, S., Thomsen, M., Phelps-Goodman, E., Lee, R., & Landay, J. A. (2002) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/503376.503378 · Cited by 137
How Polymorphic Warnings Reduce Habituation in the Brain: Insights from an fMRI Study
Anderson, B. B., Kirwan, C. B., Jenkins, J. L., Eargle, D., Howard, S., & Vance, A. (2015) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702322 · Cited by 133
The Visualization of Uncertain Data: Methods and Problems.
Griethe, H., & Schumann, H. (2006) · SimVis · Cited by 127
Revealing Uncertainty for Information Visualization
Skeels, M. M., Lee, B., Smith, G., & Robertson, G. G. (2009) · Information Visualization · doi:10.1057/ivs.2009.1 · Cited by 123
A Design Methodology for Trust Cue Calibration in Cognitive Agents
de Visser, E. J., Cohen, M. S., Freedy, A., & Parasuraman, R. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-07458-0_24 · Cited by 121
Limited Rank Matrix Learning, discriminative dimension reduction and visualization
Bunte, K., Schneider, P., Hammer, B., Schleif, F.-M., Villmann, T., & Biehl, M. L. (2011) · Neural Networks · doi:10.1016/j.neunet.2011.10.001 · Cited by 110
A framework for uncertainty-aware visual analytics
Correa, C. D., Chan, Y., & Ma, K. (2009) · IEEE Conference on Visual Analytics Science and Technology (VAST) · doi:10.1109/vast.2009.5332611 · Cited by 104
Tenbrink, T. (2014) · Language and Cognition · doi:10.1017/langcog.2014.19 · Cited by 101
What May Visualization Processes Optimize?
Chen, M., & Golan, A. (2015) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2015.2513410 · Cited by 89
Building and applying a human cognition model for visual analytics
Green, T. M., Ribarsky, W., & Fisher, B. (2009) · doi:10.1057/palgrave.ivs.2008.28 · Cited by 87
Evaluating visual analytics with eye tracking
Kurzhals, K., Fisher, B., Burch, M., & Weiskopf, D. (2014) · Workshop on Beyond Time and Errors · doi:10.1145/2669557.2669560 · Cited by 84
Analytic Provenance for Sensemaking: A Research Agenda
Xu, K., Attfield, S., Jankun-Kelly, T. J., Wheat, A., Nguyen, P. H., & Selvaraj, N. (2015) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.2015.50 · Cited by 83
Visualization of Uncertainty and Reasoning
Zuk, T., & Carpendale, S. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-73214-3_15 · Cited by 79
Procedural annotation of uncertain information
Cedilnik, A., & Rheingans, P. L. (2000) · IEEE Visualization · doi:10.5555/375213.375221 · Cited by 78
Automated Analytical Methods to Support Visual Exploration of High-Dimensional Data
Tatu, A., Albuquerque, G., Eisemann, M., Bak, P. M., Theisel, H., Magnor, M. A., & Keim, D. A. (2010) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2010.242 · Cited by 74
Self-citationSuggested Interactivity: Seeking Perceived Affordances for Information Visualization
Boy, J., Eveillard, L., Détienne, F., & Fekete, J. (2015) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2015.2467201 · Cited by 72
Developments in statistical approaches to spatial uncertainty and its propagation
Heuvelink, G. B. M., & BURROUGH, P. A. (2002) · International Journal of Geographical Information Systems · doi:10.1080/13658810110099071 · Cited by 69
Measurement of trust in complex and dynamic systems using a quantitative approach
Uggirala, A., Gramopadhye, A. K., Melloy, B. J., & Toler, J. E. (2004) · International Journal of Industrial Ergonomics · doi:10.1016/j.ergon.2004.03.005 · Cited by 68
Procedural annotation of uncertain information
Cedilnik, A., & Rheingans, P. L. (2002) · IEEE Visualization Conference (VIS) · doi:10.1109/visual.2000.885679 · Cited by 57
Visual analytic roadblocks for novice investigators
Kwon, B. C., Fisher, B., & Yi, J. S. (2011) · IEEE Conference on Visual Analytics Science and Technology (VAST) · doi:10.1109/vast.2011.6102435 · Cited by 38
Self-citationBass, E. J., Baumgart, L. A., & Shepley, K. K. (2012) · Journal of Cognitive Engineering and Decision Making · doi:10.1177/1555343412453461 · Cited by 32
Exploratory Visualization Involving Incremental, Approximate Database Queries and Uncertainty
Fisher, D., Drucker, S. M., & Knig, A. C. (2012) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.2012.48 · Cited by 31
Visual Analytics and Uncertainty: Its Not About the Data
MacEachren, A. M. (2015) · doi:10.2312/eurova.20151104 · Cited by 24
Theoretical analysis and theory creation
Dix, A. J. (2008) · Cambridge University Press eBooks · doi:10.1017/cbo9780511814570.010 · Cited by 24
Tuning Trust Using Cognitive Cues for Better Human-Machine Collaboration
Cai, H., & Lin, Y. (2010) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193121005402816 · Cited by 19
Phillips, B. K., Prybutok, V. R., & Peak, D. A. (2014) · Informing Science The International Journal of an Emerging Transdiscipline · doi:10.28945/1932 · Cited by 18
Chatfield, C. (2006) · Wiley eBooks · doi:10.1002/9780470057339.vam030 · Cited by 14
Considerations for characterizing domain problems
Winters, K. M., Lach, D. H., & Cushing, J. B. (2014) · Workshop on Beyond Time and Errors · doi:10.1145/2669557.2669573 · Cited by 8
Ragan, E. D., & Goodall, J. R. (2014) · doi:10.1145/2669557.2669563 · Cited by 8
Capturing Reasoning Process through User Interaction
Dou, W., Ribarsky, W., & Chang, R. (2010) · doi:10.2312/pe/eurovast/eurovast10/033-038 · Cited by 6
Cited by
A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems
Mohseni, S., Zarei, N., & Ragan, E. D. (2021) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3387166 · Cited by 785
A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems
Mohseni, S., Zarei, N., & Ragan, E. D. (2018) · arXiv · doi:10.48550/arxiv.1811.11839 · Cited by 783
“Therefore, a main challenge for data-analysis and decision-support systems is to increase model transparency and user awareness with visualization and interaction techniques [109].”
BackgroundA review of volunteered geographic information quality assessment methods
Senaratne, H., Mobasheri, A., Ali, A. L., Capineri, C., & Haklay, M. (. (2017) · International Journal of Geographical Information Science · doi:10.1080/13658816.2016.1189556 · Cited by 493
Self-citationTowards better analysis of machine learning models: A visual analytics perspective
Liu, S., Wang, X., Liu, M., & Zhu, J. (2017) · Visual Informatics · doi:10.1016/j.visinf.2017.01.006 · Cited by 397
A Review of User Interface Design for Interactive Machine Learning
Dudley, J. J., & Kristensson, P. O. (2018) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3185517 · Cited by 343
Metadata
- Authors
- Sacha, D., Senaratne, H., Kwon, B. C., Ellis, G., & Keim, D. A.
- Venue
- IEEE Transactions on Visualization and Computer Graphics
- Published
- 2016-01-31
- Volume
- 22
- Issue
- 1
- Pages
- 240-249
- ISSN
- 1077-2626 (ISSN-L)
- OpenAlex
- W1914475855
- Cited by
- 267
- References
- 78
- FWCI
- 17.62
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.