Warning, Bias May Occur: A Proposed Approach to Detecting Cognitive Bias in Interactive Visual Analytics
The DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.
Abstract
Visual analytic tools combine the complementary strengths of humans and machines in human-in-the-loop systems. Humans provide invaluable domain expertise and sensemaking capabilities to this discourse with analytic models; however, little consideration has yet been given to the ways inherent human biases might shape the visual analytic process. In this paper, we establish a conceptual framework for considering bias assessment through human-in-the-loop systems and lay the theoretical foundations for bias measurement. We propose six preliminary metrics to systematically detect and quantify bias from user interactions and demonstrate how the metrics might be implemented in an existing visual analytic system, InterAxis. We discuss how our proposed metrics could be used by visual analytic systems to mitigate the negative effects of cognitive biases by making users aware of biased processes throughout their analyses.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Tversky, A., & Kahneman, D. (1974) · Science · doi:10.1126/science.185.4157.1124 · Cited by 28,810
The Framing of Decisions and the Psychology of Choice
Tversky, A., & Kahneman, D. (1981) · Science · doi:10.1126/science.7455683 · Cited by 18,497 · cited 2×
“In this way, the user interactions constitute a novel set of measurable behaviors that could be used to study and model logical fallacies and cognitive biases in the analytic process [73, 74].”
BackgroundAvailability: A heuristic for judging frequency and probability
Tversky, A., & Kahneman, D. (1973) · Cognitive Psychology · doi:10.1016/0010-0285(73)90033-9 · Cited by 9,984 · cited 1×
“They may subconsciously pay particular attention to documents that confirm a pre-existing hypothesis (confirmation bias) [53, 75] or rely heavily on documents which are most recent (availability heuristic) [72].”
BackgroundConfirmation Bias: A Ubiquitous Phenomenon in Many Guises
Nickerson, R. S. (1998) · Review of General Psychology · doi:10.1037/1089-2680.2.2.175 · Cited by 7,267
The magical number 4 in short-term memory: A reconsideration of mental storage capacity
Cowan, N. (2001) · Behavioral and Brain Sciences · doi:10.1017/s0140525x01003922 · Cited by 6,779
Gigerenzer, G., & Gaissmaier, W. (2011) · Annual Review of Psychology · doi:10.1146/annurev-psych-120709-145346 · Cited by 4,037 · cited 2×
“There are dozens of cognitive biases captured in the heuristics and biases literature [30, 41].”
BackgroundHomo Heuristicus: Why Biased Minds Make Better Inferences
Gigerenzer, G., & Brighton, H. (2009) · Topics in Cognitive Science · doi:10.1111/j.1756-8765.2008.01006.x · Cited by 1,801
Principles of mixed-initiative user interfaces
Horvitz, E. (1999) · Proceedings of the SIGCHI conference on Human factors in computing systems the CHI is the limit - CHI '99 · doi:10.1145/302979.303030 · Cited by 1,490 · cited 3×
“The design of these systems combines the complementary strengths of humans (adaptation, accommodation, and perceptual expertise) and machines (working memory and large-scale information processing) [36] and are widely considered superior to human-only or machine-only alternatives for specific tasks and domains [39].”
BackgroundInfluentialMacmillan, N. A., & Creelman, C. D. (2004) · doi:10.4324/9781410611147 · Cited by 1,269
Friedman, B., & Nissenbaum, H. (1996) · ACM Transactions on Information Systems · doi:10.1145/230538.230561 · Cited by 1,203 · cited 2×
“[27, 64], here we focus on cognitive bias injected by analysts.”
BackgroundThe Attraction Effect in Information Visualization
Dimara, E., Bezerianos, A., & Dragicevic, P. (2017) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2016.2598594 · Cited by 89
Cited by
Visual Analytics in Deep Learning: An Interrogative Survey for the Next Frontiers
Hohman, F., Kahng, M., Pienta, R., & Chau, D. H. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2018.2843369 · Cited by 582
Ethical Dimensions of Visualization Research
Correll, M. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300418 · Cited by 150
Rieger, A., Draws, T., Theune, M., & Tintarev, N. (2021) · Proceedings of the 32st ACM Conference on Hypertext and Social Media · doi:10.1145/3465336.3475101 · Cited by 48
Metadata
- Authors
- Wall, E., Blaha, L. M., Franklin, L., & Endert, A.
- Venue
- 2017 IEEE Conference on Visual Analytics Science and Technology (VAST)
- Publisher
- IEEE
- Published
- 2017-10
- Pages
- 104-115
- OpenAlex
- W2906194767
- Cited by
- 127
- References
- 11
- FWCI
- 4.04
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.