Student or educator? Sign up with your school email and get 6 months of Pro free.

The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.

Proxy tasks and subjective measures can be misleading in evaluating explainable AI systems

Open access
Buçinca, Zana, Lin, Phoebe, Gajos, Krzysztof Z., Glassman, Elena L. · 2020 · Proceedings of the 25th International Conference on Intelligent User Interfaces

Abstract

Explainable artificially intelligent (XAI) systems form part of sociotechnical systems, e.g., human+AI teams tasked with making decisions. Yet, current XAI systems are rarely evaluated by measuring the performance of human+AI teams on actual decision-making tasks. We conducted two online experiments and one in-person think-aloud study to evaluate two currently common techniques for evaluating XAI systems: (1) using proxy, artificial tasks such as how well humans predict the AI's decision from the given explanations, and (2) using subjective measures of trust and preference as predictors of actual performance. The results of our experiments demonstrate that evaluations with proxy tasks did not predict the results of the evaluations with the actual decision-making tasks. Further, the subjective measures on evaluations with actual decision-making tasks did not predict the objective performance on those same tasks. Our results suggest that by employing misleading evaluation methods, our field may be inadvertently slowing its progress toward developing human+AI teams that can reliably perform better than humans or AIs alone.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3377325.3377498

Cites

39 of 39 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

22 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 33 references

Research works printed but not linked: 8

  1. Jonathan Chang Sean Gerrish Chong Wang Jordan L Boyd-Graber and David M Blei. 2009. Reading tea leaves: How humans interpret topic models. In Advances in neural information processing systems. 288--296. Jonathan Chang Sean Gerrish Chong Wang Jordan L Boyd-Graber and David M Blei. 2009. Reading tea leaves: How humans interpret topic models. In Advances in neural information processing systems. 288--296.
  2. K Anders Ericsson and Herbert A Simon. 1984. Protocol analysis: Verbal reports as data. the MIT Press. K Anders Ericsson and Herbert A Simon. 1984. Protocol analysis: Verbal reports as data. the MIT Press.
  3. Ece Kamar. 2016. Directions in Hybrid Intelligence: Complementing AI Systems with Human Intelligence.. In IJCAI. 4070--4073. Ece Kamar. 2016. Directions in Hybrid Intelligence: Complementing AI Systems with Human Intelligence.. In IJCAI. 4070--4073.
  4. Ece Kamar , Severin Hacker , and Eric Horvitz . 2012 . Combining human and machine intelligence in large-scale crowdsourcing . In Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems-Volume 1. International Foundation for Autonomous Agents and Multiagent Systems, 467--474 . Ece Kamar, Severin Hacker, and Eric Horvitz. 2012. Combining human and machine intelligence in large-scale crowdsourcing. In Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems-Volume 1. International Foundation for Autonomous Agents and Multiagent Systems, 467--474.
  5. Wouter Kool and Matthew Botvinick . 2018. Mental labour. Nature human behaviour 2, 12 ( 2018 ), 899--908. Wouter Kool and Matthew Botvinick. 2018. Mental labour. Nature human behaviour 2, 12 (2018), 899--908.
  6. Forough Poursabzi-Sangdeh , Daniel G Goldstein , Jake M Hofman , Jennifer Wortman Vaughan, and Hanna Wallach . 2018 . Manipulating and measuring model interpretability. arXiv preprint arXiv:1802.07810 (2018). Forough Poursabzi-Sangdeh, Daniel G Goldstein, Jake M Hofman, Jennifer Wortman Vaughan, and Hanna Wallach. 2018. Manipulating and measuring model interpretability. arXiv preprint arXiv:1802.07810 (2018). · arXiv
  7. Anuj K Shah and Daniel M Oppenheimer . 2008. Heuristics made easy: An effort-reduction framework. Psychological bulletin 134, 2 ( 2008 ), 207. Anuj K Shah and Daniel M Oppenheimer. 2008. Heuristics made easy: An effort-reduction framework. Psychological bulletin 134, 2 (2008), 207.
  8. John Zeleznikow . 2004. Building intelligent legal decision support systems: Past practice and future challenges . In Applied Intelligent Systems . Springer , 201--254. John Zeleznikow. 2004. Building intelligent legal decision support systems: Past practice and future challenges. In Applied Intelligent Systems. Springer, 201--254.

Cited by

See all 391

Metadata

Authors
Buçinca, Z., Lin, P., Gajos, K. Z., & Glassman, E. L.
Affiliations
Harvard University (US)
Venue
Proceedings of the 25th International Conference on Intelligent User Interfaces
Publisher
ACM
Published
2020-03-17
Pages
454-464
OpenAlex
W3001062618
Cited by
307
References
33
Citations by year
2026: 52 · 2025: 53 · 2024: 57 · 2023: 55 · 2022: 46 · 2021: 32 · 2020: 12
FWCI
13.83
Type
conference paper
Language
en
Open access
Locations
  • ACM International Conference on Intelligent User Interfaces (IUI) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • Digital Access to Scholarship at Harvard (DASH) (Harvard University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
  • Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
Keywords
explainable AI, XAI evaluation, proxy tasks, human-AI teaming, decision performance, trust in AI, think-aloud study, human-AI collaboration

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Proxy tasks and subjective measures can be misleading in evaluating explainable AI systems. https://app.cochat.ai/papers/10.1145/3377325.3377498
The paper
Buçinca, Z., Lin, P., Gajos, K. Z., & Glassman, E. L. (2020). Proxy tasks and subjective measures can be misleading in evaluating explainable AI systems. Proceedings of the 25th International Conference on Intelligent User Interfaces. https://doi.org/10.1145/3377325.3377498

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.