Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.

Understanding the Effect of Accuracy on Trust in Machine Learning Models

Yin, Ming, Wortman Vaughan, Jennifer, Wallach, Hanna · 2019 · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

Abstract

We address a relatively under-explored aspect of human-computer interaction: people's abilities to understand the relationship between a machine learning model's stated performance on held-out data and its expected performance post deployment. We conduct large-scale, randomized human-subject experiments to examine whether laypeople's trust in a model, measured in terms of both the frequency with which they revise their predictions to match those of the model and their self-reported levels of trust in the model, varies depending on the model's stated accuracy on held-out data and on its observed accuracy in practice. We find that people's trust in a model is affected by both its stated accuracy and its observed accuracy, and that the effect of stated accuracy can change depending on the observed accuracy. Our work relates to recent research on interpretable machine learning, but moves beyond the typical focus on model internals, exploring a different component of the machine learning pipeline.

Overview

Integrity signals

Registry

Crossref · 10.1145/3290605.3300509

Cites

31 of 31 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

9 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Citations

Cites

Show all 28 references

Research works printed but not linked: 3

  1. Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2016. Machine bias. ProPublica, May 23 (2016).
  2. Zachary C. Lipton. 2016. The mythos of model interpretability. (2016). CoRR arXiv:1606.03490. · arXiv
  3. Jennifer Wortman Vaughan and Hanna Wallach. 2017. The Inescapability of Uncertainty. In CHI Workshop on Designing for Uncertainty in HCI: When Does Uncertainty Help?

Cited by

See all 531

Metadata

Authors
Yin, M., Wortman Vaughan, J., & Wallach, H.
Affiliations
  • Purdue University West Lafayette (US)
  • Microsoft Research, New York, NY, USA
  • Microsoft Research New York City (United States) (US)
Venue
Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Published
2019-05-02
Pages
1-12
OpenAlex
W2942157335
Cited by
527
References
28
Citations by year
2026: 76 · 2025: 91 · 2024: 94 · 2023: 81 · 2022: 84 · 2021: 61 · 2020: 34 · 2019: 6
FWCI
17.70
Type
conference paper
Language
en
Open access
closed
Locations
CHI Conference on Human Factors in Computing Systems (publishedVersion)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
Keywords
model accuracy, laypeople, human-computer interaction, model interpretability

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Understanding the Effect of Accuracy on Trust in Machine Learning Models. https://app.cochat.ai/papers/10.1145/3290605.3300509
The paper
Yin, M., Wortman Vaughan, J., & Wallach, H. (2019). Understanding the Effect of Accuracy on Trust in Machine Learning Models. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3290605.3300509

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.