How Accurate Does It Feel? – Human Perception of Different Types of Classification Mistakes
Open accessThe DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Abstract
Supervised machine learning utilizes large datasets, often with ground truth labels annotated by humans. While some data points are easy to classify, others are hard to classify, which reduces the inter-annotator agreement. This causes noise for the classifier and might affect the user’s perception of the classifier’s performance. In our research, we investigated whether the classification difficulty of a data point influences how strongly a prediction mistake reduces the “perceived accuracy”. In an experimental online study, 225 participants interacted with three fictive classifiers with equal accuracy (73%). The classifiers made prediction mistakes on three different types of data points (easy, difficult, impossible). After the interaction, participants judged the classifier’s accuracy. We found that not all prediction mistakes reduced the perceived accuracy equally. Furthermore, the perceived accuracy differed significantly from the calculated accuracy. To conclude, accuracy and related measures seem unsuitable to represent how users perceive the performance of classifiers.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Tversky, A., & Kahneman, D. (1974) · Science · doi:10.1126/science.185.4157.1124 · Cited by 28,810 · cited 1×
“, heuristics) rather than undertaking a thorough and rational analysis [57].”
BackgroundAlgorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232
Classification in the Presence of Label Noise: A Survey
Frenay, B., & Verleysen, M. (2014) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2013.2292894 · Cited by 1,777
Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy
Shneiderman, B. (2020) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2020.1741118 · Cited by 1,743
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2018) · Management Science · doi:10.1287/mnsc.2016.2643 · Cited by 1,230
Power to the People: The Role of Humans in Interactive Machine Learning
Amershi, S., Cakmak, M., Knox, W. B., & Kulesza, T. (2014) · AI Magazine · doi:10.1609/aimag.v35i4.2513 · Cited by 1,084
“Everyone wants to do the model work, not the data work”: Data Cascades in High-Stakes AI
Sambasivan, N., Kapania, S., Highfill, H., Akrong, D., Paritosh, P., & Aroyo, L. M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445518 · Cited by 781 · cited 1×
“Undervalued data quality causes cascading events that – often unnoticeably – impair the performance of machine learning systems [47].”
BackgroundManipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Wortman Vaughan, J. W., & Wallach, H. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445315 · Cited by 728 · cited 1×
“Accordingly, research in HCML focuses on users’ evaluations of machine learning systems, including aspects such as users’ reliance [31] on the system, or trust [40, 68] in the system, the perceived explainability [51], interpretability [44], or fairness [20, 32] of the system, and the experienced or perceived accuracy…”
BackgroundManipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Vaughan, J. W., & Wallach, H. (2018) · arXiv · doi:10.48550/arxiv.1802.07810 · Cited by 711
Beede, E., Baylor, E., Hersch, F., Iurchenko, A., Wilcox, L., Ruamviboonsuk, P., & Vardoulakis, L. M. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376718 · Cited by 614
Understanding the Effect of Accuracy on Trust in Machine Learning Models
Yin, M., Wortman Vaughan, J., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300509 · Cited by 527 · cited 3×
“Accordingly, research in HCML focuses on users’ evaluations of machine learning systems, including aspects such as users’ reliance [31] on the system, or trust [40, 68] in the system, the perceived explainability [51], interpretability [44], or fairness [20, 32] of the system, and the experienced or perceived accuracy…”
BackgroundBeyond Accuracy: The Role of Mental Models in Human-AI Team Performance
Bansal, G., Nushi, B., Kamar, E., Lasecki, W. S., Weld, D. S., & Horvitz, E. (2019) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v7i1.5285 · Cited by 466 · cited 1×
“, [3, 4, 50, 63]), data work is recently gaining attention.”
BackgroundMadaio, M. A., Stark, L., Wortman Vaughan, J., & Wallach, H. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376445 · Cited by 436 · cited 1×
“Accordingly, research in HCML focuses on users’ evaluations of machine learning systems, including aspects such as users’ reliance [31] on the system, or trust [40, 68] in the system, the perceived explainability [51], interpretability [44], or fairness [20, 32] of the system, and the experienced or perceived accuracy…”
BackgroundThe Perceived Utility of Human and Automated Aids in a Visual Detection Task
Dzindolet, M. T., Pierce, L. G., Beck, H. P., & Dawe, L. A. (2002) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/0018720024494856 · Cited by 386 · cited 3×
“Other research found that people stop trusting an algorithm after observing prediction mistakes, even when the algorithm overall outperforms human predictions [8, 11].”
BackgroundKocielnik, R., Amershi, S., & Bennett, P. N. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300641 · Cited by 277
Inherent Disagreements in Human Textual Inferences
Pavlick, E., & Kwiatkowski, T. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00293 · Cited by 214
Grgic-Hlaca, N., Redmiles, E. M., Gummadi, K. P., & Weller, A. (2018) · Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW '18 · doi:10.1145/3178876.3186138 · Cited by 207
How do visual explanations foster end users' appropriate trust in machine learning?
Yang, F., Huang, Z., Scholtz, J., & Arendt, D. L. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377480 · Cited by 200 · cited 2×
“While in-person user studies in labs (like in [66]) are rather rare, crowdsourcing tasks are commonly used to collect feedback from the users.”
BackgroundHuman Reliance on Machine Learning Models When Performance Feedback is Limited: Heuristics and Risks
Lu, Z., & Yin, M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445562 · Cited by 131
Mental Models of AI Agents in a Cooperative Game Setting
Gero, K. I., Ashktorab, Z., Dugan, C., Pan, Q., Johnson, J., Geyer, W., Ruiz, M., Miller, S., Millen, D. R., Campbell, M., et al. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376316 · Cited by 121 · cited 1×
“[17] used the think-aloud method and questionnaires to investigate participants’ mental models of an AI agent.”
MethodSmith-Renner, A., Fan, R., Birchfield, M., Wu, T., Boyd-Graber, J., Weld, D. S., & Findlater, L. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376624 · Cited by 112 · cited 2×
“Common tasks include labeling text or image data (e.g., in [31]) or assessing given classifications (e.g., in [27, 40, 48, 51]).”
BackgroundThe Disagreement Deconvolution: Bringing Machine Learning Performance Metrics In Line With Reality
Gordon, M. L., Zhou, K., Patel, K., Hashimoto, T., & Bernstein, M. S. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445423 · Cited by 101
Designing Ground Truth and the Social Life of Labels
Muller, M., Wolf, C. T., Andres, J., Desmond, M., Joshi, N. N., Ashktorab, Z., Sharma, A., Brimijoin, K., Pan, Q., Duesterwald, E., et al. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445402 · Cited by 83
Grgić-Hlača, N., Redmiles, E. M., Gummadi, K. P., & Weller, A. (2018) · arXiv · doi:10.48550/arxiv.1802.09548 · Cited by 75
Levy, A., Agrawal, M., Satyanarayan, A., & Sontag, D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445522 · Cited by 73 · cited 2×
“[27] found that domain experts notice if system label recommendations are inadequate but are still likely to accept them for lack of suitable alternatives.”
BackgroundCrowdVerge: Predicting If People Will Agree on the Answer to a Visual Question
Gurari, D., & Grauman, K. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025781 · Cited by 45
Dataset of Natural Language Queries for E-Commerce
Papenmeier, A., Kern, D., Hienert, D., Sliwa, A., Aker, A., & Fuhr, N. (2021) · Proceedings of the 2021 Conference on Human Information Interaction and Retrieval · doi:10.1145/3406522.3446043 · Cited by 8 · cited 1×
“We, therefore, chose the VACOS_NLQ dataset [41] that provides 3,560 product descriptions”
MethodSelf-citation
Cited by
What is Human-Centered about Human-Centered AI? A Map of the Research Landscape
Capel, T., & Brereton, M. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580959 · Cited by 228
How Do We Assess the Trustworthiness of AI? Introducing the Trustworthiness Assessment Model (TrAM)
Schlicker, N., Baum, K., Uhde, A., Sterz, S., Hirsch, M. C., & Langer, M. (2022) · doi:10.31234/osf.io/qhwvx · Cited by 79
Lee, Y., Son, K., Kim, T. S., Kim, J., Chung, J. J. Y., Adar, E., & Kim, J. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3662681 · Cited by 36
The Trust Recovery Journey. The Effect of Timing of Errors on the Willingness to Follow AI Advice.
Kahr, P. K., Rooks, G., Snijders, C., & Willemsen, M. C. (2024) · Proceedings of the 29th International Conference on Intelligent User Interfaces · doi:10.1145/3640543.3645167 · Cited by 21
Zhang, Z. T., Buchner, F., Liu, Y., & Butz, A. (2024) · Proceedings of Mensch und Computer 2024 · doi:10.1145/3670653.3670660 · Cited by 6
Metadata
- Authors
- Papenmeier, A., Kern, D., Hienert, D., Kammerer, Y., & Seifert, C.
- Venue
- CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2022-04-29
- Pages
- 1-13
- OpenAlex
- W4225008476
- Cited by
- 23
- References
- 28
- FWCI
- 2.55
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.