CoChat
Sign Up
PreprintCited by 754Explainable Artificial Intelligence (XAI)Human-Automation Interaction and SafetyUsability and User Interface Design

Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance

Open access
Bansal, Gagan, Wu, Tongshuang, Zhou, Joyce, Fok, Raymond, Nushi, Besmira, Kamar, Ece, Ribeiro, Marco Tulio, Weld, Daniel S.·2020·arXiv
DOI 10.48550/arxiv.2006.14779arXiv 2006.14779
Open this paper in CoChat — free accountOpen PDF

Abstract

Many researchers motivate explainable AI with studies showing that human-AI team performance on decision-making tasks improves when the AI explains its recommendations. However, prior studies observed improvements from explanations only when the AI, alone, outperformed both the human and the best team. Can explanations help lead to complementary performance, where team accuracy is higher than either the human or the AI working solo? We conduct mixed-method user studies on three datasets, where an AI with accuracy comparable to humans helps participants solve a task (explaining itself in some conditions). While we observed complementary improvements from AI augmentation, they were not increased by explanations. Rather, explanations increased the chance that humans will accept the AI's recommendation, regardless of its correctness. Our result poses new challenges for human-centered AI: Can we develop explanatory approaches that encourage appropriate trust in AI, and therefore help generate (or improve) complementary performance?

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2006.14779

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The task in Bansal et al. (sentiment analysis) could have been too easy to complete alone and the explanation modality (logical explanations) was too complicated to understand when participants were placed in a higher-effort LSAT task.

    “we postulate that the task (sentiment analysis) could have been too easy to complete alone and that the explanation modality (logical explanations) was too complicated to understand when participants were placed in a higher-effort LSAT task.”

    From Explanations Can Reduce Overreliance on AI Systems During Decision-Making (2023) · cites this paper as [4]

    The Domain and AI Model sections describe LSAT questions as more challenging and more time-consuming than sentiment classification. Section 5.3 suggests that verifying explanations was costlier for LSAT participants. The paper does not say sentiment analysis was too easy to need help, or that participants found the logical explanations too complicated to understand; those are the citing authors’ postulates. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

  • Bansal et al. found that explanations do not decrease overreliance.

    “Bansal et al. [4] find that explanations do not decrease overreliance”

    From Explanations Can Reduce Overreliance on AI Systems During Decision-Making (2023) · cites this paper as [4]

    Section 6.2 (page not provided) says explanations increased blind trust rather than appropriate reliance, supporting the claim for some explanations. But the analysis of AI recommendations (page not provided) reports that adaptive explanations reduced blind agreement when the AI was uncertain. The unqualified claim overlooks that result; the paper separately notes that the reduction did not significantly improve team accuracy. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • Explanations intend to improve performance in part by helping a user calibrate their trust according to a more accurate mental model of the AI's ability and accuracy.

    “Explanations intend to improve performance in part by helping a user calibrate their trust according to a more accurate mental model of the AI's ability and accuracy [4].”

    From Explanations Can Reduce Overreliance on AI Systems During Decision-Making (2023) · cites this paper as [4]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 79

  1. Gregory D. Abowd, Anind K. Dey, Peter J. Brown, Nigel Davies, Mark Smith, and Pete Steggles. 1999. Towards a Better Understanding of Context and Context- Awareness. In Proceedings of the 1st International Symposium on Handheld and Ubiquitous Computing (Karlsruhe, Germany) (HUC '99). Springer-Verlag, Berlin, Heidelberg, 304-307.
  2. Oisin Mac Aodha, Shihan Su, Yuxin Chen, Pietro Perona, and Yisong Yue. 2018. Teaching Categories to Human Learners With Visual Explanations. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018. IEEE Com- puter Society, Salt Lake City, UT, USA, 3820-3828. https://doi.org/10.1109/CVPR. 2018.00402 · DOI
  3. Gagan Bansal, Besmira Nushi, Ece Kamar, Eric Horvitz, and Daniel S. Weld. 2020. Optimizing AI for Teamwork. arXiv:2004.13102 [cs.AI] · arXiv
  4. Gagan Bansal, Besmira Nushi, Ece Kamar, Walter S. Lasecki, Daniel S. Weld, and Eric Horvitz. 2019. Beyond Accuracy: The Role of Mental Models in Human-AI Team Performance. Proceedings of the AAAI Conference on Human Computation and Crowdsourcing 7, 1 (Oct. 2019), 2-11. https://ojs.aaai.org/index.php/HCOMP/ article/view/5285
  5. Gagan Bansal, Besmira Nushi, Ece Kamar, Daniel S. Weld, Walter S. Lasecki, and Eric Horvitz. 2019. Updates in Human-AI Teams: Understanding and Addressing the Performance/Compatibility Tradeoff. Proceedings of the AAAI Conference on Artificial Intelligence 33, 01 (Jul. 2019), 2429-2437. https://doi.org/10.1609/aaai. v33i01.33012429 · DOI · Paper page
  6. Richard E Barlow and Hugh D Brunk. 1972. The isotonic regression problem and its dual. J. Amer. Statist. Assoc. 67, 337 (1972), 140-147.
  7. Mohsen Bayati, Mark Braverman, Michael Gillam, Karen M. Mack, George Ruiz, Mark S. Smith, and Eric Horvitz. 2014. Data-Driven Decisions for Reducing Readmissions for Heart Failure: General Methodology and Case Study. PLOS ONE 9, 10 (10 2014), 1-9. https://doi.org/10.1371/journal.pone.0109264 · DOI · Paper page
  8. Umang Bhatt, Alice Xiang, Shubham Sharma, Adrian Weller, Ankur Taly, Yun- han Jia, Joydeep Ghosh, Ruchir Puri, José M. F. Moura, and Peter Eckersley. 2020. Explainable Machine Learning in Deployment. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (Barcelona, Spain) (FAT* '20). Association for Computing Machinery, New York, NY, USA, 648-657. https://doi.org/10.1145/3351095.3375624 · DOI · Paper page
  9. Reuben Binns, Max Van Kleek, Michael Veale, Ulrik Lyngs, Jun Zhao, and Nigel Shadbolt. 2018. 'It's Reducing a Human Being to a Percentage': Per- ceptions of Justice in Algorithmic Decisions. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI '18). Association for Computing Machinery, New York, NY, USA, 1-14. https://doi.org/10.1145/3173574.3173951 · DOI · Paper page
  10. Zana Buçinca, Phoebe Lin, Krzysztof Z. Gajos, and Elena L. Glassman. 2020. Proxy Tasks and Subjective Measures Can Be Misleading in Evaluating Explainable AI Systems. In Proceedings of the 25th International Conference on Intelligent User Interfaces (Cagliari, Italy) (IUI '20). Association for Computing Machinery, New York, NY, USA, 454-464. https://doi.org/10.1145/3377325.3377498 · DOI · Paper page
  11. Adrian Bussone, Simone Stumpf, and Dympna O'Sullivan. 2015. The role of explanations on trust and reliance in clinical decision support systems. In 2015 International Conference on Healthcare Informatics. IEEE, IEEE, Dallas, Texas, 160-169.
  12. Carrie J Cai, Samantha Winter, David Steiner, Lauren Wilcox, and Michael Terry. 2019. " Hello AI": Uncovering the Onboarding Needs of Medical Practitioners for Human-AI Collaborative Decision-Making. Proceedings of the ACM on Human- Computer Interaction 3, CSCW (2019), 1-24.
  13. Rich Caruana, Yin Lou, Johannes Gehrke, Paul Koch, Marc Sturm, and Noemie Elhadad. 2015. Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-Day Readmission. In Proceedings of the 21th ACM SIGKDD In- ternational Conference on Knowledge Discovery and Data Mining (Sydney, NSW, Australia) (KDD '15). Association for Computing Machinery, New York, NY, USA, 1721-1730. https://doi.org/10.1145/2783258.2788613 · DOI · Paper page
  14. Arjun Chandrasekaran, Viraj Prabhu, Deshraj Yadav, Prithvijit Chattopadhyay, and Devi Parikh. 2018. Do explanations make VQA models more predictable to a human?. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Brussels, Belgium, 1036-1042. https://doi.org/10.18653/v1/D18-1128 · DOI
  15. Elizabeth Clark, Anne Spencer Ross, Chenhao Tan, Yangfeng Ji, and Noah A. Smith. 2018. Creative Writing with a Machine in the Loop: Case Studies on Slogans and Stories. In 23rd International Conference on Intelligent User Interfaces (Tokyo, Japan) (IUI '18). Association for Computing Machinery, New York, NY, USA, 329-340. https://doi.org/10.1145/3172944.3172983 · DOI · Paper page
  16. Pat Croskerry. 2009. Clinical cognition and diagnostic error: applications of a dual process model of reasoning. Advances in health sciences education 14, 1 (2009), 27-35.
  17. Xiao Dong and Caroline C Hayes. 2012. Uncertainty visualizations: Helping decision makers become more aware of uncertainty and its implications. Journal of Cognitive Engineering and Decision Making 6, 1 (2012), 30-56.
  18. Birte Englich, Thomas Mussweiler, and Fritz Strack. 2006. Playing dice with crim- inal sentences: The influence of irrelevant anchors on experts' judicial decision making. Personality and Social Psychology Bulletin 32, 2 (2006), 188-200.
  19. Shi Feng and Jordan Boyd-Graber. 2019. What Can AI Do for Me? Evaluating Machine Learning Interpretations in Cooperative Play. In Proceedings of the 24th International Conference on Intelligent User Interfaces (Marina del Ray, California) (IUI '19). Association for Computing Machinery, New York, NY, USA, 229-239. https://doi.org/10.1145/3301275.3302265 · DOI · Paper page
  20. Michael Fernandes, Logan Walls, Sean Munson, Jessica Hullman, and Matthew Kay. 2018. Uncertainty Displays Using Quantile Dotplots or CDFs Improve Transit Decision-Making. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI '18). Association for Computing Machinery, New York, NY, USA, 1-12. https://doi.org/10.1145/3173574.3173718 · DOI · Paper page
  21. Katy Ilonka Gero, Zahra Ashktorab, Casey Dugan, Qian Pan, James Johnson, Werner Geyer, Maria Ruiz, Sarah Miller, David R. Millen, Murray Campbell, Sadhana Kumaravel, and Wei Zhang. 2020. Mental Models of AI Agents in a Cooperative Game Setting. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '20). Association for Computing Machinery, New York, NY, USA, 1-12. https://doi.org/10.1145/ 3313831.3376316 · DOI · Paper page
  22. Reza Ghaeini, Xiaoli Fern, and Prasad Tadepalli. 2018. Interpreting Recurrent and Attention-Based Neural Models: a Case Study on Natural Language Inference. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Brussels, Belgium, 4952- 4957. https://doi.org/10.18653/v1/D18-1537 · DOI · Paper page
  23. Dimitra Gkatzia, Oliver Lemon, and Verena Rieser. 2016. Natural Language Generation enhances human decision-making with uncertain information. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Association for Computational Linguistics, Berlin, Germany, 264-268. https://doi.org/10.18653/v1/P16-2043 · DOI · Paper page
  24. Elena L Glassman, Jeremy Scott, Rishabh Singh, Philip J Guo, and Robert C Miller. 2015. OverCode: Visualizing variation in student solutions to programming problems at scale. ACM Transactions on Computer-Human Interaction (TOCHI) 22, 2 (2015), 1-35.
  25. Ben Green and Yiling Chen. 2019. The principles and limits of algorithm-in-the- loop decision making. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (2019), 1-24.
  26. Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. 2017. On Calibration of Modern Neural Networks. In Proceedings of the 34th International Conference on Machine Learning -Volume 70 (ICML'17). JMLR.org, Sydney, NSW, Australia, 1321-1330.
  27. Peter Hase and Mohit Bansal. 2020. Evaluating Explainable AI: Which Algorith- mic Explanations Help Users Predict Model Behavior?. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, Online, 5540-5552. https://doi.org/10.18653/v1/2020. acl-main.491 · DOI · Paper page
  28. Yugo Hayashi and Kosuke Wakabayashi. 2017. Can AI Become Reliable Source to Support Human Decision Making in a Court Scene?. In Companion of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing (Portland, Oregon, USA) (CSCW '17 Companion). Association for Computing Ma- chinery, New York, NY, USA, 195-198. https://doi.org/10.1145/3022198.3026338 · DOI
  29. Ruining He and Julian McAuley. 2016. Ups and Downs: Modeling the Vi- sual Evolution of Fashion Trends with One-Class Collaborative Filtering. In Proceedings of the 25th International Conference on World Wide Web (Mon- tréal, Québec, Canada) (WWW '16). International World Wide Web Confer- ences Steering Committee, Republic and Canton of Geneva, CHE, 507-517. https://doi.org/10.1145/2872427.2883037 · DOI
  30. Dan Hendrycks and Kevin Gimpel. 2017. A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks. In 5th International Con- ference on Learning Representations, ICLR 2017. OpenReview.net, Toulon, France, 1-12. https://openreview.net/forum?id=Hkg4TI9xl
  31. Chien-Ju Ho, Aleksandrs Slivkins, Siddharth Suri, and Jennifer Wortman Vaughan. 2015. Incentivizing High Quality Crowdwork. In Proceedings of the 24th Interna- tional Conference on World Wide Web (Florence, Italy) (WWW '15). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 419-429. https://doi.org/10.1145/2736277.2741102 · DOI · Paper page
  32. Eric Horvitz. 1999. Principles of Mixed-Initiative User Interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Pittsburgh, Pennsylvania, USA) (CHI '99). Association for Computing Machinery, New York, NY, USA, 159-166. https://doi.org/10.1145/302979.303030 · DOI · Paper page
  33. Eric Horvitz and Tim Paek. 2007. Complementary computing: policies for trans- ferring callers from dialog systems to human receptionists. User Modeling and User-Adapted Interaction 17, 1-2 (2007), 159-182.
  34. Shagun Jhaver, Amy Bruckman, and Eric Gilbert. 2019. Does transparency in moderation really matter? User behavior after content removal explanations on reddit. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (2019), 1-27.
  35. Susan Joslyn and Jared LeClerc. 2013. Decisions with uncertainty: The glass half full. Current Directions in Psychological Science 22, 4 (2013), 308-315.
  36. Harmanpreet Kaur, Harsha Nori, Samuel Jenkins, Rich Caruana, Hanna Wallach, and Jennifer Wortman Vaughan. 2020. Interpreting Interpretability: Under- standing Data Scientists' Use of Interpretability Tools for Machine Learning. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '20). Association for Computing Machinery, New York, NY, USA, 1-14. https://doi.org/10.1145/3313831.3376219 · DOI · Paper page
  37. Derek J Koehler. 1991. Explanation, imagination, and confidence in judgment. Psychological bulletin 110, 3 (1991), 499.
  38. Pang Wei Koh and Percy Liang. 2017. Understanding Black-Box Predictions via Influence Functions. In Proceedings of the 34th International Conference on Machine Learning -Volume 70 (ICML'17). JMLR.org, Sydney, NSW, Australia, 1885-1894.
  39. Johannes Kunkel, Tim Donkers, Lisa Michael, Catalin-Mihai Barbu, and Jür- gen Ziegler. 2019. Let Me Explain: Impact of Personal and Impersonal Ex- planations on Trust in Recommender Systems. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI '19). Association for Computing Machinery, New York, NY, USA, 1-12. https://doi.org/10.1145/3290605.3300717 · DOI · Paper page
  40. Vivian Lai, Han Liu, and Chenhao Tan. 2020. "Why is 'Chicago' Deceptive?" Towards Building Model-Driven Tutorials for Humans. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '20). Association for Computing Machinery, New York, NY, USA, 1-13. https://doi.org/10.1145/3313831.3376873 · DOI · Paper page
  41. Vivian Lai and Chenhao Tan. 2019. On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detection. In Proceedings of the Conference on Fairness, Accountability, and Transparency (Atlanta, GA, USA) (FAT* '19). Association for Computing Machinery, New York, NY, USA, 29-38. https://doi.org/10.1145/3287560.3287590 · DOI · Paper page
  42. Himabindu Lakkaraju, Ece Kamar, Rich Caruana, and Eric Horvitz. 2017. Iden- tifying Unknown Unknowns in the Open World: Representations and Policies for Guided Exploration. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI'17). AAAI Press, San Francisco, California, USA, 2124-2132.
  43. Himabindu Lakkaraju, Ece Kamar, Rich Caruana, and Jure Leskovec. 2017. Interpretable & Explorable Approximations of Black Box Models. arXiv:1707.01154 [cs.AI] · arXiv
  44. John D. Lee and Katrina A. See. 2004. Trust in Automation: Designing for Appropriate Reliance. Human Factors 46, 1 (2004), 50-80. https://doi.org/10. 1518/hfes.46.1.50_30392 · DOI · Paper page
  45. Timothy R Levine. 2014. Truth-default theory (TDT) a theory of human deception and deception detection. Journal of Language and Social Psychology 33, 4 (2014), 378-392.
  46. B. Lim, A. Dey, and D. Avrahami. 2009. Why and Why Not Explanations Improve the Intelligibility of Context-aware Intelligent Systems. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Boston, MA, USA) (CHI '09). ACM, New York, NY, USA, 2119-2128. https://doi.org/10.1145/1518701. 1519023 · DOI · Paper page
  47. Zhouhan Lin, Minwei Feng, Cícero Nogueira dos Santos, Mo Yu, Bing Xiang, Bowen Zhou, and Yoshua Bengio. 2017. A Structured Self-Attentive Sentence Embedding. In 5th International Conference on Learning Representations, ICLR 2017, Conference Track Proceedings. OpenReview.net, Toulon, France, 1-15. https: //openreview.net/forum?id=BJC_jUqxe
  48. Zachary C. Lipton. 2018. The Mythos of Model Interpretability: In Machine Learning, the Concept of Interpretability is Both Important and Slippery. Queue 16, 3 (June 2018), 31-57. https://doi.org/10.1145/3236386.3241340 · DOI · Paper page
  49. Angli Liu, Stephen Soderland, Jonathan Bragg, Christopher H. Lin, Xiao Ling, and Daniel S. Weld. 2016. Effective Crowd Annotation for Relation Extraction. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, San Diego, California, 897-906. https://doi.org/10. 18653/v1/N16-1104 · DOI · Paper page
  50. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv:1907.11692 [cs.CL] · arXiv
  51. Scott M. Lundberg, Bala Nair, Monica S. Vavilala, Mayumi Horibe, Michael J. Eisses, Trevor Adams, David E. Liston, Daniel King-Wai Low, Shu-Fang Newman, Jerry Kim, and Su-In Lee. 2018. Explainable machine-learning predictions for the prevention of hypoxaemia during surgery. Nature Biomedical Engineering 2, 10 (01 Oct 2018), 749-760. https://doi.org/10.1038/s41551-018-0304-0 · DOI · Paper page
  52. Les Macleod. 2011. Avoiding "groupthink" A manager's challenge. Nursing management 42, 10 (2011), 44-48.
  53. David Madras, Toniann Pitassi, and Richard Zemel. 2018. Predict Responsibly: Improving Fairness and Accuracy by Learning to Defer. In Proceedings of the 32nd International Conference on Neural Information Processing Systems (Montréal, Canada) (NIPS'18).
  54. Julian McAuley, Jure Leskovec, and Dan Jurafsky. 2012. Learning Attitudes and Attributes from Multi-Aspect Reviews. In Proceedings of the 2012 IEEE 12th International Conference on Data Mining (ICDM '12). IEEE Computer Society, USA, 1020-1025. https://doi.org/10.1109/ICDM.2012.110 · DOI
  55. Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. 2019. Model Cards for Model Reporting. In Proceedings of the Conference on Fairness, Accountability, and Transparency (Atlanta, GA, USA) (FAT* '19). Association for Computing Machinery, New York, NY, USA, 220-229. https://doi.org/10.1145/ 3287560.3287596 · DOI · Paper page
  56. Hussein Mozannar and David Sontag. 2020. Consistent Estimators for Learning to Defer to an Expert. In Proceedings of the 37th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 119), Hal Daumé III and Aarti Singh (Eds.). PMLR, Virtual, 7076-7087. http://proceedings.mlr.press/v119/ mozannar20b.html
  57. Limor Nadav-Greenberg and Susan L Joslyn. 2009. Uncertainty forecasts improve decision making among nonexperts. Journal of Cognitive Engineering and Decision Making 3, 3 (2009), 209-227.
  58. Menaka Narayanan, Emily Chen, Jeffrey He, Been Kim, Sam Gershman, and Finale Doshi-Velez. 2018. How do Humans Understand Explanations from Machine Learning Systems? An Evaluation of the Human-Interpretability of Explanation. arXiv:1802.00682 [cs.AI] · arXiv
  59. Dong Nguyen. 2018. Comparing Automatic and Human Evaluation of Local Explanations for Text Classification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). Association for Computational Linguistics, New Orleans, Louisiana, 1069-1078. https://doi.org/10.18653/v1/N18- 1097 · DOI
  60. Joon Sung Park, Rick Barber, Alex Kirlik, and Karrie Karahalios. 2019. A Slow Algorithm Improves Users' Assessments of the Algorithm's Accuracy. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (2019), 1-15.
  61. Ethan Perez, Siddharth Karamcheti, Rob Fergus, Jason Weston, Douwe Kiela, and Kyunghyun Cho. 2019. Finding Generalizable Evidence by Learning to Convince Q&A Models. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Association for Computational Linguistics, Hong Kong, China, 2402-2411. https://doi.org/10.18653/v1/D19-1244 · DOI
  62. Forough Poursabzi-Sangdeh, Daniel G. Goldstein, Jake M. Hofman, Jennifer Wort- man Vaughan, and Hanna Wallach. 2019. Manipulating and Measuring Model Interpretability. arXiv:1802.07810 [cs.AI] · arXiv
  63. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. "Why Should I Trust You?": Explaining the Predictions of Any Classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (San Francisco, California, USA) (KDD '16). Association for Computing Machinery, New York, NY, USA, 1135-1144. https://doi.org/10.1145/2939672. 2939778 · DOI · Paper page
  64. Philipp Schmidt and Felix Biessmann. 2019. Quantifying Interpretability and Trust in Machine Learning Systems. arXiv:1901.08558 [cs.LG] · arXiv
  65. C. Estelle Smith, Bowen Yu, Anjali Srivastava, Aaron Halfaker, Loren Terveen, and Haiyi Zhu. 2020. Keeping Community in the Loop: Understanding Wikipedia Stakeholder Values for Machine Learning-Based Systems. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI '20). Association for Computing Machinery, New York, NY, USA, 1-14. https://doi.org/10.1145/3313831.3376783 · DOI · Paper page
  66. Simone Stumpf, Vidya Rajaram, Lida Li, Weng-Keen Wong, Margaret Burnett, Thomas Dietterich, Erin Sullivan, and Jonathan Herlocker. 2009. Interacting meaningfully with machine learning systems: Three experiments. International Journal of Human-Computer Studies 67, 8 (2009), 639-662.
  67. Sarah Tan, Julius Adebayo, Kori Inkpen, and Ece Kamar. 2018. Inves- tigating Human + Machine Complementarity for Recidivism Predictions. arXiv:1808.09123 [cs.LG] · arXiv
  68. Kush R. Varshney, Prashant Khanduri, Pranay Sharma, Shan Zhang, and Pramod K. Varshney. 2018. Why Interpretability in Machine Learn- ing? An Answer Using Distributed Detection and Data Fusion Theory. arXiv:1806.09710 [stat.ML] · arXiv
  69. J. von Neumann and O. Morgenstern. 1947. Theory of games and economic behavior. Princeton University Press, Princeton, New Jersey.
  70. Danding Wang, Qian Yang, Ashraf Abdul, and Brian Y. Lim. 2019. Design- ing Theory-Driven User-Centric Explainable AI. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland Uk) (CHI '19). Association for Computing Machinery, New York, NY, USA, 1-15. https://doi.org/10.1145/3290605.3300831 · DOI · Paper page
  71. Yequan Wang, Minlie Huang, Xiaoyan Zhu, and Li Zhao. 2016. Attention-based LSTM for Aspect-level Sentiment Classification. In Proceedings of the 2016 Confer- ence on Empirical Methods in Natural Language Processing. Association for Com- putational Linguistics, Austin, Texas, 606-615. https://doi.org/10.18653/v1/D16- 1058 · DOI · Paper page
  72. Hilde J. P. Weerts, Werner van Ipenburg, and Mykola Pechenizkiy. 2019. A Human- Grounded Evaluation of SHAP for Alert Processing. arXiv:1907.03324 [cs.LG] · arXiv
  73. Daniel S. Weld and Gagan Bansal. 2019. The Challenge of Crafting Intelligible Intelligence. Commun. ACM 62, 6 (May 2019), 70-79. https://doi.org/10.1145/ 3282486 · DOI · Paper page
  74. Jenna Wiens, John Guttag, and Eric Horvitz. 2016. Patient risk stratification with time-varying parameters: a multitask learning approach. JMLR 17, 1 (2016), 2797-2819.
  75. Bryan Wilder, Eric Horvitz, and Ece Kamar. 2020. Learning to Complement Humans. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20, Christian Bessiere (Ed.). International Joint Con- ferences on Artificial Intelligence Organization, Yokohama, Japan, 1526-1533. https://doi.org/10.24963/ijcai.2020/212 Main track. · DOI · Paper page
  76. Fumeng Yang, Zhuanyi Huang, Jean Scholtz, and Dustin L. Arendt. 2020. How Do Visual Explanations Foster End Users' Appropriate Trust in Machine Learning?. In Proceedings of the 25th International Conference on Intelligent User Interfaces (Cagliari, Italy) (IUI '20). Association for Computing Machinery, New York, NY, USA, 189-201. https://doi.org/10.1145/3377325.3377480 · DOI · Paper page
  77. Kun Yu, Shlomo Berkovsky, Ronnie Taib, Jianlong Zhou, and Fang Chen. 2019. Do I Trust My Machine Teammate? An Investigation from Perception to Decision. In Proceedings of the 24th International Conference on Intelligent User Interfaces (Marina del Ray, California) (IUI '19). Association for Computing Machinery, New York, NY, USA, 460-468. https://doi.org/10.1145/3301275.3302277 · DOI · Paper page
  78. Weihao Yu, Zihang Jiang, Yanfei Dong, and Jiashi Feng. 2020. ReClor: A Reading Comprehension Dataset Requiring Logical Reasoning. In 8th International Con- ference on Learning Representations, ICLR 2020. OpenReview.net, Addis Ababa, Ethiopia, 1-26. https://openreview.net/forum?id=HJgJtT4tvB
  79. Yunfeng Zhang, Q. Vera Liao, and Rachel K. E. Bellamy. 2020. Effect of Confi- dence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (Barcelona, Spain) (FAT* '20). Association for Computing Machin- ery, New York, NY, USA, 295-305. https://doi.org/10.1145/3351095.3372852 · DOI · Paper page

Cited by

Sorted by citations

Authors
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. S.
Affiliations
  • University of Washington (US)
  • Microsoft (United States) (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2020
Pages
1-16
DOI
10.48550/arxiv.2006.14779
arXiv
2006.14779
OpenAlex
W3037634279
Cited by
754
References
0
Citations by year
2026: 235 · 2025: 148 · 2024: 128 · 2023: 125 · 2022: 88 · 2021: 26 · 2020: 3 · 2019: 1
FWCI
42.39
Type
conference paper
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • CHI Conference on Human Factors in Computing Systems (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • National Science Foundation (2040196)
  • Microsoft Research
  • University of Washington
  • Office of Naval Research (N00014- 18-1-2193, N00014-18, N00014)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
  • Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
Keywords
explainable AI, AI explanations, human-AI teaming, AI augmentation, trust in AI, human-AI decision-making, recommendation acceptance, human-centered AI

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review