CoChat
Sign Up
PreprintCited by 3Human-Automation Interaction and SafetyReinforcement Learning in RoboticsRecommender Systems and Techniques

Towards Optimizing Human-Centric Objectives in AI-Assisted Decision-Making With Offline Reinforcement Learning

Open access
Buçinca, Zana, Swaroop, Siddharth, Paluch, Amanda E., Murphy, Susan A., Gajos, Krzysztof Z.·2024·arXiv
DOI 10.48550/arxiv.2403.05911arXiv 2403.05911
Open this paper in CoChat — free accountOpen PDF

Abstract

Imagine if AI decision-support tools not only complemented our ability to make accurate decisions, but also improved our skills, boosted collaboration, and elevated the joy we derive from our tasks. Despite the potential to optimize a broad spectrum of such human-centric objectives, the design of current AI tools remains focused on decision accuracy alone. We propose offline reinforcement learning (RL) as a general approach for modeling human-AI decision-making to optimize human-AI interaction for diverse objectives. RL can optimize such objectives by tailoring decision support, providing the right type of assistance to the right person at the right time. We instantiated our approach with two objectives: human-AI accuracy on the decision-making task and human learning about the task and learned decision support policies from previous human-AI interaction data. We compared the optimized policies against several baselines in AI-assisted decision-making. Across two experiments (N=316 and N=964), our results demonstrated that people interacting with policies optimized for accuracy achieve significantly better accuracy -- and even human-AI complementarity -- compared to those interacting with any other type of AI support. Our results further indicated that human learning was more difficult to optimize than accuracy, with participants who interacted with learning-optimized policies showing significant learning improvement only at times. Our research (1) demonstrates offline RL to be a promising approach to model human-AI decision-making, leading to policies that may optimize human-centric objectives and provide novel insights about the AI-assisted decision-making space, and (2) emphasizes the importance of considering human-centric objectives beyond decision accuracy in AI-assisted decision-making, opening up the novel research challenge of optimizing human-AI interaction for such objectives.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2403.05911

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Information synthesis can also bias decision making by emphasizing misleading, incorrect, or unimportant information.

    “Despite this perception, information synthesis can also bias decision making by emphasizing misleading, incorrect, or unimportant information [9, 48].”

    From To Recommend or Not to Recommend: Designing and Evaluating AI-Enabled Decision Support for Time-Critical Medical Events (2025) · cites this paper as [9]

    In “Generating incorrect AI recommendations” (page unavailable), the paper describes AI explanations that emphasize an arbitrary, unhelpful detail while recommending a suboptimal choice. Section 7.3 says people may find misleading explanations plausible and overrely on AI. This supports the risk of biased decisions from misleading or unimportant AI-provided information, but the paper examines recommendations and explanations rather than information synthesis broadly; its example also describes the explanation as correct, not incorrect. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • The simulated system had an accuracy of 75%, a level used in prior studies evaluating AI support strategies.

    “As a result, our simulated system had an accuracy of 75%, a level used in prior studies evaluating the effects of different AI support strategies on decision making [8, 9].”

    From To Recommend or Not to Recommend: Designing and Evaluating AI-Enabled Decision Support for Time-Critical Medical Events (2025) · cites this paper as [9]

  • AI systems may need to support multiple human-centric objectives, such as learning, collaboration, and agency.

    “In addition to improving decision making, AI systems may need to support multiple human-centric objectives, such as learning, collaboration, and agency [9].”

    From To Recommend or Not to Recommend: Designing and Evaluating AI-Enabled Decision Support for Time-Critical Medical Events (2025) · cites this paper as [9]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 80

  1. Daniel A Adler, Emily Tseng, Khatiya C Moon, John Q Young, John M Kane, Emanuel Moss, David C Mohr, and Tanzeem Choudhury. 2022. Burnout and the quantified workplace: tensions around personal sensing interventions for stress in resident physicians. Proceedings of the ACM on Human-computer Interaction 6, CSCW2 (2022), 1-48.
  2. Barbara E Ainsworth, William L Haskell, Stephen D Herrmann, Nathanael Meckes, David R Bassett, Catrine Tudor-Locke, Jennifer L Greer, Jesse Vezina, Melicia C Whitt-Glover, and Arthur S Leon. 2011. 2011 Compendium of Physical Activities: a second update of codes and MET values. Med Sci Sports Exerc 43, 8 (2011), 1575-1581.
  3. Ifeoma Ajunwa. 2020. The "black box" at work. Big Data & Society 7, 2 (2020), 2053951720966181.
  4. Yasmeen Alufaisan, Laura R Marusich, Jonathan Z Bakdash, Yan Zhou, and Murat Kantarcioglu. 2021. Does explainable artificial intelligence improve human decision-making?. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 6618-6626.
  5. Gagan Bansal, Tongshuang Wu, Joyce Zhou, Raymond Fok, Besmira Nushi, Ece Kamar, Marco Tulio Ribeiro, and Daniel Weld. 2021. Does the whole exceed its parts? the effect of ai explanations on complementary team performance. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1-16.
  6. Lucy M Berlin and Robin Jeffries. 1992. Consultants and apprentices: observations about learning and collaborative problem solving. In Proceedings of the 1992 ACM conference on Computer-supported cooperative work. 130-137.
  7. Umang Bhatt, Valerie Chen, Katherine M Collins, Parameswaran Kamalaruban, Emma Kallina, Adrian Weller, and Ameet Talwalkar. 2023. Learning Personalized Decision Support Policies. arXiv preprint arXiv:2304.06701 (2023). · arXiv
  8. Zana Buçinca, Phoebe Lin, Krzysztof Z. Gajos, and Elena L. Glassman. 2020. Proxy Tasks and Subjective Measures Can Be Misleading in Evaluating Explainable AI Systems. In Proceedings of the 25th International Conference on Intelligent User Interfaces (IUI '20). ACM, New York, NY, USA.
  9. Zana Buçinca, Maja Barbara Malaya, and Krzysztof Z. Gajos. 2021. To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-Assisted Decision-Making. Proc. ACM Hum.-Comput. Interact. 5, CSCW1, Article 188 (April 2021), 21 pages. DOI:http://dx.doi.org/10.1145/3449287 · DOI · Paper page
  10. Ángel Alexander Cabrera, Adam Perer, and Jason I Hong. 2023. Improving human-AI collaboration with descriptions of AI behavior. Proceedings of the ACM on Human-Computer Interaction 7, CSCW1 (2023), 1-21.
  11. John T Cacioppo and Richard E Petty. 1982a. The need for cognition. Journal of personality and social psychology 42, 1 (1982), 116.
  12. John T. Cacioppo and Richard E. Petty. 1982b. The need for cognition. Journal of Personality and Social Psychology 42, 1 (1982), 116-131. DOI:http://dx.doi.org/10. 1037/0022-3514.42.1.116 · DOI · Paper page
  13. Shiye Cao, Catalina Gomez, and Chien-Ming Huang. 2023. How Time Pressure in Different Phases of Decision-Making Influences Human-AI Collaboration. Proceedings of the ACM on Human-Computer Interaction 7, CSCW2 (2023), 1-26.
  14. Giuseppe Carenini. 2001. An Analysis of the Influence of Need for Cognition on Dynamic Queries Usage. In CHI '01 Extended Abstracts on Human Factors in Computing Systems (CHI EA '01). ACM, New York, NY, USA, 383-384. DOI: http://dx.doi.org/10.1145/634067.634293 · DOI · Paper page
  15. Ana-Maria Cazan and Simona Elena Indreica. 2014. Need for cognition and approaches to learning among university students. Procedia-Social and Behavioral Sciences 127 (2014), 134-138.
  16. Valerie Chen, Q Vera Liao, Jennifer Wortman Vaughan, and Gagan Bansal. 2023. Understanding the Role of Human Intuition on Reliance in Human-AI Decision- Making with Explanations. arXiv e-prints (2023), arXiv-2301.
  17. Xiuli Chen, Sandra Dorothee Starke, Chris Baber, and Andrew Howes. 2017. A cognitive model of how people make decisions through interaction with visual displays. In Proceedings of the 2017 CHI conference on human factors in computing systems. 1205-1216.
  18. Lingwei Cheng and Alexandra Chouldechova. 2023. Overcoming Algorithm Aversion: A Comparison between Process and Outcome Control. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1-27.
  19. Nick Colegrave and Graeme D Ruxton. 2003. Confidence intervals are a more useful complement to nonsignificant tests than are power calculations. Behavioral Ecology 14, 3 (2003), 446-447.
  20. William M Crocoll and Bruce G Coury. 1990. Status or recommendation: Selecting the type of information for decision aiding. In Proceedings of the human factors society annual meeting, Vol. 34. SAGE Publications Sage CA: Los Angeles, CA, 1524-1528.
  21. Valdemar Danry, Pat Pataranutaporn, Yaoli Mao, and Pattie Maes. 2023. Don't Just Tell Me, Ask Me: AI Systems that Intelligently Frame Explanations as Questions Improve Human Logical Discernment Accuracy over Causal AI explanations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1-13.
  22. Edward L Deci, Anja H Olafsen, and Richard M Ryan. 2017. Self-determination theory in work organizations: The state of a science. Annual review of organiza- tional psychology and organizational behavior 4 (2017), 19-43.
  23. Edward L Deci and Richard M Ryan. 2012. Self-determination theory. Handbook of theories of social psychology 1, 20 (2012), 416-436.
  24. Louis Deslauriers, Logan S McCarty, Kelly Miller, Kristina Callaghan, and Greg Kestin. 2019. Measuring actual learning versus feeling of learning in response to being actively engaged in the classroom. Proceedings of the National Academy of Sciences (2019), 201821936.
  25. Shayan Doroudi, Vincent Aleven, and Emma Brunskill. 2019. Where's the reward? a review of reinforcement learning for instructional sequencing. International Journal of Artificial Intelligence in Education 29 (2019), 568-620.
  26. Matthew Fisher and Daniel M Oppenheimer. 2021a. Harder than you think: How outside assistance leads to overconfidence. Psychological Science 32, 4 (2021), 598-610.
  27. Matthew Fisher and Daniel M Oppenheimer. 2021b. Who knows what? Knowl- edge misattribution in the division of cognitive labor. Journal of Experimental Psychology: Applied 27, 2 (2021), 292.
  28. Krzysztof Z Gajos and Krysta Chauncey. 2017. The influence of personality traits and cognitive load on the use of adaptive user interfaces. In Proceedings of the 22nd International Conference on Intelligent User Interfaces. 301-306.
  29. Krzysztof Z Gajos and Lena Mamykina. 2022. Do People Engage Cognitively with AI? Impact of AI Assistance on Incidental Learning. arXiv preprint arXiv:2202.05402 (2022). · arXiv
  30. Susanne Gaube, Harini Suresh, Martina Raue, Alexander Merritt, Seth J Berkowitz, Eva Lermer, Joseph F Coughlin, John V Guttag, Errol Colak, and Marzyeh Ghas- semi. 2021. Do as AI say: susceptibility in deployment of clinical decision-aids. NPJ digital medicine 4, 1 (2021), 1-8.
  31. Luke Guerdan, Amanda Coston, Zhiwei Steven Wu, and Kenneth Holstein. 2023. Ground(less) Truth: A Causal Framework for Proxy Labels in Human-Algorithm Decision-Making. In Proceedings of the 2023 ACM Conference on Fairness, Account- ability, and Transparency (FAccT '23). Association for Computing Machinery, New York, NY, USA, 688-704. DOI:http://dx.doi.org/10.1145/3593013.3594036 · DOI · Paper page
  32. J Richard Hackman and Greg R Oldham. 1976. Motivation through the design of work: Test of a theory. Organizational behavior and human performance 16, 2 (1976), 250-279.
  33. Gaole He, Lucie Kuiper, and Ujwal Gadiraju. 2023. Knowing About Knowing: An Illusion of Human Competence Can Hinder Appropriate Reliance on AI Systems. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1-18.
  34. Guido W Imbens and Donald B Rubin. 2015. Causal inference in statistics, social, and biomedical sciences. Cambridge University Press.
  35. Jussi Jokinen, Aditya Acharya, Mohammad Uzair, Xinhui Jiang, and Antti Oulasvirta. 2021. Touchscreen typing as optimal supervisory control. In Proceed- ings of the 2021 CHI conference on human factors in computing systems. 1-14.
  36. Patricia K Kahr, Gerrit Rooks, Martijn C Willemsen, and Chris CP Snijders. 2023. It seems smart, but it acts stupid: Development of trust in ai advice in a repeated legal decision-making task. In Proceedings of the 28th International Conference on Intelligent User Interfaces. 528-539.
  37. Anna Kawakami, Shreya Chowdhary, Shamsi T Iqbal, Q Vera Liao, Alexandra Olteanu, Jina Suh, and Koustuv Saha. 2023a. Sensing wellbeing in the work- place, why and for whom? envisioning impacts with organizational stakeholders. Proceedings of the ACM on Human-Computer Interaction 7, CSCW2 (2023), 1-33.
  38. Anna Kawakami, Luke Guerdan, Yanghuidi Cheng, Matthew Lee, Scott Carter, Nikos Arechiga, Kate Glazko, Haiyi Zhu, and Kenneth Holstein. 2023b. Training Towards Critical Use: Learning to Situate AI Predictions Relative to Human Knowledge. arXiv preprint arXiv:2308.15700 (2023). · arXiv
  39. Kenneth R Koedinger, Paulo F Carvalho, Ran Liu, and Elizabeth A McLaughlin. 2023. An astonishing regularity in student learning rate. Proceedings of the National Academy of Sciences 120, 13 (2023), e2221311120.
  40. Vivian Lai, Yiming Zhang, Chacha Chen, Q Vera Liao, and Chenhao Tan. 2023. Se- lective explanations: Leveraging human input to align explainable ai. Proceedings of the ACM on Human-Computer Interaction 7, CSCW2 (2023), 1-35.
  41. Markus Langer, Tim Hunsicker, Tina Feldkamp, Cornelius J König, and Nina Grgić-Hlača. 2022. "Look! It'sa computer program! It's an algorithm! It's AI!": Does terminology affect human perceptions and evaluations of algorithmic decision-making systems?. In Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems. 1-28.
  42. Dong Kyu Lee. 2016. Alternatives to P value: confidence interval and effect size. Korean journal of anesthesiology 69, 6 (2016), 555.
  43. Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu. 2020. Offline rein- forcement learning: Tutorial, review, and perspectives on open problems. arXiv preprint arXiv:2005.01643 (2020). · arXiv
  44. Zhi Li, Yu-Jung Ko, Aini Putkonen, Shirin Feiz, Vikas Ashok, IV Ramakrishnan, Antti Oulasvirta, and Xiaojun Bi. 2023. Modeling touch-based menu selection performance of blind users via reinforcement learning. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1-18.
  45. Peng Liao, Predrag Klasnja, Ambuj Tewari, and Susan A Murphy. 2016. Sample size calculations for micro-randomized trials in mHealth. Statistics in medicine 35, 12 (2016), 1944-1971.
  46. Chin-Lung Lin, Sheng-Hsien Lee, and Der-Juinn Horng. 2011. The effects of online reviews on purchasing intention: The moderating role of need for cognition. Social Behavior and Personality: an international journal 39, 1 (2011), 71-81.
  47. Zhuoran Lu, Dakuo Wang, and Ming Yin. 2024. Does more advice help? the effects of second opinions in AI-assisted decision making. arXiv preprint arXiv:2401.07058 (2024). · arXiv
  48. Zhuoran Lu and Ming Yin. 2021. Human reliance on machine learning models when performance feedback is limited: Heuristics and risks. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1-16.
  49. Shuai Ma, Ying Lei, Xinru Wang, Chengbo Zheng, Chuhan Shi, Ming Yin, and Xiaojuan Ma. 2023. Who Should I Trust: AI or Myself? Leveraging Human and AI Correctness Likelihood to Promote Appropriate Trust in AI-Assisted Decision- Making. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1-19.
  50. Victoria J. Marsick and Karen E. Watkins. 2001. Informal and Incidental Learning. New Directions for Adult and Continuing Education 2001, 89 (2001), 25. DOI: http://dx.doi.org/10.1002/ace.5 · DOI
  51. Victoria J Marsick, Karen E Watkins, Ellen Scully-Russ, and Aliki Nicolaides. 2017. Rethinking informal and incidental learning in terms of complexity and the social context. Journal of Adult Learning, Knowledge and Innovation 1, 1 (2017), 27-34.
  52. Tim Miller. 2023. Explainable AI is Dead, Long Live Explainable AI! Hypothesis- Driven Decision Support Using Evaluative AI. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (FAccT '23). Association for Computing Machinery, New York, NY, USA, 333-342. DOI:http://dx.doi.org/ 10.1145/3593013.3594001 · DOI · Paper page
  53. Frederick P Morgeson and Michael A Campion. 2003. Work design. Handbook of psychology: Industrial and organizational psychology 12, 2 (2003), 423-452.
  54. Hussein Mozannar, Jimin Lee, Dennis Wei, Prasanna Sattigeri, Subhro Das, and David Sontag. 2024. Effective Human-AI Teams via Learned Natural Language Rules and Onboarding. Advances in Neural Information Processing Systems 36 (2024).
  55. Hussein Mozannar, Arvind Satyanarayan, and David Sontag. 2022. Teaching humans when to defer to a classifier via exemplars. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 5323-5331.
  56. Gali Noti and Yiling Chen. 2022. Learning When to Advise Human Decision Makers. arXiv preprint arXiv:2209.13578 (2022). · arXiv
  57. Joon Sung Park, Rick Barber, Alex Kirlik, and Karrie Karahalios. 2019. A Slow Algorithm Improves Users' Assessments of the Algorithm's Accuracy. Proceedings of the ACM on Human-Computer Interaction 3, CSCW (2019), 1-15.
  58. Sharon K Parker and Caroline Knight. 2024. The SMART model of work design: A higher order structure to help see the wood from the trees. Human Resource Management 63, 2 (2024), 265-291.
  59. Sharon K Parker, Frederick P Morgeson, and Gary Johns. 2017. One hundred years of work design research: Looking back and looking forward. Journal of applied psychology 102, 3 (2017), 403.
  60. Samir Passi and Mihaela Vorvoreanu. 2022. Overreliance on AI Literature Review. Microsoft Research (2022).
  61. Marc Pinski, Martin Adam, and Alexander Benlian. 2023. AI Knowledge: Improv- ing AI Delegation through Human Enablement. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1-17.
  62. Forough Poursabzi-Sangdeh, Daniel G Goldstein, Jake M Hofman, Jennifer Wort- man Vaughan, and Hanna Wallach. 2018. Manipulating and measuring model interpretability. arXiv preprint arXiv:1802.07810 (2018). · arXiv
  63. Amy Rechkemmer and Ming Yin. 2022. When confidence meets accuracy: Explor- ing the effects of multiple performance indicators on trust in machine learning models. In Proceedings of the 2022 chi conference on human factors in computing systems. 1-14.
  64. Katharina Reinecke and Krzysztof Z. Gajos. 2015. LabintheWild: Conducting Large-Scale Online Experiments With Uncompensated Samples. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW '15). ACM, New York, NY, USA, 1364-1378. DOI:http://dx. doi.org/10.1145/2675133.2675246 · DOI
  65. Günter Ropohl. 1999. Philosophy of socio-technical systems. Society for Philoso- phy and Technology Quarterly Electronic Journal 4, 3 (1999), 186-194.
  66. Jerome I Rotgans and Henk G Schmidt. 2011. Cognitive engagement in the problem-based learning classroom. Advances in health sciences education 16, 4 (2011), 465-479.
  67. James Schaffer, John O'Donovan, James Michaelis, Adrienne Raglin, and Tobias Höllerer. 2019. I can do better than your AI: expertise and explanations. In Proceedings of the 24th International Conference on Intelligent User Interfaces. 240-251.
  68. Maria Sicilia, Salvador Ruiz, and Jose L Munuera. 2005. Effects of interactivity in a web site: The moderating effect of need for cognition. Journal of advertising 34, 3 (2005), 31-44.
  69. Divya Siddarth, Daron Acemoglu, Danielle Allen, Kate Crawford, James Evans, Michael Jordan, and E Weyl. 2021. How AI fails us. arXiv preprint arXiv:2201.04200 (2021). · arXiv
  70. Siddharth Swaroop, Zana Buçinca, Krzysztof Z. Gajos, and Finale Doshi-Velez. 2024. Accuracy-Time Tradeoffs in AI-Assisted Decision Making under Time Pressure. In 29th International Conference on Intelligent User Interfaces (IUI '24). ACM.
  71. Bruce Thompson. 2007. Effect sizes, confidence intervals, and confidence intervals for effect sizes. Psychology in the Schools 44, 5 (2007), 423-432.
  72. Kashyap Todi, Gilles Bailly, Luis Leiva, and Antti Oulasvirta. 2021. Adapting user interfaces with model-based reinforcement learning. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1-13.
  73. Tracy L Tuten and Michael Bosnjak. 2001. Understanding differences in web usage: The role of need for cognition and the five factor model of personality. Social Behavior and Personality: an international journal 29, 4 (2001), 391-398.
  74. Helena Vasconcelos, Matthew Jörke, Madeleine Grunde-McLaughlin, Tobias Gerstenberg, Michael S Bernstein, and Ranjay Krishna. 2023. Explanations can reduce overreliance on ai systems during decision-making. Proceedings of the ACM on Human-Computer Interaction 7, CSCW1 (2023), 1-38.
  75. Jennifer Irvin Vidrine, Vani Nath Simmons, and Thomas H. Brandon. 2007. Con- struction of smoking-relevant risk perceptions among college students: The influ- ence of need for cognition and message content. Journal of Applied Social Psychol- ogy 37, 1 (2007), 91-114. DOI:http://dx.doi.org/10.1111/j.0021-9029.2007.00149.x · DOI
  76. Guy H Walker, Neville A Stanton, Paul M Salmon, and Daniel P Jenkins. 2008. A review of sociotechnical systems theory: a classic concept for new command and control paradigms. Theoretical issues in ergonomics science 9, 6 (2008), 479-499.
  77. Pamela Williams-Piehota, Tamera R Schneider, Linda Mowad, and Peter Salovey. 2003. Matching Health Messages to Information-Processing Styles : Need for Cognition and Mammography Utilization. Health Communication 15, 4 (2003), 375-392.
  78. Qian Yang, Yuexing Hao, Kexin Quan, Stephen Yang, Yiran Zhao, Volodymyr Kuleshov, and Fei Wang. 2023. Harnessing biomedical literature to calibrate clinicians' trust in AI decision support systems. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1-14.
  79. Ming Yin, Jennifer Wortman Vaughan, and Hanna Wallach. 2019. Understanding the effect of accuracy on trust in machine learning models. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1-12.
  80. Yunfeng Zhang, Q. Vera Liao, and Rachel K. E. Bellamy. 2020. Effect of Confidence and Explanation on Accuracy and Trust Calibration in AI-Assisted Decision Making. In Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency (FAT* '20). Association for Computing Machinery, New York, NY, USA, 295-305. DOI:http://dx.doi.org/10.1145/3351095.3372852 learning-SXAI -.18 [-.48, .11] .19 [-.15, .54] combined-SXAI .08 [-.22, .38] .17 [-.16, .51] · DOI · Paper page

Cited by

Sorted by citations

Authors
Buçinca, Z., Swaroop, S., Paluch, A. E., Murphy, S. A., & Gajos, K. Z.
Affiliations
  • Harvard University (US)
  • University of Massachusetts Amherst (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2024
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2403.05911
arXiv
2403.05911
OpenAlex
W4392737061
Cited by
2
References
0
Citations by year
2025: 1 · 2024: 1
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • National Science Foundation (2107391)
  • University of Michigan
Topics
  • Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
Keywords
AI-assisted decision-making, human-AI interaction, offline reinforcement learning, human-AI complementarity, human learning, decision accuracy, personalized assistance

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review