Human-AI Collaboration with Bandit Feedback
Open accessAbstract
Human-machine complementarity is important when neither the algorithm nor the human yield dominant performance across all instances in a given domain. Most research on algorithmic decision-making solely centers on the algorithm's performance, while recent work that explores human-machine collaboration has framed the decision-making problems as classification tasks. In this paper, we first propose and then develop a solution for a novel human-machine collaboration problem in a bandit feedback setting. Our solution aims to exploit the human-machine complementarity to maximize decision rewards. We then extend our approach to settings with multiple human decision makers. We demonstrate the effectiveness of our proposed methods using both synthetic and real human responses, and find that our methods outperform both the algorithm and the human when they each make decisions on their own. We also show how personalized routing in the presence of multiple human decision-makers can further improve the human-machine team performance.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Scholars describe human-AI augmentation as humans and machines 'collaborating' together.
“Embracing the vision of multiplying diverse expertise, many scholars describe human-AI augmentation as humans and machines 'collaborating' together (e.g., Boyaci et al. 2020, Gao et al. 2021, Khadpe et al. 2020, Puranam 2021, Wilson and Daugherty 2018).”
From To Engage or Not to Engage with AI for Critical Judgments: How Professionals Deal with Opacity When Using AI for Medical Diagnosis (2022) · cites this paper as Gao et al. 2021
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
No references could be resolved yet.
Research works printed but not linked: 30
- Joshua D Angrist, Guido W Imbens, and Donald B Rubin. Identification of causal effects us- ing instrumental variables. JASA, 91(434):444-455, 1996.
- Gagan Bansal, Besmira Nushi, Ece Ka- mar, Eric Horvitz, and Daniel S Weld. Optimizing ai for teamwork. arXiv:2004.13102, 2020. · arXiv
- Yoshua Bengio, Ian Goodfellow, and Aaron Courville. Deep learning, volume 1. MIT press Massachusetts, USA:, 2017.
- Dimitris Bertsimas and Nathan Kallus. The power and limits of predictive approaches to observational-data-driven optimization. arXiv, 2016.
- Matthew R Boutell, Jiebo Luo, Xipeng Shen, and Christopher M Brown. Learning multi-label scene classification. Pattern recognition, 37(9):1757- 1771, 2004.
- Hongming Chen, Ola Engkvist, Yinhai Wang, Marcus Olivecrona, and Thomas Blaschke. The rise of deep learning in drug discovery. Drug discovery today, 23(6):1241-1250, 2018.
- Maria De-Arteaga, Artur Dubrawski, and Alexandra Chouldechova. Learning under selective labels in the presence of expert consis- tency. Workshop on FAT / ML, 2018.
- Maria De-Arteaga, Riccardo Fogliato, and Alexandra Chouldechova. A case for humans-in-the-loop: Decisions in the presence of erroneous algorithmic scores. In CHI, pages 1-12, 2020.
- Abir De, Paramita Koley, Niloy Ganguly, and Manuel Gomez-Rodriguez. Regression under human assistance. In AAAI, pages 2611-2620, 2020.
- Elisseeff and Weston, 2002] André Elisseeff and Jason We- ston. A kernel method for multi-labelled classification. In NeurIPS, pages 681-687, 2002.
- Green and Chen, 2020] Ben Green and Yiling Chen. Al- gorithmic risk assessments can alter human decision- making processes in high-stakes government contexts. arXiv:2012.05370, 2020. · arXiv
- Sheng-Jun Huang, Jia-Lve Chen, Xin Mu, and Zhi-Hua Zhou. Cost-effective active learning from diverse labelers. In IJCAI, 2017.
- Thorsten Joachims, Adith Swami- nathan, and Maarten de Rijke. Deep learning with logged bandit feedback. In ICLR, 2018.
- Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv, 2014.
- Shao-Yuan Li, Yuan Jiang, Nitesh V Chawla, and Zhi-Hua Zhou. Multi-label learning from crowds. TKDE, 31(7):1369-1382, 2018.
- David Madras, Toni Pitassi, and Richard Zemel. Predict responsibly: improving fairness and accuracy by learning to defer. NeurIPS, 31:6147- 6157, 2018.
- Mozannar and Sontag, 2020] Hussein Mozannar and David Sontag. Consistent estimators for learning to defer to an expert. arXiv:2006.01862, 2020. · arXiv
- Aldo Pacchiano, My Phan, Yasin Abbasi-Yadkori, Anup Rao, Julian Zimmert, Tor Latti- more, and Csaba Szepesvari. Model selection in contex- tual stochastic bandit problems. arXiv:2003.01704, 2020. · arXiv
- Pearl, 2017] Judea Pearl. Detecting latent heterogeneity. Sociological Methods & Research, 46(3):370-389, 2017.
- Raghu et al., 2019] Maithra Raghu, Katy Blumer, Greg Corrado, Jon Kleinberg, Ziad Obermeyer, and Sendhil Mullainathan. The algorithmic automation problem: Pre- diction, triage, and human effort. arXiv:1903.12220, 2019. · arXiv
- Paul R Rosenbaum. Model-based direct adjustment. JASA, 82(398):387-394, 1987.
- Donald B Rubin. Causal inference using po- tential outcomes: Design, modeling, decisions. JASA, 100(469):322-331, 2005.
- Andrey Rzhetsky, Hagit Shatkay, and W John Wilbur. How to get the most out of your curation effort. PLoS computational biology, 5(5):e1000391, 2009.
- Megan T Stevenson and Jen- nifer L Doleac. Algorithmic risk assessment in the hands of humans. Available at SSRN, 2019.
- Swaminathan and Joachims, 2015a] Adith Swaminathan and Thorsten Joachims. Counterfactual risk minimization: Learning from logged bandit feedback. In ICML, pages 814-823, 2015.
- Swaminathan and Joachims, 2015b] Adith Swaminathan and Thorsten Joachims. The self-normalized estimator for counterfactual learning. In NeurIPS, 2015.
- Wang and Saar-Tsechansky, 2020] Tong Wang and May- tal Saar-Tsechansky. Augmented fairness: An inter- pretable model augmenting decision-makers' fairness. arXiv:2011.08398, 2020. · arXiv
- Bryan Wilder, Eric Horvitz, and Ece Kamar. Learning to complement humans. arXiv, 2020.
- Yanbo Xu, Yanxun Xu, and Suchi Saria. A bayesian nonparametric approach for estimating individ- ualized treatment-response curves. In Machine Learning for Healthcare Conference, pages 282-300, 2016.
- Yan Yan, Romer Rosales, Glenn Fung, and Jennifer G Dy. Active learning from crowds. In ICML, volume 11, pages 1161-1168, 2011.
Cited by
Sorted by citations
Lebovitz, S., Lifshitz-Assaf, H., & Levina, N. (2022) · Organization Science · doi:10.1287/orsc.2021.1549 · Cited by 632 · see citing works
- Authors
- Gao, R., Saar-Tsechansky, M., De-Arteaga, M., Han, L., Lee, M. K., & Lease, M.
- Affiliations
- The University of Texas at Austin (US)
- Rutgers, The State University of New Jersey (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2021
- Pages
- 1722-1728
- arXiv
- 2105.10614
- OpenAlex
- W3192754559
- Cited by
- 23
- References
- 0
- Citations by year
- 2026: 3 · 2025: 6 · 2024: 3 · 2023: 5 · 2022: 6
- FWCI
- 5.10
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- International Joint Conference on Artificial Intelligence (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Micron Foundation
- University of Texas at Austin
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Advanced Bandit Algorithms Research
- Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Keywords
- human-AI collaboration, bandit feedback, human-machine complementarity, algorithmic decision-making, multi-armed bandits, personalized routing, human response
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
