The Principles and Limits of Algorithm-in-the-Loop Decision Making
The DOI matches its Crossref record; 61 of the 64 references it cites were found in a registry or an index.
Abstract
The rise of machine learning has fundamentally altered decision making: rather than being made solely by people, many important decisions are now made through an "algorithm-in-the-loop'' process where machine learning models inform people. Yet insufficient research has considered how the interactions between people and models actually influence human decisions. Society lacks both clear normative principles regarding how people should collaborate with algorithms as well as robust empirical evidence about how people do collaborate with algorithms. Given research suggesting that people struggle to interpret machine learning models and to incorporate them into their decisions---sometimes leading these models to produce unexpected outcomes---it is essential to consider how different ways of presenting models and structuring human-algorithm interactions affect the quality and type of decisions made. This paper contributes to such research in two ways. First, we posited three principles as essential to ethical and responsible algorithm-in-the-loop decision making. Second, through a controlled experimental study on Amazon Mechanical Turk, we evaluated whether people satisfy these principles when making predictions with the aid of a risk assessment. We studied human predictions in two contexts (pretrial release and financial lending) and under several conditions for risk assessment presentation and structure. Although these conditions did influence participant behaviors and in some cases improved performance, only one desideratum was consistently satisfied. Under all conditions, our study participants 1) were unable to effectively evaluate the accuracy of their own or the risk assessment's predictions, 2) did not calibrate their reliance on the risk assessment based on the risk assessment's performance, and 3) exhibited bias in their interactions with the risk assessment. These results highlight the urgent need to expand our analyses of algorithmic decision making aids beyond evaluating the models themselves to investigating the full sociotechnical contexts in which people and algorithms interact.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“there has been increasing recent research efforts [8, 15, 22, 41, 58, 65] that highlight the importance of involving stakeholders to understand their practices and needs [15, 41, 65] and socio-environmental factors [8] for the design and evaluation of a system.”
From Understanding the Effect of Counterfactual Explanations on Trust and Reliance on AI for Human-AI Collaborative Clinical Decision Making (2023) · cites this paper as [22]
In the citing paper
“However, many empirical studies on human-AI decision-making did not observe complementary performance [6,16,37,52,58,61,68,82,84,87].”
From Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations (2023) · cites this paper as [37]
In the citing paper
“We note that these approaches correspond to the Update and Feedback methods of Green & Chen [27], which were effective, albeit not in the complementary zone.”
From Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance (2021) · cites this paper as [27]
In the citing paper
“More interestingly, communication failure can also stem from a deeper inability to understand how AI makes decisions and how likely AI may err in specific conditions [61] .”
From Enhancing human-AI collaboration: The case of colonoscopy (2024) · cites this paper as [61]
In the citing paper
“However, many studies have shown that human-AI teams under-perform the AI alone in tasks [3, 4, 6, 14, 18, 20, 21, 30].”
From A Decision Theoretic Framework for Measuring AI Reliance (2024) · cites this paper as [14]
7 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Greedy function approximation: A gradient boosting machine
Friedman, J. H. (2001) · The Annals of Statistics · doi:10.1214/aos/1013203451 · Cited by 30,731
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605
Strictly Proper Scoring Rules, Prediction, and Estimation
Gneiting, T., & Raftery, A. E. (2007) · Journal of the American Statistical Association · doi:10.1198/016214506000001437 · Cited by 5,793
Algorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232
What Should Be Done with Equity Theory?
Leventhal, G. S. (1980) · Social Exchange · doi:10.1007/978-1-4613-3087-5_2 · Cited by 3,010 · cited 2×
“For example, the principle of procedural justice requires that decisions be (among other things) accurate, fair, consistent, correctable, and ethical [55].”
BackgroundAlgorithm appreciation: People prefer algorithmic to human judgment
Logg, J. M., Minson, J. A., & Moore, D. A. (2019) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2018.12.005 · Cited by 2,061
Principles of mixed-initiative user interfaces
Horvitz, E. (1999) · Proceedings of the SIGCHI conference on Human factors in computing systems the CHI is the limit - CHI '99 · doi:10.1145/302979.303030 · Cited by 1,490
Lee, M. K. (2018) · Big Data & Society · doi:10.1177/2053951718756684 · Cited by 1,338
Human Decisions and Machine Predictions
Kleinberg, J., Lakkaraju, H., Leskovec, J., Ludwig, J., & Mullainathan, S. (2018) · The Quarterly Journal of Economics · doi:10.1093/qje/qjx032 · Cited by 1,040
Research works printed but not linked: 12
- Alex Albright . 2019. If You Give a Judge a Risk Score: Evidence from Kentucky Bail Decisions . The John M. Olin Center for Law , Economics , and Business Fellows' Discussion Paper Series 85 ( 2019 ). Alex Albright. 2019. If You Give a Judge a Risk Score: Evidence from Kentucky Bail Decisions. The John M. Olin Center for Law, Economics, and Business Fellows' Discussion Paper Series 85 (2019).
- Julia Angwin , Jeff Larson , Surya Mattu , and Lauren Kirchner . 2016. Machine Bias. ProPublica ( 2016 ). https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2016. Machine Bias. ProPublica (2016). https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing
- Laura Bliss . 2018. Former Uber Backup Driver: 'We Saw This Coming'. CityLab ( 2018 ). https://www.citylab.com/transportation/2018/03/former-uber-backup-driver-we-saw-this-coming/556427/ Laura Bliss. 2018. Former Uber Backup Driver: 'We Saw This Coming'. CityLab (2018). https://www.citylab.com/transportation/2018/03/former-uber-backup-driver-we-saw-this-coming/556427/
- Darwin Bond-Graham and Ali Winston . 2013. All Tomorrow's Crimes: The Future of Policing Looks a Lot Like Good Branding. SF Weekly ( 2013 ). http://archives.sfweekly.com/sanfrancisco/all-tomorrows-crimes-the-future-of-policing-looks-a-lot-like-good-branding/Content?oid=2827968 Darwin Bond-Graham and Ali Winston. 2013. All Tomorrow's Crimes: The Future of Policing Looks a Lot Like Good Branding. SF Weekly (2013). http://archives.sfweekly.com/sanfrancisco/all-tomorrows-crimes-the-future-of-policing-looks-a-lot-like-good-branding/Content?oid=2827968
- Pamela M. Casey , Roger K. Warren , and Jennifer K . Elek . 2011 . Using Offender Risk and Needs Assessment Information at Sentencing : Guidance for Courts from a National Working Group .National Center for State Courts . Pamela M. Casey, Roger K. Warren, and Jennifer K. Elek. 2011. Using Offender Risk and Needs Assessment Information at Sentencing: Guidance for Courts from a National Working Group .National Center for State Courts.
- Thomas H. Cohen , Bailey Pendergast , and Scott W. VanBenschoten . 2016 . Examining overrides of risk classifications for offenders on federal supervision . Federal Probation , Vol. 80 , 1 (2016), 12 . Thomas H. Cohen, Bailey Pendergast, and Scott W. VanBenschoten. 2016. Examining overrides of risk classifications for offenders on federal supervision. Federal Probation, Vol. 80, 1 (2016), 12.
- Mary L. Cummings . 2006. Automation and Accountability in Decision Support System Interface Design. Journal of Technology Studies ( 2006 ). Mary L. Cummings. 2006. Automation and Accountability in Decision Support System Interface Design. Journal of Technology Studies (2006).
- William Dieterich , Christina Mendoza , and Tim Brennan . 2016. COMPAS Risk Scales: Demonstrating Accuracy Equity and Predictive Parity . Northpointe Inc . ( 2016 ). William Dieterich, Christina Mendoza, and Tim Brennan. 2016. COMPAS Risk Scales: Demonstrating Accuracy Equity and Predictive Parity. Northpointe Inc. (2016).
- Jeff Larson , Surya Mattu , Lauren Kirchner , and Julia Angwin . 2016. How We Analyzed the COMPAS Recidivism Algorithm. ProPublica ( 2016 ). https://www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm Jeff Larson, Surya Mattu, Lauren Kirchner, and Julia Angwin. 2016. How We Analyzed the COMPAS Recidivism Algorithm. ProPublica (2016). https://www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm
- 10.1518/hfes.46.1.50.30392 · doi:10.1518/hfes.46.1.50.30392 · Paper page
- Frank Main . 2016. Cook County judges not following bail recommendations: study. Chicago Sun-Times ( 2016 ). https://chicago.suntimes.com/chicago-news/cook-county-judges-not-following-bail-recommendations-study-find/ Frank Main. 2016. Cook County judges not following bail recommendations: study. Chicago Sun-Times (2016). https://chicago.suntimes.com/chicago-news/cook-county-judges-not-following-bail-recommendations-study-find/
- Megan T. Stevenson and Jennifer L . Doleac . 2018 . The Roadblock to Reform. The American Constitution Society ( 2018). https://www.acslaw.org/wp-content/uploads/2018/11/RoadblockToReformReport.pdf Megan T. Stevenson and Jennifer L. Doleac. 2018. The Roadblock to Reform. The American Constitution Society (2018). https://www.acslaw.org/wp-content/uploads/2018/11/RoadblockToReformReport.pdf
Cited by
Items 101–200 of 393
When Algorithms Err : Differential Impact of Early vs. Late Errors on Users’ Reliance on Algorithms
Kim, A., Yang, M., & Zhang, J. (2020) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3557889 · Cited by 57
“In most application scenarios, however, humans are still in the łdriver’s seat," choosing whether or not to rely on algorithms for decision-making [18].”
BackgroundPainting the Black Box White: Experimental Findings from Applying XAI to an ECG Reading Setting
Cabitza, F., Campagner, A., Natali, C., Parimbelli, E., Ronzio, L., & Cameli, M. (2023) · Machine Learning and Knowledge Extraction · doi:10.3390/make5010017 · Cited by 41
Regulating Government AI and the Challenge of Sociotechnical Design
Engstrom, D. F., & Haim, A. (2023) · Annual Review of Law and Social Science · doi:10.1146/annurev-lawsocsci-120522-091626 · Cited by 48
Lu, J., Yan, Y., Huang, K., Yin, M., & Zhang, F. (2024) · Group Decision and Negotiation · doi:10.1007/s10726-024-09912-x · Cited by 44
“We can see that the two-step collaborative workflow design in human-AI collaboration demonstrates an overconfidence calibration effect for human, which is consistent with existing studies focusing on the decision-support setting (Buçinca et al. 2021; Gajos and Mamykina 2022; Green and Chen 2019).”
InfluentialDoes Explainable Artificial Intelligence Improve Human Decision-Making?
Alufaisan, Y., Marusich, L. R., Bakdash, J. Z., Zhou, Y., & Kantarcioglu, M. (2020) · doi:10.31234/osf.io/d4r9t · Cited by 30
Snow, T. (2021) · Data & Policy · doi:10.1017/dap.2020.25 · Cited by 31
“While there is significant research on the impact of technology on SLB discretion (Ellis, 2011; Hoybye-Mortensen, 2015; Busch and Henriksen, 2018), insufficient research considers how SLBs use ADTs to inform their decision processes (Green and Chen, 2019).”
BackgroundEnhancing human-AI collaboration: The case of colonoscopy
Introzzi, L., Zonca, J., Cabitza, F., Cherubini, P., & Reverberi, C. (2024) · Digestive and Liver Disease · doi:10.1016/j.dld.2023.10.018 · Cited by 28
Algorithmic Agency and Autonomy in Archaeological Practice
Huggett, J. (2021) · Open Archaeology · doi:10.1515/opar-2020-0136 · Cited by 48
“This can be characterized as the human - in - the - loop problem ( e.g. Broussard, 2018 , p. 177 ff ) or the algorithm - in - the - loop problem ( e.g. Green & Chen, 2019 ) , but the key question is where in the loop does the human or algorithm sit?”
BackgroundExplain, Edit, and Understand: Rethinking User Study Design for Evaluating Model Explanations
Arora, S., Pruthi, D., Sadeh, N., Cohen, W. W., Lipton, Z. C., & Neubig, G. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i5.20464 · Cited by 55
A Decision Theoretic Framework for Measuring AI Reliance
Guo, Z., Wu, Y., Hartline, J. D., & Hullman, J. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3658901 · Cited by 27
“However, many studies have shown that human-AI teams under-perform the AI alone in tasks where the AI’s accuracy is higher than humans [3, 4, 6, 14, 18, 20, 21, 28].”
Scholastic: Graphical Human-AI Collaboration for Inductive and Interpretive Text Analysis
Hong, M.-H., Marsh, L. A., Feuston, J. L., Ruppert, J., Brubaker, J. R., & Szafir, D. A. (2022) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3526113.3545681 · Cited by 49
“In this work, we introduce Scholastic , a visual analytics tool that instead explores a machine-in-the-loop [26, 27] approach to interpretive research, where scholars analyze text to generate codes and categories that also serve as goal-oriented user input to models that scaffold, rather than…”
MethodDorsch, J., & Deroy, O. (2024) · Philosophy & Technology · doi:10.1007/s13347-024-00752-w · Cited by 26
“The core program is that humans and AI ought to build upon each other’s strengths and compensate for their weaknesses (Bansal et al., 2021; cf. Green & Chen, 2019).”
Toward Sociotechnical AI: Mapping Vulnerabilities for Machine Learning in Context
Dobbe, R., & Wolters, A. (2024) · Minds and Machines · doi:10.1007/s11023-024-09668-y · Cited by 25
Reicherts, L., Zhang, Z. T., von Oswald, E., Liu, Y., Rogers, Y., & Hassib, M. E. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713295 · Cited by 62
“However, various challenges persist, including inappropriate reliance on AI [1, 10, 17, 19, 27, 49], deskilling [26, 57], or the difficulty of integrating AI recommendations into users’ decision-making process [4, 34, 65].”
Rapid Assisted Visual Search: Supporting Digital Pathologists with Imperfect AI
Lindvall, M., Lundström, C. F., & Löwgren, J. (2021) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3397481.3450681 · Cited by 43
“For instance, the presentation style of decision aids can affect human-AI ensemble performance [15].”
BackgroundFogliato, R., Chouldechova, A., & Lipton, Z. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479572 · Cited by 23
“In §??, we show that, in contrast with past work [38, 39], we did not find evidence of disparate-interactions in the uptake of RAI’s predictions.”
BackgroundMethodResultInfluentialHaque, M. R., Saxena, D., Weathington, K., Chudzik, J., & Guha, S. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642738 · Cited by 38
Designing meaningful human oversight in AI
Zhu, L., Lu, Q., Ding, M., Lee, S. U., & Wang, C. (2026) · AI and Ethics · doi:10.1007/s43681-026-01147-7 · Cited by 66
“Similarly, Green and Chen [11] observed that human decision-makers sometimes override accurate algorithmic recommendations, degrading overall decision quality.”
Hendrix, N., Lowry, K. P., Elmore, J. G., Lotter, W. E., Sorensen, A. G., Hsu, W. W. Y., Liao, G. J., Parsian, S., Kolb, S., Naeim, A., et al. (2022) · Journal of the American College of Radiology · doi:10.1016/j.jacr.2022.06.019 · Cited by 26
Towards providing explanations for robot motion planning
Brandão, M., Canal, G., Krivić, S., & Magazzeni, D. (2021) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48506.2021.9562003 · Cited by 37
“Finally, insights regarding user trust and reliance on algorithms [43] should be taken into account when designing explainable motion planners.”
BackgroundKaptchuk, G., Goldstein, D. G., Hargittai, E., Hofman, J. M., & Redmiles, E. M. (2021) · Digital Threats Research and Practice · doi:10.1145/3488307 · Cited by 23
Ground(less) Truth: A Causal Framework for Proxy Labels in Human-Algorithm Decision-Making
Guerdan, L., Coston, A., Wu, Z. S., & Holstein, K. (2023) · 2023 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3593013.3594036 · Cited by 28
ORES: Lowering Barriers with Participatory Machine Learning in WikipediaPreprint
Halfaker, A., & Geiger, R. S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.05189 · Cited by 22
“CSCW and HCI has also long studied human-in-the-loop systems [14, 37, 111], as well as explored implications of “algorithm-in-the-loop” socio-technical systems [44].”
Personalising AI Assistance Based on Overreliance Rate in AI-Assisted Decision Making
Swaroop, S., Buçinca, Z., Gajos, K. Z., & Doshi-Velez, F. (2025) · Proceedings of the 30th International Conference on Intelligent User Interfaces · doi:10.1145/3708359.3712128 · Cited by 20
“However, using a fxed type of AI assistance for all people does not lead to complementarity [3, 6, 10, 19, 36, 51].”
RetroLens: A Human-AI Collaborative System for Multi-step Retrosynthetic Route Planning
Shi, C., Hu, Y., Wang, S., Ma, S., Zheng, C., Ma, X., & Luo, Q. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581469 · Cited by 19
SenseMate: An Accessible and Beginner-Friendly Human-AI Platform for Qualitative Data Analysis
Overney, C., Saldías-Fuentes, B., Dimitrakopoulou, D., & Roy, D. K. (2024) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3640543.3645194 · Cited by 46
“In algorithm-in-the-loop systems, AI performs an assistive role by providing recommendations, while humans are the final decision makers [33].”
BackgroundBlack, E., Naidu, R. C. A., Ghani, R., Rodolfa, K. T., Ho, D. E., & Heidari, H. (2023) · ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO) · doi:10.1145/3617694.3623259 · Cited by 43
“Green and Chen [115, 116] show that including Mechanical Turk workers to aid algorithmic decisions consistently decreased accuracy relative to the algorithm’s performance alone, and that humans in the loop also exhibited racial bias when interacting with ML predictions.”
BackgroundMeasuring the effect of mental workload and explanations on appropriate AI reliance using EEG
Zhang, Z. T., Argın, S. K., Bilen, M. B., Urgun, D., Deniz, S. M., Liu, Y., & Hassib, M. E. (2024) · Behaviour and Information Technology · doi:10.1080/0144929x.2024.2431055 · Cited by 24
“However, people’s reliance on AI-generated decision recommendations is often inappropriate, with multiple studies observing either underreliance (rejection of correct/beneficial recommendations) [3,4,5] or overreliance (acceptance of incorrect/detrimental recommendations) [6,7,8].”
Human Delegation Behavior in Human-AI Collaboration: The Effect of Contextual Information
Spitzer, P., Holstein, J., Hemmer, P., Vössing, M., Kühl, N., Martin, D., & Satzger, G. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3710999 · Cited by 18
“Our study does not reach the benchmark of complementary team performance [7, 29, 35], where the collective capabilities of human-AI collaborations exceed the performance of each entity operating independently.”
Mok, L., Nanda, S., & Anderson, A. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610100 · Cited by 29
“We evaluate the model’s performance using the Brier score [44], for which we obtain a score of 0.17 on a 20% hold-out dataset 4 .”
BackgroundMethodInfluentialLee, M. H., Siewiorek, D. P., Smailagic, A., Bernardino, A., & Bermúdez i Badia, S. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511112 · Cited by 19
“It is necessary to further study how and what additional interactions can be provided to enable trustful usage by the domain experts and improve their decision making without introducing biases [21].”
BackgroundTo Err Is AI! Debugging as an Intervention to Facilitate Appropriate Reliance on AI Systems
He, G., Bharos, A., & Gadiraju, U. (2024) · ACM Conference on Hypertext and Social Media (HT) · doi:10.1145/3648188.3675130 · Cited by 24
“Following existing empirical study design of human-AI decision making [18, 19, 26], all participants in our study work on each trial case with two stages of decision making.”
Wang, X., Chen, L., & Yin, M. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/343 · Cited by 27
“Following prior work [Green and Chen, 2019b], we quantified the extent to which the AI model influenced participants’ decisions by comparing the bid prices made by participants who were given the AI 5 We chose $20 as the threshold to determine outlier bids since bid prices less than $20 consisted…”
BackgroundMethod“Reliance and automation for Human-AI collaborative data labeling conflict resolution,”
Brachman, M., Ashktorab, Z., Desmond, M., Duesterwald, E., Dugan, C., Joshi, N. N., Pan, Q., & Sharma, A. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555212 · Cited by 14
“Appropriately evaluating the performance of systems and calibrating reliance correctly is an essential and non-trivial task for humans [32].”
BackgroundA Taxonomy of Human and ML Strengths in Decision-Making to Investigate Human-ML Complementarity
Rastogi, C., Leqi, L., Holstein, K., & Heidari, H. (2023) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v11i1.27554 · Cited by 72
“Researchers have proposed and tested various hybrid human-ML designs, ranging from human-in-the-loop (Rus-sakovsky, Li, and Li 2015) to algorithm-in-the-loop (De-Arteaga, Fogliato, and Chouldechova 2020; Saxena et al. 2020; Brown et al. 2019; Green and Chen 2019) arrangements.”
MethodBeyond Recommendations: From Backward to Forward AI Support of Pilots' Decision-Making Process
Zhang, Z. T., Feger, S. S., Dullenkopf, L., Liao, R., Süsslin, L. E., Liu, Y., & Butz, A. M. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3687024 · Cited by 30
“This is the result of inappropriate reliance , including both overreliance (human relies on AI even when it is disadvantageous to do so) [2, 6, 8, 24, 32, 39, 43, 47, 56, 66] and underreliance (human rejects AI even when it would be beneficial to rely on it) [12–14 In practice, AI support often…”
BackgroundHussain, M. H., Iacovides, I., Lawton, T., Sharma, V., Porter, Z., Cunningham, A., Habli, I., Hickey, S., Jia, Y., Morgan, P., et al. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3660697 · Cited by 24
“Combining humans and AI in collaborative decision-making is thought to enhance decision quality in healthcare [26].”
BackgroundImproving Human Sequential Decision Making with Reinforcement Learning
Bastani, H., Bastani, O., & Sinchaisri, W. P. (2025) · Management Science · doi:10.1287/mnsc.2022.02455 · Cited by 67
“…predictions can improve human performance in one-shot decision-making—where the current decision does not affect future outcomes—e.g., bail decisions (Green and Chen 2019), visual question answering (Chandrasekaran et al. 2017, 2018), satellite image analysis (Kneusel and Mozer 2017), and…”
InfluentialLearning When to Advise Human Decision Makers
Noti, G., & Chen, Y. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/339 · Cited by 43
Self-citationThe Ethics of Data-Driven Personas
Salminen, J. O., Froneman, W., Jung, S., Chowdhury, S. A., & Jansen, B. J. J. (2020) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3334480.3382790 · Cited by 21
“The underlying assumption in our analysis is one of procedural justice [20], which requires that every step of the DDPC is accurate, fair, consistent, correctable, and ethical [12, 13].”
BackgroundFerreira, J. J., & Monteiro, M. D. S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.05460 · Cited by 38
“Therefore, there is an urgent need to expand our analyses of algorithmic decision-making aids beyond evaluating the models themselves to investigating the full socio-technical contexts in which people and algorithms interact [6] .”
InfluentialPreference-Driven Classification Measure
Kozák, J. T., Probierz, B., Kania, K., & Juszczuk, P. (2022) · Entropy · doi:10.3390/e24040531 · Cited by 21
“In practice, this applies to all issues except physical or technical ones, where only objective laws are in force [10].”
BackgroundTraining Towards Critical Use: Learning to Situate AI Predictions Relative to Human Knowledge
Kawakami, A., Guerdan, L., Cheng, Y., Glazko, K. S., Lee, M. L., Carter, S., Aréchiga, N., Zhu, H., & Holstein, K. (2023) · ACM Collective Intelligence Conference (CI) · doi:10.1145/3582269.3615595 · Cited by 25
“However, realizing these tools’ potential in practice is far from guaranteed [4, 15, 20, 24, 38].”
BackgroundFramework for human–XAI symbiosis: extended self from the dual-process theory perspective
Litvinova, Y., Mikalef, P., & Luo, X. R. (2024) · Journal of Business Analytics · doi:10.1080/2573234x.2024.2396366 · Cited by 20
“We presuppose human autonomy: This is the human who makes the final decision and takes over responsibility for it (Green & Chen, 2019a, 2019b; Jacobs et al., 2021; Lai & Tan, 2019; Schemmer et al., 2022).”
InfluentialUnfair! Perceptions of Fairness in Human-Robot Teams
Chang, M. L., Trafton, J. G., McCurry, J. M., & Thomaz, A. L. (2021) · IEEE RO-MAN · doi:10.1109/ro-man50785.2021.9515428 · Cited by 22
“Another study investigated how users collaborate with risk assessment tools, focusing on a tool that aides judges’ pretrail release decisions ([23], [24]).”
BackgroundNot Just a Preference: Reducing Biased Decision-making on Dating Websites
Ma, Z., & Gajos, K. Z. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517587 · Cited by 19
“In prior work, this intervention resulted in improved decision-making [14, 37, 63] presumably by making people less vulnerable to cognitive biases [14].”
BackgroundMethodInfluentialAlgorithm-in-the-Loop Decision Making
Green, B., & Chen, Y. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i09.7115 · Cited by 15
“Disparate Interactions Our first experiments studied algorithm-in-the-loop decision making in the context of pretrial release (Green and Chen 2019a).”
BackgroundMethodSelf-citationAlvero, A., Pal, J., & Moussavian, K. M. (2022) · Journal of Computational Social Science · doi:10.1007/s42001-022-00185-5 · Cited by 16
“Given the strength of the computational results, there is both opportunity but also risk [60] in the downstream application of these methods to actual systems.”
BackgroundYacoby, Y., Green, B., Griffin, C. L., Jr., & Doshi‐Velez, F. (2022) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v10i1.22001 · Cited by 33
Self-citationOn prediction-modelers and decision-makers: why fairness requires more than a fair prediction model
Scantamburlo, T., Baumann, J., & Heitz, C. (2024) · AI & Society · doi:10.1007/s00146-024-01886-3 · Cited by 28
“Further research pointed out that human decision-makers tend to deviate from algorithmic predictions (Stevenson and Doleac 2021) and struggle to assess algorithmic performance (Green and Chen 2019; Poursabzi-Sangdeh et al. 2021).”
BackgroundLi, C., Zhang, Z., Saugstad, M., Safranchik, E., Kulkarni, M., Huang, X., Patel, S., Iyer, V., Althoff, T., & Froehlich, J. E. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642089 · Cited by 21
“Because there is an anchoring bias [32] that occurs when presenting users with AI’s recommendations, one effective strategy is to ask the user to make a decision prior to seeing the AI’s recommendation [10].”
BackgroundCrowdsourcing Scholarly Discourse Annotations
Oelen, A., Stocker, M., & Auer, S. (2021) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3397481.3450685 · Cited by 19
“With this machine-in-the-loop approach [23] synergy is achieved between crowdsourcing and autonomous NLP extraction.”
MethodThe Influences of Task Design on Crowdsourced Judgement: A Case Study of Recidivism Risk Evaluation
Duan, X., Ho, C.-J., & Yin, M. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3485447.3512239 · Cited by 16
AI and Holistic Review: Informing Human Reading in College AdmissionsPreprint
Alvero, A., Arthurs, N., antonio, A. L., Domingue, B. W., Gebre-Medhin, B., Giebel, S., & Stevens, M. L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.09318 · Cited by 12
“The process of combining algorithms and humans in decision making processes also requires more understanding before data auditing in college admissions is feasible [23].”
MethodDoes Human-algorithm Feedback Loop Lead to Error Propagation? Evidence from Zillow’s Zestimate
Fu, R., Jin, G. Z., & Liu, M. (2022) · National Bureau of Economic Research · doi:10.3386/w29880 · Cited by 20
A comparative user study of human predictions in algorithm-supported recidivism risk assessment
Portela, M., Castillo, C., Tolan, S., Karimi-Haghighi, M., & Pueyo, A. A. (2024) · Artificial Intelligence and Law · doi:10.1007/s10506-024-09393-y · Cited by 23
“This is common practice and incentivizes conscientious completion of the survey (see, e.g., Green and Chen (2019), Cheng et al. (2019), Yu et al. (2020), Dressel and Farid (2018)).”
InfluentialSalimzadeh, S., & Gadiraju, U. (2024) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3627043.3659567 · Cited by 19
“Decision-making tasks can have varying levels of risk and stake across different domains, which can influence user behaviour, especially in high-stake situations where vulnerability and potential consequences are significant [6, 43, 53, 68].”
BackgroundDevOps Model Appproach for Monitoring Smart Energy Systems
Lévy, L.-N., Bosom, J., Guérard, G., Amor, S. B., Bui, M., & Tran, H. (2022) · Energies · doi:10.3390/en15155516 · Cited by 16
Grgić-Hlača, N., Castelluccia, C., & Gummadi, K. P. (2022) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v10i1.21989 · Cited by 26
Son, D., Lew, B., Choi, K., Baek, Y., Choi, S., Shin, B., Ha, S., & Chang, B. (2023) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3539597.3570439 · Cited by 17
“models and humanmoderators to reduce the risk of missing harmful content [15, 32].”
BackgroundBringing Friends into the Loop of Recommender Systems: An Exploratory Study
Piao, J., Zhang, G., Xu, F., Chen, Z., Zheng, Y., Gao, C., & Li, Y. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479583 · Cited by 12
“A trustworthy recommendation can not only simplify the decision-making process but also greatly enhance consumers’ satisfaction [21, 34].”
BackgroundInfluentialSun, T., Gao, Y., Khaladkar, S., Liu, S., Zhao, L., Kim, Y., & Hong, S.-S. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610187 · Cited by 16
“In Human-Computer Interaction (HCI) and Computer Supported Cooperative Work (CSCW), despite the rich body of research dedicated to better supporting ML engineers [22, 34], little effort has been made to design interfaces that can efficiently and effectively steer CNNs to mitigate contextual bias.”
BackgroundRobots as Managers? Organizational Justice, Power Asymmetries, and the Future of AI Supervision
Monod, E. (2025) · Management Research Quarterly · doi:10.63029/rqg5z883 · Cited by 13
AI algorithmic oversight: new frontiers in regulation
Busuioc, M. (2022) · Edward Elgar Publishing eBooks · doi:10.4337/9781839108990.00043 · Cited by 36
An Empirical Evaluation of Predicted Outcomes as Explanations in Human-AI Decision-Making
Jakubik, J., Schoeffer, J., Hoge, V., Vössing, M., & Kühl, N. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-23618-1_24 · Cited by 24
AI-Assisted Decision-Making: A Cognitive Modeling Approach to Infer Latent Reliance StrategiesPreprint
Tejeda, H., Kumar, A., Smyth, P., & Steyvers, M. (2022) · doi:10.31234/osf.io/q6yz4 · Cited by 10
Ashktorab, Z., Desmond, M., Pan, Q., Johnson, J., Brachman, M., Dugan, C., Danilevsky, M., & Geyer, W. (2025) · Proceedings of the 4th Annual Symposium on Human-Computer Interaction for Work · doi:10.1145/3729176.3729179 · Cited by 10
“Despite the remarkable accuracy of AI, human-AI teams sometimes fall short of expectations [8], particularly in areas where AI-alone teams outperform human-alone counterparts across various domains [6, 7, 19, 24, 55].”
Papantonis, I., & Belle, V. (2025) · Frontiers in Computer Science · doi:10.3389/fcomp.2025.1560448 · Cited by 14
Fanfani, M., Palesi, L. A. I., & Nesi, P. (2026) · Big Data and Cognitive Computing · doi:10.3390/bdcc10060186 · Cited by 19
“HCAI systems are designed to collaborate with users (human in the loop) by supporting instead of replacing them to improve the quality and reduce the efforts of human decision-making activities [18].”
Ferguson, S. A., Aoyagui, P. A., & Kuzminykh, A. (2023) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3544549.3585727 · Cited by 14
“Based on evidence [15, 17] that better decision-making performance can be achieved if the human counterpart is asked to make an initial decision before seeing an AI recommendation, we first asked participants to read a subtle sexism scenario and decide whether it was sexist, not sexist, or if they couldn’t decide.”
Background“ObjectivAIze: Measuring performance and biases in augmented business decision systems,”
Baudel, T., Verbockhaven, M., Cousergue, V., Roy, G., & Laarach, R. (2021) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-85613-7_22 · Cited by 10
Decision-Focused Summarization
Hsu, C.-C., & Tan, C. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.10 · Cited by 18
“It is useful to highlight our motivation in support decision making in challenging tasks towards effective human-AI collaboration (Green and Chen, 2019; Lai et al., 2020; Lai and Tan, 2019).”
BackgroundSafety Issues in Human-Machine Collaboration and Possible Countermeasures
Liang Ma, A., & Wang, C. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-05890-5_21 · Cited by 10
One Size Does Not Fit All: Mechanisms of Employees’ Acceptance of Robotic Lower-Level Managers
Wolf, F. D., & Stock‐Homburg, R. (2025) · Group & Organization Management · doi:10.1177/10596011251313568 · Cited by 11
“Finally, potential unfairness in AI and robotic decision-making raises concerns about the ability of robots to ful fi ll the important relational dimension of management, such as appropriately contextualizing work (Bankins et al., 2022; Green & Chen, 2019; Lee, 2018).”
The Influence of Curiosity Traits and On-Demand Explanations in AI-Assisted Decision-Making
Cau, F. M., & Spano, L. D. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712165 · Cited by 15
“Previous studies [27, 36, 37] have shown that the concurrent paradigm, where system advice is displayed immediately to the human decision-maker, can lead users to over-rely on its assistance.”
Cabitza, F., Campagner, A., Fregosi, C., Cameli, M., Gallazzi, E., Sconfienza, L. M., & Tontini, G. E. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757601 · Cited by 13
When Do Citizens Resist the Use of AI Algorithms in Public Policy? Theory and Evidence
Raviv, S. (2025) · The Journal of Politics · doi:10.1086/736362 · Cited by 12
Leveraging Human-AI Collaboration in Crowd-Powered Source Search: A Preliminary Study
Zhao, Y. B., Zhu, Z., Chen, B., & Qiu, S. (2023) · Journal of Social Computing · doi:10.23919/jsc.2023.0002 · Cited by 9
Ma, S., Chen, Q., Wang, X., Zheng, C., Peng, Z., Yin, M., & Ma, X. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4772689 · Cited by 8
Trustworthy human computation: a survey
Kashima, H., Oyama, S., Arai, H., & Mori, J. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10974-1 · Cited by 11
“Green et al. [74] propose three behavioural principles of algorithm-in-the-loop: “people using the algorithm should make more accurate predictions than they could without the algorithm” (accuracy), “people should accurately evaluate their own and the algorithm’s performance and should calibrate…”
Coalesce: An Accessible Mixed-Initiative System for Designing Community-Centric Questionnaires
Overney, C., Kessler, D. T., Fulay, S. P., Jasim, M., & Roy, D. K. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712118 · Cited by 14
“In such systems, AI provides recommendations, while humans are the final decision-makers [35].”
Salminen, J. O., Jung, S., Chowdhury, S. A., & Jansen, B. J. J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-50334-5_6 · Cited by 13
“While it is not clear whether this phenomenon is fundamental to HCI or the result of factors like interface design or training [67], explainability inarguably poses a critical design challenge for DDPs. Transparency has been suggested as a solution to trust concerns regarding data use and…”
BackgroundInfluentialIs Overreliance on AI Provoked by Study Design?
Zhang, Z. T., Tong, S., Liu, Y., & Butz, A. M. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-42286-7_3 · Cited by 11
“Artificial intelligence (AI) is increasingly used to support human decisions, often in high-stakes domains such as healthcare, finance, or criminal justice (e.g. [2,6,18]).”
BackgroundKang, B., & Wu, S. (2023) · Journal of Experimental Criminology · doi:10.1007/s11292-022-09512-2 · Cited by 6
“MTurk samples have been widely used in recent experimental studies examining public opinion on risk assessment (Dressel & Farid, 2018; Green & Chen, 2019; Saxena et al., 2019; Srivastava et al., 2019), as they have several advantages over other conventional convenient samples.”
Ethics and Safety of Human-Machine Teaming
Assaad, Z., & Burken, C. B.-V. (2023) · International Symposium on Trustworthy Autonomous Systems (TAS) · doi:10.1145/3597512.3600205 · Cited by 9
Graells-Garrido, E., Peña-Araya, V., & Bravo, L. (2020) · Sustainability · doi:10.3390/su12156001 · Cited by 7
“Such collaborations are needed, as a data-driven focus is not enough to evaluate the quality of a solution [8].”
BackgroundFair Governance with Humans and MachinesPreprint
Hermstrüwer, Y., & Langenbach, P. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4118650 · Cited by 7
Rosenbacke, R. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4773350 · Cited by 9
Zhang, Z. T., Buchner, F., Liu, Y., & Butz, A. (2024) · Proceedings of Mensch und Computer 2024 · doi:10.1145/3670653.3670660 · Cited by 6
“…the goal is to improve human decision-making in difficult high-stakes decisions, such as medical diagnosis [4, 17], creditworthiness assessment [16], or recidivism prediction [24, 38], where the call for explainability is particularly prominent and at the same time, overreliance is especially…”
Vicente, L., Matute, H., Fregosi, C., & Cabitza, F. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2025.103474 · Cited by 18
Berretta, S., Tausch, A., Bülow, F., Kuhlenkötter, B., Topp, M., Els, C., Peifer, C., & Kluge, A. (2025) · Applied Ergonomics · doi:10.1016/j.apergo.2025.104669 · Cited by 9
Passi, S., Dhanorkar, S., & Vorvoreanu, M. (2025) · Springer eBooks · doi:10.1007/978-981-95-3658-0_98 · Cited by 15
Evaluating the Utility of Conformal Prediction Sets for AI-Advised Image Labeling
Zhang, D., Chatzimparmpas, A., Kamali, N., & Hullman, J. R. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642446 · Cited by 27
“Consistent with prior work on AI-advised decision-making (e.g., [7, 10, 20, 34]), our results suggest that when a model is well-calibrated and more accurate than humans alone, users with access to its predictions can perform better than without the model, but not as well as the model alone for…”
Accelerated experimental design using a human–AI teaming framework
Arun Kumar, A., V, Shilton, A., Gupta, S., Ryan, S., Abdolshah, M., Lê, H., Rana, S., Berk, J., Rashid, M., & Venkatesh, S. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113138 · Cited by 6
Three Directions for the Design of Human-Centered Machine Translation
Liebling, D. J., Heller, K., Mitchell, M., Díaz, M., Lahav, M., Salehi, N., Robertson, S., Bengio, S., Gebru, T., & Deng, W. (2021) · Cited by 22
Welfarist Moral Grounding for Transparent AI
Narayanan, D. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3593977 · Cited by 8
“studies have suggested that transparency can lead decision-makers to ‘overtrust’ their AI systems, often because they take the mere fact that a system is transparent (rather than the information made transparent) as reason to trust it [11, 34, 48].”
BackgroundYou Are What You Tweet: Profiling Users by Past Tweets to Improve Hate Speech Detection
Chaudhry, P., & Lease, M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-030-96960-8_13 · Cited by 6
Pesch, P. J., Димитрова, Д., & Boehm, F. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-07315-1_4 · Cited by 13
Harnessing Explanations to Bridge AI and HumansPreprint
Lai, V., Carton, S., & Tan, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.07370 · Cited by 7
“Some experiments in this vein have included predicting apartment prices [13], detecting deceptive online reviews [11], assessing social media toxicity [3], performing various artificial tasks [9], and recidivism prediction [5].”
BackgroundEXPRESS: Causal Alignment: Augmenting Language Models with A/B Tests
Angelopoulos, P., Lee, K., & Misra, S. (2026) · Journal of Marketing Research · doi:10.2139/ssrn.4781850 · Cited by 8
Metadata
- Authors
- Green, B., & Chen, Y.
- Affiliations
- Harvard University (US)
- Venue
- Proceedings of the ACM on Human-Computer Interaction
- Published
- 2019-11-07
- Volume
- 3
- Issue
- CSCW
- Pages
- 1-24
- ISSN
- 2573-0142 (ISSN-L)
- OpenAlex
- W2979893369
- Cited by
- 392
- References
- 64
- Citations by year
- 2026: 94 · 2025: 61 · 2024: 62 · 2023: 54 · 2022: 58 · 2021: 36 · 2020: 22 · 2019: 5
- FWCI
- 52.66
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Proceedings of the ACM on Human-Computer Interaction (publishedVersion)
- Funders
- National Science Foundation (DGE1745303)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Keywords
- human-algorithm interaction, risk assessment, pretrial release, lending, interpretability, socio-technical context
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.