The Principles and Limits of Algorithm-in-the-Loop Decision Making
The DOI matches its Crossref record; 61 of the 64 references it cites were found in a registry or an index.
Abstract
The rise of machine learning has fundamentally altered decision making: rather than being made solely by people, many important decisions are now made through an "algorithm-in-the-loop'' process where machine learning models inform people. Yet insufficient research has considered how the interactions between people and models actually influence human decisions. Society lacks both clear normative principles regarding how people should collaborate with algorithms as well as robust empirical evidence about how people do collaborate with algorithms. Given research suggesting that people struggle to interpret machine learning models and to incorporate them into their decisions---sometimes leading these models to produce unexpected outcomes---it is essential to consider how different ways of presenting models and structuring human-algorithm interactions affect the quality and type of decisions made. This paper contributes to such research in two ways. First, we posited three principles as essential to ethical and responsible algorithm-in-the-loop decision making. Second, through a controlled experimental study on Amazon Mechanical Turk, we evaluated whether people satisfy these principles when making predictions with the aid of a risk assessment. We studied human predictions in two contexts (pretrial release and financial lending) and under several conditions for risk assessment presentation and structure. Although these conditions did influence participant behaviors and in some cases improved performance, only one desideratum was consistently satisfied. Under all conditions, our study participants 1) were unable to effectively evaluate the accuracy of their own or the risk assessment's predictions, 2) did not calibrate their reliance on the risk assessment based on the risk assessment's performance, and 3) exhibited bias in their interactions with the risk assessment. These results highlight the urgent need to expand our analyses of algorithmic decision making aids beyond evaluating the models themselves to investigating the full sociotechnical contexts in which people and algorithms interact.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“there has been increasing recent research efforts [8, 15, 22, 41, 58, 65] that highlight the importance of involving stakeholders to understand their practices and needs [15, 41, 65] and socio-environmental factors [8] for the design and evaluation of a system.”
From Understanding the Effect of Counterfactual Explanations on Trust and Reliance on AI for Human-AI Collaborative Clinical Decision Making (2023) · cites this paper as [22]
In the citing paper
“However, many empirical studies on human-AI decision-making did not observe complementary performance [6,16,37,52,58,61,68,82,84,87].”
From Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations (2023) · cites this paper as [37]
In the citing paper
“We note that these approaches correspond to the Update and Feedback methods of Green & Chen [27], which were effective, albeit not in the complementary zone.”
From Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance (2021) · cites this paper as [27]
In the citing paper
“More interestingly, communication failure can also stem from a deeper inability to understand how AI makes decisions and how likely AI may err in specific conditions [61] .”
From Enhancing human-AI collaboration: The case of colonoscopy (2024) · cites this paper as [61]
In the citing paper
“However, many studies have shown that human-AI teams under-perform the AI alone in tasks [3, 4, 6, 14, 18, 20, 21, 30].”
From A Decision Theoretic Framework for Measuring AI Reliance (2024) · cites this paper as [14]
7 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Greedy function approximation: A gradient boosting machine
Friedman, J. H. (2001) · The Annals of Statistics · doi:10.1214/aos/1013203451 · Cited by 30,731
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605
Strictly Proper Scoring Rules, Prediction, and Estimation
Gneiting, T., & Raftery, A. E. (2007) · Journal of the American Statistical Association · doi:10.1198/016214506000001437 · Cited by 5,793
Algorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232
What Should Be Done with Equity Theory?
Leventhal, G. S. (1980) · Social Exchange · doi:10.1007/978-1-4613-3087-5_2 · Cited by 3,010 · cited 2×
“For example, the principle of procedural justice requires that decisions be (among other things) accurate, fair, consistent, correctable, and ethical [55].”
BackgroundAlgorithm appreciation: People prefer algorithmic to human judgment
Logg, J. M., Minson, J. A., & Moore, D. A. (2019) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2018.12.005 · Cited by 2,061
Principles of mixed-initiative user interfaces
Horvitz, E. (1999) · Proceedings of the SIGCHI conference on Human factors in computing systems the CHI is the limit - CHI '99 · doi:10.1145/302979.303030 · Cited by 1,490
Lee, M. K. (2018) · Big Data & Society · doi:10.1177/2053951718756684 · Cited by 1,338
Human Decisions and Machine Predictions
Kleinberg, J., Lakkaraju, H., Leskovec, J., Ludwig, J., & Mullainathan, S. (2018) · The Quarterly Journal of Economics · doi:10.1093/qje/qjx032 · Cited by 1,040
Research works printed but not linked: 12
- Alex Albright . 2019. If You Give a Judge a Risk Score: Evidence from Kentucky Bail Decisions . The John M. Olin Center for Law , Economics , and Business Fellows' Discussion Paper Series 85 ( 2019 ). Alex Albright. 2019. If You Give a Judge a Risk Score: Evidence from Kentucky Bail Decisions. The John M. Olin Center for Law, Economics, and Business Fellows' Discussion Paper Series 85 (2019).
- Julia Angwin , Jeff Larson , Surya Mattu , and Lauren Kirchner . 2016. Machine Bias. ProPublica ( 2016 ). https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2016. Machine Bias. ProPublica (2016). https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing
- Laura Bliss . 2018. Former Uber Backup Driver: 'We Saw This Coming'. CityLab ( 2018 ). https://www.citylab.com/transportation/2018/03/former-uber-backup-driver-we-saw-this-coming/556427/ Laura Bliss. 2018. Former Uber Backup Driver: 'We Saw This Coming'. CityLab (2018). https://www.citylab.com/transportation/2018/03/former-uber-backup-driver-we-saw-this-coming/556427/
- Darwin Bond-Graham and Ali Winston . 2013. All Tomorrow's Crimes: The Future of Policing Looks a Lot Like Good Branding. SF Weekly ( 2013 ). http://archives.sfweekly.com/sanfrancisco/all-tomorrows-crimes-the-future-of-policing-looks-a-lot-like-good-branding/Content?oid=2827968 Darwin Bond-Graham and Ali Winston. 2013. All Tomorrow's Crimes: The Future of Policing Looks a Lot Like Good Branding. SF Weekly (2013). http://archives.sfweekly.com/sanfrancisco/all-tomorrows-crimes-the-future-of-policing-looks-a-lot-like-good-branding/Content?oid=2827968
- Pamela M. Casey , Roger K. Warren , and Jennifer K . Elek . 2011 . Using Offender Risk and Needs Assessment Information at Sentencing : Guidance for Courts from a National Working Group .National Center for State Courts . Pamela M. Casey, Roger K. Warren, and Jennifer K. Elek. 2011. Using Offender Risk and Needs Assessment Information at Sentencing: Guidance for Courts from a National Working Group .National Center for State Courts.
- Thomas H. Cohen , Bailey Pendergast , and Scott W. VanBenschoten . 2016 . Examining overrides of risk classifications for offenders on federal supervision . Federal Probation , Vol. 80 , 1 (2016), 12 . Thomas H. Cohen, Bailey Pendergast, and Scott W. VanBenschoten. 2016. Examining overrides of risk classifications for offenders on federal supervision. Federal Probation, Vol. 80, 1 (2016), 12.
- Mary L. Cummings . 2006. Automation and Accountability in Decision Support System Interface Design. Journal of Technology Studies ( 2006 ). Mary L. Cummings. 2006. Automation and Accountability in Decision Support System Interface Design. Journal of Technology Studies (2006).
- William Dieterich , Christina Mendoza , and Tim Brennan . 2016. COMPAS Risk Scales: Demonstrating Accuracy Equity and Predictive Parity . Northpointe Inc . ( 2016 ). William Dieterich, Christina Mendoza, and Tim Brennan. 2016. COMPAS Risk Scales: Demonstrating Accuracy Equity and Predictive Parity. Northpointe Inc. (2016).
- Jeff Larson , Surya Mattu , Lauren Kirchner , and Julia Angwin . 2016. How We Analyzed the COMPAS Recidivism Algorithm. ProPublica ( 2016 ). https://www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm Jeff Larson, Surya Mattu, Lauren Kirchner, and Julia Angwin. 2016. How We Analyzed the COMPAS Recidivism Algorithm. ProPublica (2016). https://www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm
- 10.1518/hfes.46.1.50.30392 · doi:10.1518/hfes.46.1.50.30392 · Paper page
- Frank Main . 2016. Cook County judges not following bail recommendations: study. Chicago Sun-Times ( 2016 ). https://chicago.suntimes.com/chicago-news/cook-county-judges-not-following-bail-recommendations-study-find/ Frank Main. 2016. Cook County judges not following bail recommendations: study. Chicago Sun-Times (2016). https://chicago.suntimes.com/chicago-news/cook-county-judges-not-following-bail-recommendations-study-find/
- Megan T. Stevenson and Jennifer L . Doleac . 2018 . The Roadblock to Reform. The American Constitution Society ( 2018). https://www.acslaw.org/wp-content/uploads/2018/11/RoadblockToReformReport.pdf Megan T. Stevenson and Jennifer L. Doleac. 2018. The Roadblock to Reform. The American Constitution Society (2018). https://www.acslaw.org/wp-content/uploads/2018/11/RoadblockToReformReport.pdf
Cited by
Items 201–300 of 393
Raees, M., Khan, V.-J., Lykourentzou, I., & Papangelis, K. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791467 · Cited by 5
“…several approaches for studying human-AI reliance, mainly categorized under enhancing the AI systems (explanations, performance, uncertainty) [5, 54, 118, 127], understanding users (biases, expertise, self-assessment) [48, 108, 126, 144, 151], and designing interaction methods [9, 10, 15, 60].”
InfluentialThe Importance of Modeling Data Missingness in Algorithmic Fairness: A Causal Perspective
Goel, N., Amayuelas, A., Deshpande, A., & Sharma, A. K. (2021) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v35i9.16926 · Cited by 13
Trust Calibration for Joint Human/AI Decision-Making in Dynamic and Uncertain Contexts
Marusich, L. R., Files, B. T., Bancilhon, M., Rawal, J. C., & Raglin, A. J. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-93412-4_6 · Cited by 16
“Some of the cognitive forcing strategies used in the human-AI decision-making literature include: • Prompting the user to make a decision before seeing the AI’s recommendation [14, 6] • Delaying the presentation of the AI recommendation [38, 6] • Displaying the AI recommendation on demand [12, 6].”
Exploring the impact of explainable AI and cognitive capabilities on users’ decisions
Cau, F. M., & Spano, L. D. (2025) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-025-09438-0 · Cited by 20
“Also, the loan prediction scenario has been used in other human–AI team studies (Binns et al. 2018; Green and Chen 2019b; Gomez et al. 2020; Chromik et al. 2021; van Berkel et al. 2021; He et al. 2023a; Esfahani et al. 2024a; He et al. 2025), reinforcing its validity and suitability for…”
What should AI explain to you? Visions of older adults regarding explainable AI
van Leersum, C. M. (2026) · The Gerontologist · doi:10.1093/geront/gnaf308 · Cited by 6
Reinmund, T., Kunze, L., & Jirotka, M. D. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642247 · Cited by 9
“Researchers delineate the design practices of data scientists and their collaborators [e.g., 61, 62, 96, 105, 109]; others instead look towards the experience of use, focusing on instances of interaction with ML systems [e.g., 7, 30, 40, 47, 51, 97].”
Faas, C., Uth, R., Sterz, S., Langer, M., & Feit, A. M. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812265 · Cited by 26
“AI-based systems can support users in complex or time-sensitive scenarios by providing for example decision recommendations [5, 9, 29, 70, 95].”
Beyond aversion – principles of appropriate algorithmic decision-making in human resource management
Strohmeier, S., Becker, M., & Scheer-Weller, E. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.128954 · Cited by 9
Resilience Through Appropriation: Pilots’ View on Complex Decision Support
Zhang, Z. T., Storath, C., Liu, Y., & Butz, A. M. (2023) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3581641.3584056 · Cited by 13
“Along with the impressive advances in AI, interest in intelligent decision support tools (DSTs) has surged in recent years, often in high-stakes applications like medical diagnosis [5, 33, 48], creditworthiness assessment [11, 25], university admission [9, 13], or recidivism prediction [25, 44, 58].”
BackgroundFairness as adequacy: a sociotechnical view on model evaluation in machine learning
Grote, T. (2023) · AI and Ethics · doi:10.1007/s43681-023-00280-x · Cited by 8
“As it stands, the existing literature is still dominated by one-shot online studies with laypeople (e.g., [32, 33]) or the analysis of retrospective data [46].”
A Framework for Effective AI Recommendations in Cyber-Physical-Human Systems
Dave, A., Bang, H., & Malikopoulos, A. A. (2024) · IEEE Control Systems Letters · doi:10.1109/lcsys.2024.3410145 · Cited by 9
“In such applications, the human’s actual decision may depart from a recommended decision for various reasons [5], including: (1) different interpretations of the system’s state to the AI platform; (2) different objectives than those designated for the AI; or (3) high confidence in their inherent…”
BackgroundLanger, M. (2023) · Psychologische Rundschau · doi:10.1026/0033-3042/a000626 · Cited by 5
Ex(plainable) Machina: how social-implicit XAI affects complex human-robot teaming tasks
Matarese, M., Cocchella, F., Rea, F., & Sciutti, A. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10160839 · Cited by 9
“We can easily transpose these to our future domestic robots; hence, we still need a profound discussion about the ethical implication of using such technologies in everyday life [18].”
BackgroundFrom Trust to Appropriate Reliance: Measurement Constructs in Human-AI Decision-Making
Raees, M., & Papangelis, K. (2026) · arXiv · doi:10.48550/arxiv.2604.23896 · Cited by 5
“To this end, research has explored various means (e.g., explanations [1], interaction [63]) to enhance transparency, accountability, and proper usage of AI systems; yet humans often over-or under-rely on AI, leading to inferior performance on decision-making tasks [4, 28].”
Policy-makers’ Processing of AI Algorithmic Advice: ‘Automation Bias’ versus Selective Adherence
Alon Barkat, S., & Busuioc, M. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3794660 · Cited by 6
Ogawa, N., Okafuji, Y., Hatada, Y., & Baba, J. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757686 · Cited by 10
“Interaction factors include cognitive load [84], risk perception [19], and other factors that change over time throughout the interaction with the system [18, 28] such as first impressions [68], updates of expectations [50], experience of error [54], and perceived accuracy [29].”
Mondrus, O. V., & SimanTov‐Nachlieli, I. (2025) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2025.100857 · Cited by 5
Kapoor, A., Kapoor, S., & Sindwani, R. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-2597-2.ch003 · Cited by 5
Trust to Reliance: Measurement Constructs for Human-AI Appropriate Reliance
Raees, M., & Papagelis, K. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798835 · Cited by 8
“These constructs are motivated by a common notion of humans’ ability to rely on AI when it provides correct advice instead of when it provides incorrect advice [2, 11].”
Ng, J. Y. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1745928 · Cited by 7
“One such concern is automation bias, a phenomenon in which individuals place excessive trust in outputs generated by automated systems (Bansal et al., 2021; Green and Chen, 2019).”
Norberg, P. A., & Horne, D. R. (2024) · Journal of Consumer Affairs · doi:10.1111/joca.12586 · Cited by 4
The Facets and Consequences of Uncertainty in Human–AI Interaction
Magni, F., Yang, H. H. J., & Gong, Y. (2024) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780197501061.013.22 · Cited by 10
Imperfections of XAI: Phenomena Influencing AI-Assisted Decision-Making
Spitzer, P., Morrison, K., Turri, V., Feng, M., Perer, A., & Kühl, N. (2025) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3750052 · Cited by 17
“The structure of phases one and two is corroborated with previous work [34].”
Towards Model-informed Precision Dosing with Expert-in-the-loop Machine Learning
Kang, Y., Chiu, Y., Lin, M., Su, F.-Y., & Huang, S.-T. (2021) · IEEE International Conference on Information Reuse and Integration (IRI) · doi:10.1109/iri51335.2021.00053 · Cited by 5
A Review of Gender Bias Mitigation in Credit Scoring Models
Corrales-Barquero, R., Marín‐Raventós, G., & Barrantes, E. G. (2021) · doi:10.1109/ee-rds53766.2021.9708589 · Cited by 11
Ulusan, A., Narayan, U., Snodgrass, S., Ergün, Ö., & Harteveld, C. (2022) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3490099.3511163 · Cited by 5
“As such, our work contributes to the growing amount of experimental evidence for putting the human in the algorithmic loop [35, 39, 72] and contrasts with recent work that shows that human-machine teams perform worse than the machine alone [28, 29, 48, 60].”
ResultSchönau, A. (2022) · AI and Ethics · doi:10.1007/s43681-022-00158-4 · Cited by 7
“One strategy to address those issues now is to identify sociotechnical contexts in which people and algorithms interact [14].”
BackgroundLearning Representations by Humans, for HumansPreprint
Hilgard, S., Rosenfeld, N., Banaji, M. R., Cao, J., & Parkes, D. C. (2019) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/4nvts · Cited by 8
Human-Algorithm Interactions: Evidence from Zillow.comPreprint
Fu, R., Jin, G. Z., & Liu, M. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4061116 · Cited by 11
Proceedings of The ACM Collective Intelligence Conference
(2023) · doi:10.1145/3582269 · Cited by 4
Wilson, B., Natali, C., Roach, M., Scott, D., Rahat, A., Rawlinson, D. R., & Cabitza, F. (2025) · Computer Supported Cooperative Work (CSCW) · doi:10.1007/s10606-025-09514-4 · Cited by 12
Erlei, A., Cau, F. M., Georgiev, R., Kumar, S. C., Bizer, K., & Gadiraju, U. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791420 · Cited by 7
“For tabular data, we selected the loan prediction problem dataset 10 as a testbed, given its utility in several previous works in AI-assisted decisions [19, 39, 48, 57, 59, 61, 63, 141].”
Dai, C., Yang, M., & Lee, S. J. (2026) · Systems · doi:10.3390/systems14060639 · Cited by 6
Algorithmic Status Inequality: An Integrative Perspective on AI‐Driven Social Stratification
Wu, J. (2026) · Journal of Management Studies · doi:10.1111/joms.70134 · Cited by 6
Requirements and Concepts for Interactive Media Retrieval User Interfaces
Oliveira, V. A. D. J., Rottermanner, G., Größbacher, S., Boucher, M., & Judmaier, P. (2022) · Nordic Conference on Human-Computer Interaction · doi:10.1145/3546155.3546701 · Cited by 3
“Research related to human-algorithm interactions shows that people struggle to interpret and effectively use algorithms when making decisions [16].”
BackgroundLost in the Loop: Who Is the ‘Human’ of the Human in the Loop?
Goldenfein, J. (2026) · Cambridge University Press eBooks · doi:10.1017/9781009425308.006 · Cited by 4
Human-AI Teaming: Following the IMOI Framework
Kleanthous, S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-60611-3_27 · Cited by 6
Exploring the Effects of User Trust in Generative AI on Decision-Making in Semi-structured Problem
Kamada, K., Takahashi, H., Wang, T.-Y., & Yuizono, T. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-93965-5_17 · Cited by 5
An Experimental Method for Studying Complex Choices
Gurney, N., King, T., & Miller, J. H. (2022) · Communications in computer and information science · doi:10.1007/978-3-031-19679-9_6 · Cited by 5
Milella, F., Natali, C., Scantamburlo, T., Campagner, A., & Cabitza, F. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-42283-6_19 · Cited by 6
Hooked on artificial agents: a systems thinking perspective
Ðula, I., Berberena, T., Keplinger, K., & Wirzberger, M. (2023) · Frontiers in Behavioral Economics · doi:10.3389/frbhe.2023.1223281 · Cited by 5
Human-Centered Evaluation of Explanations in AI-Assisted Decision-Making
Wang, X. (2024) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3640544.3645239 · Cited by 4
“…result in more unfair AI-assisted human decisions across racial groups, partly because they lead humans DMs to engage in “disparate interactions” [1] with the AI model, i.e., humans DMs respond to the AI model’s predictions in a biased way that disproportionately result in lower bids to Black…”
BackgroundAlgorithmic Decision Making in Education: Challenges and Opportunities
Levantis, N., & Sgora, A. (2024) · IEEE Global Engineering Education Conference (EDUCON) · doi:10.1109/educon60312.2024.10578645 · Cited by 11
“This approach also enables participatory practices [41] that should involve both academic staff and students, addressing design versatility concerns by covering all possible perspectives, but also requires educators to be actively involved in the decision-making process themselves as ”co-creators”…”
BackgroundMethodInfluentialApproaching Fairness in Digital Governance: A Multi-Dimensional, Multi-Layered Framework
Yousefi, Y. (2024) · International Conference on Theory and Practice of Electronic Governance · doi:10.1145/3680127.3680197 · Cited by 5
“All the existing definitions of the notion of fairness in algorithmic decisions -regardless of their differences-revolve around the way protected groups or categories are treated or considered [4, 14].”
Kelso, L. E., Dobolyi, D. G., Grabman, J. H., & Dodson, C. S. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2503971122 · Cited by 7
Crowd-Powered Source Searching in Complex Environments
Zhao, Y. B., Zhu, Z., Chen, B., & Qiu, S. (2023) · Communications in computer and information science · doi:10.1007/978-981-99-2385-4_15 · Cited by 3
“Previous work introduced this concept, and provided principles for human-AI decision-making and risk analysis [18, 19].”
BackgroundFace-work for Human-Agent Joint Decision-MakingPreprint
Jeong, J., & Hoffman, G. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.01969 · Cited by 3
“Achieving productive decision-making with humans and algorithms can be challenging (Green and Chen 2019) and immediate performance gain is not always the best goal to strive for.”
Things Machine Learning Models Know That They Don’t Know
Ruggieri, S., & Pugnana, A. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i27.35094 · Cited by 22
“For instance, Green and Chen (2019) investigated with a controlled experiment the effectiveness of using an ML-based risk assessment model as a decision aid.”
Behavior Matters: An Alternative Perspective on Promoting Responsible Data Science
Zhang, D., Patil, A., Shoda, Y., Battle, L., & Wall, E. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3710932 · Cited by 7
Vejandla, S., Ray, A., Sikstrom, L., Ratto, M., Zaheer, J., & Maslej, M. M. (2025) · Scientific Reports · doi:10.1038/s41598-025-30506-3 · Cited by 4
Human at the Center: A Framework for Human‐Driven AI Development
Hu, D., Acosta-Navas, D., Gaube, S., Mozannar, H., Taylor, M. E., Dvijotham, K., & Bondi, E. (2025) · AI Magazine · doi:10.1002/aaai.70043 · Cited by 4
Machine learning integration for smarter, more adaptive systems in large-scale 3D printing
Barbhuiya, S., Qazi, N., Das, B. B., & Katare, V. D. (2026) · Proceedings of the Institution of Civil Engineers - Structures and Buildings · doi:10.1680/jstbu.25.00139 · Cited by 2
Mascareño, J., Wörtler, B., Przegalińska, A., & Ciechanowski, L. (2026) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2026.101054 · Cited by 5
Pal, O., Duddu, V., Goyal, A., Goel, D., & Saha, K. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3799386 · Cited by 6
“For example, in legal and organizational contexts, reliance varies more with social accountability pressures than measured trust, and social proof often supersedes performance-based trust [18, 39, 54].”
AI, Evidentiary Authority, and the Right to a Fair Trial in Criminal Proceedings
Kocagul, H., & Kartal, M. (2026) · Laws · doi:10.3390/laws15030058 · Cited by 4
“…research conducted across different contexts points to a consistent finding: decision-makers tend to accept algorithmic recommendations rather than reject them, even when the recommendation is clearly erroneous (Green and Chen 2019; Završnik 2020, p. 574; Skitka et al. 1999; Goddard et al. 2012).”
InfluentialThe Explanatory Gap in Algorithmic News Curation
Heuer, H. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87031-7_1 · Cited by 4
“Green et al. found that insufficient research has considered how the interactions between people and models influence human decisions [19].”
BackgroundDesigning Accountable Health Care Algorithms: Lessons from Covid-19 Contact Tracing
Lu, L., D’Agostino, A., Rudman, S. L., Ouyang, D., & Ho, D. E. (2022) · NEJM Catalyst · doi:10.1056/cat.21.0382 · Cited by 3
“12 , 13 There are many ways in which these technical measures may fail.”
BackgroundExplainable AI Assisted Decision-Making and Human Behaviour
Suffian, M. (2024) · Studies in computational intelligence · doi:10.1007/978-3-031-53717-2_36 · Cited by 2
(De)Noise: Moderating the Inconsistency Between Human Decision-Makers
Grgić-Hlača, N., Ali, J., Gummadi, K. P., & Vaughan, J. W. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3686987 · Cited by 5
“In settings where decisions have important societal implications, prior work has also studied the effects of algorithmic assistance on the fairness of people’s decisions [40, 41, 102].”
BackgroundAutomating Misrecognition: The Case of Disability
Scully, J. L., van Toorn, G., & Gendera, S. (2025) · Journal of Bioethical Inquiry · doi:10.1007/s11673-025-10462-3 · Cited by 6
“This [technology and AI] Other authors have also cautioned against the assumption that having a human in the loop necessarily answers concerns about autonomous decision-making’s lack of empathy or “humanity” (Green and Chen 2019).”
Mei, P., Cannon, R., Everett, J. A. C., Liu, P., & Awad, E. (2025) · Transportation Research Interdisciplinary Perspectives · doi:10.1016/j.trip.2025.101545 · Cited by 4
Trucks Don’t Mean Trump: Diagnosing Human Error in Image Analysis
Zamfirescu-Pereira, J. D., Chen, J., Wen, E., Koenecke, A., Garg, N., & Pierson, E. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533145 · Cited by 6
“There is substantial work on creating algorithms which can learn to complement humans [24, 31, 34, 62], for example, by learning to defer to a human expert when the human will achieve better performance [47, 52, 65] or by providing automated assessments for human experts to consider in making decisions [32].”
BackgroundInvestigating the Relative Strengths of Humans and Machine Learning in Decision-Making
Rastogi, C. (2023) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3600211.3604738 · Cited by 4
Kolb, J., Srivastava, D., & Feigh, K. M. (2023) · IEEE RO-MAN · doi:10.1109/ro-man57019.2023.10309505 · Cited by 2
“Buc¸inca et. al details three cognitive forcing functions that can provide such structure [2]: requiring users to make a preliminary decision before seeing the AI’s recommendation [7], time-extending the decision-making process [15], and requiring users to solicit the AI’s recommendation [5].”
BackgroundA Bandit Model for Human-Machine Decision Making with Private Information and OpacityPreprint
Bordt, S., & Ulrike Von Luxburg, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.04800 · Cited by 11
“Of course it is an interesting question to ask how specific behavioral assumptions on the the human decision maker, such as bounded rationality (Selten, 1990) or biases when dealing with machine recommendations (Green and Chen, 2019) influence optimal interaction.”
BackgroundHow Different Explanations Impact Trust Calibration: The Case of Clinical Decision Support SystemsPreprint
Naiseh, M., Al‐Thani, D., Jiang, N., & Ali, R. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4098528 · Cited by 2
High-stakes team based public sector decision making and AI oversightPreprint
Morgan, D., Hashem, Y., Straub, V. J., & Bright, J. (2022) · SocArXiv (OSF Preprints) · doi:10.31235/osf.io/arq3w · Cited by 8
Lu, X., Huang, Y., Zhang, Y., & Shen, L. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4547893 · Cited by 2
AI Reliance and Decision Quality: Fundamentals, Interdependence, and the Effects of InterventionsPreprint
Schoeffer, J., Jakubik, J., Vössing, M., Kühl, N., & Satzger, G. (2024) · doi:10.31219/osf.io/cekm9 · Cited by 2
Action vs. Attention Signals for Human-AI Collaboration: Evidence from ChessPreprint
Poulidis, S., Ge, H., Bastani, H., & Bastani, O. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5128584 · Cited by 8
Designing Human-AI Collaboration: A Sufficient-Statistic Approach
Agarwal, N., Moehring, A., & Wolitzky, A. (2025) · National Bureau of Economic Research · doi:10.3386/w33949 · Cited by 31
Mitigating Voter Attribute Bias for Fair Opinion Aggregation
Ueda, R., Takeuchi, K., & Kashima, H. (2023) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3600211.3604660 · Cited by 1
“The recent development of decision support methods based on prediction using artificial intelligence has attracted considerable attention [11, 16], and raised some concerns about the possibility of social disadvantage resulting from unfair predictions caused by voter attribute bias.”
BackgroundEpistemic Injustice and Government Information Systems: Lessons from Two Cases
Hulstijn, J., Dong, H., & Markovich, R. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-70274-7_26 · Cited by 2
Towards Understanding Human-AI Reliance Patterns Through Explanation Styles
Casolin, E. R., & Salim, F. D. (2024) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3675094.3678996 · Cited by 2
“On the contrary, much of the prior literature has found that XAI increases over-reliance on AI [2, 14, 17, 31], leading to poorer decision outcomes overall.”
InfluentialReviseMate: Exploring Contextual Support for Digesting STEM Paper Reviews
Xu, Y., Zhang, S., Fan, Y., Shi, S., Peng, Z., & Li, Q. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757502 · Cited by 1
“Although AI-assisted systems are believed to enhance human analysis efficiency and quality significantly, some studies indicate that mixed-initiative human-AI workflows can often be inferior to standalone AI [6, 27, 45, 84].”
Algocracy and Capillaries of Power
Muhammady, I. G. E., & Rizkika, H. L. (2025) · Jurnal Penelitian Ilmu Sosial dan Eksakta · doi:10.47134/trilogi.v5i1.1699 · Cited by 1
Kachnowski, S., Khan, A. H., Floquet, S., Whitlock, K. K., Wisnivesky, J. P., Neill, D. B., Dankwa‐Mullan, I., Ortega, G., Daoud, M., Zaheer, R., et al. (2025) · Frontiers in Big Data · doi:10.3389/fdata.2025.1621526 · Cited by 1
“They may also be influenced by irrelevant information, trust inaccurate algorithms, and apply algorithms in ways they were not designed for (Green and Chen, 2019), exemplifying the need for increased participation and improved education of subject-matter experts to ensure that AI and BDA are not…”
Nanorobotics and artificial intelligence for effective nanobased intervention
Sharma, D., Singh, J., Mehta, P., Sharma, A., Thakur, A., & Chowdhury, K. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-33070-4.00002-2 · Cited by 3
Adaptive Realities: Human-in-the-Loop AI for Trustworthy XR Training in Safety-Critical Domains
Pretolesi, D., Regal, G., Schrom-Feiertag, H., & Tscheligi, M. (2026) · Multimodal Technologies and Interaction · doi:10.3390/mti10010011 · Cited by 3
“Meta-analyses suggest that HITL implementations yield measurable improvements in reliability and decision quality over fully autonomous baselines [23–25].”
Zheng, Z., You, W., Shao, Y., Zhou, M., Zhao, X., Ye, Y., Lu, Y., Zhou, Z., & Sun, L. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2623219 · Cited by 2
“Xpath (Gu et al., 2023) adopted an “algorithm-in-the-loop” approach (Green & Chen, 2019), where AI offers diagnostic suggestions and explanations, and the pathologist ultimately decides to approve, decline or remain uncertain about the suggested diagnosis.”
Raees, M., & Papagelis, K. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798834 · Cited by 1
“Hence, post-hoc explanations are used to help users understand complex AI systems [2, 17, 36].”
Algorithmic Decision-Making and Human Autonomy in Finance: A Systematic Review of AI Ethics Research
Bhatti, T. (2026) · Journal of Statistical Theory and Applications · doi:10.1007/s44199-026-00175-w · Cited by 11
“Some studies focus primarily on technical questions of model performance and fairness mitigation, whereas others emphasize regulatory compliance, human rights, institutional accountability, or stakeholder trust [2, 5, 34 , 77].”
Perceive, Prioritize, Act: Eliciting Alert Design Requirements for Fast-Paced Medical Work
Zhang, Z., Agbugba, V. F., Billings, S., Bai, E., Luo, X., & Adelgais, K. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2682917 · Cited by 1
Rahman, A. T. M. M., & Sultana, S. (2026) · Designing Interactive Systems Conference · doi:10.1145/3800645.3812955 · Cited by 1
“This implication extends HCI research on human–AI decision-making by emphasizing support for human decision-making in complex contexts, rather than replacing it [4, 32, 78].”
From Prediction to Stewardship: Framing Educational Data Science in the Age of Generative AI
McNamara, D. S., & Huynh, L. (2026) · Information · doi:10.3390/info17060610 · Cited by 1
Doh, W., Goh, Y., & Kim, S.-H. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-29548-4_13 · Cited by 1
Dhanorkar, S., Passi, S., & Vorvoreanu, M. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812402 · Cited by 32
“…tasks using proxy measures (unsurprising, given the importance attributed to the oversight work of reviewing AI outputs) [e.g. : 12, 44, 46, 63, 72, 107] or is largely conceptual, focusing on high-level arguments, frameworks, and theories of oversight [e.g. : 29, 41, 65, 68, 70, 112,…”
Digital humanism in requirements engineering for healthcare solutions in the age of AI
Ouhbi, S., Levy, M., Mondragón, O., Lessard, L., Taveter, K., Fiorini, L., Premanandan, S., Fricker, S. A., & Amyot, D. (2026) · Requirements Engineering · doi:10.1007/s00766-026-00466-9 · Cited by 1
“…the AI’s recommendation, thereby reducing anchoring bias where users’ initial decisions are influenced by the AI’s suggestion if presented first [ 54 ]; slowing down the process by delaying the presentation of the AI’s recommendation, targeting to motivate users to form their own hypotheses…”
Ersozlu, Z., Ledger, S., Babic, M., & Parkes, R. (2026) · Contemporary Educational Technology · doi:10.30935/cedtech/18970 · Cited by 2
Cybersophy: A framework for human cognition, identity, and values in the age of artificial agents
Ning, H., & Ding, J. (2026) · Computers in Human Behavior Artificial Humans · doi:10.1016/j.chbah.2026.100385 · Cited by 1
Kim, J. Y., Rowell, B. E., Whitaker, M., Chen, Q., Kontar, R. A., Lester, C. A., & Yang, X. J. (2026) · JMIR Human Factors · doi:10.2196/92822 · Cited by 1
“In contrast, ex-post systems require the human decision maker to commit to an initial decision before receiving AI advice [33-36].”
Nitchals, C. (2026) · AI and Ethics · doi:10.1007/s43681-026-01414-7 · Cited by 1
Alomair, G., & Salem, M. A. (2026) · Algorithms · doi:10.3390/a19100832 · Cited by 1
“These requirements are consistent with responsible-AI principles that emphasise human agency, meaningful human control, accountability, transparency, and oversight [20,25,26,28,29,51].”
InfluentialLindvall, M. (2021) · Linköping studies in science and technology. Dissertations · doi:10.3384/diss.diva-176117 · Cited by 1
Challenges of Applying Data-Driven Persona Development
Jansen, B. J. J., Salminen, J. O., Jung, S., & Guan, K. (2021) · Synthesis lectures on human-centered informatics · doi:10.1007/978-3-031-02231-9_6 · Cited by 3
de Santana, V. F., Quigley, L. T., Coates, J., & Graves, M. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3637130 · Cited by 1
“Green et al. [24] contrasts how sometimes decision making is done with a human supervising an algorithm’s task (human-in-the-loop) with situations when an algorithm participates in a human task (algorithm-in-the-loop) and how the former has its own characteristics.”
BackgroundOn Prediction-Modelers and Decision-Makers: Why Fairness Requires More Than a Fair Prediction ModelPreprint
Scantamburlo, T., Baumann, J., & Heitz, C. (2023) · Zürcher Hochschule für Angewandte Wissenschaften digital collection (Zurich University of Applied Sciences) · doi:10.48550/arxiv.2310.05598 · Cited by 2
“Further research pointed out that human decision-makers tend to deviate from algorithmic predictions (Stevenson and Doleac 2021) and struggle to assess algorithmic performance (Green and Chen 2019; Poursabzi-Sangdeh et al. 2021).”
BackgroundSupporting Information Integration in Human-AI Augmentation via Reflection on UnobservablesPreprint
De‐Arteaga, M., & Holstein, K. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4884604 · Cited by 2
Lane, J. N., Boussioux, L., Ayoubi, C., Chen, Y. H., Lin, C., Spens, R., Wagh, P., & Wang, P.-H. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4914367 · Cited by 7
Self-citationThe ABC’s of Who Benefits from Working with AI: Ability, Beliefs, and CalibrationPreprint
Caplin, A., Deming, D. J., Li, S., Martin, D. F., Marx, P., Weidmann, B., & Ye, K. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4978700 · Cited by 1
Metadata
- Authors
- Green, B., & Chen, Y.
- Affiliations
- Harvard University (US)
- Venue
- Proceedings of the ACM on Human-Computer Interaction
- Published
- 2019-11-07
- Volume
- 3
- Issue
- CSCW
- Pages
- 1-24
- ISSN
- 2573-0142 (ISSN-L)
- OpenAlex
- W2979893369
- Cited by
- 392
- References
- 64
- Citations by year
- 2026: 94 · 2025: 61 · 2024: 62 · 2023: 54 · 2022: 58 · 2021: 36 · 2020: 22 · 2019: 5
- FWCI
- 52.66
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Proceedings of the ACM on Human-Computer Interaction (publishedVersion)
- Funders
- National Science Foundation (DGE1745303)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Keywords
- human-algorithm interaction, risk assessment, pretrial release, lending, interpretability, socio-technical context
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.