The Principles and Limits of Algorithm-in-the-Loop Decision Making
The DOI matches its Crossref record; 61 of the 64 references it cites were found in a registry or an index.
Abstract
The rise of machine learning has fundamentally altered decision making: rather than being made solely by people, many important decisions are now made through an "algorithm-in-the-loop'' process where machine learning models inform people. Yet insufficient research has considered how the interactions between people and models actually influence human decisions. Society lacks both clear normative principles regarding how people should collaborate with algorithms as well as robust empirical evidence about how people do collaborate with algorithms. Given research suggesting that people struggle to interpret machine learning models and to incorporate them into their decisions---sometimes leading these models to produce unexpected outcomes---it is essential to consider how different ways of presenting models and structuring human-algorithm interactions affect the quality and type of decisions made. This paper contributes to such research in two ways. First, we posited three principles as essential to ethical and responsible algorithm-in-the-loop decision making. Second, through a controlled experimental study on Amazon Mechanical Turk, we evaluated whether people satisfy these principles when making predictions with the aid of a risk assessment. We studied human predictions in two contexts (pretrial release and financial lending) and under several conditions for risk assessment presentation and structure. Although these conditions did influence participant behaviors and in some cases improved performance, only one desideratum was consistently satisfied. Under all conditions, our study participants 1) were unable to effectively evaluate the accuracy of their own or the risk assessment's predictions, 2) did not calibrate their reliance on the risk assessment based on the risk assessment's performance, and 3) exhibited bias in their interactions with the risk assessment. These results highlight the urgent need to expand our analyses of algorithmic decision making aids beyond evaluating the models themselves to investigating the full sociotechnical contexts in which people and algorithms interact.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“there has been increasing recent research efforts [8, 15, 22, 41, 58, 65] that highlight the importance of involving stakeholders to understand their practices and needs [15, 41, 65] and socio-environmental factors [8] for the design and evaluation of a system.”
From Understanding the Effect of Counterfactual Explanations on Trust and Reliance on AI for Human-AI Collaborative Clinical Decision Making (2023) · cites this paper as [22]
In the citing paper
“However, many empirical studies on human-AI decision-making did not observe complementary performance [6,16,37,52,58,61,68,82,84,87].”
From Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations (2023) · cites this paper as [37]
In the citing paper
“We note that these approaches correspond to the Update and Feedback methods of Green & Chen [27], which were effective, albeit not in the complementary zone.”
From Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance (2021) · cites this paper as [27]
In the citing paper
“More interestingly, communication failure can also stem from a deeper inability to understand how AI makes decisions and how likely AI may err in specific conditions [61] .”
From Enhancing human-AI collaboration: The case of colonoscopy (2024) · cites this paper as [61]
In the citing paper
“However, many studies have shown that human-AI teams under-perform the AI alone in tasks [3, 4, 6, 14, 18, 20, 21, 30].”
From A Decision Theoretic Framework for Measuring AI Reliance (2024) · cites this paper as [14]
7 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Greedy function approximation: A gradient boosting machine
Friedman, J. H. (2001) · The Annals of Statistics · doi:10.1214/aos/1013203451 · Cited by 30,731
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605
Strictly Proper Scoring Rules, Prediction, and Estimation
Gneiting, T., & Raftery, A. E. (2007) · Journal of the American Statistical Association · doi:10.1198/016214506000001437 · Cited by 5,793
Algorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232
What Should Be Done with Equity Theory?
Leventhal, G. S. (1980) · Social Exchange · doi:10.1007/978-1-4613-3087-5_2 · Cited by 3,010 · cited 2×
“For example, the principle of procedural justice requires that decisions be (among other things) accurate, fair, consistent, correctable, and ethical [55].”
BackgroundAlgorithm appreciation: People prefer algorithmic to human judgment
Logg, J. M., Minson, J. A., & Moore, D. A. (2019) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2018.12.005 · Cited by 2,061
Principles of mixed-initiative user interfaces
Horvitz, E. (1999) · Proceedings of the SIGCHI conference on Human factors in computing systems the CHI is the limit - CHI '99 · doi:10.1145/302979.303030 · Cited by 1,490
Lee, M. K. (2018) · Big Data & Society · doi:10.1177/2053951718756684 · Cited by 1,338
Human Decisions and Machine Predictions
Kleinberg, J., Lakkaraju, H., Leskovec, J., Ludwig, J., & Mullainathan, S. (2018) · The Quarterly Journal of Economics · doi:10.1093/qje/qjx032 · Cited by 1,040
Research works printed but not linked: 12
- Alex Albright . 2019. If You Give a Judge a Risk Score: Evidence from Kentucky Bail Decisions . The John M. Olin Center for Law , Economics , and Business Fellows' Discussion Paper Series 85 ( 2019 ). Alex Albright. 2019. If You Give a Judge a Risk Score: Evidence from Kentucky Bail Decisions. The John M. Olin Center for Law, Economics, and Business Fellows' Discussion Paper Series 85 (2019).
- Julia Angwin , Jeff Larson , Surya Mattu , and Lauren Kirchner . 2016. Machine Bias. ProPublica ( 2016 ). https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2016. Machine Bias. ProPublica (2016). https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing
- Laura Bliss . 2018. Former Uber Backup Driver: 'We Saw This Coming'. CityLab ( 2018 ). https://www.citylab.com/transportation/2018/03/former-uber-backup-driver-we-saw-this-coming/556427/ Laura Bliss. 2018. Former Uber Backup Driver: 'We Saw This Coming'. CityLab (2018). https://www.citylab.com/transportation/2018/03/former-uber-backup-driver-we-saw-this-coming/556427/
- Darwin Bond-Graham and Ali Winston . 2013. All Tomorrow's Crimes: The Future of Policing Looks a Lot Like Good Branding. SF Weekly ( 2013 ). http://archives.sfweekly.com/sanfrancisco/all-tomorrows-crimes-the-future-of-policing-looks-a-lot-like-good-branding/Content?oid=2827968 Darwin Bond-Graham and Ali Winston. 2013. All Tomorrow's Crimes: The Future of Policing Looks a Lot Like Good Branding. SF Weekly (2013). http://archives.sfweekly.com/sanfrancisco/all-tomorrows-crimes-the-future-of-policing-looks-a-lot-like-good-branding/Content?oid=2827968
- Pamela M. Casey , Roger K. Warren , and Jennifer K . Elek . 2011 . Using Offender Risk and Needs Assessment Information at Sentencing : Guidance for Courts from a National Working Group .National Center for State Courts . Pamela M. Casey, Roger K. Warren, and Jennifer K. Elek. 2011. Using Offender Risk and Needs Assessment Information at Sentencing: Guidance for Courts from a National Working Group .National Center for State Courts.
- Thomas H. Cohen , Bailey Pendergast , and Scott W. VanBenschoten . 2016 . Examining overrides of risk classifications for offenders on federal supervision . Federal Probation , Vol. 80 , 1 (2016), 12 . Thomas H. Cohen, Bailey Pendergast, and Scott W. VanBenschoten. 2016. Examining overrides of risk classifications for offenders on federal supervision. Federal Probation, Vol. 80, 1 (2016), 12.
- Mary L. Cummings . 2006. Automation and Accountability in Decision Support System Interface Design. Journal of Technology Studies ( 2006 ). Mary L. Cummings. 2006. Automation and Accountability in Decision Support System Interface Design. Journal of Technology Studies (2006).
- William Dieterich , Christina Mendoza , and Tim Brennan . 2016. COMPAS Risk Scales: Demonstrating Accuracy Equity and Predictive Parity . Northpointe Inc . ( 2016 ). William Dieterich, Christina Mendoza, and Tim Brennan. 2016. COMPAS Risk Scales: Demonstrating Accuracy Equity and Predictive Parity. Northpointe Inc. (2016).
- Jeff Larson , Surya Mattu , Lauren Kirchner , and Julia Angwin . 2016. How We Analyzed the COMPAS Recidivism Algorithm. ProPublica ( 2016 ). https://www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm Jeff Larson, Surya Mattu, Lauren Kirchner, and Julia Angwin. 2016. How We Analyzed the COMPAS Recidivism Algorithm. ProPublica (2016). https://www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm
- 10.1518/hfes.46.1.50.30392 · doi:10.1518/hfes.46.1.50.30392 · Paper page
- Frank Main . 2016. Cook County judges not following bail recommendations: study. Chicago Sun-Times ( 2016 ). https://chicago.suntimes.com/chicago-news/cook-county-judges-not-following-bail-recommendations-study-find/ Frank Main. 2016. Cook County judges not following bail recommendations: study. Chicago Sun-Times (2016). https://chicago.suntimes.com/chicago-news/cook-county-judges-not-following-bail-recommendations-study-find/
- Megan T. Stevenson and Jennifer L . Doleac . 2018 . The Roadblock to Reform. The American Constitution Society ( 2018). https://www.acslaw.org/wp-content/uploads/2018/11/RoadblockToReformReport.pdf Megan T. Stevenson and Jennifer L. Doleac. 2018. The Roadblock to Reform. The American Constitution Society (2018). https://www.acslaw.org/wp-content/uploads/2018/11/RoadblockToReformReport.pdf
Cited by
Items 301–393 of 393
Elemure, I. E., Chuwa, C. C., Kolawole, L. O., Ologun, A. G., Ibidunmoye, A. F., Eneano, G. A., & Ikeakaonwu, O. M. (2026) · Research Square · doi:10.21203/rs.3.rs-9087686/v1 · Cited by 3
راشد, ز. ح. (2026) · مجلة الدراسات والبحوث القانونية · doi:10.67603/jlsr.v11i2.10268
L'acte administratif algorithmique
Stepanov, A. (2026) · doi:10.70675/adc05be3za62ez4b4fz81e7zabf744e60dbf
"A cold, technical decision-maker": Can AI provide explainability, negotiability, and humanity?Preprint
Woodruff, A., Anderson, Y. A., Armstrong, K. J., Gkiza, M., Jennings, J., Moessner, C., Viégas, F. B., WATTENBERG, M. P., Webb, A. L., Wrede, F., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.00874 · Cited by 11
“At the same time, other research has shown that handoffs between humans and AI are complex and difficult to design well [32, 46].”
Willing to Revise? Confidence and Recommendation Adoption in AI-Assisted Image Recognition
Röseler, L., Scholtes, I., Sendhoff, B., & Hannák, A. (2022) · Frontiers in artificial intelligence and applications · doi:10.3233/faia220222
Maxwell, J., & Tomlinson, J. (2022) · Bristol University Press eBooks · doi:10.51952/9781529219869.bm001
Smolenaars, C. C., & Gonçalves, T. L. V. (2023) · Casa Leiria eBooks · doi:10.29327/5321162.1-15
Learning Fair Policies for Multi-Stage Selection Problems from Observational Data
Jia, Z., Hanasusanto, G. A., Vayanos, P., & Xie, W. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i19.30112 · Cited by 8
Artificial Intelligence and the End of AutonomyPreprint
Geddes, K. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4773531 · Cited by 2
(2024) · Advances in logistics, operations, and management science book series · doi:10.4018/979-8-3693-2695-4.ch009
Ethical Leadership in the Age of AI
(2024) · Advances in logistics, operations, and management science book series · doi:10.4018/979-8-3693-2695-4.ch008
Minerva Potenziale und Produktvision
Gutzmann, J. C., Dossena, G. L., Weber, V., Rudel, T., & Prilla, M. (2024) · HMD Praxis der Wirtschaftsinformatik · doi:10.1365/s40702-024-01120-0
Ðula, I., Berberena, T., Keplinger, K., & Wirzberger, M. (2024) · Lecture notes in information systems and organisation · doi:10.1007/978-3-031-76970-2_15 · Cited by 1
XAI-Supported Decision-Making: Insights from NeuroIS Studies for a User Perspective
Litvinova, Y., & Keplinger, K. (2025) · Lecture notes in information systems and organisation · doi:10.1007/978-3-031-71385-9_13
Krause, T. A. (2025) · dms – der moderne staat – Zeitschrift für Public Policy Recht und Management · doi:10.3224/dms.vxix.403989
Improving AI-Assisted Decision-Making Through Calibrated CoarseningPreprint
Hoong, R., & Dreyfuss, B. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5286198 · Cited by 5
Schoeffer, J., De‐Arteaga, M., & Elmer, J. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732070 · Cited by 7
“Information concerning what construct is being predicted, and its differences with the construct of interest to the task, should be clearly communicated to users; otherwise, it could lead to a drift in decision-making criteria [29].”
Error in the Loop: How Human Mistakes Can Improve Algorithmic Learning
Copus, R. W., Spackman, C., & Laqueur, H. S. (2025) · Journal of law & empirical analysis. · doi:10.1177/2755323x251357643 · Cited by 2
“In medical diagnoses, academic performance prediction, lending, and many more arenas, human overrides of algorithmic determinations have been shown to signi fi cantly decrease decision-making accuracy (Rezazade Mehrizi et al., 2023; Sele & Chugunova, 2024; Green & Chen, 2019).”
AI Behavioral SciencePreprint
Jackson, M. O., Mei, Q., Wang, S., Xie, Y., Yuan, W., Benzell, S. G., Brynjolfsson, E., Evans, J. A., Jabarian, B., Kleinberg, J. M., et al. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5395006 · Cited by 12
Horton, P., Florea, A., & Stringfield, B. (2025) · Machine Learning with Applications · doi:10.1016/j.mlwa.2025.100733 · Cited by 5
User Interaction for Human–AI Interaction and Collaboration
Dan, W., & Guo, Q. (2025) · Cambridge University Press eBooks · doi:10.1017/9781009587877.003
Cabitza, F., Fregosi, C., & Vicente, L. (2025) · Communications in computer and information science · doi:10.1007/978-3-032-08317-3_11 · Cited by 3
AutoS$$^2$$earch: Unlocking the Reasoning Potential of Large Models for Web-Based Source Search
Zhu, Z., Ji, Y., Huang, J., Zhao, Y. B., Qiu, S., & Ju, R. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-97207-2_6
Improving Human-AI Collaboration in Medical Diagnosis with Combination Advice
Zhao, X., Liu, J., Yu, Z., & Guo, B. (2025) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme59968.2025.11209356
“To encourage doctors’ independent thinking, we adopt the sequential paradigm [3], presenting AI advice only after doctors decide initially.”
Designing reliable algorithms to improve patient outcomes by deploying AI in medical domain
Joshi, R., Kumari, S., & Pandey, K. (2025) · Elsevier eBooks · doi:10.1016/b978-0-443-41660-6.00019-3
Pak, R., Rovira, E. M., & McLaughlin, A. C. (2026) · Ergonomics · doi:10.1080/00140139.2026.2686284 · Cited by 3
Human-AI Proximity: Unveiling the Detrimental Effects on Idea Selection and ImplementationPreprint
Mascareño, J., Wortler, B., Przegalinska, A., & Ciechanowskie, L. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5885745
Human-AI Proximity: Unveiling the Detrimental Effects on Idea Selection and ImplementationPreprint
Mascareño, J., Wortler, B., Przegalinska, A., & Ciechanowskie, L. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5885744
Human-AI Proximity: Unveiling the Detrimental Effects on Idea Selection and ImplementationPreprint
Mascareño, J., Wortler, B., Przegalinska, A., & Ciechanowskie, L. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5893131
On the Influence of Social Robots During Ethical-Decision Making: A Preliminary Study
Matarese, M., Guerrieri, V., Kahya, R., Rea, F., & Sciutti, A. (2025) · Lecture notes in computer science · doi:10.1007/978-981-95-2382-5_41
Risks Related to Artificial Intelligence and Their Mitigation Techniques: Review on Recent Advances
Kapoor, D., & Bhatia, A. (2026) · Lecture notes in electrical engineering · doi:10.1007/978-981-96-9370-2_14
AI-Enabled ESG Scoring Models for Corporate Governance
Pradhan, U., & Anand, V. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-5891-8.ch004
Domingo, L.-F., Gilbert, S., Cao, Y. T., Daumé, H., Mazurek, M. L., & Shilton, K. (2026) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3796227
“Value Sensitive Design of LLM Support for Online Moderation”
Liew, X. Y., Hameed, N., Clos, J., & Fischer, J. E. (2026) · PeerJ Computer Science · doi:10.7717/peerj-cs.3632 · Cited by 1
Khasawneh, Y. J. A., Khasawneh, N. A. S., & Khasawneh, M. A. S. (2026) · Journal of Computer Assisted Learning · doi:10.1002/jcal.70216 · Cited by 1
Affa, A. S., & Satrio, D. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6393399
Explainability for machine learning models : from data adaptability to user perception
Delaunay, J. (2023) · theses.fr (ABES) · doi:10.70675/933366c4z576ez464cz9fcfz24627f86dd2f · Cited by 1
Advanced Clustering and AI-Driven Decision Support Systems for Smart Energy Management
Lévy, L.-N. (2024) · theses.fr (ABES) · doi:10.70675/d9814b98za715z4986z88f1z081963d08cf3 · Cited by 12
Bertrand, A. (2024) · theses.fr (ABES) · doi:10.70675/216d3647z78bcz4c68z9ba9z2d0a029cf1b4 · Cited by 8
Human-Machine Conflict in Human-Machine TeamingPreprint
Chen, M., Xiong, W., Tsao, L., & Ma, L. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6508019 · Cited by 1
Understanding and Designing Human-Centered AI Interactions for Non-Expert Users
Raees, M. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3799192
“However, research shows that people often fail to appropriately rely on AI systems due to various reasons, such as their biases [10] or system complexity [17].”
More Isn't Always Better: Balancing Decision Accuracy and Conformity Pressures in Multi-AI Advice
Tsuchiya, Y., & Baba, Y. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791648 · Cited by 4
“Recent research in HCI and AI has examined frameworks for collaborative decision-making with AI systems [44, 57].”
Seah, C., Lee, Y. C., Lee, J.-J., Yen, C. C., & Zheng, C. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790461 · Cited by 4
“In contrast, Algorithm-in-the-loop [19] and AI-in-the-loop [33] frameworks better align with our findings around therapy staff’s needs.”
Does Sycophancy Change Decisions? Effect of LLM Sycophancy on AI-Assisted Decision-Making
Li, Z., Pan, J., Liu, Q., Xi, Y., Zhou, Y., Jin, Y., Mao, R., & Chen, P. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790934 · Cited by 13
“Moreover, task risk influences the effectiveness of AI explanations [108] and user reliance patterns [37, 99].”
Kuang, E., Shen, L., Soure, E. J., Fan, M., & Shinohara, K. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2638554
The embodied ethics alignment problem of AI
Zwitter, A. (2026) · Discover Artificial Intelligence · doi:10.1007/s44163-026-01905-w · Cited by 2
“Human reviewers cannot examine every decision a deployed system makes; the explanations a model generates may not be faithful to the underlying computation [17]; and the cognitive load of meaningful oversight scales poorly with system speed and decision volume [18].”
Future Horizons in Human–AI Interaction: Joint Perspectives from HCI and AI
André, E., Conati, C., Levy‐Tzedek, S., Matera, M., & Spitale, M. (2026) · International Conference on Advanced Visual Interfaces (AVI) · doi:10.1145/3811427.3811490
“Users are no longer simply operating systems; they are engaging with adaptive, generative, and sometimes unpredictable behaviors that require interpretation, negotiation, and trust [9].”
For Those About to Rely—A Taxonomy of Experimental Studies on AI Reliance: Completed Research Paper
Schaschek, M., Spatscheck, N., & Winkelmann, A. (2026) · Lecture notes in information systems and organisation · doi:10.1007/978-3-032-08480-4_2
Vanbrabant, S., Ruiz, G. A. R., & Vanacken, D. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-26051-2_10 · Cited by 1
AI Ethics in Cybersecurity: Designing for Accountability, Trust, and Human Oversight
Ramachandran, M. (2026) · Texts in computer science · doi:10.1007/978-3-032-25874-8_4
Niazi, M., Hassani, H., & Lee, M. (2026) · Automation · doi:10.3390/automation7030096 · Cited by 1
“A critical gap in many “HITL solves accountability” narratives is that human oversight can become rubber-stamp oversight under time pressure, workload, and interface design conditions that reliably produce automation bias and deference to automated recommendations [25,26,67,68].”
Özçiçek, A. (2026) · Journal of Evidence-Based Social Work · doi:10.1080/26408066.2026.2691414
Ruiz-Amezcua, P., Franco-Barranco, D., Reigada, D., Muñoz‐Galdeano, T., Maza, R. M., & Nieto‐Díaz, M. (2026) · Scientific Reports · doi:10.1038/s41598-026-57519-w
“This effect can lead to a progressive alignment between annotations and model outputs rather than an independent approximation of ground truth 42,43 .”
Mahajan, P. T. (2026) · AI and Ethics · doi:10.1007/s43681-026-01202-3 · Cited by 1
“Green and Chen for example, show that individuals frequently adjust judgments in response to algorithmic recommendations while reducing independent evaluation [ 85 ].”
Chopra, A. (2026) · VINE Journal of Information and Knowledge Management Systems · doi:10.1108/vjikms-07-2025-0288
Yang, M., Kagawa, R., & Baba, Y. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812288
“24% (Dating), indicating that the AI outperformed participants and providing appropriate accuracy for AI decision support models [23, 66].”
Gulyamov, S. S. (2026) · Psychology and Law · doi:10.17759/psylaw.2026160211 · Cited by 1
Li, Z., & Ma, Z. (2026) · International journal of law, crime and justice · doi:10.1016/j.ijlcj.2026.100879
Möhle, M. H., Zall, M., Thorwart-Gumpert, T., Elbert, N., Thiesse, F., & Flath, C. M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-30860-3_30 · Cited by 1
Naser, M. Z. (2026) · Safety Science · doi:10.1016/j.ssci.2026.107367
Cognitive Vulnerabilities in AI-Augmented Organisational Decision-Making
Gültekin, D. G. (2026) · IGI Global eBooks · doi:10.4018/979-8-3373-8302-6.ch009
Humanism and AI-Driven Vulnerabilities: Customers, Partners, Employees
Fioravante, R., & Vaccaro, A. (2026) · Humanism in business series · doi:10.1007/978-3-032-17121-4_5
Fan, L., Miao, Y., Li, S., & Zhang, J. (2026) · Communications in computer and information science · doi:10.1007/978-981-92-3563-6_14
Digital Evidence Under Legal Realism: Extralegal Factors Determining The Mario Dandy Verdict
Wijaya, M. F., Putri, T. A. R., & Titifiani, A. I. (2026) · Smart Journal of Criminal Law Review and Analysis · doi:10.65101/scrim.v1i3.80
Adaptive switching optimization for heterogeneous decision modules under operational uncertainty
Shin, H. K., Lee, C., Cha, J., Kil, M., Choi, J., & C6 (Command, C. (2026) · Journal of Industrial and Management Optimization · doi:10.3934/jimo.2026144
Gupta, K. (2026) · Research Square · doi:10.21203/rs.3.rs-10516581/v1
Promise and Caution: Mapping Opportunities for AI Decision Support in Emergency Medical Services
Zhang, Z., Agbugba, V. F., Bai, E., Billings, S., Luo, X., & Adelgais, K. (2026) · IEEE International Conference on Healthcare Informatics (ICHI) · doi:10.1109/ichi69079.2026.00086
“In such scenarios, even technically strong AI may provide little benefit, or be ignored entirely, if its outputs are not aligned with clinicians’ needs and domain-specific workflows [31], [32], [33].”
Guo, J. (2026) · International Journal of Web-Based Learning and Teaching Technologies · doi:10.4018/ijwltt.419119
Causal attributions for ChatGPT errors: what is the role of AI literacy?
Norcia, M., & Colí, E. (2026) · Journal of Psychology and AI · doi:10.1080/29974100.2026.2712302
Wang, Z., & Hu, K. (2026) · ACM Transactions on Social Computing · doi:10.1145/3843742 · Cited by 1
Lee, M.-S., & Han, S.-Y. (2026) · Systems · doi:10.3390/systems14091126 · Cited by 1
Al-Thani, H. E. S. M. (2026) · Unveiling seven continents yearbook journal · doi:10.65326/u7y10031
Tuo, B., Miller, T., & Demartini, G. (2026) · ACM Conference on Human-AI Complementarity and Alignment · doi:10.1145/3834580.3838739
“Social media platforms therefore continue to rely on human moderators as a safety net, with AI positioned to assist human decision-making[24].”
Wang, L., Huang, Q., Wang, K., LOU, J., & Yuan, C. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2728615
Tuo, B., Wang, B. T., Miller, T., & Demartini, G. (2026) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3848504
“This overreliance reflects a risky shift in agency and accountability, especially in domains like healthcare and law, where flawed AI guidance may perpetuate machine bias under the illusion of human oversight [1, 23].”
Affa, A. S., & Satrio, D. (2026) · Social Sciences & Humanities Open · doi:10.1016/j.ssaho.2026.103685
Human and AI in Decision Making: A Systematic Review of Trust, Fairness, and Empathy
Arora, A., & Arora, A. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-9599-9.ch015
Sha, H., Song, Y., Hao, X., & Jin, G. (2026) · Information Processing & Management · doi:10.1016/j.ipm.2026.105200
User-Friendly AI-Based Decision Support Systems
Echeverry, N. (2026) · International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW) · doi:10.1109/aciiw71585.2026.11712078
“Decisions initially made by human judgment alone now have an “algorithm-in-the-loop” [6].”
InfluentialLevantis, N., Sgora, A., Tsipis, A., & Polykalas, S. E. (2026) · Information · doi:10.3390/info17100962
Liu, J., Zhong, C., Meng, F., Shank, D. B., & Wunsch, D. C. (2026) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2026.103140
Intelligent Leadership in the Age of AI: Navigating Paradox, Uncertainty, and Hybrid Decision-Making
Truong, N., Kumar, P. V., & Hasan, M. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-2600-1488-2.ch010
Meng, Y., Liu, B., Gao, Z., Ma, S., & Zhou, H. (2026) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3785651.3831520
Building responsible AI: a trilateral framework integrating ethics, design and regulation
Raghupathi, W., Ren, J., & Saharia, A. (2026) · AI and Ethics · doi:10.1007/s43681-026-01426-3
“Yet technical fairness research often treats fairness definitions as engineering choices rather than normative commitments requiring ethical justification [ 29].”
Responsible and Trusted AI: An Interdisciplinary Perspective (2026)
Baum, K., Helfer, T., Jahn, F., Kerstan, S., Sesing-Wagenpfeil, A., Speith, T., & Sterz, S. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-41269-0_7
Shoaib, S., Haq, F. U., Maaitah, N., Dhahbi, A., Badawy, M., Sheikh, A. A., & Masood, A. (2026) · Social Sciences & Humanities Open · doi:10.1016/j.ssaho.2026.103799
Kanakamedala, V. R. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7635485
A Unified Framework for Classifying Artificial Intelligence Use Cases Beyond Task Automation
Baswa, S. R. (2026) · International Conference on Networks, Computers and Communications (ICNCCom) · doi:10.1109/icnccom69117.2026.11736232
“A system with high standalone accuracy but an opaque output format that prevents expert critical engagement may generate IL below 1.0; its very competence suppresses human oversight and produces net epistemic harm [17].”
InfluentialJin, Y., Sun, Y., & Zhang, H. L. (2026) · Legal and Criminological Psychology · doi:10.1111/lcrp.70071
Garg, A. (2023) · International Journal of Computational and Experimental Science and Engineering · doi:10.22399/ijcesen.5557
Kovalchuk, A. (2026) · Computers in Human Behavior Artificial Humans · doi:10.1016/j.chbah.2026.100409
AI Reliance in the Fantasy American Football Lineup-Setting Task
Nicholson, M. B., Keegan, B., & Semaan, B. (2026) · doi:10.1145/3838843.3838850
Metadata
- Authors
- Green, B., & Chen, Y.
- Affiliations
- Harvard University (US)
- Venue
- Proceedings of the ACM on Human-Computer Interaction
- Published
- 2019-11-07
- Volume
- 3
- Issue
- CSCW
- Pages
- 1-24
- ISSN
- 2573-0142 (ISSN-L)
- OpenAlex
- W2979893369
- Cited by
- 392
- References
- 64
- Citations by year
- 2026: 94 · 2025: 61 · 2024: 62 · 2023: 54 · 2022: 58 · 2021: 36 · 2020: 22 · 2019: 5
- FWCI
- 52.66
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Proceedings of the ACM on Human-Computer Interaction (publishedVersion)
- Funders
- National Science Foundation (DGE1745303)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Keywords
- human-algorithm interaction, risk assessment, pretrial release, lending, interpretability, socio-technical context
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.