Cited by 51Human-Automation Interaction and SafetyEthics and Social Impacts of AIExplainable Artificial Intelligence (XAI)

AI Shall Have No Dominion: on How to Measure Technology Dominance in AI-supported Human decision-making

Open access
Cabitza, Federico, Campagner, Andrea, Angius, Riccardo, Natali, Chiara, Reverberi, Carlo·2023·Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

Abstract

In this article, we propose a conceptual and methodological framework for measuring the impact of the introduction of AI systems in decision settings, based on the concept of technological dominance, i.e. the influence that an AI system can exert on human judgment and decisions. We distinguish between a negative component of dominance (automation bias) and a positive one (algorithm appreciation) by focusing on and systematizing the patterns of interaction between human judgment and AI support, or reliance patterns, and their associated cognitive effects. We then define statistical approaches for measuring these dimensions of dominance, as well as corresponding qualitative visualizations. By reporting about four medical case studies, we illustrate how the proposed methods can be used to inform assessments of dominance and of related cognitive biases in real-world settings. Our study lays the groundwork for future investigations into the effects of introducing AI support into naturalistic and collaborative decision-making.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3544548.3581095

Cites

65 of 65 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 25

  1. Vicky Arnold and Steve G Sutton. 1998. The theory of technology dominance: Un- derstanding the impact of intelligent decision aids on decision maker's judgments. Advances in accounting behavioral research 1, 3 (1998), 175-194.
  2. Gilles Blondel, Marta Arias, and Ricard Gavaldà. 2017. Identifability and trans- portability in dynamic causal networks. International journal of data science and analytics 3, 2 (2017), 131-147.
  3. Federico Cabitza. 2021. Cobra AI: Exploring Some Unintended Consequences. MIT Press, Cambridge, MA (USA), 87.
  4. John M Carroll, Wendy A Kellogg, and Mary Beth Rosson. 1991. The task-artifact cycle. Cambridge University Press, Cambridge (UK), 74-102.
  5. Mary Cummings. 2004. Automation Bias in Intelligent Time Critical Decision Support Systems. In AIAA 1st Intelligent Systems Technical Conference. American Institute of Aeronautics and Astronautics, Reston, VA (USA).
  6. Brian J Fogg. 1998. Captology: the study of computers as persuasive technolo- gies. In CHI 98 Conference Summary on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY (USA), 385.
  7. Gerd Gigerenzer and Henry Brighton. 2009. Homo heuristicus: Why biased minds make better inferences. Topics in cognitive science 1, 1 (2009), 107-143.
  8. Charles AE Goodhart. 1984. Problems of monetary management: the UK experi- ence. In Monetary theory and practice. Palgrave, London (UK), 91-121.
  9. Joseph Y Halpern. 2016. Actual causality. MIT Press, Cambridge, MA (USA).
  10. Nigel Harvey and Ilan Fischer. 1997. Taking advice: Accepting help, improving judgment, and sharing responsibility. Organizational behavior and human decision processes 70, 2 (1997), 117-133.
  11. Yoyo Tsung-Yu Hou and Malte F Jung. 2021. Who is the expert? Reconciling algorithm aversion and algorithm appreciation in AI-supported decision making. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1-25.
  12. Ekaterina Jussupow, Izak Benbasat, and Armin Heinzl. 2020. Why are we averse towards algorithms? A comprehensive literature review on algorithm aversion. In Twenty-Eighth European Conference on Information Systems (ECIS2020). AIS, Atlanta, GA (USA), 1-16.
  13. K Katsikopoulos, O Simsek, Marcus Buckmann, and Gerd Gigerenzer. 2020. Clas- sifcation in the Wild. MIT Press, Cambridge, MA (USA).
  14. Christopher J Kelly, Alan Karthikesalingam, Mustafa Suleyman, Greg Corrado, and Dominic King. 2019. Key challenges for delivering clinical impact with artifcial intelligence. BMC medicine 17, 1 (2019), 1-9.
  15. Xiaoxuan Liu, Samantha Cruz Rivera, David Moher, Melanie J Calvert, and Alas- tair K Denniston. 2020. Reporting guidelines for clinical trial reports for inter- ventions involving artifcial intelligence: the CONSORT-AI extension. bmj 370 (2020), 1364--1374.
  16. David Lyell and Enrico Coiera. 2016. Automation bias and verifcation complexity: A systematic review. Journal of the American Medical Informatics Association 24 (08 2016), ocw105.
  17. Shuai Ma, Ying Lei, Xinru Wang, Chengbo Zheng, Chuhan Shi, Ming Yin, and Xiaojuan Ma. 2023. Who Should I Trust: AI or Myself? Leveraging Human and AI Correctness Likelihood to Promote Appropriate Trust in AI-Assisted Decision-Making. arXiv:2301.05809 · arXiv
  18. Anne-Sophie Mayer, Franz Strich, and Marina Fiedler. 2020. Unintended Conse- quences of Introducing AI Systems for Decision Making. MIS Quarterly Executive 19, 4 (2020).
  19. Rob McCarney, James Warner, Steve Ilife, Robbert Van Haselen, Mark Grifn, and Peter Fisher. 2007. The Hawthorne Efect: a randomised, controlled trial. BMC medical research methodology 7, 1 (2007), 1-8.
  20. Kathleen L Mosier and Linda J. Skitka. 1996. Human Decision Makers and Automated Decision Aids: Made for Each Other? In Automation and Human Performance: Theory and Applications, Raja Parasuraman and Mustapha Mouloua (Eds.). CRC Press, Boca Raton, Chapter 10, 201-220.
  21. Judea Pearl. 2012. The do-calculus revisited. In Proceedings of the Twenty-Eighth Conference on Uncertainty in Artifcial Intelligence. AUAI Press, Arlington, VA (USA), 3-11.
  22. Max Schemmer, Patrick Hemmer, Niklas Kühl, Carina Benz, and Gerhard Satzger. 2022. Should I Follow AI-based Advice? Measuring Appropriate Reliance in Human-AI Decision-Making. In CHI Conference on Human Factors in Computing Systems (CHI '22), Workshop on Trust and Reliance in AI-Human Teams (trAIt). Association for Computing Machinery, New York, NY (USA), 10.
  23. Linda J Skitka, Kathleen L Mosier, and Mark Burdick. 1999. Does automation bias decision-making? International Journal of Human-Computer Studies 51, 5 (1999), 991-1006.
  24. Magdalena Szumilas. 2010. Explaining odds ratios. Journal of the Canadian academy of child and adolescent psychiatry 19, 3 (2010), 227.
  25. Bradley Edward Williams. 2020. The Role of Complexity Within Intelligent Decision Aids on User Reliance: An Extension of the Theory of Technology Dominance. Ph.D. Dissertation. The University of North Carolina at Charlotte.

Cited by

Sorted by citations

Authors
Cabitza, F., Campagner, A., Angius, R., Natali, C., & Reverberi, C.
Affiliations
  • IRCCS Ospedale Galeazzi - Sant'Ambrogio (IT)
  • University of Padua (IT)
  • University of Milano-Bicocca (IT)
Venue
Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Published
2023-04-19
Pages
1-20
OpenAlex
W4366547500
Cited by
51
References
65
Citations by year
2026: 13 · 2025: 24 · 2024: 11 · 2023: 3
FWCI
25.52
Type
conference paper
Language
en
Open access
Locations
  • CHI Conference on Human Factors in Computing Systems (publishedVersion)
  • BOA (University of Milano-Bicocca) (submittedVersion)
Topics
  • Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
Keywords
technological hegemony, AI-supported decision-making, human-AI interaction, automation bias, algorithm appreciation, cognitive biases, measurement framework, clinical decision-making, collaborative decision-making

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review