“Look! It’s a Computer Program! It’s an Algorithm! It’s AI!”: Does Terminology Affect Human Perceptions and Evaluations of Algorithmic Decision-Making Systems?
Open accessAbstract
In the media, in policy-making, but also in research articles, algorithmic decision-making (ADM) systems are referred to as algorithms, artificial intelligence, and computer programs, amongst other terms. We hypothesize that such terminological differences can affect people’s perceptions of properties of ADM systems, people’s evaluations of systems in application contexts, and the replicability of research as findings may be influenced by terminological differences. In two studies (N = 397, N = 622), we show that terminology does indeed affect laypeople’s perceptions of system properties (e.g., perceived complexity) and evaluations of systems (e.g., trust). Our findings highlight the need to be mindful when choosing terms to describe ADM systems, because terminology can have unintended consequences, and may impact the robustness and replicability of HCI research. Additionally, our findings indicate that terminology can be used strategically (e.g., in communication about ADM systems) to influence people’s perceptions and evaluations of these systems.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Affinity to technology is consistently correlated with end users' perceptions of algorithmic capabilities.
“Langer et al. [56] showed that afnity to technology was consistently correlated with end users' perceptions of algorithmic capabilities.”
From Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability (2023) · cites this paper as [56]
End users might think about the decision-maker in binary terms, as either a human or not a human.
“The reason for this might be that end users might think about the decision-maker in binary terms, as either “a human” or “not a human” [56].”
From Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability (2023) · cites this paper as [56]
Terminology has been claimed to affect end users' fairness perceptions.
“Fifth, terminology has been claimed to afect end users' fairness perceptions [56].”
From Disentangling Fairness Perceptions in Algorithmic Decision-Making: the Effects of Explanations, Human Oversight, and Contestability (2023) · cites this paper as [56]
Prior studies investigated how conceptual metaphors influenced people's perception of algorithmic decision-making systems more broadly, as well as how anthropomorphic cues influence people's trust in robots, voice-assistants, and websites.
“Prior studies investigated how conceptual metaphors influenced people's perception of algorithmic decision-making systems more broadly [31], as well as how anthropomorphic cues influence people's trust in robots [9], voice-assistants [15], and websites [42].”
From From "AI" to Probabilistic Automation: How Does Anthropomorphization of Technical Systems Descriptions Influence Trust? (2024) · cites this paper as [31]
Even short textual descriptions can influence how humans meet and evaluate digital systems.
“Even short textual descriptions can influence how humans meet and evaluate digital systems [17, 26, 27, 29, 31, 47].”
From From "AI" to Probabilistic Automation: How Does Anthropomorphization of Technical Systems Descriptions Influence Trust? (2024) · cites this paper as [31]
In the context of probabilistic automation systems, the conceptual metaphor or 'pitch' of the system's functionality might play an especially compelling role, given the complexity of such systems.
“In the context of probabilistic automation systems, the conceptual metaphor [11, 33] or 'pitch' of the system's functionality might play an especially compelling role, given the complexity of such systems [31].”
From From "AI" to Probabilistic Automation: How Does Anthropomorphization of Technical Systems Descriptions Influence Trust? (2024) · cites this paper as [31]
We confirm earlier research that demonstrates some influence of anthropomorphization on attitude.
“although we confirm earlier research that demonstrate some influence of anthropomorphization on attitude (e.g., [29, 31])”
From From "AI" to Probabilistic Automation: How Does Anthropomorphization of Technical Systems Descriptions Influence Trust? (2024) · cites this paper as [31]
Our studies support the findings of [31].
“Our studies thus support the findings of [31].”
From From "AI" to Probabilistic Automation: How Does Anthropomorphization of Technical Systems Descriptions Influence Trust? (2024) · cites this paper as [31]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
User Acceptance of Information Technology: Toward A Unified View1
Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003) · MIS Quarterly · doi:10.2307/30036540 · Cited by 45,035 · see citing works
SciPy 1.0: fundamental algorithms for scientific computing in Python
Virtanen, P., Gommers, R., Oliphant, T. E., Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weckesser, W., Bright, J., et al. (2020) · Nature Methods · doi:10.1038/s41592-019-0686-2 · Cited by 38,430 · see citing works
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · see citing works
On the dimensionality of organizational justice: A construct validation of a measure.
Colquitt, J. A. (2001) · Journal of Applied Psychology · doi:10.1037/0021-9010.86.3.386 · Cited by 5,989 · see citing works
On seeing human: A three-factor theory of anthropomorphism
Epley, N., Waytz, A., & Cacioppo, J. T. (2007) · Psychological Review · doi:10.1037/0033-295x.114.4.864 · Cited by 4,069 · cited 1× · see citing works
“Anthropomorphism refers to whether people perceive the term describing an entity as possessing human-like characteristics [15, 21].”
BackgroundA large annotated corpus for learning natural language inference
Bowman, S. R., Angeli, G., Potts, C., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1075 · Cited by 3,377 · see citing works
Algorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232 · see citing works
Bartneck, C., Kulić, D., Croft, E., & Zoghbi, S. (2009) · International Journal of Social Robotics · doi:10.1007/s12369-008-0001-3 · Cited by 3,193 · see citing works
A Meta-Analysis of Cronbach's Coefficient Alpha
Peterson, R. A. (1994) · Journal of Consumer Research · doi:10.1086/209405 · Cited by 3,028 · see citing works
Human trust in artificial intelligence: Review of empirical research
Glikson, E., & Woolley, A. W. (2020) · Academy of Management Annals · doi:10.5465/annals.2018.0057 · Cited by 2,921 · see citing works
Trust in Automation: Integrating Empirical Evidence on Factors That Influence Trust
Hoff, K. A., & Bashir, M. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814547570 · Cited by 2,817 · see citing works
The reliability of a two-item scale: Pearson, Cronbach, or spearman-brown?
Eisinga, R., Grotenhuis, M. T., & Pelzer, B. (2013) · International Journal of Public Health · doi:10.1007/s00038-012-0416-3 · Cited by 2,695 · see citing works
Aguinis, H., & Bradley, K. J. (2014) · Organizational Research Methods · doi:10.1177/1094428114547952 · Cited by 2,047 · see citing works
Resistance to Medical Artificial Intelligence
Longoni, C., Bonezzi, A., & Morewedge, C. K. (2019) · Journal of Consumer Research · doi:10.1093/jcr/ucz013 · Cited by 2,015 · see citing works
(2000) · Cited by 1,733 · see citing works
Task-Dependent Algorithm Aversion
Castelo, N., Bos, M. W., & Lehmann, D. R. (2019) · Springer eBooks · doi:10.1007/978-3-031-88067-4_21 · Cited by 1,596 · see citing works
SemEval-2017 Task 1: Semantic Textual Similarity Multilingual and Crosslingual Focused Evaluation
Cer, D., Diab, M., Agirre, E., Lopez-Gazpio, I., & Specia, L. (2017) · Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017) · doi:10.18653/v1/s17-2001 · Cited by 1,562 · see citing works
Task-Dependent Algorithm Aversion
Castelo, N., Bos, M. W., & Lehmann, D. R. (2019) · Journal of Marketing Research · doi:10.1177/0022243719851788 · Cited by 1,500
Lee, M. K. (2018) · Big Data & Society · doi:10.1177/2053951718756684 · Cited by 1,338 · see citing works
Cer, D., Yang, Y., Kong, S.-Y., Hua, N., Limtiaco, N., John, R. S., Constant, N., Guajardo-Cespedes, M., Yuan, S., Tar, C., et al. (2018) · arXiv · doi:10.48550/arxiv.1803.11175 · Cited by 1,267 · see citing works
Araujo, T. (2018) · Computers in Human Behavior · doi:10.1016/j.chb.2018.03.051 · Cited by 1,156 · see citing works
Experimental Vignette Studies in Survey Research
Atzmüller, C., & Steiner, P. M. (2010) · Methodology · doi:10.1027/1614-2241/a000014 · Cited by 1,049 · see citing works
Franke, T., Attig, C., & Wessel, D. (2019) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2018.1456150 · Cited by 828 · see citing works
'It's Reducing a Human Being to a Percentage': Perceptions of Justice in Algorithmic Decisions
Binns, R., Van Kleek, M., Veale, M., Lyngs, U., Zhao, J., & Shadbolt, N. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173951 · Cited by 691 · see citing works
Li, J. (2015) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2015.01.001 · Cited by 652 · see citing works
Human–Autonomy Teaming: A Review and Analysis of the Empirical Literature
O’Neill, T., McNeese, N. J., Barron, A. H., & Schelble, B. G. (2020) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720820960865 · Cited by 623 · see citing works
Almost human: Anthropomorphism increases trust resilience in cognitive agents.
de Visser, E. J., Monfort, S. S., McKendrick, R., Smith, M. A. B., McKnight, P. E., Krueger, F., & Parasuraman, R. (2016) · Journal of Experimental Psychology: Applied · doi:10.1037/xap0000092 · Cited by 614 · see citing works
Citizen Science Terminology Matters: Exploring Key Terms
Eitzel, M. V., Cappadonna, J. L., Santos-Lang, C., Duerr, R. E., Virapongse, A., West, S., Kyba, C. C. M., Bowser, A., Cooper, C. B., Sforzi, A., et al. (2017) · Citizen Science Theory and Practice · doi:10.5334/cstp.96 · Cited by 605 · see citing works
Newman, D. T., Fast, N. J., & Harmon, D. J. (2020) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2020.03.008 · Cited by 550 · cited 1× · see citing works
“If the term artificial intelligence would lead to a less favorable evaluation of using ADM systems to evaluate people and make decisions over people’s careers (as found in previous work; [36, 39, 48]), this can lead to stronger public outcry and potentially even protests than the idea of algorithms doing the same thin…”
BackgroundFormalizing Trust in Artificial Intelligence: Prerequisites, Causes and Goals of Human Trust in AI
Jacovi, A., Marasović, A., Miller, T., & Goldberg, Y. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445923 · Cited by 545 · see citing works
Lee, K. M., Jung, Y., Kim, J., & Kim, S. R. (2006) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2006.05.002 · Cited by 510 · see citing works
Gell‐Mann, M. (2002) · Contributions to economics · doi:10.1007/978-3-642-50007-7_2 · Cited by 460 · see citing works
Algorithm Aversion: People Erroneously Avoid Algorithms after Seeing Them ErrPreprint
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2014) · SSRN Electronic Journal · doi:10.2139/ssrn.2466040 · Cited by 356 · see citing works
Message Framing and Persuasion: A Message Processing Analysis
Smith, S. M., & Petty, R. E. (1996) · Personality and Social Psychology Bulletin · doi:10.1177/0146167296223004 · Cited by 341 · see citing works
Langer, M., & Landers, R. N. (2021) · Computers in Human Behavior · doi:10.1016/j.chb.2021.106878 · Cited by 338 · see citing works
Self-citationde Graaf, M. M. A., Allouch, S. B., & Klamer, T. (2015) · Computers in Human Behavior · doi:10.1016/j.chb.2014.10.030 · Cited by 299 · see citing works
Motivating or stigmatizing? Public perceptions of weight-related language used by health providers
Puhl, R. M., Peterson, J. L., & Luedicke, J. (2012) · International Journal of Obesity · doi:10.1038/ijo.2012.110 · Cited by 287 · see citing works
Dietvorst, B. J., & Bharti, S. (2020) · Psychological Science · doi:10.1177/0956797620948841 · Cited by 275 · see citing works
Keel, S., Lee, P. Y., Scheetz, J., Li, Z., Kotowicz, M. A., MacIsaac, R. J., & He, M. (2018) · Scientific Reports · doi:10.1038/s41598-018-22612-2 · Cited by 220 · see citing works
Wang, R., Harper, F. M., & Zhu, H. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376813 · Cited by 211 · see citing works
Grgic-Hlaca, N., Redmiles, E. M., Gummadi, K. P., & Weller, A. (2018) · Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW '18 · doi:10.1145/3178876.3186138 · Cited by 207 · see citing works
Highly automated job interviews: Acceptance under the influence of stakes
Langer, M., König, C. J., & Papathanasiou, M. (2019) · International Journal of Selection and Assessment · doi:10.1111/ijsa.12246 · Cited by 203 · see citing works
Self-citationFactors influencing decision support system acceptance
Shibl, R., Lawley, M., & Debuse, J. C. W. (2012) · Decision Support Systems · doi:10.1016/j.dss.2012.09.018 · Cited by 194 · see citing works
Puhl, R. M. (2020) · Obesity Reviews · doi:10.1111/obr.13008 · Cited by 172 · see citing works
The impact of using algorithms for managerial decisions on public employees' procedural justice
Nagtegaal, R. (2021) · Government Information Quarterly · doi:10.1016/j.giq.2020.101536 · Cited by 166 · see citing works
Visualizing recommendations to support exploration, transparency and controllability
Verbert, K., Parra, D., Brusilovsky, P. L., et al. (2013) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/2449396.2449442 · Cited by 163 · see citing works
Attributions of morality and mind to artificial intelligence after real-world moral violations
Shank, D. B., & DeSanti, A. (2018) · Computers in Human Behavior · doi:10.1016/j.chb.2018.05.014 · Cited by 160 · see citing works
Marcinkowski, F., Kieslich, K., Starke, C., & Lünich, M. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372867 · Cited by 159 · see citing works
Ötting, S. K., & Maier, G. W. (2018) · Computers in Human Behavior · doi:10.1016/j.chb.2018.07.022 · Cited by 157 · see citing works
Lee, M. K., & Rich, K. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445570 · Cited by 145 · see citing works
“Where’s the I-O?” Artificial Intelligence and Machine Learning in Talent Management Systems
Gonzalez, M., Capman, J. F., Oswald, F. L., Theys, E. R., & Tomczak, D. L. (2019) · Personnel Assessment and Decisions · doi:10.25035/pad.2019.03.005 · Cited by 121 · see citing works
Why do patients derogate physicians who use a computer-based diagnostic support system?
Shaffer, V. A., Probst, C. A., Merkle, E. C., Arkes, H. R., & Medow, M. A. (2013) · Medical Decision Making · doi:10.1177/0272989x12453501 · Cited by 115 · see citing works
Human Decision Making with Machine Assistance: An Experiment on Bailing and Jailing
Grgić-Hlača, N., Engel, C., & Gummadi, K. P. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359280 · Cited by 113 · see citing works
Self-citationHou, Y. T.-Y., & Jung, M. F. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479864 · Cited by 108 · see citing works
A note of caution regarding anthropomorphism in HCI agents
Culley, K. E., & Madhavan, P. (2013) · Computers in Human Behavior · doi:10.1016/j.chb.2012.11.023 · Cited by 95 · see citing works
Coefficient Alpha: An Engineer's Interpretation of Test Reliability
Allen, K., Reed, T., Terry, R., Murphy, T. J., & Stone, A. (2008) · Journal of Engineering Education · doi:10.1002/j.2168-9830.2008.tb00956.x · Cited by 84 · see citing works
Langer, M., König, C. J., & Hemsing, V. (2020) · Journal of Managerial Psychology · doi:10.1108/jmp-03-2019-0156 · Cited by 83 · see citing works
Self-citationNew Information Technology and Implicit Bias
Elsbach, K. D., & Stigliani, I. (2019) · Academy of Management Perspectives · doi:10.5465/amp.2017.0079 · Cited by 74 · see citing works
Empirically understanding trust in medical technology
Montague, E. N. H., Kleiner, B. M., & Winchester, W. W. (2009) · International Journal of Industrial Ergonomics · doi:10.1016/j.ergon.2009.01.004 · Cited by 64 · see citing works
Fairness and Decision-making in Collaborative Shift Scheduling Systems
Uhde, A., Schlicker, N., Wallach, D. P., & Hassenzahl, M. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376656 · Cited by 64 · see citing works
Implementation of an automated scheduling tool improves schedule quality and resident satisfaction
Howard, F. M., Gao, C. A., & Sankey, C. B. (2020) · PLoS ONE · doi:10.1371/journal.pone.0236952 · Cited by 35 · see citing works
Shank, D. B., Bowen, M., Burns, A., & Dew, M. (2021) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2021.100092 · Cited by 24 · see citing works
Dörfler, V. (2019) · Elsevier eBooks · doi:10.1016/b978-0-12-809324-5.23863-7 · Cited by 10 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- Niklas Luhmann. 2000. Familiarity, confdence, trust: Problems and alterna- tives. In Trust: Making and breaking cooperative relations, Diego Gambetta (Ed.). Department of Sociology, University of Oxford, Oxford, UK, 94-107.
- Tatsuya Nomura, Tomohiro Suzuki, Takayuki Kanda, and Kensuke Kato. 2006. Altered attitudes of people toward robots: Investigation through the Negative Attitudes toward Robots Scale. In Proceedings of the 2006 AAAI workshop on Human Implications of Human-Robot Interaction. 29-35.
Cited by
Sorted by citations
Wischnewski, M., Krämer, N., & Müller, E. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581197 · Cited by 146 · see citing works
Yurrita, M., Draws, T., Balayn, A., Murray-Rust, D., Tintarev, N., & Bozzon, A. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581161 · Cited by 73 · see citing works
Inie, N., Druga, S., Zukerman, P., & Bender, E. M. (2024) · The 2024 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3630106.3659040 · Cited by 39 · see citing works
What’s the Appeal? Perceptions of Review Processes for Algorithmic Decisions
Lyons, H., Wijenayake, S., Miller, T., & Velloso, E. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517606 · Cited by 27 · see citing works
- Authors
- Langer, M., Hunsicker, T., Feldkamp, T., König, C. J., & Grgić-Hlača, N.
- Affiliations
- Saarland University (DE)
- Max Planck Institute for Software Systems (DE)
- Venue
- CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2022-04-29
- Pages
- 1-28
- OpenAlex
- W4225105901
- Cited by
- 55
- References
- 64
- Citations by year
- 2026: 13 · 2025: 17 · 2024: 9 · 2023: 12 · 2022: 4
- FWCI
- 10.43
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- FreiDok plus (Universitätsbibliothek Freiburg) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Publications of the UdS (Saarland University) (cc-by)
- Funders
- Bundesministerium für Bildung und Forschung (BMBF) (16SV8640)
- Volkswagen Foundation (Az. 98513)
- Deutsche Forschungsgemeinschaft (389792660 as part of TRR248)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Psychology › Social Psychology › Human-Automation Interaction and Safety
- Keywords
- terminology, lay perceptions, trust in algorithms, perceived complexity, replicability, HCI research, AI terminology, algorithmic bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
