Are Algorithmic Decisions Legitimate? The Effect of Process and Outcomes on Perceptions of Legitimacy of AI Decisions
Open accessThe DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Abstract
Firms use algorithms to make important business decisions. To date, the algorithmic accountability literature has elided a fundamentally empirical question important to business ethics and management: Under what circumstances, if any, are algorithmic decision-making systems considered legitimate? The present study begins to answer this question. Using factorial vignette survey methodology, we explore the impact of decision importance, governance, outcomes, and data inputs on perceptions of the legitimacy of algorithmic decisions made by firms. We find that many of the procedural governance mechanisms in practice today, such as notices and impact statements, do not lead to algorithmic decisions being perceived as more legitimate in general, and, consistent with legitimacy theory, that algorithmic decisions with good outcomes are perceived as more legitimate than bad outcomes. Yet, robust governance, such as offering an appeal process, can create a legitimacy dividend for decisions with bad outcomes. However, when arbitrary or morally dubious factors are used to make decisions, most legitimacy dividends are erased. In other words, companies cannot overcome the legitimacy penalty of using arbitrary or morally dubious factors, such as race or the day of the week, with a good outcome or an appeal process for individuals. These findings add new perspectives to both the literature on legitimacy and policy discussions on algorithmic decision-making in firms.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Tyler, T. R. (2006) · doi:10.1515/9781400828609 · Cited by 5,156
The Black Box Society: The Secret Algorithms That Control Money and Information
Pasquale, F. (2015) · doi:10.4159/harvard.9780674736061 · Cited by 3,796
How the machine “thinks”: Understanding opacity in machine learning algorithms
Burrell, J. (2016) · Big Data & Society · doi:10.1177/2053951715622512 · Cited by 3,196 · cited 1×
“For example, Tufekci (2015) highlights the importance of governing “gatekeep-ing” algorithms; Calo (2017), Burrell (2016), O’Neil (2016), and Martin (2019) argue that “consequential” or “pivotal” decisions should be given the most attention.”
BackgroundBarocas, S., & Selbst, A. D. (2014) · doi:10.2139/ssrn.2477899 · Cited by 3,048 · cited 1×
“For algorithmic decisions, using race-based factors in ADM contributes to illegal, and seemingly not legitimate, decisions (Barocas & Selbst, 2016).”
MethodThe role of procedural justice and legitimacy in shaping public support for policing
Sunshine, J., & Tyler, T. R. (2003) · Law & Society Review · doi:10.1111/1540-5893.3703002 · Cited by 2,973 · cited 1×
“And in canonical studies of legal legitimacy, Tyler (2006/1990) and Sunshine and Tyler (2003) showed that popular perceptions of legitimacy hinges, at least in part, on the existence of procedural safeguards and the opportunity to be heard.”
BackgroundSemantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887
The Politics of Stakeholder Theory: Some Future Directions
Freeman, R. E. (1994) · Business Ethics Quarterly · doi:10.2307/3857340 · Cited by 2,801
Algorithms of Oppression: How Search Engines Reinforce Racism
Noble, S. U. (2020) · doi:10.2307/j.ctt1pwt9w5 · Cited by 2,709
The Double-Edge of Organizational Legitimation
Ashforth, B. E., & Gibbs, B. W. (1990) · Organization Science · doi:10.1287/orsc.1.2.177 · Cited by 2,228 · cited 1×
“Legitimate firms pursue “socially acceptable goals in a socially acceptable manner” (Ashforth & Gibbs, 1990, p. 177).”
BackgroundLodge, M., & Taber, C. S. (2013) · doi:10.1017/cbo9781139032490 · Cited by 1,281
The accuracy, fairness, and limits of predicting recidivism
Dressel, J., & Farid, H. (2018) · Science Advances · doi:10.1126/sciadv.aao5580 · Cited by 1,071 · cited 1×
“Studies have also shown that ADM’s predictive capabilities may be exaggerated (Dressel & Farid, 2018; Jung et al., 2017; Salganik et al., 2020).”
BackgroundIn AI we trust? Perceptions about automated decision-making by artificial intelligence
Araujo, T., Helberger, N., Kruikemeier, S., & de Vreese, C. H. (2020) · AI & SOCIETY · doi:10.1007/s00146-019-00931-w · Cited by 1,038
Ethical Implications and Accountability of Algorithms
Martin, K. (2019) · Journal of Business Ethics · doi:10.1007/s10551-018-3921-3 · Cited by 757
Self-citationSociety-in-the-loop: programming the algorithmic social contract
Rahwan, I. (2018) · Ethics and Information Technology · doi:10.1007/s10676-017-9430-8 · Cited by 553
Newman, D. T., Fast, N. J., & Harmon, D. J. (2020) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2020.03.008 · Cited by 550 · cited 4×
“And Newman et al. (2020) have shown that individuals perceive ADM used by human resources departments as reductionist and unfair.”
BackgroundMethodInfluentialGeneralizing from Survey Experiments Conducted on Mechanical Turk: A Replication Approach
Coppock, A. (2019) · Political Science Research and Methods · doi:10.1017/psrm.2018.10 · Cited by 515 · cited 4×
“…procedural governance to legitimize algorithmic decisions in the eyes of the public, lending credibility to arguments from critical scholars that current law needs to be more robust or, perhaps, disused in certain circumstances and for certain purposes all together (Pasquale, 2015, 2018, 2021).”
BackgroundResultInfluentialZwitter, A. (2014) · Big Data & Society · doi:10.1177/2053951714559253 · Cited by 465 · cited 1×
“This creates the circumstances for invasions of privacy and erosion of privacy norms (Zwitter, 2014).”
BackgroundAlgorithmic governance: Developing a research agenda through the power of collective intelligence
Danaher, J., Hogan, M. J., Noone, C., Kennedy, R., Behan, A., De Paor, A., Felzmann, H., Haklay, M., Khoo, S.-M., Morison, J., et al. (2017) · Big Data & Society · doi:10.1177/2053951717726554 · Cited by 369 · cited 2×
“For example, Danaher et al. (2017) examine the legitimacy of ADM governance mechanisms (rather than the legitimacy of the decision), and Lünich and Kieslich (2021) studied attributes of respondents (i.e. general trust) in their perception of legitimacy of a vaccine distribution algorithm.”
BackgroundThe Challenges of Algorithm-Based HR Decision-Making for Personal Integrity
Leicht-Deobald, U., Busch, T., Schank, C., Weibel, A., Schafheitle, S., Wildhaber, I., & Kasper, G. (2019) · Journal of Business Ethics · doi:10.1007/s10551-019-04204-w · Cited by 347 · cited 1×
“…and de Fine Licht (2020) theorize that providing justifications for algorithmic decisions could provide grounds for perceived legitimacy, whereas, Leicht-Deobald et al. (2019) suggest that algorithm-based decisions may be perceived as more legitimate because it is difficult for individuals to…”
Backgroundde Fine Licht, K., & de Fine Licht, J. (2020) · AI & SOCIETY · doi:10.1007/s00146-020-00960-w · Cited by 334 · cited 1×
“For example, de Fine Licht and de Fine Licht (2020) theorize that providing justifications for algorithmic decisions could provide grounds for perceived legitimacy, whereas, Leicht-Deobald et al. (2019) suggest that algorithm-based decisions may be perceived as more legitimate because it is…”
BackgroundMTurk Character Misrepresentation: Assessment and Solutions
Sharpe Wessling, K., Huber, J., & Netzer, O. (2017) · Journal of Consumer Research · doi:10.1093/jcr/ucx053 · Cited by 330 · cited 1×
“Pivotal ally representative samples; as such the critiques focus on questions of statistical generalizability (Kang et al., 2014; Sharpe Wessling et al., 2017).”
BackgroundMeasuring the predictability of life outcomes with a scientific mass collaboration
Salganik, M. J., Lundberg, I., Kindel, A. T., Ahearn, C. E., Al-Ghoneim, K., Almaatouq, A., Altschul, D. M., Brand, J. E., Carnegie, N. B., Compton, R. J., et al. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1915006117 · Cited by 322
Ethics of the Attention Economy: The Problem of Social Media Addiction
Bhargava, V. R., & Velasquez, M. (2021) · Business Ethics Quarterly · doi:10.1017/beq.2020.32 · Cited by 292 · cited 1×
“…services (Arthur & Owen, 2019) and HR (Leicht-Deobald, et al., 2019), and have identified specific moral implications such as social-media addiction (Bhargava & Velasquez, 2020), personalized pricing (Seele et al., 2019; Steinberg, 2020), gamification (Kim, 2018), and accountability (Martin, 2019).”
BackgroundArtificial Intelligence Policy: A Primer and Roadmap
Calo, R. (2018) · University of Bologna Law Review · doi:10.6092/issn.2531-6133/8670 · Cited by 197
Mapping the Ethicality of Algorithmic Pricing: A Review of Dynamic and Personalized Pricing
Seele, P., Dierksmeier, C., Hofstetter, R., & Schultz, M. D. (2021) · Journal of Business Ethics · doi:10.1007/s10551-019-04371-w · Cited by 196
The impact of using algorithms for managerial decisions on public employees' procedural justice
Nagtegaal, R. (2021) · Government Information Quarterly · doi:10.1016/j.giq.2020.101536 · Cited by 166 · cited 4×
“…procedural governance to legitimize algorithmic decisions in the eyes of the public, lending credibility to arguments from critical scholars that current law needs to be more robust or, perhaps, disused in certain circumstances and for certain purposes all together (Pasquale, 2015, 2018, 2021).”
BackgroundMethodInfluential"The Right to Explanation, Explained"
Kaminski, M. E. (2019) · btlj · doi:10.15779/z38td9n83h · Cited by 146 · cited 4×
“This is the approach taken by many current and proposed data protection laws, which require organizations to communicate to customers the “logic behind” its algorithms and/or complete impact assessments (Kaminski, 2019a, 2019b, 2019c).”
BackgroundInfluentialTransparency as design publicity: explaining and justifying inscrutable algorithms
Loi, M., Ferrario, A., & Viganò, E. (2021) · Ethics and Information Technology · doi:10.1007/s10676-020-09564-w · Cited by 79 · cited 3×
“We chose these options because they are the most commonly proposed governance mechanisms in the legal studies literature and because they sit on a range from more robust to lax, providing a proxy for the effect on legitimacy of different types of procedural governance (Citron, 2007; Froomkin et al., 2019; Jones, 2017;…”
BackgroundMethodThe right to a human in the loop: Political constructions of computer automation and personhood
Jones, M. L. (2017) · Social Studies of Science · doi:10.1177/0306312717699716 · Cited by 75 · cited 3×
“Scholars have also assumed that good governance through procedural guardrails could legitimize decisions made on arbitrary or morally dubious ground (Froomkin et al., 2019; Jones, 2017; Kaminski, 2019a, 2019b, 2019c; Rahwan, 2018).”
Background
Cited by
Responsible Artificial Intelligence Governance: A Review and Research Framework
Papagiannidis, E., Mikalef, P., & Conboy, K. (2025) · The Journal of Strategic Information Systems · doi:10.1016/j.jsis.2024.101885 · Cited by 565
Figueroa-Armijos, M., Clark, B. B., & da Motta Veiga, S. P. (2023) · Journal of Business Ethics · doi:10.1007/s10551-022-05166-2 · Cited by 164
“A few years later, Martin and Freeman (2004) proposed that the confluence of technology and ethics carries with it an inherently value-laden situational relationship, whose impact within and beyond organizations is of particular importance to business ethics scholars (Martin & Waldman, 2022).”
BackgroundHow AI can learn from the law: putting humans in the loop only on appeal
Cohen, I. G., Babic, B., Gerke, S., Xia, Q., Evgeniou, T., & Wertenbroch, K. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00906-8 · Cited by 51
“23 for a test of allowing users to appeal to an internal review board), and collecting such evidence would be challenging as such systems are in their nascency.”
BackgroundFan, Y., & Liu, X. (2022) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.1009173 · Cited by 40
Liu, R., & Han, Y. (2026) · Public Administration · doi:10.1111/padm.70043 · Cited by 2
Metadata
- Authors
- Martin, K., & Waldman, A.
- Venue
- Journal of Business Ethics
- Published
- 2022-02-10
- Volume
- 183
- Issue
- 3
- Pages
- 653-670
- ISSN
- 0167-4544 (ISSN-L)
- 1573-0697
- OpenAlex
- W4211094900
- Cited by
- 132
- References
- 29
- FWCI
- 21.04
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.