Holding AI Accountable Like Herding Cats: The Contingent Impact on the Legitimacy of Algorithmic Bureaucracy

May combine two claims.

People are more likely to accept and trust AI systems when they produce desirable results but become skeptical when outcomes are unfavorable.

“It aligns with the theoretical rationale on algorithm aversion (Dietvorst et al. 2015;Hohensinn et al. 2024) and appreciation (Logg et al. 2019), which suggests that people are more likely to accept and trust AI systems when they produce desirable results but become skeptical when outcomes are unfavorable.”

• cites Dietvorst et al. 2015 sourcePartly supported

  The General Discussion (p. 10) reports that seeing an algorithm err reduced confidence in it and willingness to choose it, supporting the skepticism part of the claim. It does not establish that desirable results increase acceptance: participants became less confident even when they saw the algorithm outperform a human forecaster. (LLM, conf=0.86)

  Experimental — accuracy not yet measured
• cites Hohensinn et al. 2024 sourceNot checked — no full text of the source
• cites Logg et al. 2019 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

AI's opacity, proprietary design, and technical complexity exacerbate information asymmetries.

“Busuioc (2021) further emphasizes that AI's opacity, proprietary design, and technical complexity exacerbate information asymmetries.”

• cites Busuioc 2021 sourceSupported

May combine two claims.

AI agents act autonomously, creating risks of agency drift.

“Like human bureaucrats, AI agents act autonomously, creating risks of agency drift (Lechterman 2022).”

• cites Lechterman 2022 sourceSupported

Classified as probably not a substantial claim.

AI expertise and accountability are the twin foundations of algorithmic bureaucracy.

“As we have illustrated for the first two hypotheses, AI expertise and accountability are the twin foundations of algorithmic bureaucracy (Busuioc 2021;Roehl and Crompvoets 2023;Vogl et al. 2020).”

• cites Busuioc 2021 sourcePartly supported

  The Conclusion (page not provided) calls bureaucratic expertise and accountability the twin foundations of bureaucratic legitimacy. It does not call AI expertise a foundation of algorithmic bureaucracy; that substitution changes both the kind of expertise and what the foundations underpin. (LLM, conf=0.86)

  Experimental — accuracy not yet measured
• cites Roehl and Crompvoets 2023Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Vogl et al. 2020 sourceNot checked — no full text of the source

May combine two claims.

Under positive outcomes, AfC does not significantly improve legitimacy (p = 0.056), and AfR produces only a modest increase compared to no accountability (coef.= 0.067, p = 0.002).

“Regression results (Table 4) show that under positive outcomes, AfC does not significantly improve legitimacy (p = 0.056), and AfR produces only a modest increase compared to no accountability (coef.= 0.067, p = 0.002).”

May combine two claims.

Highly effective AI systems significantly increase perceived legitimacy when outcomes are positive (coef.= 0.25, p < 0.001), providing partial support for Hypothesis 1.

“Regression results (Table 2) show that highly effective AI systems significantly increase perceived legitimacy when outcomes are positive (coef.= 0.25, p < 0.001), providing partial support for Hypothesis 1.”

May combine two claims.

Group 15 had the highest legitimacy mean (1.793), combining high effectiveness, AfR, and a positive outcome.

“By contrast, Group 15 is with the highest legitimacy mean (1.793), which combined high effectiveness, AfR, and a positive outcome.”

May combine two claims.

Group 4 had the lowest legitimacy mean (0.78), combining low effectiveness, no accountability, and a negative outcome.

“Group 4 has the lowest legitimacy mean (0.78), which combined low effectiveness, no accountability, and a negative outcome.”

May combine two claims.

The positive effect of effectiveness was significant only under positive outcomes (p < 0.001), but not under negative ones (p = 0.227).

“However, when broken down by decision outcome, the positive effect of effectiveness was significant only under positive outcomes (p < 0.001), but not under negative ones (p = 0.227).”

May combine two claims.

The interaction terms AfC × Negative (p < 0.001) and AfR × Negative (p = 0.003) are both positive and significant, indicating that accountability arrangements have stronger legitimating effects when AI decisions produce negative outcomes.

“By contrast, the interaction terms AfC × Negative (p < 0.001) and AfR × Negative (p = 0.003) are both positive and significant, indicating that accountability arrangements have stronger legitimating effects when AI decisions produce negative outcomes.”

May combine two claims.

The combination of high effectiveness and Accountability for Results does not differ significantly from the combination of high effectiveness and Accountability for Compliance in legitimacy, in either the positive (mean difference = 0.003, p = 1.000) or the negative (mean difference = -0.010, p = 1.000) decision-outcome condition.

“(High_Efft×AfR) vs. | 0.003 | 0.024 | 1.000 | -0.010 | 0.079 | 1.000 |”

May combine two claims.

Under negative outcomes, both AfC and AfR significantly improved legitimacy compared to no accountability (mean differences = 0.500 and 0.365, respectively; both p < 0.01).

“By contrast, under negative outcomes, both AfC and AfR significantly improved legitimacy compared to no accountability (mean differences = 0.500 and 0.365, respectively; both p < 0.01).”

May combine two claims.

In Model 1 (positive outcomes), neither AfC × Effectiveness nor AfR × Effectiveness interactions are statistically significant.

“In Model 1 (positive outcomes), neither AfC × Effectiveness nor AfR × Effectiveness interactions are statistically significant, indicating that combining accountability mechanisms with effectiveness does not further increase legitimacy under favorable conditions.”

May combine two claims. The quote only partly states this claim.

The interaction between Accountability for Compliance (AfC) and effectiveness is not statistically significant for legitimacy in either the positive (β = -0.0759) or the negative (β = 0.173) decision-outcome condition.

“AfC × Efft | -0.0759 | 0.173”

May combine two claims.

AfC increased legitimacy by 0.27 points (p < 0.001) and AfR by 0.22 points (p < 0.001), with no significant difference between the two (p = 0.721).

“AfC increased legitimacy by 0.27 points (p < 0.001) and AfR by 0.22 points (p < 0.001), with no significant difference between the two (p = 0.721).”

Classified as probably not a substantial claim. May combine two claims.

The platform facilitated random assignment, ensuring participants were evenly distributed across the 18 experimental conditions.

“The platform facilitated random assignment, ensuring that participants were evenly distributed across the 18 experimental conditions.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The log transformation normalized the distribution of the composite legitimacy index and enhanced its suitability for OLS regression models.

“The log transformation normalized the distribution and enhanced the suitability of the index for OLS regression models.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Credamo has over two million registered users and has been widely used to obtain reliable and representative samples.

“While the platform has faced some criticisms, it boasts over two million registered users and has been widely used to obtain reliable and representative samples.”

Probably not a substantial claim: may not pass S2 (checkable).

Realistic and high-stakes settings help ensure manipulations are plausible and psychologically engaging.

“Following best practices in vignette design, realistic and high-stakes settings help ensure manipulations are both plausible and psychologically engaging (Mutz 2011).”

• cites Mutz 2011 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

Many recent public administration studies have employed Credamo for survey experiments.

“Many recent PA studies have employed Credamo for survey experiments (Deng and Sun 2025;Tao et al. 2024;Wang and Zhang 2024;Zhang et al. 2025).”

• cites Deng and Sun 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Tao et al. 2024Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Wang and Zhang 2024 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

The legitimacy items demonstrated strong internal consistency (Cronbach's alpha = 0.95).

“The items demonstrate strong internal consistency (Cronbach's alpha = 0.95).”

May combine two claims.

Separating experiments helps avoid effect size attenuation and preserves statistical power by concentrating sample size on theoretically central comparisons.

“Separating experiments helps avoid effect size attenuation, a critical concern for Type II error management, and also allows sample size to be concentrated on theoretically central comparisons instead of dispersed across redundant cells, thereby actively preserving statistical power (Cohen 1988;Aron et al. 2009).”

• cites Cohen 1988Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Aron et al. 2009Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

The study set the minimum per-group sample at 58 participants, the most conservative estimate.

“To ensure robustness, we adopted the most conservative estimate, setting the minimum per-group sample at 58 participants.”

Classified as probably not a substantial claim.

These findings support Hypothesis 3.

“These findings support our Hypothesis 3.”

May combine two claims.

Unlike human agents, algorithms lack self-interest or corrupt motives, reducing the likelihood of agent drift and making forum dynamics more critical.

“Unlike human agents, algorithms lack self-interest or corrupt motives (Heidelberg 2024), reducing the likelihood of agent drift and making forum dynamics more critical.”

• cites Heidelberg 2024 sourceNot checked — no full text of the source

Classified as probably not a substantial claim.

Citizens are likely to integrate decision outcomes into their legitimacy judgments.

“In this context, citizens are likely to integrate decision outcomes into their legitimacy judgments.”

May combine two claims.

Negative outcomes invite accountability demands, whereas favorable outcomes may discourage forums from challenging decisions, even if flawed.

“Negative outcomes invite accountability demands, whereas favorable outcomes may discourage forums from challenging decisions, even if flawed (Brandsma and Schillemans 2012;Busuioc 2013).”

• cites Brandsma and Schillemans 2012Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Busuioc 2013 sourceNot checked — no full text of the source

May combine two claims.

Negative outcomes weaken the legitimacy benefits of AI effectiveness, while positive outcomes allow performance cues to translate more directly into legitimacy perceptions.

“Negative outcomes thus weaken the legitimacy benefits of effectiveness, while positive outcomes allow performance cues to translate more directly into legitimacy perceptions.”

May combine two claims.

Negative outcomes amplify algorithm aversion and reduce the incremental effect of AI effectiveness on legitimacy, whereas positive outcomes foster algorithm appreciation and strengthen this effect.

“Collectively, these insights suggest that negative outcomes amplify algorithm aversion and reduce the incremental effect of AI effectiveness on legitimacy, whereas positive outcomes foster algorithm appreciation and strengthen this effect.”

Probably not a substantial claim: may not pass S4 (based).

A highly accountable AI system that lacks effectiveness may fail to gain public confidence due to subpar performance.

“An effective AI system without proper accountability may lead to skepticism regarding its decision-making processes, while a highly accountable system that lacks effectiveness may fail to gain public confidence due to subpar performance.”

May combine two claims.

Positive outcomes typically meet or exceed expectations, leading citizens to attribute performance success to the AI system and evaluate it as legitimate.

“The expectancy-disconfirmation model (Chen et al. 2022;Van Ryzin 2006) lends additional support: positive outcomes typically meet or exceed expectations, leading citizens to attribute performance success to the AI system and evaluate it as legitimate.”

• cites Chen et al. 2022 sourceNot checked — no full text of the source
• cites Van Ryzin 2006 sourceNot checked — no full text of the source

May combine two claims.

Negative outcomes invite causal search and blame attribution, shifting attention toward potential shortcomings in technical capacity and dampening the marginal benefits of perceived effectiveness.

“Negative outcomes, however, invite causal search and blame attribution, shifting attention toward potential shortcomings in technical capacity and dampening the marginal benefits of perceived effectiveness.”

May combine two claims.

Hypothesis 4 is not supported by the interaction test, but the pattern of simple effects suggests that decision outcomes condition the extent to which effectiveness translates into perceived legitimacy.

“Second, Hypothesis 4 is not supported by the interaction test, but the pattern of simple effects suggests that decision outcomes condition the extent to which effectiveness translates into perceived legitimacy.”

May combine two claims.

Randomization was successfully implemented, ensuring the validity of the experimental design.

“These findings confirm that randomization was successfully implemented, ensuring the validity of the experimental design.”

• Also stated in: § 6 | Research Limitation · hedged

  “While the platform ensures random assignment across experimental conditions, online samples may not fully capture the diversity of the broader population.”

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

The three experiments form an integrated and theoretically progressive design that isolates core mechanisms, preserves statistical power, and enhances the rigor and interpretability of tests of algorithmic legitimacy.

“Overall, the three experiments form an integrated and theoretically progressive design that isolates core mechanisms, preserves statistical power, and enhances the rigor and interpretability of tests of algorithmic legitimacy.”

Classified as probably not a substantial claim. May combine two claims.

After quality control, 1135 valid responses were retained, averaging approximately 63 participants per group.

“After applying all quality control measures (Section A1), we retained 1135 valid responses, averaging approximately 63 participants per group.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

The sample had a higher proportion of young and educated participants, which reflects real-world housing dynamics because these groups face greater rental housing demand and are primary beneficiaries of government housing support policies.

“Notably, the sample had a higher proportion of young and educated participants, which reflects real-world housing dynamics, as these groups face greater rental housing demand and are primary beneficiaries of government housing support policies.”

Classified as probably not a substantial claim.

Accountability arrangements are particularly effective in enhancing legitimacy perceptions when AI decisions lead to negative rather than positive outcomes for citizens.

“The studies above suggested accountability arrangements are particularly effective in enhancing legitimacy perceptions when AI decisions lead to negative outcomes for citizens.”

• cites Brummel and de Blok 2024 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

The quality control procedures enhance the internal validity of the findings by ensuring that participants meaningfully engaged with the experimental stimuli.

“We believe these procedures enhance the internal validity of our findings by ensuring that participants meaningfully engaged with the experimental stimuli.”

Classified as probably not a substantial claim. May combine two claims.

For a public decision to be perceived as legitimate, acceptability of the decision result and fairness of the decision-making process are essential.

“Together, these frameworks suggest that for a public decision to be perceived as legitimate, two elements are essential: the acceptability of the decision result (acceptance perception) and the fairness of the decision-making process (fairness perception).”

Classified as probably not a substantial claim.

Decision outcomes may moderate the impact of accountability on legitimacy perceptions.

“This suggests that decision outcomes may moderate the impact of accountability on legitimacy perceptions.”

May combine two claims.

The power analysis indicated minimum per-group sample requirements of 45 participants for the 3 × 2 design, 58 participants for the 2 × 2 design, and 28 participants for the 3 × 2 × 2 design.

“The analysis indicated the following minimum per-group sample requirements: 45 participants for the 3 × 2 design, 58 participants for the 2 × 2 design, and 28 participants for the 3 × 2 × 2 design.”

May combine two claims.

Accountability is essential for holding administrative expertise under control, reinforcing its legitimacy.

“Accountability is therefore essential for holding administrative expertise under control, which reinforcing its legitimacy (Bovens 2007;Olsen 2013).”

• cites Bovens 2007 sourceNot checked — no full text of the source
• cites Olsen 2013 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S4 (based).

An effective AI system without proper accountability may lead to skepticism regarding its decision-making processes.

“An effective AI system without proper accountability may lead to skepticism regarding its decision-making processes, while a highly accountable system that lacks effectiveness may fail to gain public confidence due to subpar performance.”

May combine two claims.

Negative outcomes create a legitimacy deficit, and AfR provides assurance that adverse results are accurate, reviewable, and therefore justifiable.

“Negative outcomes create a legitimacy deficit, and AfR provides assurance that adverse results are accurate, reviewable, and therefore justifiable (Licht et al. 2022).”

• cites Licht et al. 2022 sourceNot checked — no full text of the source

Classified as probably not a substantial claim.

Procedural fairness can mitigate negative evaluations of ineffective authorities by reassuring citizens that decisions are rule-based and impartial.

“As Tyler (2006b) argues, procedural fairness can mitigate negative evaluations of ineffective authorities by reassuring citizens that decisions are rule-based and impartial.”

• cites Tyler (2006b) sourceNot checked — no full text of the source

May combine two claims.

AfR operates as a general assurance mechanism, particularly salient when negative outcomes must be justified, whereas AfC becomes especially important when outcomes are unfavorable and citizens scrutinize procedural fairness.

“AfR operates as a general assurance mechanism, particularly salient when negative outcomes must be justified, whereas AfC becomes especially important when outcomes are unfavorable and citizens scrutinize procedural fairness.”

May combine two claims.

The findings are context-specific and most directly applicable to welfare policies, particularly in the Chinese context.

“Accordingly, our findings should be interpreted as most directly applicable to welfare policies, particularly in the Chinese context.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Citizens' perceptions of algorithmic legitimacy may differ across policy domains with different stakes, outcome structures, and accountability concerns.

“Citizens' perceptions of algorithmic legitimacy may differ across policy domains with different stakes, outcome structures, and accountability concerns.”

May combine two claims.

AfR enhances legitimacy across both positive and negative outcomes, reflecting the value citizens place on performance assurances regardless of valence.

“AfR enhances legitimacy across both positive and negative outcomes, reflecting the value citizens place on performance assurances regardless of valence.”

May combine two claims.

When decisions are favorable, citizens focus on outcomes and pay less attention to procedural safeguards, but when outcomes are unfavorable, scrutiny of processes and fairness mechanisms intensifies.

“When decisions are favorable, citizens focus on outcomes and pay less attention to procedural safeguards, but when outcomes are unfavorable, scrutiny of processes and fairness mechanisms intensifies.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Online samples may not fully capture the diversity of the broader population.

“While the platform ensures random assignment across experimental conditions, online samples may not fully capture the diversity of the broader population.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Forum drifting can take several forms.

“As Schillemans and Busuioc (2014)suggest, forum drifting can take several forms, of which our study only addressed two.”

• cites Schillemans and Busuioc (2014)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

Using data from three survey experiments involving 1135 participants in China, accountability is most critical when AI decisions lead to negative outcomes, while effectiveness plays a greater role when AI decisions result in positive outcomes.

“Using data from three survey experiments involving 1135 participants in China, we find that accountability is most critical when AI decisions lead to negative outcomes, while effectiveness plays a greater role when AI decisions result in positive outcomes.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Respondents' decision-making in an online setting may not fully reflect behaviors in real-world governance contexts.

“Additionally, respondents' decision-making in an online setting may not fully reflect behaviors in real-world governance contexts.”

May combine two claims. The quote only partly states this claim.

AfR operates as damage control, particularly valuable when negative outcomes must be justified, but adds little when AI is effective and results are favorable.

“AfR operates as”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

Legitimacy is a concept that is inherently difficult to measure directly.

“Fourth, the outcome variable in this study is legitimacy, a concept that is inherently difficult to measure directly.”

May combine two claims.

Positive outcomes shift attention toward performance, while negative outcomes heighten sensitivity to institutional safeguards.

“Positive outcomes shift attention toward performance, while negative outcomes heighten sensitivity to institutional safeguards.”

May combine two claims.

Effectiveness functions as the primary source of legitimacy when algorithmic decisions yield favorable outcomes, whereas accountability becomes central when decisions impose losses on citizens.

“Specifically, effectiveness functions as the primary source of legitimacy when algorithmic decisions yield favorable outcomes, whereas accountability becomes central when decisions impose losses on citizens.”