CoChat
Sign Up
Cited by 35Ethics and Social Impacts of AIInformation Systems Theories and ImplementationE-Government and Public Services

Bureaucracy as a Lens for Analyzing and Designing Algorithmic Systems

Open access
Pääkkönen, Juho, Nelimarkka, Matti, Haapoja, Jesse, Lampinen, Airi·2020·Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
DOI 10.1145/3313831.3376780
Open this paper in CoChat — free accountOpen PDF

Abstract

Scholarship on algorithms has drawn on the analogy between algorithmic systems and bureaucracies to diagnose shortcomings in algorithmic decision-making. We extend the analogy further by drawing on Michel Crozier's theory of bureaucratic organizations to analyze the relationship between algorithmic and human decision-making power. We present algorithms as analogous to impartial bureaucratic rules for controlling action, and argue that discretionary decision-making power in algorithmic systems accumulates at locations where uncertainty about the operation of algorithms persists. This key point of our essay connects with Alkhatib and Bernstein's theory of 'street-level algorithms', and highlights that the role of human discretion in algorithmic systems is to accommodate uncertain situations which inflexible algorithms cannot handle. We conclude by discussing how the analysis and design of algorithmic systems could seek to identify and cultivate important sources of uncertainty, to enable the human discretionary work that enhances systemic resilience in the face of algorithmic errors.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3313831.3376780

Cites

71 of 71 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Whatever the outcome type, public-sector street-level bureaucrats must still use discretion, contextualize algorithmic results for each case, and act within bureaucratic constraints as ADMAPS describes.

    “Moreover, regardless of the type of outcome, street-level bureaucrats in the public sector must still exercise discretion and contextualize the algorithmic results for each case and act within the constraints posed by bureaucratic processes as explained by the ADMAPS framework [2,62,91].”

    From A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfare (2021) · cites this paper as [91]

    The Case 1 section says qualified human decision-makers can interpret algorithmic outputs when algorithms support decisions. But Case 2 says that when algorithms automate decisions, interpretation of outcomes can no longer be controlled by discretion; discretion shifts toward those maintaining the system. The paper therefore does not support the claim that street-level bureaucrats must contextualize results for each case regardless of outcome type. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • The actions and potential inability of system designers and software developers to take into account the requirements of administrative due process may have a considerable impact on the outcomes of ADM.

    “because their actions and potential inability to take into account the requirements of administrative due process may have a considerable impact on the outcomes of ADM (Pääkkönen et al. 2020; see also Smith et al. 2010, 3; Automaattiseen päätöksentekoon liittyvät… 2020, 11)”

    From Administrative due process when using automated decision-making in public administration: some notes from a Finnish perspective (2021) · cites this paper as Pääkkönen et al. 2020

    In “Case 2: Automated housing allocation for the homeless” (page unavailable), the paper says that people who design or maintain algorithmic technologies can control uncertainties in implementation and that their discretionary work can contribute to system dysfunctions. It does not link those effects specifically to designers’ or developers’ failure to account for administrative due-process requirements. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Pääkkönen et al. argue that algorithmic system design must identify and cultivate uncertainty sources because human discretion is needed where algorithm operation remains ambiguous.

    “Pääkkönen et al. [91] further extend this argument to state that the design of algorithmic systems must identify and cultivate important sources of uncertainty because it is at these sources where the need for human discretion accumulates since ambiguity about the operation of the algorithm persists.”

    From A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfare (2021) · cites this paper as [91]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 9

  1. [6]6. Jane Bambauer and Tal Zarsky. 2018. The Algorithm Game. 94 Notre Dame Law Review 1; Arizona Legal Studies Discussion Paper No. 18-09 (2018). https://ssrn.com/abstract=3135949
  2. [7]7. Jens Beckert. 2016. Imagined Futures. Harvard University Press. Google-Books-ID: BhSSjgEACAAJ.
  3. [8]8. David Beer. 2016. Metric Power. Palgrave Macmillan Limited, London, UNITED KINGDOM.
  4. [10]10. Geoffrey C. Bowker and Susan Leigh Star. 1999. Sorting things out: Classification and its consequences. MIT Press, Cambridge, Mass. http://www.worldcat.org/title/ sorting-things-out-classification-and-its-consequences/ oclc/41273009&referer=brief_results
  5. [40]40. Eugene C Kim. 2007. YouTube: Testing the safe harbors of digital copyright law. S. Cal. Interdisc. LJ 17 (2007), 139.
  6. [52]52. Helen Nissenbaum. 2005. Values in Technical Design. In Encyclopedia of Science, Technology, and Ethics, Carl Mitcham (Ed.). MacMillan, New York, lxvi-lxx.
  7. [59]59. Luis Ignacio Reyes-Galindo. 2016. Automating the Horae: Boundary-work in the age of computers. Social Studies of Science (2016). https://arxiv.org/pdf/1603.03824.pdf · arXiv
  8. [62]62. Herbert A Simon. 1972. Theories of bounded rationality. Decision and organization 1, 1 (1972), 161-176.
  9. [71]71. Max Weber. 2013. Bureacracy. Routledge, 196-244.

Cited by

Sorted by citations

Related

Authors
Pääkkönen, J., Nelimarkka, M., Haapoja, J., & Lampinen, A.
Affiliations
  • University of Helsinki (FI)
  • Stockholm University (SE)
ORCID
Venue
Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Published
2020-04-21
Pages
1-14
DOI
10.1145/3313831.3376780
OpenAlex
W3029690672
Cited by
35
References
71
Citations by year
2026: 4 · 2025: 5 · 2024: 5 · 2023: 6 · 2022: 7 · 2021: 6 · 2020: 2
FWCI
5.45
Type
conference paper
Language
en
Open access
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations
  • CHI Conference on Human Factors in Computing Systems (publishedVersion)
  • Aaltodoc (Aalto University) (submittedVersion, other-oa)
  • Työväentutkimus Vuosikirja (submittedVersion, other-oa)
Funders
Koneen Säätiö (Project Algorithmic systems, power, and interaction)
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Social Sciences › Sociology and Political Science › Information Systems Theories and Implementation
  • Social Sciences › Political Science and International Relations › E-Government and Public Services
Keywords
algorithmic decision-making, bureaucracy, Michel Crozier, bureaucratic organization, discretionary decision-making, human discretion, algorithmic uncertainty, algorithmic errors, systemic resilience, algorithmic governance

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review