CoChat
Sign Up
Cited by 207Ethics and Social Impacts of AIE-Government and Public ServicesExplainable Artificial Intelligence (XAI)

Toward Algorithmic Accountability in Public Services: A Qualitative Study of Affected Community Perspectives on Algorithmic Decision-making in Child Welfare Services

Open access
Brown, Anna, Chouldechova, Alexandra, Putnam-Hornstein, Emily, Tobin, Andrew, Vaithianathan, Rhema·2019·Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems
DOI 10.1145/3290605.3300271
Open this paper in CoChat — free accountOpen PDF

Abstract

Algorithmic decision-making systems are increasingly being adopted by government public service agencies. Researchers, policy experts, and civil rights groups have all voiced concerns that such systems are being deployed without adequate consideration of potential harms, disparate impacts, and public accountability practices. Yet little is known about the concerns of those most likely to be affected by these systems. We report on workshops conducted to learn about the concerns of affected communities in the context of child welfare services. The workshops involved 83 study participants including families involved in the child welfare system, employees of child welfare agencies, and service providers. Our findings indicate that general distrust in the existing system contributes significantly to low comfort in algorithmic decision-making. We identify strategies for improving comfort through greater transparency and improved communication strategies. We discuss the implications of our study for accountable algorithm design for child welfare applications.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3290605.3300271

Cites

34 of 34 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Co-design workshops with child-welfare stakeholders found algorithms largely increased distrust, perpetuated bias, and created black-box systems, raising concerns about negative effects on caseworkers' decisions.

    “Brown et al. [20] conducted co-design workshops with stakeholders within the CW community (e.g., families, frontline providers, and specialists) and found that such algorithms largely bolstered distrust, perpetuated bias, and created black-boxed systems, which accelerated concerns about how these tools may negatively impact child-welfare workers' decisions.”

    From A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfare (2021) · cites this paper as Brown et al. [20]

    The Participants and Findings sections describe workshops with families, frontline providers, and specialists, including concerns about bias and effects on caseworker judgment. The Address system-level concerns section says existing distrust was projected onto algorithmic scenarios, while the section on how the algorithm weighs factors describes discomfort with the tool’s black-box presentation. The paper reports these concerns, not findings that algorithms themselves largely increased distrust, perpetuated bias, or created black-box systems. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • Prior studies have found that caseworkers distrust risk assessment algorithms such as CANS because those tools are deficit-based.

    “Prior studies have found that caseworkers distrust risk assessment algorithms such as CANS because of their deficit-based nature [20].”

    From A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfare (2021) · cites this paper as [20]

    In “4 FINDINGS” (page not provided), frontline providers express concern that child-welfare approaches emphasize deficits and risks; one calls a scenario a “deficit model.” The paper does not identify CANS or establish that caseworkers distrust that tool because it is deficit-based. Its findings concern participants’ responses to child-welfare decision-making scenarios. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • No technological solution can be inherently fair or just; complementary policy changes are required to affect community perceptions.

    “Thus, any technological solution cannot be inherently deemed 'fair' or 'just' and require complementary policy changes to affect community perceptions [20].”

    From A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfare (2021) · cites this paper as [20]

  • This deficit-based focus diverts attention from strength-based frameworks.

    “This is driving attention away from strength-based frameworks [9,20,99,107].”

    From A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfare (2021) · cites this paper as [20]

  • Differing socio-technical imaginations of public-sector algorithms are consistent with existing literature.

    “Differing socio-technical imaginations of algorithms in the public sector are aligned well with existing literature [20,62,103,107,112].”

    From A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfare (2021) · cites this paper as [20]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 4

  1. [28]28. Christopher T Lowenkamp. 2009. The development of an actuarial risk assessment instrument for US Pretrial Services. Fed. Probation 73 (2009), 33.
  2. [38]38. Douglas Schuler and Aki Namioka. 1993. Participatory design: Principles and practices. CRC Press.
  3. [41]41. Halil Toros and Daniel Flaming. 2018. Prioritizing Homeless Assistance Using Predictive Algorithms: An Evidence-Based Approach. Cityscape 20, 1 (2018), 117-146.
  4. [45]45. Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, and Krishna P Gummadi. 2016. Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreat- ment. arXiv preprint arXiv:1610.08452 (2016). · arXiv

Cited by

Sorted by citations

Related

Authors
Brown, A., Chouldechova, A., Putnam-Hornstein, E., Tobin, A., & Vaithianathan, R.
Affiliations
  • Massey University (NZ)
  • Microsoft (United States) (US)
  • University of Southern California, Los Angeles, CA, USA
  • Auckland University of Technology (NZ)
ORCID
Venue
Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Published
2019-05-02
Pages
1-12
DOI
10.1145/3290605.3300271
OpenAlex
W2942227409
Cited by
207
References
34
Citations by year
2026: 20 · 2025: 30 · 2024: 33 · 2023: 38 · 2022: 37 · 2021: 30 · 2020: 15 · 2019: 4
FWCI
22.39
Type
conference paper
Language
en
Open access
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations
CHI Conference on Human Factors in Computing Systems (publishedVersion)
Funders
SIGKDD (Social Impact Award)
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Social Sciences › Political Science and International Relations › E-Government and Public Services
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
Keywords
algorithmic accountability, algorithmic decision-making, public services, child welfare, algorithmic transparency, public trust, affected communities, algorithmic harms, indirect discrimination, public accountability, algorithmic fairness, stakeholder concerns

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review