Cited by 117Ethics and Social Impacts of AIChild Welfare and AdoptionArtificial Intelligence in Law

A Human-Centered Review of Algorithms used within the U.S. Child Welfare System

Open access
Saxena, Devansh, Badillo-Urquiola, Karla, Wisniewski, Pamela J., Guha, Shion·2020·Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

Abstract

The U.S. Child Welfare System (CWS) is charged with improving outcomes for foster youth; yet, they are overburdened and underfunded. To overcome this limitation, several states have turned towards algorithmic decision-making systems to reduce costs and determine better processes for improving CWS outcomes. Using a human-centered algorithmic design approach, we synthesize 50 peer-reviewed publications on computational systems used in CWS to assess how they were being developed, common characteristics of predictors used, as well as the target outcomes. We found that most of the literature has focused on risk assessment models but does not consider theoretical approaches (e.g., child-foster parent matching) nor the perspectives of caseworkers (e.g., case notes). Therefore, future algorithms should strive to be context-aware and theoretically robust by incorporating salient factors identified by past research. We provide the HCI community with research avenues for developing human-centered algorithms that redirect attention towards more equitable outcomes for CWS.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3313831.3376229

Cites

93 of 93 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Public-sector practice often uses aggregate administrative data that is poorly collected.

    “Second, current practices of using aggregate administrative data that is often poorly collected [31,53,107] and biased to predict an individual's behavior is a complex and hard task that may lead to unfairness in decision-making outcomes and is, in most western, liberal democratic systems unconstitutional and/or illegal [123].”

    From A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfare (2021) · cites this paper as [107]

    The “Sociotechnical Systems for Child-Welfare” section describes state child-welfare data systems used for decision-making and national databases. The “Inferential Statistics approaches” section says data in early child-welfare algorithm studies was limited and of questionable quality because collection was not standardized. That supports a data-quality concern in this setting, but not the broader claim that aggregate administrative data in public-sector practice is often poorly collected. (LLM, conf=0.84)

    Experimental — accuracy not yet measured
  • Public-sector outcomes such as recidivism risk or welfare need are poorly and inconsistently defined.

    “First, outcomes in the public sector like assessing risk of recidivism in criminal justice or assessing need in welfare services are poorly and inconsistently defined [11,42,55,107,129].”

    From A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfare (2021) · cites this paper as [107]

    In “Improve Lives and not just ‘Minimize Risk’ (RQ3)” (page unavailable), the review says that the child-welfare outcome “risk of maltreatment” is poorly defined and that state definitions vary. This supports the child-welfare risk example, but the passage does not establish the claim for recidivism risk or for definitions of welfare need. (LLM, conf=0.87)

    Experimental — accuracy not yet measured
  • Prior research shows workers let decision-aids supplant their own decisions when they lack confidence or experience.

    “However, prior research shows that workers allow decision-aids to supplant their own decisions when they lack confidence and/or experience [42,107,113].”

    From A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfare (2021) · cites this paper as [107]

    The Risk Assessment section says child-welfare caseworkers must rely on decision tools, especially in cases of high uncertainty. The section on theoretical and context-aware algorithms says those tools can force workers to abandon contextual judgments. Neither passage says this happens because workers lack confidence or experience. (LLM, conf=0.84)

    Experimental — accuracy not yet measured
  • Child-welfare risk-assessment algorithms have traditionally used only child and parent characteristics as predictors.

    “For instance, risk assessment algorithms in child-welfare have traditionally only used a narrow set of predictors (child and parent characteristics) to assess risk [107].”

    From A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfare (2021) · cites this paper as [107]

    In “Going Beyond what is ‘Easily Quantifiable’ (RQ2)” (page not provided), the review says a majority of algorithms used predictors concerning child and parent characteristics. It also criticizes risk estimates based on a narrow set of predictors, but it does not say these algorithms used only child and parent characteristics; “majority” does not support that absolute claim. (LLM, conf=0.88)

    Experimental — accuracy not yet measured
  • Law determines which data are available for predictive modeling and how target outcomes are defined.

    “For instance, law dictates which data is available for predictive modeling and how target outcomes are defined [107,114].”

    From A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfare (2021) · cites this paper as [107]

    The “Improve Lives and not just ‘Minimize Risk’ (RQ3)” section says federal and state law define child abuse and neglect, and that state laws and policies affect substantiation judgments used to train algorithms. This supports the point about target outcomes. The “Going Beyond what is ‘Easily Quantifiable’ (RQ2)” section discusses which information becomes available for modeling, but does not say law determines that availability; that part of the claim is unsupported. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 11

  1. [23]23. Leo Breiman and others. 2001. Statistical modeling: The two cultures (with comments and a rejoinder by the author). Statistical science 16, 3 (2001), 199-231.
  2. [50]50. Administration for Children and Families. Comprehensive Child Welfare Information System (106 ed.). Vol. 81. Federal Register: The Daily Journal of the United States.
  3. [56]56. Stuart Gray, Kirsten Cater, Chloe Meineck, Rachel Hahn, Debbie Watson, and Tom Metcalfe. 2019. trove: A digitally enhanced memory box for looked after and adopted children. In Proceedings of the 18th ACM International Conference on Interaction Design and Children. ACM, 458-463.
  4. [57]57. Robin Gregory, Lee Failing, Michael Harstone, Graham Long, Tim McDaniels, and Dan Ohlson. 2012. Structured decision making: a practical guide to environmental management choices. John Wiley & Sons.
  5. [74]74. Michael J MacKenzie, Jonathan B Kotch, and Li-Ching Lee. 2011. Toward a cumulative ecological risk model for the etiology of child maltreatment. Children and youth services review 33, 9 (2011), 1638-1647.
  6. [75]75. Diana MacLean, Sonal Gupta, Anna Lembke, Christopher Manning, and Jeffrey Heer. 2015. Forum77: An analysis of an online health forum dedicated to addiction recovery. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing. ACM, 1511-1526.
  7. [86]86. US Department of Health and Human Services. 2017. Child Maltreatment 2017. Children's Bureau (Ed.) (2017).
  8. [87]87. US Department of Health, Human Services, and others. 2017. The AFCARS report: Preliminary FY 2016 estimates as of Oct 2017. Children's Bureau (Ed.) 21 (2017), 6.
  9. [97]97. A James Schwab Jr, Michael E Bruce, and Ruth G McRoy. 1984. Matching children with placements. Children and youth services review 6, 2 (1984), 125-133.
  10. [105]105. Susan Leigh Star and Anselm Strauss. 1999. Layers of silence, arenas of voice: The ecology of visible and invisible work. Computer supported cooperative work (CSCW) 8, 1-2 (1999), 9-30.
  11. [117]117. Jacob O Wobbrock and Julie A Kientz. 2016. Research contributions in human-computer interaction. interactions 23, 3 (2016), 38-44.

Cited by

Sorted by citations

Authors
Saxena, D., Badillo-Urquiola, K., Wisniewski, P. J., & Guha, S.
Affiliations
  • Marquette University (US)
  • University of Central Florida (US)
  • University of Central Florida, Orlando, FL, USA
Venue
Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Published
2020-04-21
Pages
1-15
OpenAlex
W3009647070
Cited by
116
References
93
Citations by year
2026: 8 · 2025: 20 · 2024: 28 · 2023: 23 · 2022: 22 · 2021: 12 · 2020: 3
FWCI
15.56
Type
conference paper
Language
en
Open access
Locations
  • CHI Conference on Human Factors in Computing Systems (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • e-Publications@Marquette (Marquette University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • Facebook Computational Social Science Methodology Research Award
  • Northwestern Mutual Data Science Institute
  • William T. Grant Foundation (1067525)
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Social Sciences › Safety Research › Child Welfare and Adoption
  • Social Sciences › Political Science and International Relations › Artificial Intelligence in Law
Keywords
child welfare system, algorithmic decision-making, human-centered design, risk assessment model, foster youth, case law commentary, algorithmic fairness, predictive modeling, child welfare casework

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review