The DOI matches its Crossref record; 93 of the 93 references it cites were found in a registry or an index.
A Human-Centered Review of Algorithms used within the U.S. Child Welfare System
Open accessAbstract
The U.S. Child Welfare System (CWS) is charged with improving outcomes for foster youth; yet, they are overburdened and underfunded. To overcome this limitation, several states have turned towards algorithmic decision-making systems to reduce costs and determine better processes for improving CWS outcomes. Using a human-centered algorithmic design approach, we synthesize 50 peer-reviewed publications on computational systems used in CWS to assess how they were being developed, common characteristics of predictors used, as well as the target outcomes. We found that most of the literature has focused on risk assessment models but does not consider theoretical approaches (e.g., child-foster parent matching) nor the perspectives of caseworkers (e.g., case notes). Therefore, future algorithms should strive to be context-aware and theoretically robust by incorporating salient factors identified by past research. We provide the HCI community with research avenues for developing human-centered algorithms that redirect attention towards more equitable outcomes for CWS.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Second, current practices of using aggregate administrative data that is often poorly collected [31,53,107] and biased to predict an individual's behavior is a complex and hard task that may lead to unfairness in decision-making outcomes and is, in most western, liberal democratic systems unconstitutional and/or illegal [123].”
From A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfare (2021) · cites this paper as [107]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“First, outcomes in the public sector like assessing risk of recidivism in criminal justice or assessing need in welfare services are poorly and inconsistently defined [11,42,55,107,129].”
From A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfare (2021) · cites this paper as [107]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, prior research shows that workers allow decision-aids to supplant their own decisions when they lack confidence and/or experience [42,107,113].”
From A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfare (2021) · cites this paper as [107]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For instance, risk assessment algorithms in child-welfare have traditionally only used a narrow set of predictors (child and parent characteristics) to assess risk [107].”
From A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfare (2021) · cites this paper as [107]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For instance, law dictates which data is available for predictive modeling and how target outcomes are defined [107,114].”
From A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfare (2021) · cites this paper as [107]
Experimental — accuracy not yet measured
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Prior work has argued that algorithms in the public sector is a domain in its own right [34,62,107,109,121] and must be characterized differently as compared to algorithms in the private sector where the decision outcomes are well-defined.”
From A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfare (2021) · cites this paper as [107]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Consequently, they have also been scrutinized for achieving worse outcomes, exacerbating racial biases, and strengthening structural inequalities [42,101,107] within systems that are overburdened and under-resourced, yet critically needed [33,75].”
From A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfare (2021) · cites this paper as [107]
The cited source says what the claim attributes to it.
In the citing paper
“This is driving attention away from strength-based frameworks [9,20,99,107].”
From A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfare (2021) · cites this paper as [107]
The cited source says what the claim attributes to it.
In the citing paper
“Saxena et al. [107] recently conducted a systematic review of algorithms employed in the U.S. child-welfare system and uncovered several discrepancies in regard to data, computational methods, and target outcomes of these algorithms [107].”
From A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfare (2021) · cites this paper as [107]
The cited source says what the claim attributes to it.
In the citing paper
“However, a more comprehensive understanding about the accumulation of risk is necessary to account for the family's social support system as well as the risk posed by the system itself [107].”
From A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfare (2021) · cites this paper as [107]
The cited source says what the claim attributes to it.
In the citing paper
“Differing socio-technical imaginations of algorithms in the public sector are aligned well with existing literature [20,62,103,107,112].”
From A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfare (2021) · cites this paper as [107]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“The benefits of strength-based approaches have been supported in prior literature [9,99,107,119] as well as the pitfalls for reappropriating algorithmic outcomes for different purposes [79,106].”
From A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfare (2021) · cites this paper as [107]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
Exploration and Exploitation in Organizational Learning
March, J. G. (1991) · Organization Science · doi:10.1287/orsc.2.1.71 · Cited by 21,942 · cited 1×
“For instance, understanding contextual knowledge with respect to an organization requires incorporating the organizational memory of the organization and its people [76, 5] which is inherently HCI research.”
BackgroundApplied Multiple Regression/Correlation Analysis for the Behavioral Sciences
Cohen, P., Cohen, P., West, S. G., & Aiken, L. S. (2014) · Psychology Press eBooks · doi:10.4324/9781410606266 · Cited by 20,870
Analyzing the Past to Prepare for the Future: Writing a Literature Review
Webster, J., & Watson, R. T. (2002) · MIS Quarterly · doi:10.2307/4132319 · Cited by 7,449
Writing Culture: The Poetics and Politics of Ethnography
Clifford, J., & Marcus, G. E. (1986) · Cited by 6,160
An Introduction to Statistical Methods and Data Analysis
Ziegel, E. R., & Ott, L. (1994) · Technometrics · doi:10.1080/00401706.1994.10485832 · Cited by 5,474
Wallach, H. (2006) · International Conference on Machine Learning · doi:10.1145/1143844.1143967 · Cited by 1,096
Lambrecht, A., & Tucker, C. (2019) · Management Science · doi:10.1287/mnsc.2018.3093 · Cited by 854
Psychometrics: An Introduction.
Furr, R. M., & Bacharach, V. R. (2007) · Cited by 846
Donoho, D. (2017) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.2017.1384734 · Cited by 695
Research works printed but not linked: 25
- 2018. Using Data To Help Protect Children and Families Act. 115th Congress, Senate of the United States.
- 2019. Child Welfare Goals, Legislation, and Monitoring. (2019). Retrieved April 2, 2019 from https://www.nccdglobal.org/what-we-do/children-s-research-center
- Leo Breiman and others. 2001. Statistical modeling: The two cultures (with comments and a rejoinder by the author). Statistical science 16, 3 (2001), 199--231.
- US Congress. 2008. Fostering connections to success and increasing adoptions act of 2008. (2008).
- Michael Corrigan. 2019. Building A Comprehensive Child Welfare Information System. (Jan 2019). https://chronicleofsocialchange.org/child-welfare-2/building-comprehensive-child-welfare-information-system/
- EW Danktert and Kristen Johnson. 2013. Risk assessment validation: A prospective study. Los Angeles: California Department of Social Services, Children and Family Services Division (2013).
- Administration for Children and Families. Comprehensive Child Welfare Information System (106 ed.). Vol. 81. Federal Register: The Daily Journal of the United States.
- Patrick J Fowler, Katherine E Marcal, Saras Chung, Derek S Brown, Melissa Jonson-Reid, and Peter S Hovmand. 2019. Scaling up housing services within the child welfare system: policy insights from simulation modeling. Child maltreatment (2019), 1077559519846431.
- Robin Gregory, Lee Failing, Michael Harstone, Graham Long, Tim McDaniels, and Dan Ohlson. 2012. Structured decision making: a practical guide to environmental management choices. John Wiley & Sons.
- Luma Institute. 2012. Innovating for people: Handbook of human-centered design methods. LUMA Institute, LLC.
- David Jackson and Gary Marx. 2017. Data mining program designed to predict child abuse proves unreliable, DCFS says. (dec 2017). https://www.chicagotribune.com/investigations/ct-dcfs-eckerd-met-20171206-story.html
- K Johnson and D Wagner. 2003. California Structured Decision Making. Risk Assessment Revalidation: A Prospective Study. Children's Research Center. SAMSHA's National Registry of Evidence-based Programs and Practices (NREPP). (2003).
- Will Johnson. 2004. Effectiveness of California's child welfare structured decision making (SDM) model: a prospective study of the validity of the California Family Risk Assessment. Madison (Wisconsin, USA): Children's Research Center (2004).
- John S Lyons, Dana Aron Weiner, and Melanie Buddin Lyons. 2004. Measurement as communication in outcomes management: The child and adolescent needs and strengths (CANS). The Use of Psychological Testing for Treatment Planning and Outcomes Assessment. Volume 2: Instruments for Children and Adolescents (2004).
- Michael J MacKenzie, Jonathan B Kotch, and Li-Ching Lee. 2011. Toward a cumulative ecological risk model for the etiology of child maltreatment. Children and youth services review 33, 9 (2011), 1638--1647.
- Judge Michael Nash. 2017. Examination of using Structured Decision Making and Predictive Analytics in assessing Safety and Risk in Child Welfare. County of Los Angeles Office of Child Protection (May 2017).
- US Department of Health and Human Services. 2017. Child Maltreatment 2017. Children's Bureau (Ed.) (2017).
- US Department of Health, Human Services, and others. 2017. The AFCARS report: Preliminary FY 2016 estimates as of Oct 2017. Children's Bureau (Ed.) 21 (2017), 6.
- A James Schwab Jr, Michael E Bruce, and Ruth G McRoy. 1984. Matching children with placements. Children and youth services review 6, 2 (1984), 125--133.
- Susan Leigh Star and Anselm Strauss. 1999. Layers of silence, arenas of voice: The ecology of visible and invisible work. Computer supported cooperative work (CSCW) 8, 1--2 (1999), 9--30.
- Rebecca Tushnet. 2018. The Difference Engine: Perpetuating Poverty through Algorithms. Jotwell: J. Things We Like (2018), 1.
- Rhema Vaithianathan, Emily Putnam-Hornstein, Nan Jiang, Parma Nand, and Tim Maloney. 2017. Developing predictive models to support child maltreatment hotline screening decisions: Allegheny County methodology and implementation. Center for Social data Analytics (2017).
- D Wagner, K Johnson, and W Johnson. 1998. Using actuarial risk assessment to target service nterventions in pilot California Counties. 13th National Roundtable on CPS Risk Assessment, San Francisco, CA (1998).
- Jacob O Wobbrock and Julie A Kientz. 2016. Research contributions in human-computer interaction. interactions 23, 3 (2016), 38--44.
- Susan J Zuravin. 1999. Child neglect. Neglected children: Research, practice, and policy (1999), 24--46.
Cited by
Sorted by citations
Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction
Shelby, R., Rismani, S., Henne, K., Moon, A., Rostamzadeh, N., Nicholas, P., Yilla-Akbari, N., Gallegos, J., Smart, A., Garcia, E., et al. (2023) · Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3600211.3604673 · Cited by 279
Jacobs, M., He, J., F. Pradier, M., Lam, B., Ahn, A. C., McCoy, T. H., Perlis, R. H., Doshi-Velez, F., & Gajos, K. Z. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445385 · Cited by 177
Trauma-Informed Computing: Towards Safer Technology Experiences for All
Chen, J. X., McDonald, A., Zou, Y., Tseng, E., Roundy, K., Tamersoy, A., Schaub, F., Ristenpart, T., & Dell, N. L. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517475 · Cited by 201
Kawakami, A., Sivaraman, V., Cheng, H.-F., Stapleton, L., Cheng, Y., Qing, D., Perer, A., Wu, Z. S., Zhu, H., & Holstein, K. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517439 · Cited by 143
Saxena, D., Badillo-Urquiola, K., Wisniewski, P. J., & Guha, S. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3476089 · Cited by 134
“This is driving attention away from strength-based frameworks [9, 20, 99, 107].”
BackgroundMethodInfluentialSelf-citation
Metadata
- Authors
- Saxena, D., Badillo-Urquiola, K., Wisniewski, P. J., & Guha, S.
- Affiliations
- Marquette University (US)
- University of Central Florida (US)
- University of Central Florida, Orlando, FL, USA
- Venue
- Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2020-04-21
- Pages
- 1-15
- OpenAlex
- W3009647070
- Cited by
- 116
- References
- 93
- Citations by year
- 2026: 8 · 2025: 20 · 2024: 28 · 2023: 23 · 2022: 22 · 2021: 12 · 2020: 3
- FWCI
- 15.56
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- e-Publications@Marquette (Marquette University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Facebook Computational Social Science Methodology Research Award
- Northwestern Mutual Data Science Institute
- William T. Grant Foundation (1067525)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Sociology and Political Science › Child Welfare and Adoption
- Social Sciences › Law › Artificial Intelligence in Law
- Keywords
- child welfare system, algorithmic decision-making, human-centered design, risk assessment model, foster youth, case law commentary, algorithmic fairness, predictive modeling, child welfare casework
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
