A Framework of High-Stakes Algorithmic Decision-Making for the Public Sector Developed through a Case Study of Child-Welfare

Algorithmic decision tools are used in high-stakes child-welfare decisions such as assessing child-abuse risk and determining placement stability.

“As such, algorithmic decision-making tools are now being used in high-stakes CWS situations, including making risk assessments of child abuse [34] and determining placement stability [86].”

• cites [34] 34. Maria De-Arteaga, Riccardo Fogliato, and Alexandra Chouldechova. 2020. A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1-12. sourceSupported
• cites [86] 86. Terry D Moore, Thomas P McDonald, and Kari Cronbaugh-Auld. 2016. Assessing risk of placement instability to aid foster care placement decision making. Journal of Public Child Welfare 10, 2 (2016), 117-131. sourceNot checked — no full text of the source

High-stakes public-sector decisions require a distinct algorithmic framework that accounts for bureaucratic processes and human discretion.

“we argue that the high-stakes decisions being made within the public sector necessitate the critical need for a distinctly unique framework for algorithmic systems that accounts for the complexities of public sector bureaucratic processes [43,46,77] and the delicate application of human discretion that has historically been a cornerstone of social services [74,116].”

• cites [43] 43. Ali Farazmand. 2009. Bureaucracy and administration. CRC Press. sourceNot checked — no full text of the source
• cites [46] 46. H George Frederickson, Kevin B Smith, Christopher W Larimer, and Michael J Licari. 2015. The public administration theory primer. Westview Press.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [77] 77. Kim Loyens and Jeroen Maesschalck. 2010. Toward a theoretical framework for ethical decision making of street-level bureaucracy: Existing models reconsidered. Administration & Society 42, 1 (2010), 66-100. sourceSupported
• cites [74] 74. Michael Lipsky. 2010. Street-level bureaucracy: Dilemmas of the individual in public service. Russell Sage Foundation.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [116] 116. Michael R Sosin. 2010. Discretion in human service organizations. Human services as complex organizations (2010), 381-403.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

A systematic review of U.S. child-welfare algorithms found discrepancies in data, computational methods, and target outcomes.

“Saxena et al. [107] recently conducted a systematic review of algorithms employed in the U.S. child-welfare system and uncovered several discrepancies in regard to data, computational methods, and target outcomes of these algorithms [107].”

• cites [107] 107. Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2020. A Human-Centered Review of Algorithms used within the US Child Welfare System. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1-15. sourceSupported

Public-sector outcomes such as recidivism risk or welfare need are poorly and inconsistently defined.

“First, outcomes in the public sector like assessing risk of recidivism in criminal justice or assessing need in welfare services are poorly and inconsistently defined [11,42,55,107,129].”

• cites [11] 11. Solon Barocas and Andrew D Selbst. 2016. Big data's disparate impact. Calif. L. Rev. 104 (2016), 671.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [42] 42. Virginia Eubanks. 2018. Automating inequality: How high-tech tools profile, police, and punish the poor. St. Martin's Press.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [55] 55. Travis Greene, Galit Shmueli, Jan Fell, Ching-Fu Lin, Mark L Shope, and Han-Wei Liu. 2020. The Hidden Inconsistencies Introduced by Predictive Algorithms in Judicial Decision Making. arXiv preprint arXiv:2012.00289 (2020). sourcePartly supported

  Section 4.1 (pp. 15–16) says there is no generally accepted legal definition of recidivism and documents differing ways to measure it. That supports the criminal-justice example, but the paper does not establish that need in welfare services is poorly or inconsistently defined. (LLM, conf=0.86)

  Experimental — accuracy not yet measured
• cites [107] 107. Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2020. A Human-Centered Review of Algorithms used within the US Child Welfare System. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1-15. sourcePartly supported

  In “Improve Lives and not just ‘Minimize Risk’ (RQ3)” (page unavailable), the review says that the child-welfare outcome “risk of maltreatment” is poorly defined and that state definitions vary. This supports the child-welfare risk example, but the passage does not establish the claim for recidivism risk or for definitions of welfare need. (LLM, conf=0.87)

  Experimental — accuracy not yet measured
• cites [129] 129. Kristen M Zgoba and Nishi P Dayal. 2015. Recidivism. The Encyclopedia of Crime and Punishment (2015), 1-5. sourceNot checked — no full text of the source

These new skills are not accounted for in what constitutes professional expertise.

“Practitioners in the public sector are continually acquiring new skills as they learn to make decisions through data systems and interpret algorithmic outputs [17,21], however, these new skills are not being accounted for with respect to what constitutes professional expertise [62].”

• cites [62] 62. Naja Holten Møller, Irina Shklovski, and Thomas T Hildebrandt. 2020. Shifting concepts of value: Designing algorithmic decision-support systems for public services. In Proceedings of the 11th Nordic Conference on Human- Computer Interaction: Shaping Experiences, Shaping Society. 1-12. sourceSupported

This deficit-based focus diverts attention from strength-based frameworks.

“This is driving attention away from strength-based frameworks [9,20,99,107].”

• cites [9] 9. Karla Badillo-Urquiola, Xinru Page, and Pamela Wisniewski. 2019. Risk vs. Restriction: The Digital Divide between Providing a Sense of Normalcy and Keeping Foster Teens Safe Online. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM. sourceNot checked — no full text of the source
• cites [20] 20. Anna Brown, Alexandra Chouldechova, Emily Putnam-Hornstein, Andrew Tobin, and Rhema Vaithianathan. 2019. Toward Algorithmic Accountability in Public Services: A Qualitative Study of Affected Community Perspectives on Algorithmic Decision-making in Child Welfare Services. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, 41. sourceSupported
• cites [99] 99. Anthony T Pinter, Pamela J Wisniewski, Heng Xu, Mary Beth Rosson, and Jack M Caroll. 2017. Adolescent online safety: Moving beyond formative evaluations to designing solutions for the future. In Proceedings of the 2017 Conference on Interaction Design and Children. ACM, 352-357. sourceNot checked — no full text of the source
• cites [107] 107. Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2020. A Human-Centered Review of Algorithms used within the US Child Welfare System. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1-15. sourceSupported

Public-sector practice often uses aggregate administrative data that is poorly collected.

“Second, current practices of using aggregate administrative data that is often poorly collected [31,53,107] and biased to predict an individual's behavior is a complex and hard task that may lead to unfairness in decision-making outcomes and is, in most western, liberal democratic systems unconstitutional and/or illegal [123].”

• cites [31] 31. Roxanne Connelly, Christopher J Playford, Vernon Gayle, and Chris Dibben. 2016. The role of administrative data in the big data revolution in social science research. Social science research 59 (2016), 1-12. sourcePartly supported

  Section 6.4 (Data quality; page unavailable) says some administrative data contain errors and gives an example of information that was not collected diligently. It also says quality varies between systems. The paper does not establish that public-sector practice often uses aggregate administrative data that is poorly collected. (LLM, conf=0.85)

  Experimental — accuracy not yet measured
• cites [53] 53. Philip Gillingham. 2018. Decision-making about the adoption of information technology in social welfare agencies: Some key considerations. European Journal of Social Work 21, 4 (2018), 521-529. sourceNot checked — no full text of the source
• cites [107] 107. Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2020. A Human-Centered Review of Algorithms used within the US Child Welfare System. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1-15. sourcePartly supported

  The “Sociotechnical Systems for Child-Welfare” section describes state child-welfare data systems used for decision-making and national databases. The “Inferential Statistics approaches” section says data in early child-welfare algorithm studies was limited and of questionable quality because collection was not standardized. That supports a data-quality concern in this setting, but not the broader claim that aggregate administrative data in public-sector practice is often poorly collected. (LLM, conf=0.84)

  Experimental — accuracy not yet measured

A person's situation can destabilize repeatedly, making successful outcomes or interventions hard to assess.

“Moreover, an individual's personal situation can (de)stabilize several times making it hard to assess what constitutes a successful outcome or intervention [62].”

• cites [62] 62. Naja Holten Møller, Irina Shklovski, and Thomas T Hildebrandt. 2020. Shifting concepts of value: Designing algorithmic decision-support systems for public services. In Proceedings of the 11th Nordic Conference on Human- Computer Interaction: Shaping Experiences, Shaping Society. 1-12. sourceSupported

Practitioners' beliefs and moral values are important factors in street-level decision-making.

“Practitioner's beliefs and moral values are important factors in regard to how street-level decision-making unfolds [77,83,105].”

• cites [77] 77. Kim Loyens and Jeroen Maesschalck. 2010. Toward a theoretical framework for ethical decision making of street-level bureaucracy: Existing models reconsidered. Administration & Society 42, 1 (2010), 66-100. sourceSupported
• cites [83] 83. Steven Maynard-Moody and Michael Musheno. 2003. Cops, teachers, counselors: Narratives of street-level judgment. Ann Arbor, MI: University of Michigan Press (2003). sourceNot checked — no full text of the source
• cites [105] 105. Jodi R Sandfort. 2000. Moving beyond discretion and outcomes: Examining public management from the front lines of the welfare system. Journal of Public Administration Research and Theory 10, 4 (2000), 729-756. sourcePartly supported

  The 'Implications for Public Management Research and Practice' section says front-line staff develop collective beliefs that shape how they interpret events and carry out their work. The copy does not establish practitioners’ moral values as a distinct factor in their decisions; its emphasis is on shared, experience-based beliefs and organizational context. Page numbers were not provided. (LLM, conf=0.85)

  Experimental — accuracy not yet measured

Prior research shows workers let decision-aids supplant their own decisions when they lack confidence or experience.

“However, prior research shows that workers allow decision-aids to supplant their own decisions when they lack confidence and/or experience [42,107,113].”

• cites [42] 42. Virginia Eubanks. 2018. Automating inequality: How high-tech tools profile, police, and punish the poor. St. Martin's Press.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [107] 107. Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2020. A Human-Centered Review of Algorithms used within the US Child Welfare System. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1-15. sourcePartly supported

  The Risk Assessment section says child-welfare caseworkers must rely on decision tools, especially in cases of high uncertainty. The section on theoretical and context-aware algorithms says those tools can force workers to abandon contextual judgments. Neither passage says this happens because workers lack confidence or experience. (LLM, conf=0.84)

  Experimental — accuracy not yet measured
• cites [113] 113. Aron Shlonsky and Dennis Wagner. 2005. The next step: Integrating actuarial risk assessment and clinical judgment into an evidence-based practice framework in CPS case management. Children and youth services review 27, 4 (2005), 409-427. sourceNot checked — no full text of the source

Simple heuristics can be more successful in uncertain and complex spaces because not all decision-relevant information may be available.

“Simple heuristics can be more successful especially in uncertain and complex spaces since all the information required to make a decision might not be available as a result of uncertainty [4].”

• cites [4] 4. Florian Artinger, Malte Petersen, Gerd Gigerenzer, and Jürgen Weibler. 2015. Heuristics as adaptive decision strategies in management. Journal of Organizational Behavior 36, S1 (2015), S33-S52. sourceSupported
  Experimental — accuracy not yet measured

Child-welfare risk-assessment algorithms have traditionally used only child and parent characteristics as predictors.

“For instance, risk assessment algorithms in child-welfare have traditionally only used a narrow set of predictors (child and parent characteristics) to assess risk [107].”

• cites [107] 107. Devansh Saxena, Karla Badillo-Urquiola, Pamela J Wisniewski, and Shion Guha. 2020. A Human-Centered Review of Algorithms used within the US Child Welfare System. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1-15. sourcePartly supported

  In “Going Beyond what is ‘Easily Quantifiable’ (RQ2)” (page not provided), the review says a majority of algorithms used predictors concerning child and parent characteristics. It also criticizes risk estimates based on a narrow set of predictors, but it does not say these algorithms used only child and parent characteristics; “majority” does not support that absolute claim. (LLM, conf=0.88)

  Experimental — accuracy not yet measured

Availability of essential administrative, financial, personnel, and political resources directly affects organizational performance.

“Availability of essential resources (administrative, financial, personnel, political) directly impact organizational performance [45,69,82].”

• cites [45] 45. Patricia Fredericksen and Rosanne London. 2000. Disconnect in the hollow state: The pivotal role of organizational capacity in community-based development organizations. Public Administration Review 60, 3 (2000), 230-239. sourceNot checked — no full text of the source
• cites [69] 69. Soo-Young Lee and Andrew B Whitford. 2013. Assessing the effects of organizational resources on public agency performance: Evidence from the US federal government. Journal of Public Administration Research and Theory 23, 3 (2013), 687-712. sourcePartly supported

  The Model Estimation and Results section examines administrative, human, financial, and political resources in relation to US federal agency effectiveness. It reports positive relationships for some resources, but negative or insignificant effects for others. Thus, the paper supports a link between resources and performance, not an unqualified claim that the availability of each type directly affects performance. (LLM, conf=0.86)

  Experimental — accuracy not yet measured
• cites [82] 82. Azhar Manzoor. 2014. A look at efficiency in public administration: Past and future. Sage Open 4, 4 (2014), 2158244014564936. sourceNot checked — no full text of the source

Recent explainable-AI work supports focusing on explanations of users' daily interactions with algorithms.

“Recent work on explainable AI has found support for focusing on explanations of daily interactions of users with algorithms [73,88].”

• cites [73] 73. Q Vera Liao, Daniel Gruen, and Sarah Miller. 2020. Questioning the AI: Informing Design Practices for Explainable AI User Experiences. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1-15. sourceSupported
• cites [88] 88. Henrik Mucha, Sebastian Robert, Rüdiger Breitschwerdt, and Michael Fellmann. [n.d.]. Towards Participatory Design Spaces for Explainable AI Interfaces in Expert Domains. ([n. d.]).Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

The fluid character of legal reasoning often conflicts with the discrete predictive character of algorithms.

“The fluid nature of legal reasoning is often at odds with the discrete predictive nature of algorithms [55].”

• cites [55] 55. Travis Greene, Galit Shmueli, Jan Fell, Ching-Fu Lin, Mark L Shope, and Han-Wei Liu. 2020. The Hidden Inconsistencies Introduced by Predictive Algorithms in Judicial Decision Making. arXiv preprint arXiv:2012.00289 (2020). sourceSupported

A core contribution is showing that the theoretically derived ADMAPS framework, empirically validated in this child-welfare case study, can be used in other public-sector domains to assess high-stakes algorithms.

“Thus, a core contribution of this paper is that we demonstrate how the theoretically derived ADMAPS framework (empirically validated through this case study in child-welfare) can be used within other public sector domains (e.g., criminal justice [58,59]) to assess the strengths and weaknesses of algorithms making high-stakes decisions in the lives of people.”

• cites [58] 58. MD Romael Haque, Katherine Weathington, Joseph Chudzik, and Shion Guha. 2020. Understanding Law Enforcement and Common Peoples' Perspectives on Designing Explainable Crime Mapping Algorithms. In Conference Companion Publication of the 2020 on Computer Supported Cooperative Work and Social Computing. 269-273. sourceNot checked — no full text of the source
• cites [59] 59. MD Romael Haque, Katherine Weathington, and Shion Guha. 2019. Exploring the Impact of (Not) Changing Default Settings in Algorithmic Crime Mapping-A Case Study of Milwaukee, Wisconsin. In Conference Companion Publication of the 2019 on Computer Supported Cooperative Work and Social Computing. 206-210. sourceSupported

Prior studies have found that caseworkers distrust risk assessment algorithms such as CANS because those tools are deficit-based.

“Prior studies have found that caseworkers distrust risk assessment algorithms such as CANS because of their deficit-based nature [20].”

• cites [20] 20. Anna Brown, Alexandra Chouldechova, Emily Putnam-Hornstein, Andrew Tobin, and Rhema Vaithianathan. 2019. Toward Algorithmic Accountability in Public Services: A Qualitative Study of Affected Community Perspectives on Algorithmic Decision-making in Child Welfare Services. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. ACM, 41. sourcePartly supported

  In “4 FINDINGS” (page not provided), frontline providers express concern that child-welfare approaches emphasize deficits and risks; one calls a scenario a “deficit model.” The paper does not identify CANS or establish that caseworkers distrust that tool because it is deficit-based. Its findings concern participants’ responses to child-welfare decision-making scenarios. (LLM, conf=0.85)

  Experimental — accuracy not yet measured

ADMAPS can help design algorithms for job placement centers where caseworkers must exercise discretion in applying legal frameworks, assessing resource constraints, and resolving organizational contradictions to extend unemployment benefits.

“Similarly, ADMAPS can help design algorithms for job placement centers where the caseworkers must exercise discretion in applying complex legal frameworks, assessing resource constraints, as well as resolving organizational contradictions (i.e., bureaucratic processes) to extend unemployment benefits to citizens [62].”

• cites [62] 62. Naja Holten Møller, Irina Shklovski, and Thomas T Hildebrandt. 2020. Shifting concepts of value: Designing algorithmic decision-support systems for public services. In Proceedings of the 11th Nordic Conference on Human- Computer Interaction: Shaping Experiences, Shaping Society. 1-12. sourcePartly supported

  Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

  Experimental — accuracy not yet measured

The theme that CANS leaves no room for heuristic judgment emerged in 80% (n=16) of interviews and 73% (n=11) of planning meetings.

“This theme emerged in 80% (n=16) of the interviews and 73% (n=11) of the planning meetings.”

75% of interviewees said CANS reduced their ability to make flexible value judgments about child outcomes.

“75% of interviewees saying that it reduced their ability to make flexible value judgments on child outcomes”

Two data specialists said caseworkers' yearly job performance is tied to how timely they complete and submit CANS assessments because funds must be allocated promptly.

“Moreover, two data specialists shared that the caseworkers' yearly job performance is tied to the timeliness with which they complete and submit CANS assessments since funds need to be allocated in a timely manner.”

The CANS algorithm is used in these meetings to establish a baseline for foster children's mental health well-being.

“The CANS algorithm is also used at these meetings to establish a baseline for foster children with respect to mental health well-being.”

CANS inhibited professional expertise because caseworkers felt it turned them into data brokers who collect information and feed it to the algorithm.

“CANS further inhibited professional expertise because caseworkers felt that it turned them into data brokers who must collect information about children and feed it to the algorithm to make the decisions.”

50% of participants (n=10) said CANS recommends the level of foster care a child should be placed in.

“Some participants (50%, n=10) also shared that CANS algorithm recommends the level of foster care that the child should be placed in (see Figure 2).”

After CANS scores are entered, the algorithm generates a monetary value for foster parents.

“After CANS scores are entered in, the algorithm generates a monetary value to be offered to foster parents.”

The CANS algorithm has been re-appropriated to measure additional outcomes beyond its original purposes.

“However, as depicted in Figure 2, CANS algorithm has been re-appropriated to measure additional outcomes discussed below.”

Caseworkers cannot prioritize properly conducting CANS because generating an adequate rate takes precedence to prevent placement disruption.

“Here, caseworkers are unable to prioritize properly conducting CANS because generating the adequate rate takes precedence to ensure the placement is not disrupted.”

70% of participants (n=14) said CANS does not account for much of the data they consider pertinent when assessing cases.

“Several participants (70%, n=14) shared that CANS does not account for much of the data that they consider pertinent when assessing cases (for e.g., understanding about trauma and social support-system).”

CANS focuses on emotional/behavioral needs and risk behaviors to assess mental health needs.

“CANS algorithm focuses on the child emotional/behavioral needs (for e.g., anxiety, anger control, substance use, behavioral regression) and child risk behaviors (for e.g., suicide risk, self-harm, delinquent behavior, runaway tendencies) to assess the mental health needs of the child.”

CANS does not allow heuristic decision-making about outcomes.

“However, CANS does not allow for heuristic decision-making with respect to the outcomes.”

80% of participants (n=16) said CANS leaves no room for heuristic judgment calls.

“80% Heuristic Decision-Making.CANS leaves no room for heuristic judgment calls.”

80% of participants (n=16) said every case is contextually different with salient factors that may be central to one case and peripheral to another.

“Participants (80%, n=16) shared that every case is contextually different with salient factors that might be central to one case but peripheral to another.”

CANS had an overall negative impact on human discretion, with 80% of interviewees saying it negatively affected professional expertise by limiting value judgments and heuristic decision-making.

“we found that CANS had an overall negative impact on human discretion with 80% of the interviewees indicating that it negatively impacted professional expertise by limiting the scope for value judgments and heuristic decision-making”

90% of participants (n=18) expressed frustration with CANS's inflexible and predictive nature.

“Most of the participants (90%, n=18) shared their frustration regarding the inflexible and predictive nature of CANS.”

The child-welfare teams also use an Anti Sex-Trafficking algorithm to assess foster youths' risk of sex trafficking.

“Anti Sex-Trafficking (AST) algorithm is used to assess the risk of sex-trafficking for foster youth.”

In all planning meetings and permanency consultations, teams adaptively focused on information pertinent to the case.

“This was also apparent in all the planning meetings and permanency consultations where the teams adaptively focused on information pertinent to that case.”

CANS introduced new constraints in the case planning process.

“Resources & Constraints.CANS has introduced new constraints in the case planning process.”

CANS supports an organizational process of allocating resources.

“CANS offers a way to support an organizational process of allocating resources.”

Using CANS to calculate foster parent compensation was a prominent theme in 73% (n=11) of planning meetings and 85% (n=17) of interviews.

“Using the CANS algorithm for calculating foster parents' compensation was another prominent theme that emerged in 73% (n=11) of the planning meetings and 85% (n=17) of the interviews.”

Participants noted CANS focuses on current behaviors but not underlying trauma or traumatic triggers.

“Moreover, participants noted that CANS focuses on current behaviors but not the underlying trauma or traumatic triggers.”

75% of participants (n=17) said re-appropriating CANS to calculate foster parent compensation led to unintended consequences.

“Most of the participants (75%, n=17) shared how the re-appropriation of CANS to calculate the foster parent compensations has led to several unintended consequences.”

The first CANS assessment facilitates sharing information with legal parties and family preservation specialists involved in case planning.

“It also facilitates sharing this information with other parties such as legal parties and family preservation specialists who also play a role in case planning.”

Child-welfare teams in planning meetings and permanency consultations briefly discussed child needs and behaviors but then shifted to a more extensive conversation about trauma using trauma-informed care.

“Child-welfare teams in both the planning meetings (n=15) and permanency consultations (n=40) briefly discussed child needs and behaviors but then shifted to a more extensive conversation about trauma using trauma-informed care.”

60% of participants (n=12) said the first CANS assessment was most useful because it established a baseline for mental-health needs and the level of care needed.

“Participants (60%, n=12) stated that the first CANS assessment was most useful because it helped establish a baseline for the mental-health needs of a child and the level of care that the child needed.”

80% of interviewees asserted they were no longer given discretion to make decisions for children assigned to them because CANS made several of those decisions for them.

“80% of interviewees asserting that they were no longer given the discretion to make decisions on behalf of the children assigned to them because CANS made several of these decisions for them.”

Because suitable higher-level foster homes are scarce, the placement decision often depends on resource availability.

“However, due to a lack of such homes, this decision often comes down to the availability of resources.”

80% of participants (n=18) said their trauma-informed care training conflicted with CANS.

“However, most participants (80% n=18) shared that their training in trauma-informed care (see Section 6.3), which offers a more comprehensive understanding of trauma and the child's environment often conflicted with CANS.”

60% of participants (n=12) said the agency uses CANS to conduct mental health assessments of foster children based on risks and exhibited behaviors.

“several participants (60%, n=12) shared that the agency uses CANS algorithm to conduct a mental health assessment of foster children based on risks posed and exhibited behaviors.”

Participants explained that higher CANS mental health needs lead to higher compensation for foster parents.

“Participants explained that the higher the mental health needs of a child per CANS, higher the compensation offered to foster parents.”

Many participants felt gaming the algorithm was the only viable way for caseworkers to exercise discretion.

“Many felt that gaming the algorithm was the only viable option for caseworkers to exercise discretion.”

40% of participants (n=8) found value in the first CANS assessment because it established a baseline for mental health and compensation.

“Some participants (40%, n=8) found value in the first assessment since it established the baseline for mental health and compensation for foster parents.”

The first CANS assessment is used to devise service plans for foster children based on exhibited behaviors.

“The first assessment is used to devise service plans for foster children (for e.g, behavioral therapy) based on exhibited behaviors.”

If foster parents disagree with the rate and want more, caseworkers manipulate CANS scores to produce a higher rate.

“However, if foster parents disagree with the rate and believe they should be paid more, the caseworkers manipulate the scores to produce a higher rate.”

80% of participants (n=16) said caseworkers are trained to conduct CANS but not to manage constraints.

“Caseworkers are trained to conduct CANS but not on managing constraints.This theme emerged in 80% (n=16) of the interviews.”

The Department of Children and Families associated foster parent compensation with a child's mental health needs, so higher mental health needs lead to higher compensation.

“The Department of Children and Families (DCF) decided to associate this compensation with the mental health needs of a child, that is, the higher the mental needs of a child, the higher the compensation offered to foster parents.”

Conversations with agency leadership revealed that assessments are more likely to be manipulated if tied to singular metrics.

“Ongoing conversations with agency leadership (program directors, quality improvement leaders) revealed that assessments are more likely to be manipulated if they are tied to singular metrics.”

For children in the system for a few months, teams spent more time on Prevalence and Impact and then focused on Perspective Shift.

“For instance, in case of children who had been in the system for a few months, the team spent more time on Prevalence and Impact and then focused on Perspective Shift (see Figure 3).”

50% of participants (n=10) said new caseworkers socially learn from other caseworkers and experience to manage conflicts by gaming CANS.

“Participants (50%, n=10) shared that new caseworkers socially learn from other caseworkers (and through experience) to manage these conflicts by gaming CANS.”

Teams focused more on some 7ei domains than others depending on how long the child had been in the system.

“The teams focused more on some domains than others based on how long the child had been in the system.”

7ei was used in all 55 observed meetings.

“We witnessed 7ei being utilized in all meetings (N=55).”

Participants said 7ei offers a comprehensive view of the foster child, caregivers, traumatic event impacts, and social ecosystem interactions.

“Participants shared that 7ei algorithm offers a comprehensive view of the foster child, caregivers, impact of traumatic events in their life, as well as interactions in their social ecosystem.”

The case timeline determined which 7ei domains the teams converged on.

“Timeline of the case established which 7ei domains the child-welfare teams converged on.”

Child-welfare teams in all 40 permanency consultations and 15 planning meetings could adaptively select the factors most pertinent to the case.

“Moreover, the child-welfare teams in all the permanency consultations (n=40) and planning meetings (n=15) were able to adaptively select factors that were most pertinent to that case.”

For cases in the system longer or with multiple placement moves, teams focused more on Regulation, Perspective Shift, and Reasons to Be.

“However, cases that had been in the system for a longer period or had experienced multiple placement moves, the team focused more on Regulation, Perspective Shift, and Reasons to Be.”

80% of participants (n=18) recognized that CANS is legally mandated by the state.

“80% Laws & Policies.The use of CANS is legally mandated.Most participants (80%, n=18) recognized (and often complained) that CANS was legally mandated by the state and offered them a convenient means to allocate resources to foster parents every six months.”

Agency leadership uses 7ei to assess progress within the agency on trauma-informed care.

“Agency leadership also uses 7ei to assess progress within the agency with respect to trauma-informed care.”

Caseworkers are not trained to manage the systemic constraints and value conflicts introduced by CANS.

“However, they are not trained on how to manage the systemic constraints and value conflicts introduced by the algorithm itself.”

For cases with multiple placement moves, the team focused on Regulation and Reasons to Be to create plans improving the child's emotional, behavioral, and cognitive functioning.

“For instance, cases where the child had experienced multiple placement moves, the team focused on Regulation and Reasons to Be to devise plans that would help improve the child's emotional, behavioral, and cognitive functioning.”

7ei accounts for organizational resources.

“7ei accounts for the resources at the organization.”

AST is used predictively: selecting certain risk indicators requires reporting the case to HART.

“AST is used in a predictive capacity such that if certain risk indicators are selected then the case must be reported to HART.”

85% of participants valued that 7ei gave them flexibility and autonomy in interacting with it.

“Most participants (85%, n=17) appreciated that 7ei offered them flexibility and autonomy in how they interact with it.”

75% of participants who valued AST as a guide also were frustrated by its mandatory reporting nature.

“Several participants (75%) who found value in the algorithm as a guide also shared their frustrations in regard to its mandatory reporting nature.”

AST made caseworkers more aware of risk indicators.

“AST has made caseworkers more aware of risk indicators.”

65% of participants said AST offered a new perspective by capturing risk indicators they had not considered.

“Several participants (65%, n=13) shared that the algorithm offers a new perspective on sex-trafficking by capturing risk indicators that they had not considered before.”

80% of participants (n=16) felt every case was contextually different and required individual attention.

“Most participants (80%, n=16) felt that every case was contextually different and required their individual attention.”

7ei is used every time a case is discussed in a team setting.

“we observed that 7ei is embedded into daily work processes because it is utilized every time a case is discussed in a team setting.”

Reporting to HART removes the case from the child-welfare team that built a relationship with the child and caregivers.

“Reporting to HART takes the case away from the child-welfare team who have spent significant amount of time building a relationship with the child and their caregivers.”

AST improved professional expertise because caseworkers are more cognizant of risks.

“AST has improved professional expertise since caseworkers are more cognizant of such risks.”

75% of participants said AST misses context about each case.

“However, participants (75%, n=15) also shared that the algorithm misses context about each case”

Assessing sex-trafficking risk for foster children over 10 is an emerging high-stakes decision using an algorithm.

“An emerging high-stakes decision for which an algorithm is used is assessing the risk of sextrafficking for a foster child over 10 years of age.”

The theme that 7ei allows informed value judgments emerged in 70% (n=14) of interviews and 72% (n=40) of meetings.

“This theme emerged in 70% (n=14) of the interviews and 72% (n=40) of the meetings.”

75% of participants found continual engagement with 7ei in meetings helpful.

“Participants (75%, n=15) recognized that continually engaging with the tool in meetings was helpful”

85% of participants (n=17) appreciated that 7ei allowed open discussions and brainstorming of solutions.

“Most participants (85%, n=17) appreciated that 7ei allowed for open discussions and brainstorming of solutions.”

AST's predictive nature frustrates caseworkers.

“Predictive nature of AST frustrates caseworkers.”

55% of participants said 7ei added tasks to an already heavy workload.

“some participants (55%, n=11) shared that the tool also added more tasks to an already heavy workload.”

The theme of adaptive information selection appeared in all planning meetings and permanency consultations.

“This theme also emerged in all the planning meeting (n=15) and permanency consultations (n=40) where the child-welfare team adaptively selected information that was most pertinent to the case”

70% of participants appreciated that AST taught new caseworkers to be aware of sex-trafficking indicators.

“Most participants (70%, n=14) appreciated that this decision-tool taught new caseworkers to be actively aware of indicators that are associated with sex trafficking.”

7ei increased workload for case managers and supervisors who must discuss and complete an additional tool as a job requirement.

“However, 7ei has added to the workload of both the case managers and supervisors who must discuss and complete an additional tool as part of their job requirement at the agency.”

75% of participants said that even though AST was useful as a guide, risk indicators did not mean the child was at risk of trafficking.

“Several participants (75%, n=15) shared that even though AST was useful as a guide, the presence of risk indicators did not mean the child was at risk of being trafficked.”

Lack of context about a case leads to high uncertainty in AST.

“Lack of context about the case leads to high degree of uncertainty.”

Pertinent information beyond AST indicators was necessary to determine trafficking risk.

“There was still a lot of pertinent information that was necessary to make such a determination.”

60% of participants said LPS did not predict or recommend an outcome from inputs.

“Participants (60%, n=12) shared that the tool did not predict or recommend an outcome based on the inputs”

The theme of devising specific family plans through trauma-informed care appeared in all planning meetings and permanency consultations.

“This was also a dominant theme in all the planning meetings (n=15) and permanency consultations (n=40) where the child-welfare team devised specific plans for each family through trauma-informed care (i.e.-using 7ei).”

85% of participants appreciated that 7ei facilitated brainstorming and idea generation rather than predicting an outcome.

“Participants (85%, n=17) especially appreciated that 7ei facilitated brainstorming and idea generation instead of predicting an outcome of interest.”

Lack of relevant information leads to high uncertainty in LPS.

“Lack of relevant information leads to high degree of uncertainty.”

How LPS is conducted depends on the team's perspective.

“Participants (65%, n=13) also shared that how LPS was conducted depended on the perspective of the team.”

AST is inadequately supported by bureaucratic processes, causing caseworker frustrations.

“Overall, we found that AST is inadequately supported by bureaucratic processes which leads to frustrations on part of caseworkers.”

65% of participants said DCF mandated LPS to track placement quality and permanency.

“Participants (65%, n=13) shared that the Department of Children and Families (DCF) has mandated using this decision-tool as a way to track the important outcomes of quality of placements and permanency.”

LPS builds professional expertise by establishing urgency toward permanency.

“LPS builds professional expertise by establishing an urgency towards permanency.”

Arbitrary constraints are placed on LPS regarding how a placement can be rated.

“Moreover, there are arbitrary constraints placed on the decision-tool in regard to how a placement can be rated.”

80% of participants were indifferent toward how LPS is administered.

“Most participants (80%, n=16) were indifferent towards LPS because of the several constraints and utility issues discussed above.”

65% of participants said 7ei scores translated into actionable steps directly helping children and families.

“Several participants (65%, n=13) explained that 7ei scores translated into actionable steps that directly sought to help children and their families.”

80% of participants felt strongly about the need to support each family differently.

“Most of the participants (80%, n=16) felt strongly about the need to support each family in a different capacity and through different practices.”

LPS tracks an outcome over time rather than predicting an outcome from input data.

“This algorithm is used to track an outcome over time instead of predicting an outcome of interest using input data.”

In 87% of meetings (n=35), LPS was used near the end as a requirement to rate placement quality.

“we noticed that for majority of these meetings (87%, n=35), LPS was used towards the end as a requirement to rate the quality of the placement.”

65% of participants said AST is routinely conducted for foster youth over 10.

“Most participants (65%, n=13) shared that AST is routinely conducted for foster youth over 10 years of age.”

LPS teaches new caseworkers to prioritize placement options even if it upsets biological parents.

“It teaches new caseworkers that they needed to prioritize finding placement options even if that upset biological parents.”

Participants said 7ei offers flexibility and autonomy in how they interact with it and which domains they focus on.

“Participants noted that the tool offers flexibility and autonomy in how they interact with it and which 7ei domains they focused on.”

80% of participants said 7ei augmented their decision-making when making difficult decisions.

“Most of our participants (80%, n=16) mentioned that 7ei algorithm augmented their decision-making processes when they were making difficult decisions.”

Tracking placement stability and permanency emerged as another decision outcome using an algorithm.

“Tracking performance metrics such as placement stability and permanency emerged as another decision outcome for which an algorithm is being used.”

In meetings, 7ei brainstorming produced solutions the team could directly affect rather than just referring to therapy.

“In the meetings, 7ei brainstorming sessions resulted in solutions that the child-welfare team could affect directly and not simply refer children and parents to therapy.”

40% of participants recognized LPS's utility.

“Some participants recognized (40%, n=8) the utility of this tool”

High uncertainty about timeliness of addressing systemic barriers makes assessing placement quality hard.

“Moreover, there is high uncertainty associated with the timeliness with which some systemic barriers can be addressed making it hard to assess the quality of the placement”

7ei prioritizes and enhances value judgments and heuristic decision-making caseworkers use when devising action steps for families.

“It prioritizes and enhances the value judgments and heuristic decision-making that caseworkers must engage in when devising action steps to help families.”

65% of participants said ambiguity around inputs and lack of agency turned LPS into documentation every team must complete at permanency consultations.

“Participants shared (65%, n=13) that ambiguity around input variables and lack of agency with respect to permanency decisions has turned this decision-tool into documentation that every child-welfare team must complete at permanency consultations.”

Legal parties such as district attorneys and judges can fully disregard the child-welfare team's permanency recommendation.

“critical decision-making power in regard to permanency decisions sits with the legal parties (district attorneys and judges) who can choose to fully disregard the child-welfare team's recommendation.”

40% of participants said LPS established urgency and insistence toward achieving permanency.

“Some participants (40%, n=8) shared that the decision-tool was useful in that it established a sense of urgency and insistence towards achieving permanency for foster children.”

CANS's inflexible and predictive nature shifted focus from mental health screening to the secondary outcome of resource allocation.

“This inflexible and predictive nature has shifted focus away from the primary outcome of interest (i.e.-mental health screening) and towards the secondary outcome that allocates resources.”

AST caused an influx of cases referred to HART.

“AST has caused an influx of cases referred to HART.”

Different people can look at the same facts and reach different conclusions about whether a placement is good, fair, or poor.

“Different people might look at the same set of facts and reach different conclusions as to why the placement should be rated as good, fair, or poor.”

65% of participants were most frustrated by LPS's lack of relevant information needed for proper placement determination.

“Participants (65%, n=13) were most frustrated by the lack of relevant information that LPS needed to account for in order to make a proper determination about the quality of the placement.”

60% of participants said AST added to HART's frustrations because HART lacks resources to manage all reported cases due to mandatory reporting.

“Several participants (60%, n=12) shared that AST has added to the frustrations of HART who do not have enough resources to manage all the cases that are being reported to them because of the mandatory reporting aspect of AST.”

60% of participants said LPS does not allow value judgments or heuristics because permanency decision power rests with legal parties.

“Participants (60%, n=12) shared that the tool does not allow for value judgments or heuristics on part of the child-welfare team since decision-making power about permanency decisions sits with the legal parties.”

Caseworkers often receive inconsistent information from parents, relatives, and neighbors.

“Oftentimes, caseworkers might receive conflicting information from parents, relatives, and neighbors.”

Most participants, 85% (n=17), did not know which algorithms were being used.

“Most of the participants (85%, n=17) were unaware of the algorithms being used.”

When frustrated with CANS, caseworkers often questioned why they still had to use it when 7ei was producing better outcomes.

“When frustrated by algorithms such as CANS, caseworkers often asked why they were still expected to use CANS when 7ei was leading to better outcomes.”

60% of participants recognized that this meant unnecessary services were requested for children.

“several participants (60%, n=12) recognized that this also meant unnecessary services being requested for children.”

Participants did not see CANS or 7ei as algorithmic systems even though they used them every day.

“They did not recognize CANS or 7ei as algorithmic systems, even though they interact with them on a daily basis.”

Continuous engagement with 7ei in trauma-informed meetings under proper supervision ensures caseworkers continually discuss trauma and develop TIC-based solutions.

“Continuous engagement with 7ei in trauma-informed meetings under proper supervision ensures that caseworkers are continually having conversations centered in trauma and are coming up with solutions founded in TIC.”

When CANS was discussed explicitly, every interviewee recognized it as an algorithm that plays an important role in decision-making.

“For instance, when explicitly discussing CANS, all the interviewees realized it to be an algorithm that plays a pertinent role in decision-making.”

70% of participants recognized algorithms as essential training mechanisms.

“Most participants (70%, n=14) recognized the value of algorithms as essential training mechanisms.”

80% of participants said they could not use algorithms as intended because of organizational constraints such as limited time due to high caseloads.

“Most of the participants (80%, n=16) stated that they were unable to use algorithms as intended because of several organizational constraints such as limited time availability due to high caseloads.”

A dominant theme in 70% of interviews, 53% of planning meetings, and 50% of permanency consultations was that algorithms diminish the need for caseworker expertise and family-centered care in a contextual domain.

“Another dominant theme that emerged in 70% (n=14) of the interviews, 53% (n=8) of the planning meetings, and 50% (n=20) of the permanency consultations was that the algorithms in use were diminishing the need for caseworker expertise and family-centered care in a very contextual domain.”

Participants who valued AST as training were equally frustrated by its mandatory reporting nature.

“However, participants who found great value in the Anti Sex-Trafficking algorithm as a training mechanism were also equally frustrated by the mandatory reporting nature of it.”

70% of participants expressed concern that algorithmic systems might act as a safe default, making questioning an algorithmic decision add more work.

“Most of the participants (70%, n=14) expressed concerns that algorithmic systems might simply act as a safe default for most caseworkers such that questioning an algorithmic decision would add more work to their plate.”

Except for the program director, no participant could locate and share the study with the researchers.

“However, interestingly, none of the participants except the program director were able to locate the study and share it with us.”

Several participants, 75% (n=15), recognized that CANS, AST, and LPS decision tools had different meanings in different contexts.

“Several participants (75%, n=15) recognized that CANS, AST, and LPS decision tools meant different things in different contexts.”

Participants, 70% (n=14), noted that algorithms had different meanings for people depending on their goals.

“Participants (70%, n=14) also alluded to the fact that algorithms meant different things to people based on their intended goals.”

A majority of participants, 75% (n=15), expressed serious worries about the quality of data used to build algorithms.

“Most participants (75%, n=15) also shared significant concerns about the fidelity of data being used to develop algorithms.”

70% of participants recognized AST as a good training tool that keeps caseworkers aware of sex-trafficking red flags.

“Several participants (70%, n=14) also recognized the Anti Sex-Trafficking algorithm as a good training tool such that caseworkers are always aware of and looking for red flags associated with sex-trafficking.”

Heavy caseloads make it difficult for caseworkers to spend extra time on each CANS assessment.

“The supervisor or worker can take a little extra time and put an exceptional amount in.But again, everybody has too much to get done and it's unlikely for somebody to even think about that until they are in the middle of submitting CANS and its due the next day.”

Predictive versus prescriptive decision-support affected administration and training differently.

“For example, predictive versus prescriptive types of decision-support affected administration and training in different ways.”

Participants who disliked the deficit-based nature of risk assessment still found value in using risk assessment to understand a family's history.

“Interestingly, participants who were averse to this deficit-based nature did find value in risk assessment in regard to understanding a family's history.”

Each ADMAPS dimension had cross-dependencies with others, showing practical trade-offs at macro and micro levels.

“Additionally, each dimension of the framework had cross-dependencies with others, demonstrating practical trade-offs at both the macro-and micro-levels of the model.”

Caseworkers have varying perceptions of the design, outcomes, and intended uses of the different algorithms.

“Our results suggest that caseworkers have differing perceptions of the design, outcomes, and intended uses of the various algorithms.”

The ADMAPS dimensions helped reveal caseworker perspectives on different aspects of algorithms.

“ADMAPS dimensions helped uncover caseworker perspectives with respect to different aspects of algorithms.”

CANS is now used to track mental-health progress, an unintended result of continuous data collection.

“Another challenging aspect of using CANS where it leads to unsound decision-making is that it is now being used to track progress with respect to mental health; an unintended outcome of the continuous data collection.”

Examining the agency's bureaucratic processes showed that the most frustrating aspect of CANS is its use in calculating foster-parent financial resources.

“For instance, examining the bureaucratic processes at the agency revealed that the most frustrating part about CANS is that it is being used to calculate financial resources for foster parents.”

Optimizing one aspect of the ADMAPS framework often harmed other aspects.

“We found that when one aspect of the framework was optimized, the other aspects of the framework often suffered.”

The results show different approaches to algorithmic implementation between the state-mandated system and the non-profit agency contracted by the state.

“Our results show that there are differing approaches in algorithmic implementation that is mandated by the state as opposed to the non-profit agency that is contracted by the state, to provide child-welfare services.”

For the CANS algorithm, 80% of participants stated that laws and policies were impacted by the algorithm.

“Laws & Policies | 80%”

For the 7ei algorithm, 85% of participants stated that decision-support was impacted by the algorithm.

“Decision-Support 85%”

In the interviews, 50% of participants (n=10) recognized and discussed identifying systemic barriers as an outcome associated with algorithms.

“Identify systemic barriers (50%, n=10)”

For the CANS algorithm, 80% of participants stated that heuristic decisions were impacted by the algorithm.

“Heuristic Decisions | 80%”

CANS is used to recommend foster-care level, reported by 50% (n=10).

“Level of Foster Care (50%, n=10)”

For the AST algorithm, 75% of participants stated that heuristic decisions were impacted by the algorithm.

“Heuristic Decisions | 75%”

For the LPS algorithm, 65% of participants stated that resources/constraints were impacted by the algorithm.

“Resources/Constraints 65%”

For the CANS algorithm, 70% of participants stated that relevant data was impacted by the algorithm.

“Relevant Data | 70%”

For the LPS algorithm, 60% of participants stated that heuristic decisions were impacted by the algorithm.

“Heuristic Decisions | 60%”

For the CANS algorithm, 75% of participants stated that value judgments were impacted by the algorithm.

“CANS | Value Judgments | 75%”

In the interviews, 60% of participants (n=12) recognized and discussed quality of current placement as an outcome associated with algorithms.

“Quality of current placement (60%, n=12)”

The Anti Sex-Trafficking Response Tool is used to monitor cases for sex-trafficking, reported by 65% (n=13).

“Monitor case for sex-trafficking (65%, n=13)”

CANS is used for compensation calculations, reported by 85% (n=17).

“Compensation Calculations (85%, n=17)”

For the AST algorithm, 70% of participants stated that professional expertise was impacted by the algorithm.

“Professional Expertise 70%”

For the 7ei algorithm, 85% of participants stated that heuristic decisions were impacted by the algorithm.

“Heuristic Decisions | 85%”

For the 7ei algorithm, 70% of participants stated that value judgments were impacted by the algorithm.

“7ei | Value Judgments | 70%”

For the CANS algorithm, 70% of participants stated that uncertainty was impacted by the algorithm.

“Uncertainty | 70%”

For the CANS algorithm, 80% of participants stated that professional expertise was impacted by the algorithm.

“Professional Expertise 80%”

For the AST algorithm, 75% of participants stated that value judgments were impacted by the algorithm.

“AST | Value Judgments | 75%”

For the AST algorithm, 60% of participants stated that laws and policies were impacted by the algorithm.

“Laws & Policies | 60%”

For the LPS algorithm, 60% of participants stated that value judgments were impacted by the algorithm.

“LPS | Value Judgments | 60%”

For the AST algorithm, 75% of participants stated that decision-support was impacted by the algorithm.

“Decision-Support 75%”

For the CANS algorithm, 90% of participants stated that decision-support was impacted by the algorithm.

“Decision-Support 90%”

CANS is used for mental-health assessment, reported by 65% (n=13).

“Mental Heath Assessment (65%, n=13)”

Over-reliance on algorithmic decision-making to support bureaucratic processes can harm human discretion.

“In addition, an over-reliance on algorithmic decision-making to support bureaucratic processes can be detrimental to human discretion, however, algorithmic decision-making can support human discretionary work if they are fully supported by bureaucratic processes (RQ2).”

Algorithmic systems are used for high-stakes decisions about human lives and welfare across public-sector domains including child welfare, criminal justice, education, job placement, benefits, and housing.

“Algorithmic systems are being used to make high-stakes decisions about human lives and welfare in the public sector ranging across child-welfare, criminal justice, public education, job placement centers, welfare benefits, and housing among others.”

Public-sector practitioners continually acquire new skills for making decisions through data systems and interpreting algorithmic outputs.

“Practitioners in the public sector are continually acquiring new skills as they learn to make decisions through data systems and interpret algorithmic outputs [17,21], however, these new skills are not being accounted for with respect to what constitutes professional expertise [62].”

• cites [17] 17. Mark Bovens and Stavros Zouridis. 2002. From street-level to system-level bureaucracies: how information and communication technology is transforming administrative discretion and constitutional control. Public administration review 62, 2 (2002), 174-184. sourceNot checked — no full text of the source
• cites [21] 21. Justin B Bullock. 2019. Artificial intelligence, discretion, and bureaucracy. The American Review of Public Administration 49, 7 (2019), 751-761. sourceNot checked — no full text of the source

Several studies treat algorithms as peripheral end products, a new part of bureaucracy to which human discretion must adapt.

“However, several of these studies, rich in their understanding of human discretion and bureaucracy, continue to treat algorithms as peripheral end products; a new part of bureaucracy to which human discretion must adapt.”

Child-welfare workers are often frustrated by algorithms because they do not account for scarce public-sector resources.

“For instance, child-welfare workers are often frustrated by algorithms because they do not account for the scarce resources in the public sector [50].”

• cites [50] 50. Eileen Gambrill and Aron Shlonsky. 2001. The need for comprehensive risk management systems in child welfare. Children and Youth Services Review 23, 1 (2001), 79-107. sourceNot checked — no full text of the source

Decisions must be made within policy and systemic constraints that algorithms do not account for.

“Moreover, decisions must be made within the constraints of policies and systemic barriers; characteristics not accounted for by algorithms.”

Aggregate administrative data is biased for predicting individual behavior.

“Second, current practices of using aggregate administrative data that is often poorly collected [31,53,107] and biased to predict an individual's behavior is a complex and hard task that may lead to unfairness in decision-making outcomes and is, in most western, liberal democratic systems unconstitutional and/or illegal [123].”

Child-welfare algorithms have historically used a narrow set of psychometric predictors to assess foster children's and parents' risks and needs.

“Algorithms in child-welfare have historically relied on a narrow set of psychometric predictors that are used to assess the risks and needs of foster children and parents.”

Using aggregate administrative data to predict individual behavior is unconstitutional or illegal in most western liberal democracies.

“Second, current practices of using aggregate administrative data that is often poorly collected [31,53,107] and biased to predict an individual's behavior is a complex and hard task that may lead to unfairness in decision-making outcomes and is, in most western, liberal democratic systems unconstitutional and/or illegal [123].”

• cites [123] 123. Paul Voigt and Axel Von dem Bussche. 2017. The eu general data protection regulation (gdpr). A Practical Guide, 1st Ed., Cham: Springer International Publishing (2017). sourceNot checked — no full text of the source

These factors make public-sector algorithmic decision-making a high-stakes environment with real consequences for people's lives and liberties.

“These two factors combine to make algorithmic decision-making in the public sector, a high stakes decision-making environment that has real repercussions for the lives and liberties of people.”

Many public-sector algorithm implementations have life-altering or life-and-death consequences.

“In short, many of the ways in which algorithms are being implemented in the public sector result in life altering, if not life and/or death consequences.”

Practitioners' personal values are often mediated by organizational culture, which can yield results different from those centered in personal values.

“Moreover, practitioner's personal values are often mediated by organizational culture which subsequently yields results that can be significantly different than results centered in personal values [48].”

• cites [48] 48. David J Fritzsche. 1991. A model of decision-making incorporating ethical values. Journal of Business Ethics 10, 11 (1991), 841-852. sourceNot checked — no full text of the source

Organizational training establishes worker expertise by helping workers mediate practice and bureaucratic processes.

“Training at the organization also establishes the basis for worker expertise by ensuring that the workers skillfully mediate both the nature of practice and bureaucratic processes [124].”

• cites [124] 124. Sandra K Welch. 2008. A metasynthesis of the transition from novice to expert: Can instructional interventions shorten the process? Ph.D. Dissertation. Capella University.Not checked yet

  Not checked in the free run.

Calls for human-in-the-loop may be moot without understanding how algorithms affect human decision-making and how decision-support type affects human intervention.

“Moreover, the calls for human-in-the-loop might be moot if there is a lack of understanding about how algorithms impact human decision-making and how the type of decision-support (i.e.-algorithm design) impacts the practical possibilities for human intervention [96,111,113].”

• cites [96] 96. Rik Peeters. 2020. The agency of algorithms: Understanding human-algorithm interaction in administrative decision- making. Information Polity Preprint (2020), 1-16. sourceNot checked — no full text of the source
• cites [111] 111. Craig Schwalbe. 2004. Re-visioning risk assessment for human service decision making. Children and Youth Services Review 26, 6 (2004), 561-576. sourceNot checked — no full text of the source
• cites [113] 113. Aron Shlonsky and Dennis Wagner. 2005. The next step: Integrating actuarial risk assessment and clinical judgment into an evidence-based practice framework in CPS case management. Children and youth services review 27, 4 (2005), 409-427. sourceNot checked — no full text of the source

Organizational processes help explain how an agency makes decisions within policy mandates and meets diverse public needs.

“Organizational processes offer the means to understand how an agency makes decisions within policy mandates as well as how it meets diverse public needs [16].”

• cites [16] 16. Herman L Boschken. 1994. Organizational performance and multiple constituencies. Public Administration Review 54, 3 (1994), 308-312. sourceNot checked — no full text of the source

Organizational decision-making must involve professionals' heuristics because ideal conditions for rational and reductive models rarely hold in an uncertain world.

“Moreover, decision-making in organizations must involve professionals' heuristics because the ideal conditions required for rational and reductive models rarely hold true in an uncertain world [95].”

• cites [95] 95. John W Payne, James R Bettman, and Eric J Johnson. 1993. The use of multiple strategies in judgment and choice. (1993).Not checked yet

  Not checked in the free run.

Processes followed by practitioners daily continually shape policy on the ground.

“Moreover, processes followed by practitioners in their daily lives also continually shape policy on the ground [18,74].”

• cites [18] 18. Evelyn Z Brodkin. 2008. Accountability in street-level organizations. Intl Journal of Public Administration 31, 3 (2008), 317-336. sourceNot checked — no full text of the source
• cites [74] 74. Michael Lipsky. 2010. Street-level bureaucracy: Dilemmas of the individual in public service. Russell Sage Foundation.Not checked yet

  Not checked in the free run.

The incentive for foster parents to exaggerate behaviors is a constraint introduced by CANS's implementation.

“This is a constraint introduced by the implementation of the algorithm itself.”

CWS employees were aware that cost-benefit analyses tied to subjective mental health assessments were problematic.

“CWS employees were also well-aware that such cost-benefit analyses tied directly to something as subjective as mental health assessments were problematic.”

The lack of relevant data and decision-making latitude turned CANS primarily into a rate-setting tool for compensation.

“The lack of relevant data and lack of decisionmaking latitude on part of caseworkers has turned CANS into primarily a rate-setting tool to calculate compensations.”

CANS was re-appropriated to calculate compensation for caring for a foster child's mental health needs, even though its primary purposes were mental health assessment and level of foster care.

“Thus, even though the primary purpose of CANS was for conducting mental health assessments and for the level of foster care, it was re-appropriated to also calculate compensation associated with caring for a foster child's mental health needs.”

The CANS assessment was designed based on communication theory rather than psychometric measurement theory.

“CANS assessment is designed based on communication theory rather than psychometric theories centered in measurement development [78].”

• cites [78] 78. John S Lyons. 2009. Communimetrics: A communication theory of measurement in human service settings. Springer Science & Business Media. sourceNot checked — no full text of the source

The 7ei staffing protocol helps the child-welfare team apply trauma-informed care principles and practices to each case.

“Central to these meetings is the 7ei staffing protocol that helps the child-welfare team apply principles and practices derived from trauma-informed care (TIC) [60] to each case.”

• cites [60] 60. Alison Hendricks, Lisa Conradi, and Charles Wilson. 2011. Creating trauma-informed child welfare systems using a community assessment process. Child Welfare 90, 6 (2011), 187.Not checked yet

  Not checked in the free run.

7ei, based in trauma-informed care, has been proven to improve child outcomes such as placement stability and permanence.

“7ei algorithm based in TIC has also been proven to improve child outcomes such as placement stability and permanence [119].”

• cites [119] 119. James Topitzes, Timothy Grove, Erika E Meyer, Stacey M Pangratz, and Caitlin M Sprague. 2019. Trauma-responsive child welfare services: A mixed methods study assessing safety, stability, and permanency. Journal of Child Custody 16, 3 (2019), 291-312. sourceNot checked — no full text of the source

7ei tracks outcomes over time to assess a child-welfare case's trajectory rather than predicting an outcome from other factors.

“Instead of predicting an outcome of interest using other factors, 7ei is used to track outcomes (i.e.-7ei domains) over time to assess the trajectory of a child-welfare case.”

The agency uses trauma-informed care as its core guiding principle, embedded in its practice and trauma-responsive service model.

“The agency uses trauma-informed care [60] as their core guiding principle, which is embedded in their practice and trauma-responsive service model [119].”

• cites [60] 60. Alison Hendricks, Lisa Conradi, and Charles Wilson. 2011. Creating trauma-informed child welfare systems using a community assessment process. Child Welfare 90, 6 (2011), 187.Not checked yet

  Not checked in the free run.

• cites [119] 119. James Topitzes, Timothy Grove, Erika E Meyer, Stacey M Pangratz, and Caitlin M Sprague. 2019. Trauma-responsive child welfare services: A mixed methods study assessing safety, stability, and permanency. Journal of Child Custody 16, 3 (2019), 291-312. sourceNot checked — no full text of the source

The prescriptive nature of 7ei helps teams adaptively select data and address uncertainties.

“We found that the prescriptive nature of 7ei helps adaptively select data and address uncertainties.”

The child-welfare system lacks experienced caseworkers because of high turnover.

“This is especially important for CWS since the system lacks experienced caseworkers due to high turnover [24].”

• cites [24] 24. Sarah Carnochan, Megan Moore, and Michael J Austin. 2013. Achieving placement stability. Journal of evidence-based social work 10, 3 (2013), 235-253. sourceNot checked — no full text of the source

The 7ei domains are also the outcomes of interest the team seeks to track and affect over time.

“The 7ei domains are also the outcomes of interest that the team seeks to track and affect over time.”

Tensions arise when algorithms attempt to supplant human discretion.

“On the other hand, tensions arise when algorithms attempts to supplant human discretion.”

Algorithms that do not embed human values may minimize them.

“On the contrary, algorithms that do not embed human values into its design may consequentially end up minimizing them.”

The influx leaves HART with fewer resources for cases that need them.

“This leaves HART with significantly less resources to focus on the cases that need them.”

7ei is centered in social work values of service, dignity and worth of the person, and importance of human relationships.

“7ei algorithm is centered in some of social work's core values of service, dignity and worth of the person, and importance of human relationships [119]”

• cites [119] 119. James Topitzes, Timothy Grove, Erika E Meyer, Stacey M Pangratz, and Caitlin M Sprague. 2019. Trauma-responsive child welfare services: A mixed methods study assessing safety, stability, and permanency. Journal of Child Custody 16, 3 (2019), 291-312. sourceNot checked — no full text of the source

When aligned, algorithms can help embed important value judgments into decision-making.

“When aligned, algorithms can help embed important value judgments into the decisionmaking process.”

Without autonomy, gaming the algorithm is the only way caseworkers can exercise discretion and produce desired outcomes.

“With a lack of autonomy, gaming the algorithm is the only way caseworkers are able to exercise discretion and produce the desired outcome.”

Algorithms that replace the need for caseworker expertise can lead to inadequate and unreliable decision-making.

“Algorithms that replace the need for caseworker expertise can lead to inadequate and unreliable decision-making.”

When algorithmic decision-making and bureaucratic processes align, it can help train caseworkers.

“When algorithmic decision-making and bureaucratic processes are aligned, it can help train caseworkers.”

A large portion of the information used in child-welfare reflects the caseworker's own perception of the family.

“Therefore, much of the information is based on the caseworker's perception of the family.”

The agency created a four-part evidence-based service program centered on trauma-informed care, with the algorithmic tool as only one part.

“The agency developed a comprehensive four-part evidence-based service program centered in trauma-informed care of which the algorithmic tool is just one component.”

Many psychometric assessments routinely used in child-welfare to evaluate child and parent risks and needs now have algorithmic counterparts in risk assessment tools.

“Moreover, several psychometric assessments that are routinely used in child-welfare to assess the risks and needs of children and parents have adopted algorithmic analogues in the form of risk assessment algorithms.”

The agency invested substantial resources so that TIC practice and 7ei were fully supported by bureaucratic processes.

“In sum, the agency significantly invested in resources to ensure that TIC practice and 7ei were fully-supported by bureaucratic processes.”

The requirement to conduct CANS every six months makes it unfeasible for caseworkers to interview anyone beyond foster parents.

“Moreover, the bureaucratic policy that requires CANS to be conducted every six months makes it unfeasible for caseworkers to interview anyone beyond the foster parents.”

• AssumptionRests on an assumption: CANS must be conducted every six months.

CWS began using algorithms to track performance measures such as permanency and placement stability because federal legislation required it.

“CWS started using algorithms as a means to track important performance metrics such as permanency and placement stability as mandated by federal legislation [127].”

• cites [127] 127. C Wilson. 2014. Integrating safety, permanency, and well-being: A view from the field. Integrating safety, permanency and well-being series (2014).Not checked yet

  Not checked in the free run.

Caseworkers developed collective buy-in because they better understood families' psychological experiences and knew how to help them.

“A better understanding of what a family might be going through at a psychological level and knowing exactly how to help them has led to the collective buy-in from caseworkers.”

Administrative data produced through bureaucratic processes should not be used uncritically to develop algorithms.

“Administrative data curated through bureaucratic processes can not be uncritically used to develop algorithms.”

When an algorithmic decision-support system is not aligned with bureaucratic constraints, it leads to utility issues.

“When the algorithmic decision-support system is not aligned with the constraints of the bureaucratic system, it leads to utility issues.”

The utility of algorithmic decision-support systems is severely limited when they do not account for organizational constraints.

“The utility of algorithmic decision-support systems is severely limited when they do not account for the organizational constraints within which they must operate.”

Caseworkers who feel frustrated and powerless with CANS are more likely to manipulate scores to obtain a higher rate than to seek a supervisor-approved exceptional rate.

“Caseworkers who continue to feel disempowered and frustrated by CANS find it easier to manipulate CANS scores to produce a higher rate than request the addition of an exceptional rate which needs to be approved by the supervisor.”

Professional expertise in the public sector deteriorates when algorithms limit value judgments and heuristic decision-making.

“Professional expertise in the public sector deteriorates when algorithms limit the scope for value judgments and heuristic decision-making.”

Because algorithms cannot account for all information, categorizing children and families leads to lost information about the final outcome.

“However, this inadvertently leads to a loss of information with respect to the final outcome since all the information cannot be accounted for by these algorithms.”

• AssumptionRests on an assumption: Algorithms attempt to generalize child and family characteristics and place them into categories in order to make determinations.
• AssumptionRests on an assumption: Algorithms cannot account for all information.

Repeated encounters with algorithms that miss important nuances and context cause distrust to build up.

“Continued interactions illustrated by these examples leads to an accumulation of distrust towards algorithms that often miss important nuances and context.”

Contradictory incentives and conflicting values often force caseworkers into positions where they make such decisions.

“With contradictory incentive structures and conflicting values, caseworkers are often put in a position where they are forced to make such decisions.”

Knowing a family's past helps caseworkers determine which services would benefit the family most.

“Understanding a family's past helps caseworkers assess the type of services that would be most beneficial for them.”

A key implication of ADMAPS is that algorithmic decisions improve when algorithms account for and balance complex socio-technical interdependencies instead of operating alone.

“A key implication of using ADMAPS is that better algorithmic decisions are made when algorithms account for and balance the complex interdependencies within socio-technical systems, rather than operate in isolation.”

ADMAPS revealed that CANS has multiple conflicting roles as both a compensation calculator and a mental-health assessment tool.

“That is, ADMAPS uncovered the multiple and conflicting roles of CANS as a compensation calculator as well as a mental health assessment tool.”

As a result, improvements or deteriorations in mental health affect placement decisions.

“Consequently, an improvement (or deterioration) in mental health is impacting placement decisions.”