Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index.

Improving Human-AI Partnerships in Child Welfare: Understanding Worker Practices, Challenges, and Desires for Algorithmic Decision Support

Open access

Abstract

AI-based decision support tools (ADS) are increasingly used to augment human decision-making in high-stakes, social contexts. As public sector agencies begin to adopt ADS, it is critical that we understand workers’ experiences with these systems in practice. In this paper, we present findings from a series of interviews and contextual inquiries at a child welfare agency, to understand how they currently make AI-assisted child maltreatment screening decisions. Overall, we observe how workers’ reliance upon the ADS is guided by (1) their knowledge of rich, contextual information beyond what the AI model captures, (2) their beliefs about the ADS’s capabilities and limitations relative to their own, (3) organizational pressures and incentives around the use of the ADS, and (4) awareness of misalignments between algorithmic predictions and their own decision-making objectives. Drawing upon these findings, we discuss design implications towards supporting more effective human-AI decision-making.

Overview

Integrity signals

Registry

Crossref · 10.1145/3491102.3517439

Cites

45 of 45 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 42 references

Research works printed but not linked: 24

  1. Gagan Bansal , Besmira Nushi , Ece Kamar , Walter S Lasecki , Daniel S Weld , and Eric Horvitz . 2019 . Beyond Accuracy: The Role of Mental Models in Human-AI Team Performance . Proceedings of the AAAI Conference on Human Computation and Crowdsourcing 7, 1(2019) , 19. www.aaai.org Gagan Bansal, Besmira Nushi, Ece Kamar, Walter S Lasecki, Daniel S Weld, and Eric Horvitz. 2019. Beyond Accuracy: The Role of Mental Models in Human-AI Team Performance. Proceedings of the AAAI Conference on Human Computation and Crowdsourcing 7, 1(2019), 19. www.aaai.org
  2. Hugh Beyer and Karen Holtzblatt . 1999. Contextual design. interactions 6, 1 ( 1999 ), 32–42. Hugh Beyer and Karen Holtzblatt. 1999. Contextual design. interactions 6, 1 (1999), 32–42.
  3. 10.1145/3449287 · doi:10.1145/3449287 · Paper page
  4. Marc Cherna. [n.d.]. We will use all resources to keep children safe. Pittsburgh Post-Gazette([n. d.]). https://www.post-gazette.com/opinion/letters/2018/03/23/We-will-use-all-resources-to-keep-children-safe/stories/201803230094 Online; accessed 8-September-2021. Marc Cherna. [n.d.]. We will use all resources to keep children safe. Pittsburgh Post-Gazette([n. d.]). https://www.post-gazette.com/opinion/letters/2018/03/23/We-will-use-all-resources-to-keep-children-safe/stories/201803230094 Online; accessed 8-September-2021.
  5. Maria De-Arteaga , Artur Dubrawski , and Alexandra Chouldechova . 2021. Leveraging expert consistency to improve algorithmic decision support. arXiv ( 2021 ), 1–33. arxiv:2101.09648http://arxiv.org/abs/2101.09648 Maria De-Arteaga, Artur Dubrawski, and Alexandra Chouldechova. 2021. Leveraging expert consistency to improve algorithmic decision support. arXiv (2021), 1–33. arxiv:2101.09648http://arxiv.org/abs/2101.09648
  6. Maria De-Arteaga , Riccardo Fogliato , and Alexandra Chouldechova . 2020. A case for humans-in-the-loop: Decisions in the presence of erroneous algorithmic scores. arXiv ( 2020 ), 1–12. Maria De-Arteaga, Riccardo Fogliato, and Alexandra Chouldechova. 2020. A case for humans-in-the-loop: Decisions in the presence of erroneous algorithmic scores. arXiv (2020), 1–12.
  7. Berkeley J Dietvorst , Joseph P Simmons , and Cade Massey . 2015 . Algorithm aversion: People erroneously avoid algorithms after seeing them err.Journal of Experimental Psychology : General 144 , 1 (2015), 114 . Berkeley J Dietvorst, Joseph P Simmons, and Cade Massey. 2015. Algorithm aversion: People erroneously avoid algorithms after seeing them err.Journal of Experimental Psychology: General 144, 1 (2015), 114.
  8. Ben Green . 2021. The flaws of policies requiring human oversight of government algorithms. Available at SSRN ( 2021 ). Ben Green. 2021. The flaws of policies requiring human oversight of government algorithms. Available at SSRN (2021).
  9. D. Jackson and G. Marx. 2017. Data mining program designed to predict child abuse proves unreliable DCFS says.http://www.jstor.org/stable/10.7758/9781610447713 D. Jackson and G. Marx. 2017. Data mining program designed to predict child abuse proves unreliable DCFS says.http://www.jstor.org/stable/10.7758/9781610447713 · doi:10.7758/9781610447713
  10. Daniel Kahneman , Olivier Sibony , and Cass R Sunstein . 2021 . Noise: A flaw in human judgment. Little, Brown. Daniel Kahneman, Olivier Sibony, and Cass R Sunstein. 2021. Noise: A flaw in human judgment. Little, Brown.
  11. Emily Keddell . 2019. Algorithmic Justice in Child Protection: Statistical Fairness, Social Justice and the Implications for Practice. Social Sciences ( 2019 ). Emily Keddell. 2019. Algorithmic Justice in Child Protection: Statistical Fairness, Social Justice and the Implications for Practice. Social Sciences (2019).
  12. Jon Kleinberg , Himabindu Lakkaraju , Jure Leskovec , Jens Ludwig , and Sendhil Mullainathan . 2018. Human decisions and machine predictions. The quarterly journal of economics 133, 1 ( 2018 ), 237–293. Jon Kleinberg, Himabindu Lakkaraju, Jure Leskovec, Jens Ludwig, and Sendhil Mullainathan. 2018. Human decisions and machine predictions. The quarterly journal of economics 133, 1 (2018), 237–293.
  13. Vivian Lai Chacha Chen Q Vera Liao Alison Smith-Renner and Chenhao Tan. 2021. Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies. arXiv preprint arXiv:2112.11471(2021). Vivian Lai Chacha Chen Q Vera Liao Alison Smith-Renner and Chenhao Tan. 2021. Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies. arXiv preprint arXiv:2112.11471(2021). · arXiv
  14. John D Lee and Katrina A See . 2004. Trust in automation: Designing for appropriate reliance. Human factors 46, 1 ( 2004 ), 50–80. John D Lee and Katrina A See. 2004. Trust in automation: Designing for appropriate reliance. Human factors 46, 1 (2004), 50–80.
  15. Emma Lurie and Deirdre K Mulligan . 2020 . Crowdworkers are not judges: Rethinking crowdsourced vignette studies as a risk assessment evaluation technique . Proceedings of the Workshop on Fair and Responsible AI at CHI 2020 (2020). Emma Lurie and Deirdre K Mulligan. 2020. Crowdworkers are not judges: Rethinking crowdsourced vignette studies as a risk assessment evaluation technique. Proceedings of the Workshop on Fair and Responsible AI at CHI 2020 (2020).
  16. Hussein Mozannar Arvind Satyanarayan and David Sontag. 2021. Teaching Humans When To Defer to a Classifier via Examplars. arXiv preprint arXiv:2111.11297(2021). Hussein Mozannar Arvind Satyanarayan and David Sontag. 2021. Teaching Humans When To Defer to a Classifier via Examplars. arXiv preprint arXiv:2111.11297(2021). · arXiv
  17. Allegheny County Department of Human Services. [n.d.]. The Allegheny Family Screening Tool. https://www.alleghenycounty.us/Human-Services/News-Events/Accomplishments/Allegheny-Family-Screening-Tool.aspx. Online ; accessed 7- September - 2021 . Allegheny County Department of Human Services. [n.d.]. The Allegheny Family Screening Tool. https://www.alleghenycounty.us/Human-Services/News-Events/Accomplishments/Allegheny-Family-Screening-Tool.aspx. Online; accessed 7-September-2021.
  18. Allegheny County Department of Human Services. [n.d.]. Allegheny Family Screening Tool , Frequently-Asked Questions | Updated August 2018 . https://www.alleghenycountyanalytics.us/wp-content/uploads/2018/10/17-ACDHS-11_AFST_102518.pdf. Online; accessed 8-September-2021. Allegheny County Department of Human Services. [n.d.]. Allegheny Family Screening Tool, Frequently-Asked Questions | Updated August 2018. https://www.alleghenycountyanalytics.us/wp-content/uploads/2018/10/17-ACDHS-11_AFST_102518.pdf. Online; accessed 8-September-2021.
  19. Anjana Samant , Aaron Horowitz , Kath Xu , and Sophie Beiers . 2021. Family surveillance by algorithm: The rapidly spreading tools few have heard of . American Civil Liberties Union (ACLU)( 2021 ). https://www.aclu.org/sites/default/files/field_document/2021.09.28a_family_surveillance_by_algorithm.pdf Anjana Samant, Aaron Horowitz, Kath Xu, and Sophie Beiers. 2021. Family surveillance by algorithm: The rapidly spreading tools few have heard of. American Civil Liberties Union (ACLU)(2021). https://www.aclu.org/sites/default/files/field_document/2021.09.28a_family_surveillance_by_algorithm.pdf
  20. Devansh Saxena , Karla Badillo-Urquiola , Pamela Wisniewski , and Shion Guha . 2021. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child welfare. arXiv 5 , October ( 2021 ). arxiv:arXiv:2107.03487v2 Devansh Saxena, Karla Badillo-Urquiola, Pamela Wisniewski, and Shion Guha. 2021. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child welfare. arXiv 5, October (2021). arxiv:arXiv:2107.03487v2 · arXiv
  21. Sarah Tan , Julius Adebayo , and Kori Inkpen . 2018. Investigating human + machine complementarity for recidivism predictions. arXiv ( 2018 ). arxiv:arXiv:1808.09123v2 Sarah Tan, Julius Adebayo, and Kori Inkpen. 2018. Investigating human + machine complementarity for recidivism predictions. arXiv (2018). arxiv:arXiv:1808.09123v2 · arXiv
  22. Rhema Vaithianathan , Emily Putnam-Hornstein , Nan Jiang , Parma Nand , and Tim Maloney . 2017. Developing predictive models to support child maltreatment hotline screening decisions: Allegheny County methodology and implementation . Center for Social data Analytics( 2017 ). Rhema Vaithianathan, Emily Putnam-Hornstein, Nan Jiang, Parma Nand, and Tim Maloney. 2017. Developing predictive models to support child maltreatment hotline screening decisions: Allegheny County methodology and implementation. Center for Social data Analytics(2017).
  23. Elmira van den Broek , Anastasia Sergeeva , and Marleen Huysman . 2020 . Hiring algorithms: An ethnography of fairness in practice . 40th International Conference on Information Systems, ICIS 2019 (2020). Elmira van den Broek, Anastasia Sergeeva, and Marleen Huysman. 2020. Hiring algorithms: An ethnography of fairness in practice. 40th International Conference on Information Systems, ICIS 2019 (2020).
  24. MW Van Someren , YF Barnard , and JAC Sandberg . 1994. The think aloud method: a practical approach to modelling cognitive . London : AcademicPress( 1994 ). MW Van Someren, YF Barnard, and JAC Sandberg. 1994. The think aloud method: a practical approach to modelling cognitive. London: AcademicPress(1994).

Cited by

See all 150

Metadata

Authors
Kawakami, A., Sivaraman, V., Cheng, H.-F., Stapleton, L., Cheng, Y., Qing, D., Perer, A., Wu, Z. S., Zhu, H., & Holstein, K.
Affiliations
  • Carnegie Mellon University (US)
  • University of Minnesota (US)
  • University of California, Berkeley (US)
Venue
CHI Conference on Human Factors in Computing Systems
Publisher
ACM
Published
2022-04-27
Pages
1-18
OpenAlex
W4225001143
Cited by
143
References
42
Citations by year
2026: 25 · 2025: 37 · 2024: 44 · 2023: 30 · 2022: 7
FWCI
29.32
Type
conference paper
Language
en
Open access
Locations
  • CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by-nc-sa)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
NSF (National Science Foundation) (1939606)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Social Sciences › Business, Management and Accounting › Management Information Systems › Information Systems Theories and Implementation
  • Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
Keywords
human-AI collaboration, algorithmic decision support, child welfare, child maltreatment screening, AI-assisted decision-making, contextual information, trust in AI predictions, organizational incentives, public sector organizations, design implications

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Improving Human-AI Partnerships in Child Welfare: Understanding Worker Practices, Challenges, and Desires for Algorithmic Decision Support. https://app.cochat.ai/papers/10.1145/3491102.3517439
The paper
Kawakami, A., Sivaraman, V., Cheng, H.-F., Stapleton, L., Cheng, Y., Qing, D., Perer, A., Wu, Z. S., Zhu, H., & Holstein, K. (2022). Improving Human-AI Partnerships in Child Welfare: Understanding Worker Practices, Challenges, and Desires for Algorithmic Decision Support. CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3491102.3517439

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.