The DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index.
Improving Human-AI Partnerships in Child Welfare: Understanding Worker Practices, Challenges, and Desires for Algorithmic Decision Support
Open accessAbstract
AI-based decision support tools (ADS) are increasingly used to augment human decision-making in high-stakes, social contexts. As public sector agencies begin to adopt ADS, it is critical that we understand workers’ experiences with these systems in practice. In this paper, we present findings from a series of interviews and contextual inquiries at a child welfare agency, to understand how they currently make AI-assisted child maltreatment screening decisions. Overall, we observe how workers’ reliance upon the ADS is guided by (1) their knowledge of rich, contextual information beyond what the AI model captures, (2) their beliefs about the ADS’s capabilities and limitations relative to their own, (3) organizational pressures and incentives around the use of the ADS, and (4) awareness of misalignments between algorithmic predictions and their own decision-making objectives. Drawing upon these findings, we discuss design implications towards supporting more effective human-AI decision-making.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“researchers have explored the feasibility of human-AI collaborative decision-making, in which an AI model provides humans new data-driven insights on a task for achieving complementary performance, outperforming neither of the AI or the human alone [5, 30, 42, 43].”
From Understanding the Effect of Counterfactual Explanations on Trust and Reliance on AI for Human-AI Collaborative Clinical Decision Making (2023) · cites this paper as [30]
In the citing paper
“Concerns about provider liability have been a long-recognized challenge in recommendation-driven AI assistance.”
From To Recommend or Not to Recommend: Designing and Evaluating AI-Enabled Decision Support for Time-Critical Medical Events (2025) · cites this paper as [37]
In the citing paper
“If there is any institutional friction to disagreeing with AI recommendations, such as increased chance of liability for a medical error resulting from overriding AI recommendation, the presence of the algorithmic recommendation can be seen as taking away some authority from the human decision-maker without assuming any of the responsibility for the decision [30, 37, 72].”
From To Recommend or Not to Recommend: Designing and Evaluating AI-Enabled Decision Support for Time-Critical Medical Events (2025) · cites this paper as [37]
In the citing paper
“Similar to findings from prior work, our participants raised concerns about the effects of AI-enabled tools on their authority and liability [30, 37, 72].”
From To Recommend or Not to Recommend: Designing and Evaluating AI-Enabled Decision Support for Time-Critical Medical Events (2025) · cites this paper as [37]
Citations
Cites
Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor
Lebovits, H. (2019) · Public Integrity · doi:10.1080/10999922.2018.1511671 · Cited by 3,279 · cited 2×
“, reflecting biases in the administrative data on which they are trained and operate upon) [13, 17] or introducing absurd errors into decision processes, which humans would not otherwise make [1, 15, 28].”
BackgroundResultBuilding Machines That Learn and Think Like People
Lake, B. M., Ullman, T. D., Tenenbaum, J. B., & Gershman, S. J. (2016) · arXiv · doi:10.48550/arxiv.1604.00289 · Cited by 2,550
McDonald, N., Schoenebeck, S., & Forte, A. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359174 · Cited by 1,206 · cited 1×
“Following a shared open coding session to calibrate coding granularity, each participant’s datawere qualitatively coded by two to three researchers [42].”
MethodBuçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021) · arXiv · doi:10.48550/arxiv.2102.09692 · Cited by 1,124
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445717 · Cited by 757 · cited 8×
“Whereas prior work has often studied human reliance on ADS tools as a function of an individual’s trust [5, 9, 38, 50], this conceptualization overlooks the potential impacts that organizational pressures and incentive structures may have on workers’ reliance, independent of their trust in the technology itself.”
BackgroundResultInfluentialDoes the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. S. (2020) · arXiv · doi:10.48550/arxiv.2006.14779 · Cited by 754 · cited 8×
“Whereas prior work has often studied human reliance on ADS tools as a function of an individual’s trust [5, 9, 38, 50], this conceptualization overlooks the potential impacts that organizational pressures and incentive structures may have on workers’ reliance, independent of their trust in the technology itself.”
BackgroundResultInfluentialManipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Wortman Vaughan, J. W., & Wallach, H. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445315 · Cited by 728 · cited 3×
“Prior work has often understood human reliance on algorithmic decision supports in terms of patterns of “over-reliance,” “underreliance,” or “appropriate reliance”, that are shaped according to how well-calibrated an individual’s trust is in a given algorithmic system [5, 9, 38, 50].”
BackgroundCai, C. J., Winter, S., Steiner, D., Wilcox, L., & Terry, M. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359206 · Cited by 594 · cited 1×
“, where it is likely to err and what factors it is not able to take into account) [4, 10, 43].”
BackgroundThe What-If Tool: Interactive Probing of Machine Learning Models
Wexler, J., Pushkarna, M., Bolukbasi, T., Wattenberg, M., Viegas, F., & Wilson, J. (2019) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2019.2934619 · Cited by 505 · cited 1×
“[65]) in order to learn how particular factors impact the AFST score in specific cases (C1, C6, C8, S1).”
BackgroundVeale, M., Van Kleek, M., & Binns, R. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174014 · Cited by 490 · cited 1×
“At the same time, AI-based judgments are themselves likely to be imperfect and biased, even if in different ways than humans [15, 32, 39, 64].”
Background
Research works printed but not linked: 24
- Gagan Bansal , Besmira Nushi , Ece Kamar , Walter S Lasecki , Daniel S Weld , and Eric Horvitz . 2019 . Beyond Accuracy: The Role of Mental Models in Human-AI Team Performance . Proceedings of the AAAI Conference on Human Computation and Crowdsourcing 7, 1(2019) , 19. www.aaai.org Gagan Bansal, Besmira Nushi, Ece Kamar, Walter S Lasecki, Daniel S Weld, and Eric Horvitz. 2019. Beyond Accuracy: The Role of Mental Models in Human-AI Team Performance. Proceedings of the AAAI Conference on Human Computation and Crowdsourcing 7, 1(2019), 19. www.aaai.org
- Hugh Beyer and Karen Holtzblatt . 1999. Contextual design. interactions 6, 1 ( 1999 ), 32–42. Hugh Beyer and Karen Holtzblatt. 1999. Contextual design. interactions 6, 1 (1999), 32–42.
- 10.1145/3449287 · doi:10.1145/3449287 · Paper page
- Marc Cherna. [n.d.]. We will use all resources to keep children safe. Pittsburgh Post-Gazette([n. d.]). https://www.post-gazette.com/opinion/letters/2018/03/23/We-will-use-all-resources-to-keep-children-safe/stories/201803230094 Online; accessed 8-September-2021. Marc Cherna. [n.d.]. We will use all resources to keep children safe. Pittsburgh Post-Gazette([n. d.]). https://www.post-gazette.com/opinion/letters/2018/03/23/We-will-use-all-resources-to-keep-children-safe/stories/201803230094 Online; accessed 8-September-2021.
- Maria De-Arteaga , Artur Dubrawski , and Alexandra Chouldechova . 2021. Leveraging expert consistency to improve algorithmic decision support. arXiv ( 2021 ), 1–33. arxiv:2101.09648http://arxiv.org/abs/2101.09648 Maria De-Arteaga, Artur Dubrawski, and Alexandra Chouldechova. 2021. Leveraging expert consistency to improve algorithmic decision support. arXiv (2021), 1–33. arxiv:2101.09648http://arxiv.org/abs/2101.09648
- Maria De-Arteaga , Riccardo Fogliato , and Alexandra Chouldechova . 2020. A case for humans-in-the-loop: Decisions in the presence of erroneous algorithmic scores. arXiv ( 2020 ), 1–12. Maria De-Arteaga, Riccardo Fogliato, and Alexandra Chouldechova. 2020. A case for humans-in-the-loop: Decisions in the presence of erroneous algorithmic scores. arXiv (2020), 1–12.
- Berkeley J Dietvorst , Joseph P Simmons , and Cade Massey . 2015 . Algorithm aversion: People erroneously avoid algorithms after seeing them err.Journal of Experimental Psychology : General 144 , 1 (2015), 114 . Berkeley J Dietvorst, Joseph P Simmons, and Cade Massey. 2015. Algorithm aversion: People erroneously avoid algorithms after seeing them err.Journal of Experimental Psychology: General 144, 1 (2015), 114.
- Ben Green . 2021. The flaws of policies requiring human oversight of government algorithms. Available at SSRN ( 2021 ). Ben Green. 2021. The flaws of policies requiring human oversight of government algorithms. Available at SSRN (2021).
- D. Jackson and G. Marx. 2017. Data mining program designed to predict child abuse proves unreliable DCFS says.http://www.jstor.org/stable/10.7758/9781610447713 D. Jackson and G. Marx. 2017. Data mining program designed to predict child abuse proves unreliable DCFS says.http://www.jstor.org/stable/10.7758/9781610447713 · doi:10.7758/9781610447713
- Daniel Kahneman , Olivier Sibony , and Cass R Sunstein . 2021 . Noise: A flaw in human judgment. Little, Brown. Daniel Kahneman, Olivier Sibony, and Cass R Sunstein. 2021. Noise: A flaw in human judgment. Little, Brown.
- Emily Keddell . 2019. Algorithmic Justice in Child Protection: Statistical Fairness, Social Justice and the Implications for Practice. Social Sciences ( 2019 ). Emily Keddell. 2019. Algorithmic Justice in Child Protection: Statistical Fairness, Social Justice and the Implications for Practice. Social Sciences (2019).
- Jon Kleinberg , Himabindu Lakkaraju , Jure Leskovec , Jens Ludwig , and Sendhil Mullainathan . 2018. Human decisions and machine predictions. The quarterly journal of economics 133, 1 ( 2018 ), 237–293. Jon Kleinberg, Himabindu Lakkaraju, Jure Leskovec, Jens Ludwig, and Sendhil Mullainathan. 2018. Human decisions and machine predictions. The quarterly journal of economics 133, 1 (2018), 237–293.
- Vivian Lai Chacha Chen Q Vera Liao Alison Smith-Renner and Chenhao Tan. 2021. Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies. arXiv preprint arXiv:2112.11471(2021). Vivian Lai Chacha Chen Q Vera Liao Alison Smith-Renner and Chenhao Tan. 2021. Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies. arXiv preprint arXiv:2112.11471(2021). · arXiv
- John D Lee and Katrina A See . 2004. Trust in automation: Designing for appropriate reliance. Human factors 46, 1 ( 2004 ), 50–80. John D Lee and Katrina A See. 2004. Trust in automation: Designing for appropriate reliance. Human factors 46, 1 (2004), 50–80.
- Emma Lurie and Deirdre K Mulligan . 2020 . Crowdworkers are not judges: Rethinking crowdsourced vignette studies as a risk assessment evaluation technique . Proceedings of the Workshop on Fair and Responsible AI at CHI 2020 (2020). Emma Lurie and Deirdre K Mulligan. 2020. Crowdworkers are not judges: Rethinking crowdsourced vignette studies as a risk assessment evaluation technique. Proceedings of the Workshop on Fair and Responsible AI at CHI 2020 (2020).
- Hussein Mozannar Arvind Satyanarayan and David Sontag. 2021. Teaching Humans When To Defer to a Classifier via Examplars. arXiv preprint arXiv:2111.11297(2021). Hussein Mozannar Arvind Satyanarayan and David Sontag. 2021. Teaching Humans When To Defer to a Classifier via Examplars. arXiv preprint arXiv:2111.11297(2021). · arXiv
- Allegheny County Department of Human Services. [n.d.]. The Allegheny Family Screening Tool. https://www.alleghenycounty.us/Human-Services/News-Events/Accomplishments/Allegheny-Family-Screening-Tool.aspx. Online ; accessed 7- September - 2021 . Allegheny County Department of Human Services. [n.d.]. The Allegheny Family Screening Tool. https://www.alleghenycounty.us/Human-Services/News-Events/Accomplishments/Allegheny-Family-Screening-Tool.aspx. Online; accessed 7-September-2021.
- Allegheny County Department of Human Services. [n.d.]. Allegheny Family Screening Tool , Frequently-Asked Questions | Updated August 2018 . https://www.alleghenycountyanalytics.us/wp-content/uploads/2018/10/17-ACDHS-11_AFST_102518.pdf. Online; accessed 8-September-2021. Allegheny County Department of Human Services. [n.d.]. Allegheny Family Screening Tool, Frequently-Asked Questions | Updated August 2018. https://www.alleghenycountyanalytics.us/wp-content/uploads/2018/10/17-ACDHS-11_AFST_102518.pdf. Online; accessed 8-September-2021.
- Anjana Samant , Aaron Horowitz , Kath Xu , and Sophie Beiers . 2021. Family surveillance by algorithm: The rapidly spreading tools few have heard of . American Civil Liberties Union (ACLU)( 2021 ). https://www.aclu.org/sites/default/files/field_document/2021.09.28a_family_surveillance_by_algorithm.pdf Anjana Samant, Aaron Horowitz, Kath Xu, and Sophie Beiers. 2021. Family surveillance by algorithm: The rapidly spreading tools few have heard of. American Civil Liberties Union (ACLU)(2021). https://www.aclu.org/sites/default/files/field_document/2021.09.28a_family_surveillance_by_algorithm.pdf
- Devansh Saxena , Karla Badillo-Urquiola , Pamela Wisniewski , and Shion Guha . 2021. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child welfare. arXiv 5 , October ( 2021 ). arxiv:arXiv:2107.03487v2 Devansh Saxena, Karla Badillo-Urquiola, Pamela Wisniewski, and Shion Guha. 2021. A framework of high-stakes algorithmic decision-making for the public sector developed through a case study of child welfare. arXiv 5, October (2021). arxiv:arXiv:2107.03487v2 · arXiv
- Sarah Tan , Julius Adebayo , and Kori Inkpen . 2018. Investigating human + machine complementarity for recidivism predictions. arXiv ( 2018 ). arxiv:arXiv:1808.09123v2 Sarah Tan, Julius Adebayo, and Kori Inkpen. 2018. Investigating human + machine complementarity for recidivism predictions. arXiv (2018). arxiv:arXiv:1808.09123v2 · arXiv
- Rhema Vaithianathan , Emily Putnam-Hornstein , Nan Jiang , Parma Nand , and Tim Maloney . 2017. Developing predictive models to support child maltreatment hotline screening decisions: Allegheny County methodology and implementation . Center for Social data Analytics( 2017 ). Rhema Vaithianathan, Emily Putnam-Hornstein, Nan Jiang, Parma Nand, and Tim Maloney. 2017. Developing predictive models to support child maltreatment hotline screening decisions: Allegheny County methodology and implementation. Center for Social data Analytics(2017).
- Elmira van den Broek , Anastasia Sergeeva , and Marleen Huysman . 2020 . Hiring algorithms: An ethnography of fairness in practice . 40th International Conference on Information Systems, ICIS 2019 (2020). Elmira van den Broek, Anastasia Sergeeva, and Marleen Huysman. 2020. Hiring algorithms: An ethnography of fairness in practice. 40th International Conference on Information Systems, ICIS 2019 (2020).
- MW Van Someren , YF Barnard , and JAC Sandberg . 1994. The think aloud method: a practical approach to modelling cognitive . London : AcademicPress( 1994 ). MW Van Someren, YF Barnard, and JAC Sandberg. 1994. The think aloud method: a practical approach to modelling cognitive. London: AcademicPress(1994).
Cited by
Shi, Y., Gao, T., Jiao, X., & Cao, N. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610217 · Cited by 170
Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations
Chen, V., Liao, Q. V., Wortman Vaughan, J., & Bansal, G. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610219 · Cited by 220
“Different from a small but growing number of qualitative studies that aim to understand the experience of decision-makers retrospectively [14, 41, 43, 60, 71], we adopt a think-aloud protocol to investigate the real-time process as participants engage with AI output [39, 77], and use the data to understand why two typ…”
MethodSivaraman, V., Bukowski, L. A., Levin, J., Kahn, J. M., & Perer, A. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581075 · Cited by 141
“While expert consensus can serve as a useful proxy for the ground truth [40], the accuracy of a real-world decision is often fundamentally unknowable and contentious [47].”
BackgroundSelf-citationYang, Z., Xu, X., Yao, B., Rogers, E., Zhang, S., Intille, S., Shara, N., Gao, G. G., & Wang, D. (2024) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3659625 · Cited by 120
Challenges of responsible AI in practice: scoping review and recommended actions
Sadek, M., Kallina, E., Bohné, T., Mougenot, C., Calvo, R. A., & Cave, S. (2024) · AI & Society · doi:10.1007/s00146-024-01880-9 · Cited by 142
“…or not to detain criminal defend-ants (Dressel and Farid 2018); analysing which child protection requests seem credible (Chouldechova et al. 2018; Kawakami et al. 2022); in HR related topics (Mujtaba and Mahapatra 2019); facial recognition (Lohr 2018); detecting hate speech on social media…”
Metadata
- Authors
- Kawakami, A., Sivaraman, V., Cheng, H.-F., Stapleton, L., Cheng, Y., Qing, D., Perer, A., Wu, Z. S., Zhu, H., & Holstein, K.
- Affiliations
- Carnegie Mellon University (US)
- University of Minnesota (US)
- University of California, Berkeley (US)
- ORCID
- Venue
- CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2022-04-27
- Pages
- 1-18
- OpenAlex
- W4225001143
- Cited by
- 143
- References
- 42
- Citations by year
- 2026: 25 · 2025: 37 · 2024: 44 · 2023: 30 · 2022: 7
- FWCI
- 29.32
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by-nc-sa)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- NSF (National Science Foundation) (1939606)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Business, Management and Accounting › Management Information Systems › Information Systems Theories and Implementation
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Keywords
- human-AI collaboration, algorithmic decision support, child welfare, child maltreatment screening, AI-assisted decision-making, contextual information, trust in AI predictions, organizational incentives, public sector organizations, design implications
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Improving Human-AI Partnerships in Child Welfare: Understanding Worker Practices, Challenges, and Desires for Algorithmic Decision Support. https://app.cochat.ai/papers/10.1145/3491102.3517439
- The paper
- Kawakami, A., Sivaraman, V., Cheng, H.-F., Stapleton, L., Cheng, Y., Qing, D., Perer, A., Wu, Z. S., Zhu, H., & Holstein, K. (2022). Improving Human-AI Partnerships in Child Welfare: Understanding Worker Practices, Challenges, and Desires for Algorithmic Decision Support. CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3491102.3517439
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.