Heterogeneity in Algorithm-Assisted Decision-Making: A Case Study in Child Abuse Hotline Screening
Open accessAbstract
Algorithmic risk assessment tools are now commonplace in public sector domains such as criminal justice and human services. These tools are intended to aid decision makers in systematically using rich and complex data captured in administrative systems. In this study we investigate sources of heterogeneity in the alignment between worker decisions and algorithmic risk scores in the context of a real world child abuse hotline screening use case. Specifically, we focus on heterogeneity related to worker experience. We find that senior workers are far more likely to screen in referrals for investigation, even after we control for the observed algorithmic risk score and other case characteristics. We also observe that the decisions of less-experienced workers are more closely aligned with algorithmic risk scores than those of senior workers who had decision-making experience prior to the tool being introduced. While screening decisions vary across child race, we do not find evidence of racial differences in the relationship between worker experience and screening decisions. Our findings indicate that it is important for agencies and system designers to consider ways of preserving institutional knowledge when introducing algorithms into high employee turnover settings such as child welfare call screening.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137 · see citing works
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · see citing works
A Practitioner’s Guide to Cluster-Robust Inference
Colin Cameron, A., & Miller, D. L. (2015) · Journal of Human Resources · doi:10.3368/jhr.50.2.317 · Cited by 5,037 · see citing works
Algorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232 · see citing works
Human trust in artificial intelligence: Review of empirical research
Glikson, E., & Woolley, A. W. (2020) · Academy of Management Annals · doi:10.5465/annals.2018.0057 · Cited by 2,921 · see citing works
“Clinical versus Actuarial Judgment.”
Dawes, R. M., Faust, D., & Meehl, P. E. (1989) · Science · doi:10.1126/science.2648573 · Cited by 2,298 · see citing works
Explaining Job Polarization: Routine-Biased Technological Change and Offshoring
Goos, M., Manning, A., & Salomons, A. (2014) · American Economic Review · doi:10.1257/aer.104.8.2509 · Cited by 2,111 · see citing works
Algorithm appreciation: People prefer algorithmic to human judgment
Logg, J. M., Minson, J. A., & Moore, D. A. (2019) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2018.12.005 · Cited by 2,061 · see citing works
The role of trust in automation reliance
Dzindolet, M. T., Peterson, S. A., Pomranky, R. A., Pierce, L. G., & Beck, H. P. (2003) · International Journal of Human-Computer Studies · doi:10.1016/s1071-5819(03)00038-7 · Cited by 1,276 · see citing works
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2018) · Management Science · doi:10.1287/mnsc.2016.2643 · Cited by 1,230 · see citing works
Automation bias: a systematic review of frequency, effect mediators, and mitigators
Goddard, K., Roudsari, A., & Wyatt, J. C. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000089 · Cited by 1,162 · see citing works
Human Decisions and Machine Predictions
Kleinberg, J., Lakkaraju, H., Leskovec, J., Ludwig, J., & Mullainathan, S. (2018) · The Quarterly Journal of Economics · doi:10.1093/qje/qjx032 · Cited by 1,040 · see citing works
A systematic review of algorithm aversion in augmented decision making
Burton, J. W., Stein, M., & Jensen, T. B. (2020) · Journal of Behavioral Decision Making · doi:10.1002/bdm.2155 · Cited by 814 · see citing works
Two Strikes: Race and the Disciplining of Young Students
Okonofua, J. A., & Eberhardt, J. L. (2015) · Psychological Science · doi:10.1177/0956797615570365 · Cited by 731 · see citing works
Manipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Wortman Vaughan, J. W., & Wallach, H. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445315 · Cited by 728 · see citing works
Human Decisions and Machine Predictions
Kleinberg, J., Lakkaraju, H., Leskovec, J., Ludwig, J., & Mullainathan, S. (2017) · doi:10.3386/w23180 · Cited by 545 · see citing works
Understanding the Effect of Accuracy on Trust in Machine Learning Models
Yin, M., Wortman Vaughan, J., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300509 · Cited by 527 · see citing works
Veale, M., Van Kleek, M., & Binns, R. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174014 · Cited by 490 · see citing works
Transparency and trust in artificial intelligence systems
Schmidt, P., Biessmann, F., & Teubner, T. (2020) · Journal of Decision Systems · doi:10.1080/12460125.2020.1819094 · Cited by 440 · see citing works
A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores
De-Arteaga, M., Fogliato, R., & Chouldechova, A. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376638 · Cited by 221 · see citing works
Self-citationWang, R., Harper, F. M., & Zhu, H. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376813 · Cited by 211 · see citing works
Brown, A., Chouldechova, A., Putnam-Hornstein, E., Tobin, A., & Vaithianathan, R. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300271 · Cited by 207 · see citing works
Self-citationInvestigating the effects of caseworker characteristics in child welfare
Ryan, J. P., Garnier, P. C., Zyphur, M. J., & Zhai, F. (2005) · Children and Youth Services Review · doi:10.1016/j.childyouth.2005.10.013 · Cited by 204 · see citing works
Kawakami, A., Sivaraman, V., Cheng, H.-F., Stapleton, L., Cheng, Y., Qing, D., Perer, A., Wu, Z. S., Zhu, H., & Holstein, K. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517439 · Cited by 143 · see citing works
Goal-based Course Recommendation
Jiang, W., Pardos, Z. A., & Wei, Q. (2019) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3303772.3303814 · Cited by 128 · see citing works
A Human-Centered Review of Algorithms used within the U.S. Child Welfare System
Saxena, D., Badillo-Urquiola, K., Wisniewski, P. J., & Guha, S. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376229 · Cited by 116 · see citing works
Human Decision Making with Machine Assistance: An Experiment on Bailing and Jailing
Grgić-Hlača, N., Engel, C., & Gummadi, K. P. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359280 · Cited by 113 · see citing works
How to Talk When a Machine is Listening?: Corporate Disclosure in the Age of AI
CAO, S. S., Jiang, W., Yang, B., & Zhang, A. L. (2020) · National Bureau of Economic Research · doi:10.3386/w27950 · Cited by 92 · see citing works
How Child Welfare Workers Reduce Racial Disparities in Algorithmic Decisions
Cheng, H.-F., Stapleton, L., Kawakami, A., Sivaraman, V., Cheng, Y., Qing, D., Perer, A., Holstein, K., Wu, Z. S., & Zhu, H. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501831 · Cited by 88 · see citing works
Green, B., & Chen, Y. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479562 · Cited by 69 · see citing works
Allocating Interventions Based on Predicted Outcomes: A Case Study on Homelessness Services
Kube, A., Das, S., & Fowler, P. J. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.3301622 · Cited by 58 · see citing works
Sühr, T., Hilgard, S., & Lakkaraju, H. (2021) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3461702.3462602 · Cited by 54 · see citing works
Carnochan, S., Moore, M., & Austin, M. J. (2013) · Journal of Evidence-Based Social Work · doi:10.1080/15433714.2013.788953 · Cited by 50 · see citing works
Babitsch, B., Braun, T., Borde, T., & David, M. (2008) · BMC Health Services Research · doi:10.1186/1472-6963-8-82 · Cited by 49 · see citing works
McBeath, B., Chuang, E., Bunger, A. C., & Blakeslee, J. E. (2014) · Social Service Review · doi:10.1086/675373 · Cited by 38 · see citing works
Systemic Oversimplification Limits the Potential for Human-AI Partnership
Metcalfe, J. S., Perelman, B. S., Boothe, D. L., & McDowell, K. (2021) · IEEE Access · doi:10.1109/access.2021.3078298 · Cited by 35 · see citing works
Patterns of out-of-home placement decision-making in child welfare
Chor, K. H. B., McClelland, G. M., Weiner, D. A., Jordan, N., & Lyons, J. S. (2013) · Child Abuse & Neglect · doi:10.1016/j.chiabu.2013.04.016 · Cited by 31 · see citing works
What counts? quantification, worker judgment, and divergence in child welfare decision making
Bosk, E. A. (2018) · Human Services Organizations Management Leadership & Governance · doi:10.1080/23303131.2017.1422068 · Cited by 31 · see citing works
Conducting Participatory Design to Improve Algorithms in Public Services: Lessons and Challenges
Saxena, D., & Guha, S. (2020) · Companion Publication of the 2020 Conference on Computer Supported Cooperative Work and Social Computing · doi:10.1145/3406865.3418331 · Cited by 30 · see citing works
Saxena, D., Repaci, C., Sage, M., & Guha, S. (2022) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3491101.3519771 · Cited by 23 · see citing works
(2003) · CQ Press eBooks · doi:10.4135/9781483345192.n5 · Cited by 5 · see citing works
Flagged first, then by citations
Research works printed but not linked: 5
- Heather M Giannini, Jennifer C Ginestra, Corey Chivers, Michael Draugelis, Asaf Hanish, William D Schweickert, Barry D Fuchs, Laurie Meadows, Michael Lynch, Patrick J Donnelly, et al. 2019. A machine learning algorithm to predict severe sepsis and septic shock: development, implementation, and impact on clinical practice. Critical care medicine 47, 11 (2019), 1485-1492.
- B. Green and Y. Chen. 2019. Disparate Interactions: An Algorithm-in-the-Loop Analysis of Fairness in Risk Assessments . FATML Proceedings (2019). https://doi.org/files/19-fat.pdf
- Rebecca C. Hetey, Benoit Monin, Amrita Maitreyi, and Jennifer L. Eberhardt. 2016. Data for Change: A Statistical Analysis of Police Stops, Searches, Handcuffings, and Arrests in Oakland. (2016). https://doi.org/view/by412gh2838
- Jennifer Skeem, Nicholas Scurich, and John Monahan. 2020. Impact of risk assessment on judges' fairness in sentencing relatively poor defendants. Law and human behavior 44, 1 (2020), 51.
- H James Wilson and Paul R Daugherty. 2018. Collaborative intelligence: humans and AI are joining forces. Harvard Business Review 96, 4 (2018), 114-123.
Cited by
Sorted by citations
Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations
Chen, V., Liao, Q. V., Wortman Vaughan, J., & Bansal, G. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610219 · Cited by 145 · see citing works
Who Goes First? Influences of Human-AI Workflow on Decision Making in Clinical Imaging
Fogliato, R., Chappidi, S., Lungren, M. P., Fisher, P., Wilson, D., Fitzke, M., Parkinson, M., Horvitz, E., Inkpen, K. M., & Nushi, B. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533193 · Cited by 115 · see citing works
Herrera, F. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103133 · Cited by 61 · see citing works
The Devil is in the Details: Interrogating Values Embedded in the Allegheny Family Screening Tool
Gerchick, M. K., Jegede, T., Shah, T., Gutiérrez, A. B., Beiers, S., Shemtov, N., Xu, K., Samant, A., & Horowitz, A. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3594081 · Cited by 35 · see citing works
Bridging the gap: Towards an expanded toolkit for AI-driven decision-making in the public sector
Fischer-Abaigar, U., Kern, C. A., Barda, N., & Kreuter, F. (2024) · Government Information Quarterly · doi:10.1016/j.giq.2024.101976 · Cited by 49 · see citing works
Overcoming Algorithm Aversion: A Comparison between Process and Outcome Control
Cheng, L., & Chouldechova, A. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581253 · Cited by 24 · see citing works
Self-citationKuehnert, B., Kim, R., Forlizzi, J., & Heidari, H. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732191 · Cited by 28 · see citing works
The Fall of an Algorithm: Characterizing the Dynamics Toward Abandonment
Johnson, N., Moharana, S., Harrington, C., Andalibi, N., Heidari, H., & Eslami, M. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658910 · Cited by 20 · see citing works
Ben‐Michael, E., Greiner, D. J., Huang, M., Imai, K., Jiang, Z., & Shin, S. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2505106122 · Cited by 13 · see citing works
Field Reconstruction-Based Non-Rendezvous Calibration for Low Cost Mobile Sensors
Luo, J., Hu, Y., Yu, C., Hong, C., Zhang, X.-P. S., & Chen, X. (2023) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3594739.3612908 · Cited by 9 · see citing works
Improving Model Fairness with Time-Augmented Bayesian Knowledge Tracing
Barrett, J., Day, A., & Gal, K. (2024) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3636555.3636849 · Cited by 8 · see citing works
Human Discernment of Algorithmic Errors: A Case Study in Child WelfarePreprint
Fogliato, R., De‐Arteaga, M., & Chouldechova, A. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4050125 · Cited by 7 · see citing works
Self-citationBull, C. F., Kisely, S. R., Betts, K. S., & Hu, Y. (2025) · Child Abuse & Neglect · doi:10.1016/j.chiabu.2025.107630 · Cited by 8 · see citing works
The Algorithm and the Org Chart: How Algorithms Can Conflict with Organizational Structures
Valentine, M., Pratt, A. L., Hinds, R., & Bernstein, M. S. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3686903 · Cited by 4 · see citing works
Proceedings of The ACM Collective Intelligence Conference
(2023) · doi:10.1145/3582269 · Cited by 4 · see citing works
Decision-making in Cybersecurity: A Bibliometric analysis
Aljanabi, M., Ali, A. H., Yaseen, M. G., Mohammed, M. A., & Mohammed, S. Y. (2023) · Wasit Journal of Computer and Mathematics Science · doi:10.31185/wjcms.208 · Cited by 3 · see citing works
Bull, C. F., Kisely, S. R., Betts, K. S., & Hu, Y. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5132480
Li, X., Ma, B., Bao, W., & Tan, Z. (2026) · Children and Youth Services Review · doi:10.1016/j.childyouth.2026.108884
Moura, I., Powell, R. M., & Kaye, H. (2026) · Journal of Public Child Welfare · doi:10.1080/15548732.2026.2689958
Supporting Bayesian Reasoning to Mitigate Over-Reliance in AI-Assisted Decision Making
Mikhaylova, D., Turchi, T., Cevolani, G., & Malizia, A. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-30849-8_6
dos Santos, R. M., Ferreira, V. M. D. S., de Miranda, M. G., Avelar, K. E. S., Dusek, P. M., Boente, A. N. P., & Moreno, A. M. (2026) · Research Society and Development · doi:10.33448/rsd-v15i8.51435
Routinized deference: predictive screening and bureaucratic capacity in child protective services
Wu, J. (2026) · Journal of Public Administration Research and Theory · doi:10.1093/jopart/muag032
- Authors
- Cheng, L., & Chouldechova, A.
- Affiliations
- Carnegie Mellon University (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2022
- Volume
- 6
- Issue
- CSCW2
- Pages
- 1-33
- ISSN
- 2573-0142 (ISSN-L)
- arXiv
- 2204.05478
- OpenAlex
- W4308791796
- Cited by
- 20
- References
- 40
- Citations by year
- 2026: 4 · 2025: 5 · 2024: 4 · 2023: 5 · 2022: 2
- FWCI
- 4.63
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Proceedings of the ACM on Human-Computer Interaction (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (2040929, 1939606)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Social Sciences › Political Science and International Relations › Artificial Intelligence in Law
- Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › AI and HR Technologies
- Keywords
- algorithmic risk assessment, child welfare, employee experience, institutional knowledge, employee turnover
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
