Do I Look Like a Criminal? Examining how Race Presentation Impacts Human Judgement of Recidivism
Open accessAbstract
Understanding how racial information impacts human decision making in online systems is critical in today's world. Prior work revealed that race information of criminal defendants, when presented as a text field, had no significant impact on users' judgements of recidivism. We replicated and extended this work to explore how and when race information influences users' judgements, with respect to the saliency of presentation. Our results showed that adding photos to the race labels had a significant impact on recidivism predictions for users who identified as female, but not for those who identified as male. The race of the defendant also impacted these results, with black defendants being less likely to be predicted to recidivate compared to white defendants. These results have strong implications for how system-designers choose to display race information, and cautions researchers to be aware of gender and race effects when using Amazon Mechanical Turk workers.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Tasks that involve inherent uncertainty are often those where humans tend to rely on AI systems for advice, such as loan approval, recidivism prediction, house price estimation, and student admission.
“Tasks that involve inherent uncertainty are often those where humans tend to rely on AI systems for advice, such as loan approval [27, 34, 124], recidivism prediction [31, 47, 86], house price estimation [2, 13, 25], and student admission [13, 24].”
From Dealing with Uncertainty: Understanding the Impact of Prognostic Versus Diagnostic Tasks on Trust and Reliance in Human-AI Decision Making (2024) · cites this paper as [86]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
(2018) · Cited by 3,510 · see citing works
Conducting behavioral research on Amazon’s Mechanical Turk
Mason, W., & Suri, S. (2012) · Behavior Research Methods · doi:10.3758/s13428-011-0124-6 · Cited by 3,172 · see citing works
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887 · see citing works
Model Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681 · see citing works
Fair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159 · see citing works
The Chicago face database: a free stimulus set of faces and norming data
Ma, D. S., Correll, J., & Wittenbrink, B. (2015) · Behavior Research Methods · doi:10.3758/s13428-014-0532-5 · Cited by 1,788 · see citing works
Friedman, B., & Nissenbaum, H. (1996) · ACM Transactions on Information Systems · doi:10.1145/230538.230561 · Cited by 1,203 · see citing works
The accuracy, fairness, and limits of predicting recidivism
Dressel, J., & Farid, H. (2018) · Science Advances · doi:10.1126/sciadv.aao5580 · Cited by 1,071 · see citing works
Zenk, S. N., Schulz, A. J., Israel, B. A., James, S. A., Bao, S., & Wilson, M. L. (2005) · American Journal of Public Health · doi:10.2105/ajph.2004.042150 · Cited by 982 · see citing works
Race and health: Basic questions, emerging directions
Williams, D. R. (1997) · Annals of Epidemiology · doi:10.1016/s1047-2797(97)00051-3 · Cited by 825 · see citing works
Implicit Bias: Scientific Foundations
Greenwald, A. G., & Krieger, L. H. (2006) · California Law Review · doi:10.2307/20439056 · Cited by 808 · see citing works
Donath, J. (2007) · Journal of Computer-Mediated Communication · doi:10.1111/j.1083-6101.2007.00394.x · Cited by 801 · see citing works
Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries
Olteanu, A., Castillo, C., Diaz, F., & Kıcıman, E. (2019) · Frontiers in Big Data · doi:10.3389/fdata.2019.00013 · Cited by 763 · cited 2× · see citing works
“Developing solutions to mitigate bias is currently a major challenge in AI [6, 1, 38, 35, 4].”
BackgroundImplicit anti-fat bias among health professionals: is anyone immune?
Teachman, B., & Kd, B. (2001) · International Journal of Obesity · doi:10.1038/sj.ijo.0801745 · Cited by 598 · see citing works
Cai, C. J., Winter, S., Steiner, D., Wilcox, L., & Terry, M. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359206 · Cited by 594 · see citing works
Unequal Representation and Gender Stereotypes in Image Search Results for Occupations
Kay, M., Matuszek, C., & Munson, S. A. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702520 · Cited by 509 · see citing works
Implicit Bias: Scientific Foundations
Greenwald, A. G. (2006) · bepress Legal Repository · doi:10.15779/z38gh7f · Cited by 411 · see citing works
Bakhshi, S., Shamma, D. A., & Gilbert, É. (2014) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557403 · Cited by 408 · see citing works
Racial bias in judgments of physical size and formidability: From size to threat.
Wilson, J. P., Hugenberg, K., & Rule, N. O. (2017) · Journal of Personality and Social Psychology · doi:10.1037/pspi0000092 · Cited by 334 · see citing works
Does Unconscious Racial Bias Affect Trial Judges
Rachlinski, J. J., Johnson, S. L., Wistrich, A. J., & Guthrie, C. (2009) · Scholarship @ Cornell Law (Cornell University) · Cited by 311 · see citing works
Explaining models: an empirical study of how explanations impact fairness judgment
Dodge, J., Liao, Q. V., Zhang, Y., Bellamy, R. K. E., & Dugan, C. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302310 · Cited by 286 · see citing works
Disparate Interactions: An Algorithm-in-the-Loop Analysis of Fairness in Risk Assessments
Green, B., & Chen, Y. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287563 · Cited by 275 · see citing works
Dixon, T. L., & Maddox, K. B. (2005) · Journal of Applied Social Psychology · doi:10.1111/j.1559-1816.2005.tb02184.x · Cited by 272 · see citing works
Equity and Discretionary Justice: The Influence of Race on Police Arrest Decisions
Smith, D. A., Visher, C. A., & Davidson, L. A. (1984) · The Journal of Criminal Law and Criminology (1973-) · doi:10.2307/1143211 · Cited by 266 · see citing works
(2018) · Cited by 230 · see citing works
IMAGES OF CRIMINALS AND VICTIMS
Madriz, E. (1997) · Gender & Society · doi:10.1177/089124397011003005 · Cited by 218 · see citing works
Risk, Race, and Recidivism: Predictive Bias and Disparate Impact
Skeem, J., & Lowenkamp, C. T. (2016) · Criminology · doi:10.1111/1745-9125.12123 · Cited by 217 · see citing works
Implicit Bias in the Courtroom
Kang, J., Bennett, M. W., Carbado, D. W., Casey, P., Dasgupta, N., Faigman, D. L., Godsil, R. D., Greenwald, A. G., Levinson, J. D., & Mnookin, J. L. (2012) · eScholarship (California Digital Library) · Cited by 162 · see citing works
Fagerstrøm, A., Pawar, S., Sigurðsson, V., Foxall, G. R., & Yani‐de‐Soriano, M. (2017) · Computers in Human Behavior · doi:10.1016/j.chb.2017.02.029 · Cited by 158 · see citing works
Layers of bias: A unified approach for understanding problems with risk assessment
Eckhouse, L., Lum, K., Conti-Cook, C., & Ciccolini, J. (2019) · Criminal Justice and Behavior · doi:10.1177/0093854818811379 · Cited by 156 · see citing works
Does Unconscious Racial Bias Affect Trial Judges
Guthrie, C., Rachlinski, J. J., Johnson, S. L., & Wistrich, A. J. (2009) · The Notre Dame law review · Cited by 151 · see citing works
Schlesinger, A., O'Hara, K. P., & Taylor, A. S. (2018) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173889 · Cited by 142 · see citing works
Hankerson, D., Marshall, A., Booker, J., Elmimouni, H., Walker, I., & Rode, J. A. (2016) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/2851581.2892578 · Cited by 137 · see citing works
Tan, S., Caruana, R., Hooker, G., & Lou, Y. (2018) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3278721.3278725 · Cited by 127 · see citing works
Self-citationHong, W., Thong, J. Y. L., & Tam, K. Y. (2004) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2004.01.006 · Cited by 117 · see citing works
Classifying Juvenile Offenders According to Risk of Recidivism
Schwalbe, C. S. J., Fräser, M. W., Day, S. H., & Cooley, V. (2006) · Criminal Justice and Behavior · doi:10.1177/0093854806286451 · Cited by 114 · see citing works
Human Decision Making with Machine Assistance: An Experiment on Bailing and Jailing
Grgić-Hlača, N., Engel, C., & Gummadi, K. P. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359280 · Cited by 113 · see citing works
Is a Picture Worth a Thousand Words?
Barrett, A. W., & Barrington, L. W. (2005) · Harvard International Journal of Press/Politics · doi:10.1177/1081180x05281392 · Cited by 83 · see citing works
Hartmann, J., Angeli, A. D., & Sutcliffe, A. (2008) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1357054.1357190 · Cited by 82 · see citing works
Crowdsourcing Perceptions of Fair Predictors for Machine Learning
van Berkel, N., Gonçalves, J., Hettiachchi, D., Wijenayake, S., Kelly, R., & Kostakos, V. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359130 · Cited by 73 · see citing works
Looking like a criminal: Stereotypical black facial features promote face source memory error
Kleider, H. M., Cavrak, S., & Knuycky, L. R. (2012) · Memory & Cognition · doi:10.3758/s13421-012-0229-x · Cited by 63 · see citing works
Race, concentrated disadvantage, and recidivism: A test of interaction effects
Wehrman, M. M. (2010) · Journal of Criminal Justice · doi:10.1016/j.jcrimjus.2010.04.024 · Cited by 57 · see citing works
The effects of mixing machine learning and human judgment
Vaccaro, M., & Waldo, J. (2019) · Communications of the ACM · doi:10.1145/3359338 · Cited by 49 · see citing works
What You See Is What You Get? The Impact of Representation Criteria on Human Bias in Hiring
Peng, A., Nushi, B., Kıcıman, E., Inkpen, K. M., Suri, S., & Kamar, E. S. (2019) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v7i1.5281 · Cited by 45 · see citing works
Self-citationKleider‐Offutt, H. M., Bond, A. D., & Hegerty, S. E. A. (2017) · Current Directions in Psychological Science · doi:10.1177/0963721416667916 · Cited by 43 · see citing works
A socio-economic perspective on gender bias in technology
Kilbourne, W. E., & Weeks, S. L. (1997) · The Journal of Socio-Economics · doi:10.1016/s1053-5357(97)90018-4 · Cited by 39 · see citing works
Investigating Human + Machine Complementarity for Recidivism PredictionsPreprint
Tan, S., Adebayo, J., Inkpen, K. M., & Kamar, E. S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.09123 · Cited by 25 · see citing works
Self-citationWobbrock, J. O., Hsu, A. K., Burger, M. A., & Magee, M. J. (2019) · ACM Conference on Hypertext and Social Media (HT) · doi:10.1145/3342220.3343663 · Cited by 11 · see citing works
Holsinger, A. M., Lowenkamp, C. T., Latessa, E. J., Serin, R. C., Cohen, T. H., Robinson, C. R., Flores, A. W., & VanBenschoten, S. W. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3271682 · Cited by 9 · see citing works
Bias Misinformation and Disinformation: Mental Health Employment and Human Computer Interaction
Martin, J., McKay, E., & Shankar, J. (2006) · Informing Science and IT Education Conference · doi:10.28945/3016 · Cited by 4 · see citing works
Flagged first, then by citations
3 of the 53 references couldn’t be matched to a known paper.
Research works printed but not linked: 2
- Yujia Cao, Mariët Theune, and Anton Nijholt. 2009. Modality effects on cognitive load and performance in high-load information presentation. In International Conference on Intelligent User Interfaces. 335-344.
- Farnaz Jahanbakhsh, Justin Cranshaw, Kori Inkpen, Scott Counts, and Walter Lasecki. 2020. Bias in social rating systems: The role of performance quality and gender. In CHI Conference on Human Factors in Computing Systems.
Cited by
Sorted by citations
Lai, V., Chen, C., Smith-Renner, A., Liao, Q. V., & Tan, C. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594087 · Cited by 204 · see citing works
Lee, M. K., & Rich, K. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445570 · Cited by 145 · see citing works
Explanations, Fairness, and Appropriate Reliance in Human-AI Decision-Making
Schoeffer, J., De‐Arteaga, M., & Kühl, N. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642621 · Cited by 87 · see citing works
Toward involving end-users in interactive human-in-the-loop AI fairness
Nakao, Y., Stumpf, S., Ahmed, S., Naseer, A., & Strappelli, L. (2022) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3514258 · Cited by 69 · see citing works
Nakao, Y., Strappelli, L., Stumpf, S., Naseer, A., Regoli, D., & Gamba, G. D. (2022) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2022.2067936 · Cited by 55 · see citing works
A Missing Piece in the Puzzle: Considering the Role of Task Complexity in Human-AI Decision Making
Salimzadeh, S., He, G., & Gadiraju, U. (2023) · Proceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization · doi:10.1145/3565472.3592959 · Cited by 42 · see citing works
van Berkel, N., Sarsenbayeva, Z., & Gonçalves, J. (2022) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2022.102954 · Cited by 41 · see citing works
Salimzadeh, S., He, G., & Gadiraju, U. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641905 · Cited by 41 · see citing works
“Tasks that involve inherent uncertainty are often those where humans tend to rely on AI systems for advice, such as loan approval [27, 34, 124], recidivism prediction [31, 47, 86], house price estimation [2, 13, 25], and student admission [13, 24].”
Diversity in sociotechnical machine learning systems
Fazelpour, S., & De‐Arteaga, M. (2022) · Big Data & Society · doi:10.1177/20539517221082027 · Cited by 46 · see citing works
Fogliato, R., Chouldechova, A., & Lipton, Z. C. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479572 · Cited by 26 · see citing works
Cultural Considerations in AI Systems for the Global South: A Systematic Review
Anuyah, O., Wan, R., Adejoro, C. O., YEH, T. P., Metoyer, R., & Badillo-Urquiola, K. A. (2023) · African Human Computer Interaction Conference (AfriCHI) · doi:10.1145/3628096.3629046 · Cited by 20 · see citing works
Ashktorab, Z., Hoover, B., Agarwal, M., Dugan, C., Geyer, W., Yang, H., & Yurochkin, M. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581227 · Cited by 15 · see citing works
The Influences of Task Design on Crowdsourced Judgement: A Case Study of Recidivism Risk Evaluation
Duan, X., Ho, C.-J., & Yin, M. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3485447.3512239 · Cited by 12 · see citing works
A comparative user study of human predictions in algorithm-supported recidivism risk assessment
Portela, M., Castillo, C., Tolan, S., Karimi-Haghighi, M., & Pueyo, A. A. (2024) · Artificial Intelligence and Law · doi:10.1007/s10506-024-09393-y · Cited by 13 · see citing works
Salimzadeh, S., & Gadiraju, U. (2024) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3627043.3659567 · Cited by 11 · see citing works
Does My AI Help or Hurt? Exploring Human-AI Complementarity
Inkpen, K. M. (2020) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3340631.3395384 · Cited by 6 · see citing works
Self-citationYokota, T., & Nakao, Y. (2022) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3511047.3538033 · Cited by 4 · see citing works
Antar, A. D., Molaei, S., Chen, Y.-Y., Lee, M. L., & Banovic, N. (2024) · ACM Symposium on User Interface Software and Technology · doi:10.1145/3654777.3676323 · Cited by 8 · see citing works
Explanations, Fairness, and Appropriate Reliance in Human-AI Decision-MakingPreprint
Schoeffer, J., De‐Arteaga, M., & Kuehl, N. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2209.11812 · Cited by 3 · see citing works
On Meritocracy in Optimal Set Selection
Buening, T. K., Segal, M., Basu, D., George, A.-M., & Christos Dimitrakakis, A. (2022) · ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO) · doi:10.1145/3551624.3555305 · Cited by 1 · see citing works
Dhungel, A.-K. (2022) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata55660.2022.10020323 · Cited by 1 · see citing works
Fair Engineering of Machine Learning Systems – Lessons Learned from a Literature Review
Sengewald, J., & Lackes, R. (2022) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2022.605
Guided Reflection in AI-Assisted Decision-Making: Effects on AI Overreliance and Decision Accuracy
Li, S., Li, J., Li, H., Zhu, H., & Li, X. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790632
The Consensus Trap: Dissecting Subjectivity and the “Ground Truth” Illusion in Data Annotation
Munir, S., Mah, B., Kalsi, K., Kapania, S., Posada, J., Law, E., Wang, D., & Ahmed, S. I. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3806510
- Authors
- Mallari, K., Inkpen, K., Johns, P., Tan, S., Ramesh, D., & Kamar, E.
- Affiliations
- University of Washington (US)
- Microsoft (United States) (US)
- Cornell University (US)
- University of Michigan (US)
- Venue
- Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2020-04-21
- Pages
- 1-13
- OpenAlex
- W3005080571
- Cited by
- 24
- References
- 53
- Citations by year
- 2026: 2 · 2024: 5 · 2023: 4 · 2022: 10 · 2021: 2 · 2020: 1
- FWCI
- 4.42
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Social Sciences › Sociology and Political Science › Criminal Justice and Corrections Analysis
- Social Sciences › Sociology and Political Science › Names, Identity, and Discrimination Research
- Keywords
- recidivism prediction, racial bias, criminal defendants, salience, gender differences, Black defendants, online decision making, algorithmic fairness, Amazon Mechanical Turk
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
