Disparate Interactions: An Algorithm-in-the-Loop Analysis of Fairness in Risk Assessments
The DOI matches its Crossref record; 62 of the 62 references it cites were found in a registry or an index.
Abstract
Despite vigorous debates about the technical characteristics of risk assessments being deployed in the U.S. criminal justice system, remarkably little research has studied how these tools affect actual decision-making processes. After all, risk assessments do not make definitive decisions---they inform judges, who are the final arbiters. It is therefore essential that considerations of risk assessments be informed by rigorous studies of how judges actually interpret and use them. This paper takes a first step toward such research on human interactions with risk assessments through a controlled experimental study on Amazon Mechanical Turk. We found several behaviors that call into question the supposed efficacy and fairness of risk assessments: our study participants 1) underperformed the risk assessment even when presented with its predictions, 2) could not effectively evaluate the accuracy of their own or the risk assessment's predictions, and 3) exhibited behaviors fraught with "disparate interactions," whereby the use of risk assessments led to higher risk predictions about black defendants and lower risk predictions about white defendants. These results suggest the need for a new "algorithm-in-the-loop" framework that places machine learning decision-making aids into the sociotechnical context of improving human decisions rather than the technical context of generating the best prediction in the abstract. If risk assessments are to be used at all, they must be grounded in rigorous evaluations of their real-world impacts instead of in their theoretical potential.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“What is more, the authors also report important limitations in participants’ ability to evaluate the algorithm's performance (97).”
From Accountable Artificial Intelligence: Holding Algorithms to Account (2021) · cites this paper as Green and Chen 2019
In the citing paper
“They document a propensity for selective adherence to the algorithm by participants, resulting in discriminatory outcomes.”
From Accountable Artificial Intelligence: Holding Algorithms to Account (2021) · cites this paper as Green and Chen 2019
In the citing paper
“However, such audits tend to be conducted after model deployment, when the system has already negatively impacted users [26,51].”
From Closing the AI accountability gap: defining an end-to-end framework for internal algorithmic auditing (2020) · cites this paper as [26]
In the citing paper
“In their experimental study (with lay participants) about receptiveness to algorithmic advice in the context of criminal justice, the authors find evidence of “disparate interactions” with the risk assessment algorithm, leading to higher risk predictions for black defendants and lower risk predictions for white defendants.”
From Accountable Artificial Intelligence: Holding Algorithms to Account (2021) · cites this paper as Green and Chen 2019
In the citing paper
“Early empirical evidence does seem to suggest that far from being used as “neutral” devices, algorithmic outputs can accentuate disparate and discriminatory outcomes in human decision‐making (Green and Chen 2019).”
From Accountable Artificial Intelligence: Holding Algorithms to Account (2021) · cites this paper as Green and Chen 2019
8 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Greedy function approximation: A gradient boosting machine
Friedman, J. H. (2001) · The Annals of Statistics · doi:10.1214/aos/1013203451 · Cited by 30,731 · cited 1×
“, the risk assessment) using gradient boosted trees [26].”
BackgroundCrenshaw, K. (2018) · Feminist Legal Theory · doi:10.4324/9780429500480-5 · Cited by 9,510
Strictly Proper Scoring Rules, Prediction, and Estimation
Gneiting, T., & Raftery, A. E. (2007) · Journal of the American Statistical Association · doi:10.1198/016214506000001437 · Cited by 5,793 · cited 1×
“[28], participants were incentivized to report their true estimates of crime risk.”
BackgroundCrenshaw, K. (1998) · Feminism And Politics · doi:10.1093/oso/9780198782063.003.0016 · Cited by 5,672
Bertrand, M., & Mullainathan, S. (2004) · American Economic Review · doi:10.1257/0002828042002561 · Cited by 4,504
Algorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232 · cited 1×
“algorithms than errors made by other people [17].”
BackgroundTowards A Rigorous Science of Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2017) · arXiv · doi:10.48550/arxiv.1702.08608 · Cited by 3,121 · cited 1×
“Recent work related to interpretability provides one important direction where progress is already being made [20, 52, 53].”
BackgroundEquality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536 · cited 1×
“Prompted by these concerns, machine learning researchers have developed a rapidly-growing body of technical work focused on topics such as characterizing the incompatibility of dierent fairness metrics [6, 44] and developing new algorithms to reduce bias [24, 33].”
BackgroundClinical versus statistical prediction: A theoretical analysis and a review of the evidence.
Meehl, P. E. (1954) · University of Minnesota Press eBooks · doi:10.1037/11281-000 · Cited by 2,171 · cited 1×
“Judges may also be more reluctant to rely on risk assessments, believing that their own judgment is superior: previous research has shown that people with more expertise are less willing to take advice [38, 48, 50], and a recent survey found that less than 10%”
BackgroundFair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159 · cited 2×
“Prompted by these concerns, machine learning researchers have developed a rapidly-growing body of technical work focused on topics such as characterizing the incompatibility of dierent fairness metrics [6, 44] and developing new algorithms to reduce bias [24, 33].”
BackgroundAlgorithm appreciation: People prefer algorithmic to human judgment
Logg, J. M., Minson, J. A., & Moore, D. A. (2019) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2018.12.005 · Cited by 2,061
Certifying and Removing Disparate Impact
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783311 · Cited by 1,827 · cited 1×
“Prompted by these concerns, machine learning researchers have developed a rapidly-growing body of technical work focused on topics such as characterizing the incompatibility of dierent fairness metrics [6, 44] and developing new algorithms to reduce bias [24, 33].”
BackgroundPrinciples of mixed-initiative user interfaces
Horvitz, E. (1999) · Proceedings of the SIGCHI conference on Human factors in computing systems the CHI is the limit - CHI '99 · doi:10.1145/302979.303030 · Cited by 1,490 · cited 1×
“Yet research within human-computer interaction and crowdsourcing suggests that alternative approaches could lead to a better synthesis of human and computer intelligence [8, 35, 39, 40].”
BackgroundDietvorst, B. J., Simmons, J. P., & Massey, C. (2018) · Management Science · doi:10.1287/mnsc.2016.2643 · Cited by 1,230 · cited 1×
“One study found that people could not distinguish between reliable and unreliable predictions [30], and another found that people often deviate incorrectly from algorithmic fore-casts [18].”
BackgroundThe accuracy, fairness, and limits of predicting recidivism
Dressel, J., & Farid, H. (2018) · Science Advances · doi:10.1126/sciadv.aao5580 · Cited by 1,071 · cited 1×
“This is in contrast to a recent study suggesting that humans are just as accurate as COMPAS: that result holds only when the predictions of humans are aggregated to create a “wisdom of the crowd” eect; in fact, that study similarly found COMPAS to be more accurate than individuals [21].”
ResultHuman Decisions and Machine Predictions
Kleinberg, J., Lakkaraju, H., Leskovec, J., Ludwig, J., & Mullainathan, S. (2018) · The Quarterly Journal of Economics · doi:10.1093/qje/qjx032 · Cited by 1,040
Inherent Trade-Offs in the Fair Determination of Risk Scores
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2016) · arXiv · doi:10.48550/arxiv.1609.05807 · Cited by 941 · cited 2×
“We focused on calibration not as an ideal metric for fairness (recognizing that no perfect metric for fairness can exist [32]), but because it is the most commonly-used approach for evaluating risk assessments in practice [16, 25, 44].”
BackgroundMethodEberhardt, J. L., Davies, P. G., Purdie-Vaughns, V. J., & Johnson, S. L. (2006) · Psychological Science · doi:10.1111/j.1467-9280.2006.01716.x · Cited by 818
Proceedings of the 28th International Conference on Neural Information Processing Systems
Cortes, C., Lee, D., Sugiyama, M., & Garnett, R. (2015) · Cited by 747
Manipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Wortman Vaughan, J. W., & Wallach, H. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445315 · Cited by 728 · cited 2×
“terpretability provides one important direction where progress is already beingmade [20, 52, 53].”
BackgroundManipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Vaughan, J. W., & Wallach, H. (2018) · arXiv · doi:10.48550/arxiv.1802.07810 · Cited by 711
Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence
Chen, Y., Diethe, T., & Flach, P. A. (2016) · National Conference on Artificial Intelligence · Cited by 680
Self-citationReceiving other people’s advice: Influence and benefit
Yaniv, I. (2004) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2003.08.002 · Cited by 652
Human Decisions and Machine Predictions
Kleinberg, J., Lakkaraju, H., Leskovec, J., Ludwig, J., & Mullainathan, S. (2017) · doi:10.3386/w23180 · Cited by 545
Ban the Box, Criminal Records, and Racial Discrimination: A Field Experiment*
Agan, A., & Starr, S. (2018) · The Quarterly Journal of Economics · doi:10.1093/qje/qjx028 · Cited by 545
Making sense of recommendations
Yeomans, M., Shah, A., Mullainathan, S., & Kleinberg, J. (2019) · Journal of Behavioral Decision Making · doi:10.1002/bdm.2118 · Cited by 459 · cited 1×
“Even when algorithms are more accurate, people do not appropriately incorporate algorithmic recommendations to improve their decisions, instead preferring to rely on their own or other people’s judgment [47, 66].”
BackgroundDo Judges Vary in Their Treatment of Race?
Abrams, D. S., Bertrand, M., & Mullainathan, S. (2012) · The Journal of Legal Studies · doi:10.1086/666006 · Cited by 379 · cited 1×
“Studies have shown that judges harbor implicit biases and that racial disparities in incarceration rates are due in part to dierential judicial decisions across race [1, 54].”
BackgroundReconstructing Technologies as Social Practice
SUCHMAN, L., BLOMBERG, J., ORR, J. E., & TRIGG, R. (1999) · American Behavioral Scientist · doi:10.1177/00027649921955335 · Cited by 369 · cited 1×
“To answer these questions, it is necessary to study algorithms following the notion of “technologies as social practice,” which is grounded in the understanding that technologies “are constituted through and inseparable from the speci � cally situated practices of their use” [60].”
BackgroundEvidence-Based Sentencing and the Scientific Rationalization of Discrimination
Starr, S. B. (2013) · bepress Legal Repository · Cited by 356
Combining human and machine intelligence in large-scale crowdsourcing
Kamar, E. S., Hacker, S., & Horvitz, E. (2012) · doi:10.5555/2343576.2343643 · Cited by 318
Does Unconscious Racial Bias Affect Trial Judges
Rachlinski, J. J., Johnson, S. L., Wistrich, A. J., & Guthrie, C. (2009) · Scholarship @ Cornell Law (Cornell University) · Cited by 311
(2007) · Digital Commons at University of Maryland Carey Law (University of Maryland Francis King Carey School of Law) · Cited by 277
Creative Writing with a Machine in the Loop: Case Studies on Slogans and Stories
Clark, E., Ross, A. S., Tan, C., Ji, Y., & Smith, N. A. (2018) · 23rd International Conference on Intelligent User Interfaces · doi:10.1145/3172944.3172983 · Cited by 257 · cited 1×
“Yet research within human-computer interaction and crowdsourcing suggests that alternative approaches could lead to a better synthesis of human and computer intelligence [8, 35, 39, 40].”
BackgroundFlores, A. W., Bechtel, K., & Lowenkamp, C. T. (2016) · Federal probation · Cited by 240
Doleac, J. L., & Hansen, B. (2020) · Journal of Labor Economics · doi:10.1086/705880 · Cited by 233 · cited 1×
“how many black applicants have criminal records [2, 19].”
Assessing Risk Assessment in Action
Stevenson, M. (2018) · Minnesota Law Review · doi:10.24926/265535.958 · Cited by 193 · cited 2×
“This is an important factor to consider when deploying algorithmic systems, especially given research demonstrating that the changes instigated by risk assessments are short-lived [59].”
BackgroundClinical versus Statistical Prediction: A Theoretical Analysis and a Review of the Evidence
McNemar, Q., & Meehl, P. E. (1955) · The American Journal of Psychology · doi:10.2307/1418552 · Cited by 188
Judgemental adjustment of initial forecasts: Its effectiveness and biases
Lim, J. S., & O'Connor, M. (1995) · Journal of Behavioral Decision Making · doi:10.1002/bdm.3960080302 · Cited by 171 · cited 1×
“Even when algorithms are more accurate, people do not appropriately incorporate algorithmic recommendations to improve their decisions, instead preferring to rely on their own or other people’s judgment [47, 66].”
BackgroundAutomation and Accountability in Decision Support System Interface Design
Cummings, M. L. (2006) · Journal of Technology Studies · doi:10.21061/jots.v32i1.a.4 · Cited by 155 · cited 3×
“Heavy reliance on automated systems can alter people’s relationship to a task by creating a “moral bu � er” between their decisions and the impacts of those decisions [11].”
BackgroundInfluentialDirections in hybrid intelligence: complementing AI systems with human intelligence
Kamar, E. S. (2016) · International Joint Conference on Artificial Intelligence · Cited by 154
Narayanan, M., Chen, E., He, J., Kim, B., Gershman, S., & Doshi-Velez, F. (2018) · arXiv · doi:10.48550/arxiv.1802.00682 · Cited by 153 · cited 1×
“Recent work related to interpretability provides one important direction where progress is already being made [20, 52, 53].”
BackgroundDoes Unconscious Racial Bias Affect Trial Judges
Guthrie, C., Rachlinski, J. J., Johnson, S. L., & Wistrich, A. J. (2009) · The Notre Dame law review · Cited by 151
Goodwin, P., & Fildes, R. (1999) · Journal of Behavioral Decision Making · doi:10.1002/(sici)1099-0771(199903)12:1<37::aid-bdm319>3.0.co;2-8 · Cited by 141 · cited 1×
“One study found that people could not distinguish between reliable and unreliable predictions [30], and another found that people often deviate incorrectly from algorithmic forecasts [18].”
BackgroundHard Bargains: The Coercive Power of Drug Laws in Federal Court
Lynch, M. (2016) · Project Muse (Johns Hopkins University) · Cited by 137
A Punishing Look: Skin Tone and Afrocentric Features in the Halls of Justice
King, R. D., & Johnson, B. D. (2016) · American Journal of Sociology · doi:10.1086/686941 · Cited by 134 · cited 1×
“Studies have shown that employers discriminate based on raciallyindicative names [4] and that judges are harsher toward defendants with darker skin and more Afrocentric features [22, 42].”
BackgroundThe Myth in the Methodology: Towards a Recontextualization of Fairness in Machine Learning
Green, B., & Hu, L. J. (2018) · International Conference on Machine Learning · Cited by 133
Self-citationAlgorithms in the Criminal Justice System: Assessing the Use of Risk Assessments in Sentencing
Kehl, D. L., & Kessler, S. A. (2017) · Digital Access to Scholarship at Harvard (DASH) (Harvard University) · Cited by 132
Performance of recidivism risk assessment instruments in U.S. correctional settings.
Desmarais, S. L., Johnson, K. L., & Singh, J. P. (2016) · Psychological Services · doi:10.1037/ser0000075 · Cited by 118 · cited 1×
“67 on the test set, indicating comparable accuracy to COMPAS [37, 45], the Public Safety Assessment [13], and other risk assessments [14, 15].”
Result(2024) · CQ Press eBooks · doi:10.4135/9781071920541.n114 · Cited by 65
Algorithm Appreciation: People Prefer Algorithmic To Human JudgmentPreprint
Logg, J. M., Minson, J., & Moore, D. A. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.2941774 · Cited by 59
DeMichele, M. T., Baumgartner, P., Wenger, M., Barrick, K., Comfort, M. L., & Misra, S. (2018) · SSRN Electronic Journal · doi:10.2139/ssrn.3168452 · Cited by 52 · cited 2×
“The model achieves an area under the curve (AUC) of 0.67 on the test set, indicating comparable accuracy to COMPAS [37, 45], the Public Safety Assessment [13], and other risk assessments [14, 15].”
ResultInfluential“Fair” Risk Assessments: A Precarious Approach for Criminal Justice Reform
Green, B. (2018) · Cited by 50
Self-citationDoleac, J. L., & Hansen, B. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.2812811 · Cited by 49
Automation Bias and Errors: Are Teams Better than Individuals?
Mosier, K. L., Dunbar, M., McDonnell, L., Skitka, L. J., Burdick, M., & Rosenblatt, B. (1998) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193129804200304 · Cited by 28 · cited 1×
“Two types of errors are particularly common: omission errors, in which people do not recognize when automated systems err, and commission errors, in which people follow automated systems without considering contradictory information [51].”
BackgroundPredicting Civil Jury Verdicts: How Attorneys Use (and Misuse) a Second Opinion
Jacobson, J., Dobbs‐Marsh, J., Liberman, V., & Minson, J. A. (2011) · Journal of Empirical Legal Studies · doi:10.1111/j.1740-1461.2011.01229.x · Cited by 20 · cited 1×
“Judges may also be more reluctant to rely on risk assessments, believing that their own judgment is superior: previous research has shown that people with more expertise are less willing to take advice [38, 48, 50], and a recent survey found that less than 10%”
BackgroundThe Use of Risk Assessment at Sentencing: Implications for Research and Policy
Hyatt, J. M., & Chanenson, S. L. (2016) · bepress Legal Repository · Cited by 19
Will Putting Cameras on Police Reduce Polarization
Sommers, R. C. (2016) · bepress Legal Repository · Cited by 17
Elmalech, A., Sarne, D., Rosenfeld, A., & Erez, E. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9335 · Cited by 15 · cited 1×
“This may involve altering the algorithm in unintuitive ways: previous research suggests that in certain situations a seemingly suboptimal algorithm actually leads to better outcomes when provided to people as advice [23].”
BackgroundArtificial intelligence could identify gang crimes—and ignite an ethical firestorm
Hutson, M. (2018) · Science · doi:10.1126/science.aat4510 · Cited by 11
Data-Reconciliation Based Fault-Tolerant Model Predictive Control for a Biomass Boiler
Sarkar, P., Kortela, J., Boriouchkine, A., Zattoni, E., & Jämsä‐Jounela, S. (2017) · Energies · doi:10.3390/en10020194 · Cited by 8 · cited 1×
“7 Although previous studies have used the phrase “algorithm-in-the-loop,” they have dened it in the context of simulation and modeling rather than in relation to humanin-the-loop computations and human-algorithm interactions [56, 64].”
Criminal Justice Reform Annual Report to the Governor and the Legislature 2017
Judiciary, N. J. (2018) · Cited by 6
Yan, Q.-Z., Kao, M., & Barrera, M. (2010) · SAE technical papers on CD-ROM/SAE technical paper series · doi:10.4271/2010-01-0367 · Cited by 2
Research works printed but not linked: 11
- Julia Angwin , Jeff Larson , Surya Mattu , and Lauren Kirchner . 2016. Machine Bias. ProPublica ( 2016 ). https:/www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2016. Machine Bias. ProPublica (2016). https:/www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing
- Mary L. Cummings . 2006. Automation and Accountability in Decision Support System Interface Design. Journal of Technology Studies ( 2006 ). Mary L. Cummings. 2006. Automation and Accountability in Decision Support System Interface Design. Journal of Technology Studies (2006).
- Mona J.E. Danner , Marie VanNostrand , and Lisa M . Spruance . 2015 . Risk-Based Pretrial Release Recommendation and Supervision Guidelines . Mona J.E. Danner, Marie VanNostrand, and Lisa M. Spruance. 2015. Risk-Based Pretrial Release Recommendation and Supervision Guidelines.
- Sarah L. Desmarais and Jay P . Singh . 2013 . Risk Assessment Instruments Validated and Implemented in Correctional Settings in the United States . (2013). Sarah L. Desmarais and Jay P. Singh. 2013. Risk Assessment Instruments Validated and Implemented in Correctional Settings in the United States. (2013).
- William Dieterich , Christina Mendoza , and Tim Brennan . 2016. COMPAS Risk Scales: Demonstrating Accuracy Equity and Predictive Parity . Northpointe Inc . ( 2016 ). William Dieterich, Christina Mendoza, and Tim Brennan. 2016. COMPAS Risk Scales: Demonstrating Accuracy Equity and Predictive Parity. Northpointe Inc. (2016).
- Gideon's Promise , The National Legal Aid and Defenders Association , The National Association for Public Defense, and The National Association of Criminal Defense Lawyers . 2017 . Joint Statement in Support of the Use of Pretrial Risk Assessment Instruments. (2017). http://www.publicdefenders.us/files/Defenders%20Statement%20on%20Pretrial%20RAI%20May%202017.pdf Gideon's Promise, The National Legal Aid and Defenders Association, The National Association for Public Defense, and The National Association of Criminal Defense Lawyers. 2017. Joint Statement in Support of the Use of Pretrial Risk Assessment Instruments. (2017). http://www.publicdefenders.us/files/Defenders%20Statement%20on%20Pretrial%20RAI%20May%202017.pdf
- John S. Goldkamp and Michael R . Gottfredson . 1984 . Judicial Decision Guidelines for Bail: The Philadelphia Experiment. National Institute of Justice Research Report ( 1984). John S. Goldkamp and Michael R. Gottfredson. 1984. Judicial Decision Guidelines for Bail: The Philadelphia Experiment. National Institute of Justice Research Report (1984).
- Kamala Harris and Rand Paul . 2017 . Pretrial Integrity and Safety Act of 2017 . 115th Congress (2017). Kamala Harris and Rand Paul. 2017. Pretrial Integrity and Safety Act of 2017. 115th Congress (2017).
- Jeff Larson , Surya Mattu , Lauren Kirchner , and Julia Angwin . 2016. How We Analyzed the COMPAS Recidivism Algorithm. ProPublica ( 2016 ). https:/www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm Jeff Larson, Surya Mattu, Lauren Kirchner, and Julia Angwin. 2016. How We Analyzed the COMPAS Recidivism Algorithm. ProPublica (2016). https:/www.propublica.org/article/how-we-analyzed-the-compas-recidivism-algorithm
- Josh Salman , Emily Le Coz, and Elizabeth Johnson . 2016 . Florida's broken sentencing system. Sarasota Herald-Tribune ( 2016). http://projects.heraldtribune.com/bias/sentencing/ Josh Salman, Emily Le Coz, and Elizabeth Johnson. 2016. Florida's broken sentencing system. Sarasota Herald-Tribune (2016). http://projects.heraldtribune.com/bias/sentencing/
- Tom Upchurch . 2018. To work for society, data scientists need a hippocratic oath with teeth. Wired ( 2018 ). https:/www.wired.co.uk/article/data-ai-ethics-hippocratic-oath-cathy-o-neil-weapons-of-math-destruction Tom Upchurch. 2018. To work for society, data scientists need a hippocratic oath with teeth. Wired (2018). https:/www.wired.co.uk/article/data-ai-ethics-hippocratic-oath-cathy-o-neil-weapons-of-math-destruction
Cited by
Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372873 · Cited by 2,245
Accountable Artificial Intelligence: Holding Algorithms to Account
Busuioc, M. (2021) · Public Administration Review · doi:10.1111/puar.13293 · Cited by 652
“Early empirical evidence does seem to suggest that far from being used as “neutral” devices, algorithmic outputs can accentuate disparate and discriminatory outcomes in human decision-making (Green and Chen 2019).”
BackgroundThe Ethics of Artificial Intelligence: Principles, Challenges, and Opportunities
Floridi, L. (2023) · doi:10.1093/oso/9780198883098.001.0001 · Cited by 507
Alon-Barkat, S., & Busuioc, M. (2023) · Journal of Public Administration Research and Theory · doi:10.1093/jopart/muac007 · Cited by 469
The Principles and Limits of Algorithm-in-the-Loop Decision Making
Green, B., & Chen, Y. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359152 · Cited by 392
“In the context of algorithm-assisted human decision making, prior research has explored topics such as what interactions can facilitate the development of machine learning models [9, 29, 47], how to improve human performance with an algorithm’s assistance [12, 37, 48], and the ways in which laypeople perceive algorith…”
BackgroundInfluentialSelf-citation
Metadata
- Authors
- Green, B., & Chen, Y.
- Affiliations
- Harvard University (US)
- Venue
- Proceedings of the Conference on Fairness, Accountability, and Transparency
- Publisher
- ACM
- Published
- 2019-01-29
- Pages
- 90-99
- OpenAlex
- W2901481055
- Cited by
- 275
- References
- 61
- Citations by year
- 2026: 25 · 2025: 37 · 2024: 32 · 2023: 59 · 2022: 44 · 2021: 29 · 2020: 30 · 2019: 19
- FWCI
- 31.82
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Law › Artificial Intelligence in Law
- Social Sciences › Sociology and Political Science › Criminal Justice and Corrections Analysis
- Keywords
- risk assessment tools, criminal justice decision-making, algorithmic fairness, indirect discrimination, human-machine interaction, forecast evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.