The Flaws of Policies Requiring Human Oversight of Government Algorithms
Open accessThe DOI matches its Crossref record; 105 of the 110 references it cites were found in a registry or an index.
Abstract
As algorithms become an influential component of government decision-making around the world, policymakers have debated how governments can attain the benefits of algorithms while preventing the harms of algorithms. One mechanism that has become a centerpiece of global efforts to regulate government algorithms is to require human oversight of algorithmic decisions. Despite the widespread turn to human oversight, these policies rest on an uninterrogated assumption: that people are able to effectively oversee algorithmic decision-making. In this article, I survey 41 policies that prescribe human oversight of government algorithms and find that they suffer from two significant flaws. First, evidence suggests that people are unable to perform the desired oversight functions. Second, as a result of the first flaw, human oversight policies legitimize government uses of faulty and controversial algorithms without addressing the fundamental issues with these tools. Thus, rather than protect against the potential harms of algorithmic decision-making in government, human oversight policies provide a false sense of security in adopting algorithms and enable vendors and agencies to shirk accountability for algorithmic harms. In light of these flaws, I propose a shift from human oversight to institutional oversight as the central mechanism for regulating government algorithms. This institutional approach operates in two stages. First, agencies must justify that it is appropriate to incorporate an algorithm into decision-making and that any proposed forms of human oversight are supported by empirical evidence. Second, these justifications must receive democratic public review and approval before the agency can adopt the algorithm.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Blaming individual users of the technology will not help to develop safety improvements as this approach overlooks the important systemic issues and shifts responsibility from chiefs and vendors, which Green points out in relation to human oversight and algorithms (2022).”
From Just accountability structures – a way to promote the safe use of automated decision-making in the public sector (2024) · cites this paper as Green 2022
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Street-Level Bureaucracy: Dilemmas of the Individual in Public Services
Lipsky, M. (1981) · Michigan Law Review · doi:10.2307/1288305 · Cited by 8,981
The global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0088-2 · Cited by 6,220
Normal Accidents: Living with High Risk Technologies
Grimes, A. J., & Perrow, C. (1985) · Academy of Management Review · doi:10.5465/amr.1985.4278477 · Cited by 4,963
Bureaucracy: what government agencies do and why they do it
(1990) · Choice Reviews Online · doi:10.5860/choice.27-4796 · Cited by 3,795
(2018) · Cited by 3,510
Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor
Lebovits, H. (2019) · Public Integrity · doi:10.1080/10999922.2018.1511671 · Cited by 3,279
Algorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232
Normal Accidents: Living with High-Risk Technologies
Cummings, L. L., & Perrow, C. (1984) · Administrative Science Quarterly · doi:10.2307/2392945 · Cited by 2,861
Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems
(2017) · doi:10.1145/3025453 · Cited by 2,207
Bainbridge, L. (1983) · Automatica · doi:10.1016/0005-1098(83)90046-8 · Cited by 2,205
Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
(2020) · doi:10.1145/3313831 · Cited by 1,951
Complacency and Bias in Human Use of Automation: An Attentional Integration
Parasuraman, R., & Manzey, D. H. (2010) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720810376055 · Cited by 1,710
Lee, M. K. (2018) · Big Data & Society · doi:10.1177/2053951718756684 · Cited by 1,338
(2018) · Cited by 1,167
Buçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021) · arXiv · doi:10.48550/arxiv.2102.09692 · Cited by 1,124
(2019) · Cited by 1,082
Human Decisions and Machine Predictions
Kleinberg, J., Lakkaraju, H., Leskovec, J., Ludwig, J., & Mullainathan, S. (2018) · The Quarterly Journal of Economics · doi:10.1093/qje/qjx032 · Cited by 1,040
Fjeld, J., Achten, N., Hilligoss, H., Nagy, A., & Srikumar, M. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3518482 · Cited by 998
The General Data Protection Regulation: An Opportunity for the HCI Community?
Thelisson, E., Sharma, K., Salam, H., & Dignum, V. (2018) · Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3170427.3170632 · Cited by 991
Bovens, M., & Zouridis, S. (2002) · Public Administration Review · doi:10.1111/0033-3352.00168 · Cited by 964
When the State Meets the Street: Public Service and Moral Agency
Zacka, B. (2017) · Harvard University Press eBooks · doi:10.4159/9780674981423 · Cited by 789
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445717 · Cited by 757
The level of service inventory – revised
Andrews, D. A., Bonta, J. L., & Andrews, D. J. (1995) · Cited by 733
Manipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Wortman Vaughan, J. W., & Wallach, H. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445315 · Cited by 728
Does automation bias decision-making?
SKITKA, L. J., MOSIER, K. L., & BURDICK, M. (1999) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.1999.0252 · Cited by 694
'It's Reducing a Human Being to a Percentage': Perceptions of Justice in Algorithmic Decisions
Binns, R., Van Kleek, M., Veale, M., Lyngs, U., Zhao, J., & Shadbolt, N. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173951 · Cited by 691
Kleinberg, J., Ludwig, J., Mullainathan, S., & Obermeyer, Z. (2015) · American Economic Review · doi:10.1257/aer.p20151023 · Cited by 584
Human Decisions and Machine Predictions
Kleinberg, J., Lakkaraju, H., Leskovec, J., Ludwig, J., & Mullainathan, S. (2017) · doi:10.3386/w23180 · Cited by 545
Making sense of recommendations
Yeomans, M., Shah, A., Mullainathan, S., & Kleinberg, J. (2019) · Journal of Behavioral Decision Making · doi:10.1002/bdm.2118 · Cited by 459
ETHICS GUIDELINES FOR TRUSTWORTHY AI
Mason, C. L. (2018) · Cited by 414
Ethics Guidelines for Trustworthy AI
Cannarsa, M. (2021) · The Cambridge Handbook of Lawyering in the Digital Age · doi:10.1017/9781108936040.022 · Cited by 404
Street-Level Bureaucracy and E-Government
Buffat, A. (2015) · Public Management Review · doi:10.1080/14719037.2013.771699 · Cited by 394
The Principles and Limits of Algorithm-in-the-Loop Decision Making
Green, B., & Chen, Y. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359152 · Cited by 392
Self-citationLai, V., & Tan, C. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287590 · Cited by 377
Reconstructing Technologies as Social Practice
SUCHMAN, L., BLOMBERG, J., ORR, J. E., & TRIGG, R. (1999) · American Behavioral Scientist · doi:10.1177/00027649921955335 · Cited by 369
Jacobs, A. Z., & Wallach, H. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445901 · Cited by 362
Evidence-Based Sentencing and the Scientific Rationalization of Discrimination
Starr, S. B. (2013) · bepress Legal Repository · Cited by 356
Impact of a deep learning assistant on the histopathologic classification of liver cancer
Kiani, A., Uyumazturk, B., Rajpurkar, P., Wang, A., Gao, R., Jones, E., Yu, Y., Langlotz, C. P., Ball, R. L., Montine, T. J., et al. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0232-8 · Cited by 325
Moral crumple zones: cautionary tales in human–robot interaction
Elish, M. C. (2019) · Engaging Science, Technology, and Society · doi:10.17351/ests2019.260 · Cited by 321
Bainbridge, L. (1982) · IFAC Proceedings Volumes · doi:10.1016/s1474-6670(17)62897-0 · Cited by 321
Technologies of Crime Prediction: The Reception of Algorithms in Policing and Criminal Courts
Brayne, S., & Christin, A. (2021) · Social Problems · doi:10.1093/socpro/spaa004 · Cited by 279
(2007) · Digital Commons at University of Maryland Carey Law (University of Maryland Francis King Carey School of Law) · Cited by 277
Disparate Interactions: An Algorithm-in-the-Loop Analysis of Fairness in Risk Assessments
Green, B., & Chen, Y. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287563 · Cited by 275
Self-citationGovernment by Algorithm: Artificial Intelligence in Federal Administrative Agencies
Engstrom, D. F., Ho, D. E., Sharkey, C. M., & Cuéllar, M.-F. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3551505 · Cited by 262
Jacobs, M., Pradier, M. F., McCoy, T. H., Perlis, R. H., Doshi-Velez, F., & Gajos, K. Z. (2021) · Translational Psychiatry · doi:10.1038/s41398-021-01224-x · Cited by 247
Yang, Q., Steinfeld, A., & Zimmerman, J. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300468 · Cited by 247
A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores
De-Arteaga, M., Fogliato, R., & Chouldechova, A. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376638 · Cited by 221
Street-Level Bureaucracy: Dilemmas of the Individual in Public Services.
Goldner, F. H., & Lipsky, M. (1982) · Administrative Science Quarterly · doi:10.2307/2392554 · Cited by 212
Algorithmic Profiling of Job Seekers in Austria: How Austerity Politics Are Made Effective
Allhutter, D., Cech, F., Fischer, F., Grill, G., & Mager, A. (2020) · Frontiers in Big Data · doi:10.3389/fdata.2020.00005 · Cited by 199
Assessing Risk Assessment in Action
Stevenson, M. (2018) · Minnesota Law Review · doi:10.24926/265535.958 · Cited by 193
Automation technology and sense of control: A window on human agency
Berberian, B., Sarrazin, J.-C., Le Blaye, P., & Haggard, P. (2012) · PLoS ONE · doi:10.1371/journal.pone.0034075 · Cited by 190
Street-Level Algorithms: A Theory at the Gaps Between Policy and Decisions
Alkhatib, A., & Bernstein, M. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300760 · Cited by 188
Judgemental adjustment of initial forecasts: Its effectiveness and biases
Lim, J. S., & O'Connor, M. (1995) · Journal of Behavioral Decision Making · doi:10.1002/bdm.3960080302 · Cited by 171
The Smart Enough City: Putting Technology in Its Place to Reclaim Our Urban Future
Green, B. (2019) · doi:10.7551/mitpress/11555.001.0001 · Cited by 170
Self-citationRulemaking: How Government Agencies Write Law and Make Policy
Kerwin, C. M., & Furlong, S. R. (2019) · CQ Press eBooks · doi:10.4135/9781544332918 · Cited by 168
Veale, M., & Edwards, L. (2018) · Computer Law & Security Review · doi:10.1016/j.clsr.2017.12.002 · Cited by 162
'It's Reducing a Human Being to a Percentage'; Perceptions of Justice in Algorithmic Decisions
(2018) · doi:10.31235/osf.io/9wqxr · Cited by 161
Automation and Accountability in Decision Support System Interface Design
Cummings, M. L. (2006) · Journal of Technology Studies · doi:10.21061/jots.v32i1.a.4 · Cited by 155
Goodwin, P., & Fildes, R. (1999) · Journal of Behavioral Decision Making · doi:10.1002/(sici)1099-0771(199903)12:1<37::aid-bdm319>3.0.co;2-8 · Cited by 141
Liable, but not in control? Ensuring meaningful human agency in automated decision-making systems
Wagner, B. (2019) · Policy & Internet · doi:10.1002/poi3.198 · Cited by 136
Facial recognition is the plutonium of AI
Stark, L. (2019) · XRDS: Crossroads, The ACM Magazine for Students · doi:10.1145/3313129 · Cited by 134
Human Judgment in algorithmic loops: Individual justice and automated decision‐making
Binns, R. (2022) · Regulation & Governance · doi:10.1111/rego.12358 · Cited by 134
Malgieri, G. (2019) · Computer Law & Security Review · doi:10.1016/j.clsr.2019.05.002 · Cited by 124
Is the Most Accurate AI the Best Teammate? Optimizing AI for Teamwork
Bansal, G., Nushi, B., Kamar, E., Horvitz, E., & Weld, D. S. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i13.17359 · Cited by 121
Global Data Privacy: The EU WayPreprint
Schwartz, P. M. (2019) · SSRN Electronic Journal · Cited by 117
When the State Meets the Street: Public Service and Moral Agency
Stout, M. (2018) · Public Integrity · doi:10.1080/10999922.2018.1439656 · Cited by 115
Human Decision Making with Machine Assistance: An Experiment on Bailing and Jailing
Grgić-Hlača, N., Engel, C., & Gummadi, K. P. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359280 · Cited by 113
Wrongfully Accused by an Algorithm *
Hill, K. (2022) · Ethics of Data and Analytics · doi:10.1201/9781003278290-21 · Cited by 110
(2020) · bepress Legal Repository · Cited by 92
Escaping the Impossibility of Fairness: From Formal to Substantive Algorithmic Fairness
Green, B. (2022) · Philosophy & Technology · doi:10.1007/s13347-022-00584-6 · Cited by 90
Self-citationWhen the State Meets the Street: Public Service and Moral Agency
Zacka, B. (2017) · Cited by 90
The false promise of risk assessments: epistemic reform and the limits of fairness
Green, B. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372869 · Cited by 80
Self-citationThe right to a human in the loop: Political constructions of computer automation and personhood
Jones, M. L. (2017) · Social Studies of Science · doi:10.1177/0306312717699716 · Cited by 75
Impact of risk assessment on judges’ fairness in sentencing relatively poor defendants
Skeem, J., Scurich, N., & Monahan, J. (2020) · Law and Human Behavior · doi:10.1037/lhb0000360 · Cited by 70
Green, B., & Chen, Y. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479562 · Cited by 69
Self-citationYang, Q., Steinfeld, A. M., & Zimmerman, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.09612 · Cited by 68
The Administrative Conference of the United States
Gardner, W. W. (1972) · The Annals of the American Academy of Political and Social Science · doi:10.1177/000271627240000105 · Cited by 68
Catalyzing Privacy Law, by Anupam Chander, Margot E. Kaminski, and William McGeveran here.
Chander, A., Kaminski, M. E., & McGeveran, W. (2021) · Minnesota law review · doi:10.24926/265535.4205 · Cited by 68
Algorithmic Risk Assessment in the Hands of Humans
Stevenson, M., & Doleac, J. L. (2019) · doi:10.2139/ssrn.3489440 · Cited by 60
People’s Councils for Ethical Machine Learning
McQuillan, D. (2018) · Social Media + Society · doi:10.1177/2056305118768303 · Cited by 59
Catalyzing Privacy LawPreprint
Chander, A., Kaminski, M. E., & McGeveran, W. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3433922 · Cited by 57
McMahon, G. (2018) · Routledge eBooks · doi:10.4324/9780429477591-5 · Cited by 50
Algorithmic Risk Assessment in the Hands of Humans
Stevenson, M., & Doleac, J. L. (2020) · doi:10.2139/ssrn.3513695 · Cited by 42
Huq, A. Z. (2019) · eYLS (Yale Law School) · Cited by 39
Christie, G. C. (1986) · Duke Law Journal · doi:10.2307/1372667 · Cited by 35
Lai, V., & Tan, C. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.07901 · Cited by 35
(2015) · SSRN Electronic Journal · Cited by 32
Dice in the Black Box: User Experiences with an Inscrutable Algorithm
Springer, A., Hollis, V., & Whittaker, S. (2018) · arXiv · doi:10.48550/arxiv.1812.03219 · Cited by 29
Human Judgement in Algorithmic Loops; Individual Justice and Automated Decision-MakingPreprint
Binns, R. (2019) · SSRN Electronic Journal · doi:10.31228/osf.io/kz4s2 · Cited by 20
Machines We Trust: Perspectives on Dependable Ai
Pelillo, M., & Scantamburlo, T. (2021) · Cited by 18
Elmalech, A., Sarne, D., Rosenfeld, A., & Erez, E. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9335 · Cited by 15
Escaping the Impossibility of Fairness: From Formal to Substantive Algorithmic FairnessPreprint
Green, B. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3883649 · Cited by 14
Self-citationThe Ironies of Automation Law: Tying Policy Knots with Fair Automation Practices PrinciplesPreprint
Jones, M. L. (2015) · SSRN Electronic Journal · doi:10.2139/ssrn.2549285 · Cited by 13
Strange Loops: Apparent Versus Actual Human Involvement in Automated Decision Making
Brennan-Marquez, K. (2019) · OpenCommons at University of Connecticut (University of Connecticut) · doi:10.15779/z385x25d2w · Cited by 13
(2020) · London School of Economics and Political Science Research Online (London School of Economics and Political Science) · Cited by 11
Heath, A. (2011) · Cited by 10
Human rights and technology: discussion paper
Farthing, S., Howell, J., Lecchi, K., Paleologos, Z., Saintilan, P., & Santow, E. (2019) · Cited by 9
McCradden, M. D. (2021) · Translational Psychiatry · doi:10.1038/s41398-021-01479-4 · Cited by 8
Adjudicating with Inscrutable Decision Rules
Strandburg, K. J. (2021) · The MIT Press eBooks · doi:10.7551/mitpress/12186.003.0010 · Cited by 4
Is the Most Accurate AI the Best Teammate? Optimizing AI for Teamwork
Bansal, G., Nushi, B., Kamar, E., Horvitz, E., & Weld, D. S. (2020) · arXiv · doi:10.48550/arxiv.2004.13102
(2020) · ePrints Soton (University of Southampton)
Escaping the Impossibility of Fairness: From Formal to Substantive Algorithmic Fairness
Green, B. (2021) · arXiv · doi:10.48550/arxiv.2107.04642
Self-citationGoodwin, P., & Fildes, R. (1999) · Journal of Behavioral Decision Making · doi:10.1002/(sici)1099-0771(199903)12:1<37::aid-bdm319>3.3.co;2-#
Research works printed but not linked: 37
- Albright A. If you give a judge a risk score: Evidence from Kentucky bail decisions; 2019 https: //thelittledataset.com/about _ files/albright _ judge _ score.pdf.
- Belgian Federal Parliament. Loi Relative à la Protection des Personnes Physiques à L'égard des Traitements de Données à Caractère Personnel (Law on the Protection of Natural Persons with regard to the Processing of Personal Data); 2018 https:// www.ejustice.just.fgov.be/eli/loi/2018/07/30/2018040581/justel .
- Bundestag, 2019, Federal Data Protection Act (BDSG)
- California Legislature, 2021, AB-13 Public Contracts: Automated Decision Systems
- Campbell ET, editor. Detroiters Want to be Seen, not Watched. Riverwise; 2019 https://detroitcommunitytech.org/system/tdf/ librarypdfs/2019-2206 _ Riverwise-Surveillance.pdf?file=1 . CBS News. Police Departments Adopting Facial Recognition Tech Amid Allegations of Wrongful Arrests; 2021 60 Minutes https://www.cbsnews.com/news/ facial-recognition-60-minutes-2021-05-16/ .
- Charette RN. Michigan's MiDAS unemployment system: Algorithm alchemy created lead, not gold. IEEE Spectrum 2018 https://spectrum.ieee.org/riskfactor/computing/software/ michigans-midas-unemployment-system-algorithm- alchemy-that-created-lead-not-gold .
- Cambridge City, 2018, Surveillance technology ordinance
- Cummings ML. Automation and accountability in decision support system interface design. J. Technol. Stud. 2006;32(1):23-31 https://scholar.lib.vt.edu/ejournals/JOTS/v32/ v32n1/pdf/cummings.pdf.
- 2021, Proposal for a Regulation Laying Down Harmonised Rules on Artificial Intelligence, Artificial Intelligence Act
- 2021, Civil Society Calls for AI red lines in the European Union's Artificial Intelligence Proposal
- 2021, EU's New Artificial Intelligence Law Risks Enabling Orwellian Surveillance States
- National Congress of Brazil. General Data Protection Law; 2019 https://iapp.org/media/pdf/resource _ center/ Brazilian _ General _ Data _ Protection _ Law.pdf. New Jersey Courts. One Year Criminal Justice Reform Report to the Governor and the Legislature; 2017 https: //www.njcourts.gov/courts/assets/criminal/2017cjrannual.pdf.
- 2021, Ban Facial Recognition
- French Parliament, 2018, French Data Protection Act
- K Hill, 2018, How one state managed to actually write rules on facial recognition, Houses of the Oireachtas
- Houses of the Oireachtas. (2018). Data Protection Act 2018. https://data.oireachtas.ie/ie/oireachtas/act/2018/7/eng/ enacted/a0718.pdf Human Rights Watch. "Not in it for Justice": How California's Pretrial Detention and Bail System Unfairly Punishes Poor People; 2017 https://www.hrw.org/report/2017/04/11/not-it-justice/ how-californias-pretrial-detention-and-bail-system-unfairly . Hungarian Parliament. Data Protection Act; 2018 https://www.parlament.hu/irom41/00623/00623.pdf.
- Ibarra N. Santa Cruz, Calif., Bans predictive policing technology. Govern. Technol. 2020 https://www.govtech.com/public-safety/ santa-cruz-calif-bans-predictive-policing-technology.html . Indiana Supreme Court. Malenchik v. State; 2010 928
- Kelly E. EU lawmaker vows to tighten use of biometric tracking tech in Europe. Sifted 2021 https://sifted.eu/articles/eu-facial-recognition-tech/ .
- Kingdom of Bahrain. Personal Data Protection Law; 2018 https://bahrainbusinesslaws.com/laws/ Personal-Data-Protection-Law .
- Koebler J. Detroit police chief: facial recognition software misidentifies 96% of the time. Vice 2020 https://www.vice.com/en/article/dyzykz/ detroit-police-chief-facial-recognition-software-misidentifies -96-of-the-time .
- Metzinger T. Ethics washing made in Europe. Der Tagesspiegel 2019 https://www.tagesspiegel.de/politik/ eu-guidelines-ethics-washing-made-in-europe/24195496. html .
- L Miller, 2020, LAPD will end controversial program that aimed to predict where crimes would occur, Los Angeles Times
- 2017, An Act to Modernize Legislative Provisions as Regards the Protection of Personal Information, Bill, 64
- Northpointe, Inc.. Practitioner's Guide to COMPAS Core; 2015 http://www.northpointeinc.com/downloads/compas/ Practitioners-Guide-COMPAS-Core-_ 031915.pdf. Office of the Privacy Commissioner of Canada. A Regulatory Framework for AI: Recommendations for PIPEDA Reform; 2020 https://www.priv.gc.ca/en/about-the-opc/what-we-do/ consultations/completed-consultations/consultation-ai/ reg-fw _ 202011/ .
- Pasquale F . New Laws of Robotics: Defending Human Expertise in the Age of AI. Belknap Press; 2020 .
- Argentina Republic, 2013, Ley De Protecci�n De Los Datos Personales (Personal Data Protection Law)
- Richardson R, editor. Confronting Black Boxes: A Shadow Report of the New York City Automated Decision System Task Force. AI Now Institute; 2019 https://ainowinstitute.org/ads-shadowreport-2019.pdf.
- Richardson R, Schultz JM, Crawford K. Dirty data, bad predictions: How civil rights violations impact police data, predictive policing systems, and justice. New York University Law Rev. 2019;94:15-55 https://www.nyulawreview.org/online-features/ dirty-data-bad-predictions-how-civil-rights-violations-impact- police-data-predictive-policing-systems-and-justice/ .
- Scharfenberg D. Computers can solve your problem. You may not like the answer. Boston Globe 2018 https://apps.bostonglobe. com/ideas/graphics/2018/09/equity-machine/ .
- J Skeem, 2016, Impact of risk assessment on judges' fairness in sentencing relatively poor defendants, Central Bond Court Report, 44, 51
- Slovenian Parliament. Zakon o varstvu Osebnih Podatkov na Podro čju Obravnavanja Kaznivih Dejanj (Personal Data Protection Act in the Field of Dealing with Criminal Offenses); 2020 http://www.pisrs.si/Pis.web/pregledPredpisa?id=ZAKO8157 .
- Solum L. Legal theory lexicon: Rules, standards, and principles. Legal Theory Blog 2009 https://lsolum.typepad.com/legaltheory/2009/09/ legal-theory-lexicon-rules-standards-and-principles.html .
- 2018, Statistics New Zealand. Algorithm Assessment Report
- Stein MI. New Orleans City Council bans facial recognition, predictive policing and other surveillance tech. Lens 2020 https://thelensnola.org/2020/12/18/new-orleans-city- council-approves-ban-on-facial-recognition-predictive- policing-and-other-surveillance-tech/ .
- Steinhart D. Juvenile detention risk assessment: A practice guide for juvenile detention reform. Annie E. Casey Found. 2006 https://www.aecf.org/m/resourceimg/ aecf-juveniledetentionriskassessment1-2006.pdf.
- Stop LAPD Spying Coalition. Before the Bullet Hits the Body: Dismantling Predictive Policing in Los Angeles; 2018 https://stoplapdspying.org/wp-content/uploads/2018/05/ Before-the-Bullet-Hits-the-Body-May-8-2018.pdf.
- Author Information Ben Green is a Postdoctoral Scholar in the Michigan Society of Fellows and an Assistant Professor in the Gerald R. Ford School of Public Policy. He holds a PhD in Applied Mathematics, with a secondary field in Science, Technology, and Society, from Har- vard University. His book, The Smart Enough City: Putting Tech- nology in Its Place to Reclaim Our Urban Future , was published in 2019 by MIT Press. Ben is also an Affiliate at the Berkman Klein Center for Internet & Society at Harvard and a Fellow at the Center for Democracy & Technology.
Cited by
Towards a standard for identifying and managing bias in artificial intelligence
Schwartz, R., Vassilev, A., Greene, K., Perine, L., Burt, A., & Hall, P. (2022) · doi:10.6028/nist.sp.1270 · Cited by 548
Laux, J., Wachter, S., & Mittelstadt, B. (2024) · Regulation & Governance · doi:10.1111/rego.12512 · Cited by 365
OECD (2025) · OECD eBooks · doi:10.1787/795de142-en · Cited by 324
Is human oversight to AI systems still possible?
Holzinger, A., Zatloukal, K., & Müller, H. (2024) · New Biotechnology · doi:10.1016/j.nbt.2024.12.003 · Cited by 179
“How knowledge workers think generative AI will (not) transform their industries,”
Woodruff, A., Shelby, R., Kelley, P. G., Rousso-Schindler, S., Smith-Loud, J., & Wilcox, L. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642700 · Cited by 105
Metadata
- Authors
- Green, B.
- Affiliations
- University of Michigan (US)
- Corresponding author
- Green, Ben
- ORCID
- Green, Ben
- Venue
- Computer law & security review
- Publisher
- Elsevier BV
- Published
- 2021
- Volume
- 45
- Pages
- 105681-105681
- ISSN
- 2212-473X (ISSN-L)
- OpenAlex
- W3200421165
- Cited by
- 259
- References
- 110
- Citations by year
- 2026: 108 · 2025: 56 · 2024: 43 · 2023: 28 · 2022: 24
- FWCI
- 43.80
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Computer law & security review (publishedVersion, cc-by)
- SSRN Electronic Journal (acceptedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Law › Digitalization, Law, and Regulation
- Social Sciences › Political Science and International Relations › E-Government and Public Services
- Keywords
- human oversight, algorithmic decision-making, algorithmic regulation, institutional oversight, accountability, algorithmic harms, public trust in government, algorithmic governance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.