Problematic Machine Behavior: A Systematic Literature Review of Algorithm Audits
The DOI matches its Crossref record; 130 of the 130 references it cites were found in a registry or an index.
Abstract
While algorithm audits are growing rapidly in commonality and public importance, relatively little scholarly work has gone toward synthesizing prior work and strategizing future research in the area. This systematic literature review aims to do just that, following PRISMA guidelines in a review of over 500 English articles that yielded 62 algorithm audit studies. The studies are synthesized and organized primarily by behavior (discrimination, distortion, exploitation, and misjudgement), with codes also provided for domain (e.g. search, vision, advertising, etc.), organization (e.g. Google, Facebook, Amazon, etc.), and audit method (e.g. sock puppet, direct scrape, crowdsourcing, etc.). The review shows how previous audit studies have exposed public-facing algorithms exhibiting problematic behavior, such as search algorithms culpable of distortion and advertising algorithms culpable of discrimination. Based on the studies reviewed, it also suggests some behaviors (e.g. discrimination on the basis of intersectional identities), domains (e.g. advertising algorithms), methods (e.g. code auditing), and organizations (e.g. Twitter, TikTok, LinkedIn) that call for future audit attention. The paper concludes by offering the common ingredients of successful audits, and discussing algorithm auditing in the context of broader research working toward algorithmic justice.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The challenges associated with selecting subgroups for fairness assessment are well-documented by Raji et al. [52] and Barocas et al. [5], who note that intersectional and/or rare groups are often hard to identify and underrepresented in the data.”
From Who Audits the Auditors? Recommendations from a field scan of the algorithmic auditing ecosystem (2022) · cites this paper as [5]
In the citing paper
“and Bandy (2021) pointed out that, in the absence of agreed standards, technology providers' claims that their AI systems have been audited are hard to verify.”
From Auditing of AI: Legal, Ethical and Technical Approaches (2023) · cites this paper as Bandy, 2021
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & the PRISMA Group* (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00135 · Cited by 37,554
Comparison of PubMed, Scopus, Web of Science, and Google Scholar: strengths and weaknesses
Falagas, M. E., Pitsouni, E. I., Malietzis, G. A., & Pappas, G. (2008) · The FASEB Journal · doi:10.1096/fj.07-9492lsf · Cited by 5,039
The ethics of algorithms: Mapping the debate
Mittelstadt, B. D., Allo, P., Taddeo, M., Wachter, S., & Floridi, L. (2016) · Big Data & Society · doi:10.1177/2053951716679679 · Cited by 4,567
(2018) · Cited by 3,510
Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor
Lebovits, H. (2019) · Public Integrity · doi:10.1080/10999922.2018.1511671 · Cited by 3,279
Exposure to ideologically diverse news and opinion on Facebook
Bakshy, E., Messing, S., & Adamic, L. A. (2015) · Science · doi:10.1126/science.aaa1160 · Cited by 3,245
Big other: Surveillance Capitalism and the Prospects of an Information Civilization
Zuboff, S. (2015) · Journal of Information Technology · doi:10.1057/jit.2015.5 · Cited by 3,205
The New Jim Crow: Mass Incarceration in the Age of Colorblindness
Hauhart, R. C. (2011) · Contemporary Sociology: A Journal of Reviews · doi:10.1177/0094306111419111 · Cited by 2,732
Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372873 · Cited by 2,245
Fair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159 · cited 2×
“After ProPublica’s 2016 "Machine Bias" report [91] suggested that a publiclyused recidivism prediction instrument (RPI) discriminated on the basis of race, researchers discussed and explored appropriate fairness criteria and methods for balancing error rates [29, 61].”
BackgroundThe relation between 21st-century skills and digital skills: A systematic literature review
van Laar, E., van Deursen, A. J. A. M., van Dijk, J. A. G. M., & de Haan, J. (2017) · Computers in Human Behavior · doi:10.1016/j.chb.2017.03.010 · Cited by 2,006
Data colonialism: rethinking big data’s relation to the contemporary subject
Couldry, N., & Mejias, U. A. (2019) · Television & New Media · doi:10.1177/1527476418796632 · Cited by 1,617 · cited 4×
“see Zuboff [165], Couldry and Mejias [31], and West [159]), only five studies addressed how algorithmic systems exploit media content and/or personal information (Section 4.”
BackgroundInfluentialInvestigating Ad Transparency Mechanisms in Social Media: A Case Study of Facebook's Explanations
Andreou, A., Venkatadri, G., Goga, O., Gummadi, K. P., Loiseau, P., & Mislove, A. (2018) · Proceedings 2018 Network and Distributed System Security Symposium · doi:10.14722/ndss.2018.23191 · Cited by 1,477
The Netflix Recommender System: Algorithms, Business Value, and Innovation
Gomez-Uribe, C. A., & Hunt, N. (2016) · ACM Transactions on Management Information Systems · doi:10.1145/2843948 · Cited by 1,418
The Extent and Consequences of P-Hacking in Science
Head, M. L., Holman, L., Lanfear, R., Kahn, A. T., & Jennions, M. D. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002106 · Cited by 1,388
Rahwan, I., Cebrian, M., Obradovich, N., Bongard, J., Bonnefon, J.-F., Breazeal, C., Crandall, J. W., Christakis, N. A., Couzin, I. D., Jackson, M. O., et al. (2019) · Nature · doi:10.1038/s41586-019-1138-y · Cited by 1,364 · cited 2×
“In their piece describing the study of "machine behavior," Rahwan et al. [131] explicitly suggest price discrimination as a research area.”
BackgroundThinking critically about and researching algorithms
Kitchin, R. (2017) · Information, Communication & Society · doi:10.1080/1369118x.2016.1154087 · Cited by 1,186
Gillespie, T. (2014) · Media Technologies · doi:10.7551/mitpress/9042.003.0013 · Cited by 1,016
Decolonial AI: Decolonial Theory as Sociotechnical Foresight in Artificial Intelligence
Mohamed, S., Png, M.-T., & Isaac, W. (2020) · Philosophy & Technology · doi:10.1007/s13347-020-00405-8 · Cited by 912
Lambrecht, A., & Tucker, C. (2019) · Management Science · doi:10.1287/mnsc.2018.3093 · Cited by 854 · cited 3×
“Lambrecht and Tucker [98] found that Facebook advertisements for STEM career ads, even with gender-neutral targeting and bidding options, reached fewer women than men.”
BackgroundWorking with Machines: The Impact of Algorithmic and Data-Driven Management on Human Workers
Lee, M. K., Kusbit, D., Metsky, E., & Dabbish, L. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702548 · Cited by 823
The design of eco-feedback technology
Froehlich, J., Findlater, L., & Landay, J. (2010) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1753326.1753629 · Cited by 813 · cited 2×
“Literature reviews (sometimes referred to as literature surveys [163]) have been a prominent part of computing research [1, 39, 63, 76, 102], and a number of reviews are closely related to this work.”
BackgroundHuman computation: a survey and taxonomy of a growing field
Quinn, A. J., & Bederson, B. B. (2011) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1979148 · Cited by 801
Abdul, A., Vermeulen, J., Wang, D., Lim, B. Y., & Kankanhalli, M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174156 · Cited by 743
Shaping the Web: Why the Politics of Search Engines Matters
Introna, L. D., & Nissenbaum, H. (2017) · Computer Ethics · doi:10.4324/9781315259697-19 · Cited by 708
Eye-tracking analysis of user behavior in WWW search
Granka, L. A., Joachims, T., & Gay, G. (2004) · Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1008992.1009079 · Cited by 696
Automated Experiments on Ad Privacy Settings: A Tale of Opacity, Choice, and Discrimination
Datta, A., Tschantz, M. C., & Datta, A. (2015) · Proceedings on Privacy Enhancing Technologies · doi:10.1515/popets-2015-0007 · Cited by 676
Raji, I. D., & Buolamwini, J. (2019) · Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3306618.3314244 · Cited by 651
Auditing radicalization pathways on YouTube
Ribeiro, M. H., Ottoni, R., West, R., Almeida, V. A. F., & Meira, W. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372879 · Cited by 580
Where fairness fails: data, algorithms, and the limits of antidiscrimination discourse
Hoffmann, A. L. (2019) · Information, Communication & Society · doi:10.1080/1369118x.2019.1573912 · Cited by 535
Unequal Representation and Gender Stereotypes in Image Search Results for Occupations
Kay, M., Matuszek, C., & Munson, S. A. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702520 · Cited by 509 · cited 5×
“For example, one of the earliest audits reviewed compared occupational gender representation compared to statistics from the U.S. Census Bureau [88].”
BackgroundResultInfluentialData Capitalism: Redefining the Logics of Surveillance and Privacy
West, S. M. (2019) · Business & Society · doi:10.1177/0007650317718185 · Cited by 488
Sweeney, L. (2013) · Queue · doi:10.1145/2460276.2460278 · Cited by 477
Research contributions in human-computer interaction
Wobbrock, J. O., & Kientz, J. A. (2016) · Interactions · doi:10.1145/2907069 · Cited by 416
Measuring price discrimination and steering on e-commerce web sites
Hannak, A., Soeller, G., Lazer, D., Mislove, A., & Wilson, C. (2014) · Proceedings of the 2014 Conference on Internet Measurement Conference · doi:10.1145/2663716.2663744 · Cited by 405
Burst of the Filter Bubble?: Effects of personalization on the diversity of Google News
Haim, M., Graefe, A., & Brosius, H.-B. (2018) · Digital Journalism · doi:10.1080/21670811.2017.1338145 · Cited by 399
Measuring personalization of web search
Hannak, A., Sapiezynski, P., Molavi Kakhki, A., Krishnamurthy, B., Lazer, D., Mislove, A., & Wilson, C. (2013) · Proceedings of the 22nd international conference on World Wide Web · doi:10.1145/2488388.2488435 · Cited by 396
Ogbonnaya-Ogburu, I. F., Smith, A. D. R., To, A., & Toyama, K. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376392 · Cited by 396
Möller, J., Trilling, D., Helberger, N., & van Es, B. (2018) · Information, Communication & Society · doi:10.1080/1369118x.2018.1444076 · Cited by 387
Breaking the filter bubble: democracy and design
Bozdag, E., & van den Hoven, J. (2015) · Ethics and Information Technology · doi:10.1007/s10676-015-9380-y · Cited by 375
Fairness-Aware Ranking in Search & Recommendation Systems with Application to LinkedIn Talent Search
Geyik, S. C., Ambler, S., & Kenthapadi, K. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330691 · Cited by 350
Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting
De-Arteaga, M., Romanov, A., Wallach, H., Chayes, J., Borgs, C., Chouldechova, A., Geyik, S., Kenthapadi, K., & Kalai, A. T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287572 · Cited by 337 · cited 1×
“Some of these non-public audits found age-related discrimination in sentiment analysis models [44], gender discrimination in semantic representation algorithms that may be used in hiring [37], and age-related disparities in landmark detection algorithms often used for facial analysis [147].”
BackgroundBias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting
De-Arteaga, M., Romanov, A., Wallach, H., Chayes, J., Borgs, C., Chouldechova, A., Geyik, S., Kenthapadi, K., & Kalai, A. T. (2019) · arXiv · doi:10.48550/arxiv.1901.09451 · Cited by 337
Design Justice: towards an intersectional feminist framework for design theory and practice
Costanza-Chock, S. (2018) · Proceedings of DRS · doi:10.21606/drs.2018.679 · Cited by 325
Wieringa, M. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372833 · Cited by 321 · cited 2×
“One related literature review by Wieringa [160] outlines a theory of "algorithmic accountability" through a synthesis of over 200 English articles.”
BackgroundSmart, useful, scary, creepy: perceptions of online behavioral advertising
Ur, B., Leon, P. G., Cranor, L. F., Shay, R., & Wang, Y. (2012) · Proceedings of the Eighth Symposium on Usable Privacy and Security · doi:10.1145/2335356.2335362 · Cited by 313
An Empirical Analysis of Algorithmic Pricing on Amazon Marketplace
Chen, L., Mislove, A., & Wilson, C. (2016) · Proceedings of the 25th International Conference on World Wide Web · doi:10.1145/2872427.2883089 · Cited by 297
Towards a critical race methodology in algorithmic fairness
Hanna, A., Denton, R., Smart, A., & Smith-Loud, J. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372826 · Cited by 295
All in the Family: systematic reviews, rapid reviews, scoping reviews, realist reviews, and more
Moher, D., Stewart, L., & Shekelle, P. (2015) · Systematic Reviews · doi:10.1186/s13643-015-0163-7 · Cited by 288
Bias in Online Freelance Marketplaces: Evidence from TaskRabbit and Fiverr
Hannák, A., Wagner, C., Garcia, D., Mislove, A., Strohmaier, M., & Wilson, C. (2017) · Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing · doi:10.1145/2998181.2998327 · Cited by 286 · cited 3×
“[72] 2017 Direct scrape Search Intersectional TaskRabbit Fiverr”
BackgroundInfluentialRieder, B., Matamoros-Fernández, A., & Coromina, Ò. (2018) · Convergence: The International Journal of Research into New Media Technologies · doi:10.1177/1354856517736982 · Cited by 285
DeVito, M. A. (2017) · Digital Journalism · doi:10.1080/21670811.2016.1178592 · Cited by 280
The Ins and Outs of HCI for Development
Dell, N., & Kumar, N. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858081 · Cited by 254
Detecting price and search discrimination on the internet
Mikians, J., Gyarmati, L., Erramilli, V., & Laoutaris, N. (2012) · Proceedings of the 11th ACM Workshop on Hot Topics in Networks · doi:10.1145/2390231.2390245 · Cited by 244
Newspaper Closures Polarize Voting Behavior
Darr, J. P., Hitt, M. P., & Dunaway, J. L. (2018) · Journal of Communication · doi:10.1093/joc/jqy051 · Cited by 244
Nechushtai, E., & Lewis, S. C. (2019) · Computers in Human Behavior · doi:10.1016/j.chb.2018.07.043 · Cited by 237
How computer systems embody values
Nissenbaum, H. (2001) · Computer · doi:10.1109/2.910905 · Cited by 232
Measuring Misinformation in Video Search Platforms: An Audit Study on YouTube
Hussein, E., Juneja, P., & Mitra, T. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3392854 · Cited by 226
Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy
Verma, S. (2019) · Vikalpa: The Journal for Decision Makers · doi:10.1177/0256090919853933 · Cited by 219
Roles for computing in social change
Abebe, R., Barocas, S., Kleinberg, J., Levy, K., Raghavan, M., & Robinson, D. G. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372871 · Cited by 219
Seeking and sharing health information online: comparing search engines and social media
De Choudhury, M., Morris, M. R., & White, R. W. (2014) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557214 · Cited by 216
Auditing Partisan Audience Bias within Google Search
Robertson, R. E., Jiang, S., Joseph, K., Friedland, L., Lazer, D., & Wilson, C. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3274417 · Cited by 209
Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · arXiv · doi:10.48550/arxiv.2001.00973 · Cited by 179
Algorithmically recognizable: Santorum’s Google problem, and Google’s Santorum problem
Gillespie, T. (2017) · Information, Communication & Society · doi:10.1080/1369118x.2016.1199721 · Cited by 178
Investigating the Impact of Gender on Rank in Resume Search Engines
Chen, L., Ma, R., Hannák, A., & Wilson, C. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174225 · Cited by 155 · cited 2×
“Algorithm audits targeting recommender systems will benefit from the rich literature in search discrimination, on platforms like LinkedIn, Monster, and CareerBuilder (e.g., [26, 64]).”
BackgroundThe visibility of scientific misconduct: A review of the literature on retracted journal articles
Hesselmann, F., Graf, V., Schmidt, M., & Reinhart, M. (2017) · Current Sociology · doi:10.1177/0011392116663807 · Cited by 153
Quantifying Search Bias: Investigating Sources of Bias for Political Searches in Social Media
Kulshrestha, J., Eslami, M., Messias, J., Zafar, M. B., Ghosh, S., Gummadi, K. P., & Karahalios, K. (2017) · Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing · doi:10.1145/2998181.2998321 · Cited by 148 · cited 1×
“[94] 2017 Direct scrape Search Twitter Google”
Data science ethical considerations: a systematic literature review and proposed project framework
Saltz, J., & Dewar, N. (2019) · Ethics and Information Technology · doi:10.1007/s10676-019-09502-5 · Cited by 144
Addressing age-related bias in sentiment analysis,
Diaz, M., Johnson, I., Lazar, A., Piper, A. M., & Gergle, D. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173986 · Cited by 138 · cited 1×
“Some of these non-public audits found age-related discrimination in sentiment analysis models [44], gender discrimination in semantic representation algorithms that may be used in hiring [37], and age-related disparities in landmark detection algorithms often used for facial analysis [147].”
BackgroundGillingham, P. (2016) · British Journal of Social Work · doi:10.1093/bjsw/bcv031 · Cited by 133
The Sharing Economy in Computing: A Systematic Literature Review
Dillahunt, T. R., Wang, X., Wheeler, E., Cheng, H. F., Hecht, B., & Zhu, H. (2017) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3134673 · Cited by 130
Quantifying Search Bias: Investigating Sources of Bias for Political Searches in Social Media
Kulshrestha, J., Eslami, M., Messias, J., Zafar, M. B., Ghosh, S., Gummadi, K. P., & Karahalios, K. G. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.01347 · Cited by 129
Location, Location, Location: The Impact of Geolocation on Web Search Personalization
Kliman-Silver, C., Hannak, A., Lazer, D., Wilson, C., & Mislove, A. (2015) · Proceedings of the 2015 Internet Measurement Conference · doi:10.1145/2815675.2815714 · Cited by 125
Beyond the Bubble: Assessing the Diversity of Political Search Results
Puschmann, C. (2019) · Digital Journalism · doi:10.1080/21670811.2018.1539626 · Cited by 119
Gender differences in participation and reward on Stack Overflow
May, A., Wachs, J., & Hannák, A. (2019) · Empirical Software Engineering · doi:10.1007/s10664-019-09685-x · Cited by 116
Like Trainer, Like Bot? Inheritance of Bias in Algorithmic Content Moderation
Binns, R., Veale, M., Kleek, M. G. V., & Shadbolt, N. (2017) · UCL Discovery (University College London) · Cited by 113
Search as News Curator: The Role of Google in Shaping Attention to News Information
Trielli, D., & Diakopoulos, N. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300683 · Cited by 107
Automated Serendipity: The effect of using search engines on news repertoire balance and diversity
Fletcher, R., & Nielsen, R. K. (2018) · Digital Journalism · doi:10.1080/21670811.2018.1502045 · Cited by 105
Decentering technology in discourse on discrimination
Peña Gangadharan, S., & Niklas, J. (2019) · Information, Communication & Society · doi:10.1080/1369118x.2019.1593484 · Cited by 104
Search bias quantification: investigating political bias in social media and web search
Kulshrestha, J., Eslami, M., Messias, J., Zafar, M. B., Ghosh, S., Gummadi, K. P., & Karahalios, K. (2019) · Information Retrieval Journal · doi:10.1007/s10791-018-9341-2 · Cited by 103
Facebook Ads Monitor: An Independent Auditing System for Political Ads on Facebook
Silva, M., de Oliveira, L. S., Andreou, A., de Melo, P. O. S. V., Goga, O., & Benevenuto, F. (2020) · International World Wide Web Conference (WWW) · doi:10.1145/3366423.3380109 · Cited by 103
An FDA for AlgorithmsPreprint
Tutt, A. (2016) · SSRN Electronic Journal · doi:10.2139/ssrn.2747994 · Cited by 102
Bechmann, A., & Nielbo, K. L. (2018) · Digital Journalism · doi:10.1080/21670811.2018.1510741 · Cited by 102
Social Justice and Design: Power and oppression in collaborative systems
Fox, S., Dimond, J., Irani, L., Hirsch, T., Muller, M., & Bardzell, S. (2017) · Companion of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing · doi:10.1145/3022198.3022201 · Cited by 90
Suppressing the Search Engine Manipulation Effect (SEME)
Epstein, R., Robertson, R. E., Lazer, D., & Wilson, C. (2017) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3134677 · Cited by 86
Wilson, M. L. L., Resnick, P., Coyle, D., & Chi, E. H. (2013) · CHI '13 Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/2468356.2479636 · Cited by 84
Crowd-assisted search for price discrimination in e-commerce: first results
Mikians, J., Gyarmati, L., Erramilli, V., & Laoutaris, N. (2013) · Proceedings of the ninth ACM conference on Emerging networking experiments and technologies · doi:10.1145/2535372.2535415 · Cited by 82
Like Trainer, Like Bot? Inheritance of Bias in Algorithmic Content Moderation
Binns, R., Veale, M., Van Kleek, M., & Shadbolt, N. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-67256-4_32 · Cited by 81
Tolan, S., Miron, M., Gómez, E., & Castillo, C. (2019) · Proceedings of the Seventeenth International Conference on Artificial Intelligence and Law · doi:10.1145/3322640.3326705 · Cited by 75
Keyes, O., Hutson, J., & Durbin, M. (2019) · Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290607.3310433 · Cited by 72
Erratum for Discovering Order Dependencies through Order Compatibility (EDBT 2019)
Szlichta, J., Godfrey, P., Golab, L., Kargar, M., & Srivastava, D. (2020) · Movebank · doi:10.5441/002 · Cited by 72
Auditing local news presence on Google News
Fischer, S., Jaidka, K., & Lelkes, Y. (2020) · Nature Human Behaviour · doi:10.1038/s41562-020-00954-0 · Cited by 72
Taati, B., Zhao, S., Ashraf, A. B., Asgarian, A., Browne, M. E., Prkachin, K. M., Mihailidis, A., & Hadjistavropoulos, T. (2019) · IEEE Access · doi:10.1109/access.2019.2900022 · Cited by 70
Joining Together Online: The Trajectory of CSCW Scholarship on Group Formation
Harris, A. M., Gómez-Zará, D., DeChurch, L. A., & Contractor, N. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359250 · Cited by 69
Auditing Race and Gender Discrimination in Online Housing Markets
Asplund, J., Eslami, M., Sundaram, H., Sandvig, C., & Karahalios, K. (2020) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v14i1.7276 · Cited by 65
When people and algorithms meet: user-reported problems in intelligent everyday applications
Eiband, M., Völkel, S. T., Buschek, D., Cook, S., & Hussmann, H. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302262 · Cited by 62
Duwe, G., & Kim, K. (2017) · Criminal Justice Policy Review · doi:10.1177/0887403415604899 · Cited by 59
Courtois, C., Slechten, L., & Coenen, L. (2018) · Telematics and Informatics · doi:10.1016/j.tele.2018.07.004 · Cited by 59
Algorithms of Oppression: How Search Engines Reinforce Racism
Hankwitz, M. (2023) · Leonardo · doi:10.1162/leon_r_02331 · Cited by 59
Vincent, N., Hecht, B. J., & Sen, S. (2019) · International World Wide Web Conference (WWW) · doi:10.1145/3308558.3313742 · Cited by 52
Awareness Supporting Technologies used in Collaborative Systems: A Systematic Literature Review
Lopez, G., & Guerrero, L. A. (2017) · Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing · doi:10.1145/2998181.2998281 · Cited by 51
Auditing the Partisanship of Google Search Snippets
Hu, D., Jiang, S., E. Robertson, R., & Wilson, C. (2019) · The World Wide Web Conference · doi:10.1145/3308558.3313654 · Cited by 51
Auditing Autocomplete: Suggestion Networks and Recursive Algorithm Interrogation
Robertson, R. E., Jiang, S., Lazer, D. M. J., & Wilson, C. (2019) · ACM Web Science Conference (WebSci) · doi:10.1145/3292522.3326047 · Cited by 46
The case for voter-centered audits of search engines during political elections
Mustafaraj, E., Lurie, E., & Devine, C. (2020) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3351095.3372835 · Cited by 46
Coding the News: The role of computer code in filtering and distributing news
Weber, M. S., & Kosterich, A. (2018) · Digital Journalism · doi:10.1080/21670811.2017.1366865 · Cited by 45
Privacy practices in collaborative environments: a study of emergency department staff
Murphy, A. R., Reddy, M. C., & Xu, H. (2014) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2531602.2531643 · Cited by 42
Duwe, G. (2017) · Criminal Justice Policy Review · doi:10.1177/0887403417718608 · Cited by 42
3 Stars on Yelp, 4 Stars on Google Maps
Li, H., & Hecht, B. J. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3432953 · Cited by 40
González-Cabañas, J., Cuevas, √., & Cuevas, R. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025903 · Cited by 39
An Empirical Study on Online Price Differentiation
Hupperich, T., Tatang, D., Wilkop, N., & Holz, T. (2018) · ACM Conference on Data and Application Security and Privacy (CODASPY) · doi:10.1145/3176258.3176338 · Cited by 39
Quantity vs. Quality: Evaluating User Interest Profiles Using Ad Preference Managers
Bashir, M. A., Farooq, U., Shahid, M., Zaffar, M. F., & Wilson, C. (2019) · Network and Distributed System Security Symposium · doi:10.14722/ndss.2019.23392 · Cited by 39
Search Media and Elections: A Longitudinal Investigation of Political Search Results
Metaxa, D., Park, J. S., Landay, J. A., & Hancock, J. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359231 · Cited by 39
Auditing Offline Data Brokers via Facebook's Advertising Platform
Venkatadri, G., Sapiezynski, P., Redmiles, E. M., Mislove, A., Goga, O., Mazurek, M., & Gummadi, K. P. (2019) · The World Wide Web Conference · doi:10.1145/3308558.3313666 · Cited by 35
The Accuracy of the Demographic Inferences Shown on Google's Ad Settings
Tschantz, M. C., Egelman, S., Choi, J., Weaver, N. C., & Friedland, G. (2018) · Workshop on Privacy in the Electronic Society · doi:10.1145/3267323.3268962 · Cited by 31
Ad Delivery Algorithms: The Hidden Arbiters of Political Messaging
Ali, M., Sapiezynski, P., Korolova, A., Mislove, A., & Rieke, A. (2019) · arXiv · doi:10.48550/arxiv.1912.04255 · Cited by 30
More of the Same – On Spotify Radio
Snickars, P. (2017) · Culture Unbound Journal of Current Cultural Research · doi:10.3384/cu.2000.1525.1792184 · Cited by 28
Who's the Guinea Pig? Investigating Online A/B/n Tests in-the-Wild
Jiang, S., Martin, J., & Wilson, C. (2019) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3287560.3287565 · Cited by 28
Comparing Platform “Ranking Cultures” Across Languages: The Case of Islam on YouTube in Scandinavia
Moe, H. (2019) · Social Media + Society · doi:10.1177/2056305118817038 · Cited by 27
Cano‐Orón, L. (2019) · El Profesional de la Informacion · doi:10.3145/epi.2019.mar.13 · Cited by 20
MapWatch: Detecting and Monitoring International Border Personalization on Online Maps
Soeller, G., Karahalios, K., Sandvig, C., & Wilson, C. (2016) · Proceedings of the 25th International Conference on World Wide Web · doi:10.1145/2872427.2883016 · Cited by 19
The Right To Confront Your Accusers
Matthews, J. N., Babaeianjelodar, M., Lorenz, S., Matthews, A., Njie, M., Adams, N., Krane, D. E., Goldthwaite, J., & Hughes, C. (2019) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3306618.3314279 · Cited by 18
PrCP: Pre-recommendation Counter-Polarization
Badami, M., Nasraoui, O., & Shafto, P. (2018) · International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management · doi:10.5220/0006938702820289 · Cited by 13
Lai, C., & Luczak–Roesch, M. (2019) · Lecture notes in computer science · doi:10.26686/wgtn.12522023 · Cited by 12
Landau, I. (2014) · Encyclopedia of Educational Theory and Philosophy · doi:10.4135/9781483346229.n142 · Cited by 10
Lai, C., & Luczak–Roesch, M. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-34971-4_17 · Cited by 10
Analyzing the News Coverage of Personalized Newspapers
Chakraborty, A., & Ganguly, N. (2018) · IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) · doi:10.1109/asonam.2018.8508812 · Cited by 9
Whose Walkability? Challenges in Algorithmically Measuring Subjective Experience
Díaz, M., & Diakopoulos, N. A. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359228 · Cited by 9
Studying Ad Targeting with Digital Methods: The Case of Spotify
Mähler, R., & Vonderau, P. (2017) · Culture Unbound Journal of Current Cultural Research · doi:10.3384/cu.2000.1525.1792212 · Cited by 7
Eriksson, M., & Johansson, A. (2017) · GenderOpen - Repositorium für die Geschlechterforschung · doi:10.25595/1449 · Cited by 2
Research works printed but not linked: 33
- Penelope Muse Abernathy. 2018. The Expanding News Desert. University of North Carolina Press Chapel Hill. https://www.cislm.org/wp-content/uploads/2018/10/The-Expanding-News-Desert-10_14-Web.pdf
- Michael Armbrust Ion Stoica Matei Zaharia Armando Fox Rean Griffith Anthony D. Joseph Randy Katz Andy Konwinski Gunho Lee David Patterson and Ariel Rabkin. 2018. It's Time to Do Something: Mitigating the Negative Impacts of Computing Through a Change to the Peer Review Process. https://acm-fca.org/2018/03/29/negativeimpacts/
- Jack Bandy and Nicholas Diakopoulos. 2020. Auditing News Curation Systems: A Case Study Examining Algorithmic and Editorial Logic in Apple News. International conference on web and social media (ICWSM) (2020).
- Pinar Barlas, Kyriakos Kyriakou, Styliani Kleanthous, and Jahna Otterbacher. 2019. Social B(eye)as: Human and machine descriptions of people images. In Proceedings of the 13th International Conference on Web and Social Media, ICWSM 2019. 583--591. https://www.aaai.org/ojs/index.php/ICWSM/article/view/3255/3123
- Solon Barocas, Kate Crawford, Aaron Shapiro, and Hanna Wallach. 2017. The problem with bias: Allocative versus representational harms in machine learning. In 9th Annual Conference of the Special Interest Group for Computing, Information and Society.
- Ruha Benjamin. 2019. Race After Technology. Polity Press.
- José González Cabañas, Ángel Cuevas, and Rubén Cuevas. 2018. Unveiling and Quantifying Facebook Exploitation of Sensitive Personal Data for Advertising Purposes. In USENIX Security Symposium. 479--495. https://www.usenix.org/conference/usenixsecurity18/presentation/cabanas
- 10.4324/9781315582924--10 · doi:10.4324/9781315582924--10
- 10.1093/joc · doi:10.1093/joc
- Amit Datta, Anupam Datta, Jael Makagon, Deirdre K Mulligan, and Michael Carl Tschantz. 2018. Discrimination in Online Advertising: A Multidisciplinary Inquiry. Proceedings of the 1st Conference on Fairness, Accountability and Transparency, Vol. 81 (2018), 20--34. http://proceedings.mlr.press/v81/datta18a/datta18a.pdf
- Terrance DeVries, Ishan Misra, Changhan Wang, and Laurens van der Maaten. 2019. Does Object Recognition Work for Everyone?. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 52---59. arxiv: 1906.02659 · arXiv
- Nicholas Diakopoulos, Daniel Trielli, Jennifer Stark, and Sean Mussenden. 2018. I Vote For-How Search Informs Our Choice of Candidate. Digital Dominance: The Power of Google, Amazon, Facebook, and Apple (2018).
- Catherine D'Ignazio and Lauren F Klein. 2020. Data Feminism. MIT Press.
- Editors. 2020. Price Gouging Laws by State. https://consumer.findlaw.com/consumer-transactions/price-gouging-laws-by-state.html
- Elsevier. 2016. Scopus | The largest database of peer-reviewed literature.
- Motahhare Eslami, Kristen Vaccaro, Karrie Karahalios, and Kevin Hamilton. 2017. "Be careful; Things can be worse than they appear" - Understanding biased algorithms and users' behavior around them in rating platforms. In Proceedings of the 11th International Conference on Web and Social Media, ICWSM 2017. 62--71. https://ojs.aaai.org/index.php/ICWSM/article/view/14898
- Anthony Flores, Christopher T. Lowenkamp, and Kristin Bechtel. 2016. False Positives, False Negatives, and False Analyses: A Rejoinder to "Machine Bias". HeinOnline (2016).
- 10.7551/mitpress · doi:10.7551/mitpress
- 10.1093/bjsw · doi:10.1093/bjsw
- 10.1080/01972240050133634 · DOI
- Shan Jiang, Ronald E. Robertson, and Christo Wilson. 2019 b. Bias misperceived: The role of partisanship and misinformation in YouTube comment moderation. In Proceedings of the 13th International Conference on Web and Social Media, ICWSM 2019, Vol. 13(01). 278--289. https://ojs.aaai.org/index.php/ICWSM/article/view/3229
- Michael Kearns, Seth Neel, Aaron Roth, and Zhiwei Steven Wu. 2018. Preventing fairness gerrymandering: Auditing and learning for subgroup fairness. In International Conference on Machine Learning. http://proceedings.mlr.press/v80/kearns18a.html
- Kyriakos Kyriakou, Pinar Barlas, Styliani Kleanthous, and Jahna Otterbacher. 2019. Fairness in proprietary image tagging algorithms: A cross-platform audit on people images. In Proceedings of the 13th International Conference on Web and Social Media, ICWSM 2019. 313--322. https://www.aaai.org/ojs/index.php/ICWSM/article/view/3232/3100
- Emma Lurie and Eni Mustafaraj. 2019. Opening Up the Black Box: Auditing Google's Top Stories Algorithm. In Proceedings of the International Florida Artificial Intelligence Research Society Conference. https://par.nsf.gov/biblio/10101277
- Connor Mcmahon, Isaac Johnson, and Brent Hecht. 2017. The Substantial Interdependence of Wikipedia and Google: A Case Study on the Relationship Between Peer Production Communities and Information Technologies. In Proceedings of the 11th International AAAI Conference on Web and Social Media. https://ojs.aaai.org/index.php/ICWSM/article/view/14883
- 10.1145/2535372.2535415 · DOI · Paper page
- Safiya Noble. 2013. Google search: Hyper-visibility as a means of rendering black women and girls invisible. InVisible Culture (2013).
- 10.1145/3292522.3326047 · DOI
- Christian Sandvig, Kevin Hamilton, Karrie Karahalios, and Cedric Langbort. 2014. Auditing Algorithms: Research Methods for Detecting Discrimination on Internet Platforms. In Data and discrimination: converting critical concerns into productive inquiry.
- Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, and Noah A. Smith. 2020. The risk of racial bias in hate speech detection. In ACL 2019 - 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference. 1668--1678.
- 10.2139/ssrn.2208240 · DOI
- Nicholas Vincent, Isaac Johnson, Patrick Sheehan, and Brent Hecht. 2019 b. Measuring the Importance of User-Generated Content to Search Engines. In Proceedings of the International AAAI Conference on Web and Social Media (ICWSM). https://ojs.aaai.org/index.php/ICWSM/article/view/3248
- Sara Wachter-Boettcher. 2017. Technically Wrong: Sexist Apps, Biased Algorithms, and Other Threats of Toxic Tech Sexism. WW Norton & Company.
Cited by
Auditing large language models: a three-layered approach
Mökander, J., Schuett, J., Kirk, H. R., & Floridi, L. (2024) · AI and Ethics · doi:10.1007/s43681-023-00289-2 · Cited by 289
Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction
Shelby, R., Rismani, S., Henne, K., Moon, A., Rostamzadeh, N., Nicholas, P., Yilla-Akbari, N., Gallegos, J., Smart, A., Garcia, E., et al. (2023) · Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3600211.3604673 · Cited by 279
Milli, S., Carroll, M., Wang, Y., Pandey, S., & Dragan, A. D. (2023) · arXiv.org · doi:10.48550/arxiv.2305.16941 · Cited by 200
Who Audits the Auditors? Recommendations from a field scan of the algorithmic auditing ecosystem
Costanza-Chock, S., Raji, I. D., & Buolamwini, J. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533213 · Cited by 277
AI auditing: The Broken Bus on the Road to AI Accountability
Birhane, A., Steed, R., Ojewale, V., Vecchione, B., & Raji, I. D. (2024) · 2024 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml59370.2024.00037 · Cited by 158
Metadata
- Authors
- Bandy, J.
- Affiliations
- Northwestern University (US)
- Corresponding author
- Bandy, Jack
- ORCID
- Bandy, Jack
- Venue
- Proceedings of the ACM on Human-Computer Interaction
- Published
- 2021-04-13
- Volume
- 5
- Issue
- CSCW1
- Pages
- 1-34
- ISSN
- 2573-0142 (ISSN-L)
- OpenAlex
- W3159435787
- Cited by
- 193
- References
- 127
- Citations by year
- 2026: 31 · 2025: 60 · 2024: 45 · 2023: 32 · 2022: 18 · 2021: 7
- FWCI
- 31.92
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Proceedings of the ACM on Human-Computer Interaction (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Keywords
- algorithmic auditing, algorithmic discrimination, search algorithms, intersectional discrimination, audit procedures, algorithmic justice
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.