CoChat

Auditing Algorithms: Understanding Algorithmic Systems from the Outside In

Metaxa, Danaë, Park, Joon Sung, Robertson, Ronald E., Karahalios, Karrie, Wilson, Christo, Hancock, Jeff, Sandvig, Christian · 2021 · Foundations and Trends® in Human–Computer Interaction
Cited by 140

The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.

Abstract

Algorithms are ubiquitous and critical sources of information online, increasingly acting as gatekeepers for users accessing or sharing information about virtually any topic, including their personal lives and those of friends and family, news and politics, entertainment, and even information about health and well-being. As a result, algorithmically-curated content is drawing increased attention and scrutiny from users, the media, and lawmakers alike. However, studying such content poses considerable challenges, as it is both dynamic and ephemeral: these algorithms are constantly changing, and frequently changing silently, with no record of the content to which users have been exposed over time. One strategy that has proven effective is the algorithm audit: a method of repeatedly querying an algorithm and observing its output in order to draw conclusions about the algorithm’s opaque inner workings and possible external impact. In this work, we present an overview of the algorithm audit methodology, including the history of audit studies in the social sciences from which this method is derived; a summary of key algorithm audits over the last two decades in a variety of domains, including health, politics, discrimination, and others; and a set of best practices for conducting algorithm audits today, contextualizing these practices using search engine audits as a case study. Finally, we conclude by discussing the social, ethical, and political dimensions of auditing algorithms, and propose normative standards for the use of this method.

Overview

Integrity signals

Registry

Crossref · 10.1561/1100000083

Cites

15 of 15 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 15 references

Research works printed but not linked: 89

  1. Abebe R. Hill S. Vaughan J. W. Small P. M. Schwartz H. A. 2019 “Using Search Queries to Understand Health Information Needs in Africa” Proceedings of the International AAAI Conference on Web and Social Media 13 01 3–14https://ojs.aaai.org/index.php/ICWSM/article/view/3360 · DOI
  2. Abid, (2021), “Large language models associate Muslims with violence”, Nature Machine Intelligence, 3, 461, 10.1038/s42256-021-00359-2 · DOI
  3. ACLU (2020) “Federal Court Rules ‘Big Data’ Discrimination Studies Do Not Violate Federal Anti-Hacking Law” Mar. https://www.aclu.org/press-releases/federal-court-rules-big-data-discrimination-studies-do-not-violate-federal-anti (accessed on 08/26/2020)
  4. Allam, (2014), “The Impact of Search Engine Selection and Sorting Criteria on Vaccination Beliefs and Attitudes:Two Experiments Manipulating Google Output”, Journal of Medical Internet Research, 16, 10.2196/jmir.2642 · DOI
  5. Asplund, 2020, “Auditing Race and Gender Discrimination in Online Housing Markets”, Proceedings of the Eleventh International AAAI Conference on Web and Social Media (ICWSM 2020), 10
  6. Asplund, 2020, “Auditing race and gender discrimination in online housing markets” In:, Proceedings of the International AAAI Conference on Web and Social Media, 14, 24, 10.1609/icwsm.v14i1.7276 · DOI
  7. Ballatore, (2015), “Google Chemtrails: A Methodology to Analyze Topic Representation in Search Engine Results”, First Monday, 20, 10.5210/fm.v20i7.5597 · DOI
  8. Ballatore, (2017), “Digital Hegemonies: The Localness of Search Engine ”, Annals of the American Association of Geographers, 107, 1194, 10.1080/24694452.2017.1308240 · DOI
  9. Bandy J. (2021) “Problematic Machine Behavior: A Systematic Literature Review of Algorithm Audits” arXiv:2102.04256 [cs.CY] · DOI · Paper page
  10. Bandy, 2020, “Auditing News Curation Systems: A Case Study Examining Algorithmic and Editorial Logic in Apple News”, Proceedings of the International AAAI Conference on Web and Social Media, 14, 36, 10.1609/icwsm.v14i1.7277 · DOI
  11. Barocas, 2017, “Fairness in machine ”, NIPS Tutorial, 1
  12. Bechmann, 2018, “Are We Exposed to the Same “News” in the News Feed?”, Digital Journalism, 6, 990, 10.1080/21670811.2018.1510741 · DOI
  13. Bernstein, 2021, “ESR: Ethics and Society Review of Artificial Intelligence Research”
  14. Bobrowsky, 2021, “Facebook Disables Access for NYU Research Into Political-Ad Targeting”, The Wall Street Journal
  15. Brown, 2021, “The algorithm audit: Scoring the algorithms that score us”, Big Data & Society, 8, 10.1177/2053951720983865 · DOI · Paper page
  16. Buolamwini, 2018, “Gender shades: Intersectional accuracy disparities in commercial gender classification” In:, Conference on Fairness, Accountability and Transparency, 77
  17. Cartright, 2011, “Intentions and Attention in Exploratory Health Search” In:, Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information Retrieval, 65
  18. Chen, 2018, “Investigating the Impact of Gender on Rank in Resume Search Engines” In:, Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18), 1, 10.1145/3173574.3174225 · DOI · Paper page
  19. Chen, 2015, “Peeking beneath the hood of Uber” In:, Proceedings of the 2015 Internet Measurement Conference, 495, 10.1145/2815675.2815681 · DOI · Paper page
  20. Cobbe, 2021, “Reviewable Automated Decision-Making: A Framework for Accountable Algorithmic Systems”, Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (FAccT '21), 598, 10.1145/3442188.3445921 · DOI · Paper page
  21. Association for Computing Machinery 1992 “ACM Code of Ethics and Professional Conduct” Code of Ethics
  22. Daniel, 1968, “Racial discrimination in England: based on the PEP report”, 1084
  23. DeChoudhury, 2014, “Seeking and Sharing Health Information Online: Comparing Search Engines and Social Media”, Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems, 1365
  24. Diakopoulos, 2018, “I vote for—how search informs our choice of candidate”, Digital Dominance: The Power of Google, Amazon, Facebook, and Apple, 22
  25. Díaz, 2018, “Addressing age-related bias in sentiment analysis”In:, Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, 1
  26. Edelman, 2017, “Racial discrimination in the sharing economy: Evidence from a field ”, American Economic Journal: Applied Economics, 9, 1
  27. Edelman B. G. Luca M. 2014 “Digital discrimination: The case of Airbnb.com” Harvard Business School NOM Unit Working Paper 14 54 · DOI · Paper page
  28. Epstein, 2015, “The Search Engine Manipulation Effect (SEME) and Its Possible Impact on the Outcomes of Elections”, Proceedings of the National Academy of Sciences, 112, E4512, 10.1073/pnas.1419828112 · DOI · Paper page
  29. Epstein, 2017, “Suppressing the Search Engine Manipulation Effect (SEME)”, Proceedings of the ACM on Human-Computer Interaction, 1, 1, 10.1145/3134677 · DOI
  30. Eriksson, 2017, “Tracking gendered streams”, Culture Unbound, 9, 163, 10.3384/cu.2000.1525.1792163 · DOI
  31. Fortunato, 2006, “Topical Interests and the Mitigation of Search Engine Bias”, Proceedings of the National Academy of Sciences, 103, 12684, 10.1073/pnas.0605525103 · DOI
  32. Gaddis, 2018, Audit studies: Behind the scenes with theory, method, and nuance, 10.1007/978-3-319-71153-9 · DOI · Paper page
  33. Ginsberg, 2009, “Detecting Influenza Epidemics Using Search Engine Query dataprivacylab”, Nature, 457, 1012, 10.1038/nature07634 · DOI · Paper page
  34. Gneezy, 2004, “the Are the disabled discriminated against in product markets? Evidence from field experiments”, American Economic Association Annual Meeting
  35. Goldman, 2008, Web Search, 121, 10.1007/978-3-540-75829-7_8 · DOI · Paper page
  36. Granka, 2010, The“ Politics of Search: A Decade Retrospective”, The Information Society, 26, 364, 10.1080/01972243.2010.511560 · DOI
  37. Hannak, 2013, “Measuring Personalization of Web Search”, Proceedings of the 22nd International Conference on World Wide Web (WWW '13), 527, 10.1145/2488388.2488435 · DOI · Paper page
  38. Hannak, 2014, “Measuring price discrimination and steering on e-commerce web sites”, Proceedings of the 2014 Internet Measurement Conference, 305, 10.1145/2663716.2663744 · DOI · Paper page
  39. Hannak, 2017, “Bias in Online Freelance Marketplaces: Evidence from TaskRabbit and Fiverr”, 20th ACM Conference on Computer-Supported Cooperative Work and Social Computing (CSCW 2017), 10.1145/2998181.2998327 · DOI · Paper page
  40. Hu, 2019, “Auditing the Partisanship of Google Search Snippets”, The Web Conference (WWW ’19), 693, 10.1145/3308558.3313654 · DOI
  41. Hussein, 2020, “Measuring Misinformation in Video Search Platforms: An Audit Study on YouTube”, Proceedings of the ACM on Human-Computer Interaction, 4, 1, 10.1145/3392854 · DOI · Paper page
  42. Hutchinson, 2019, “50 Years of Test (Un)Fairness: Lessons for Machine Learning”, Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT* '19), 49, 10.1145/3287560.3287600 · DOI · Paper page
  43. Introna, 2000, “Shaping the Web: Why the Politics of Search Engines Matters”, The Information Society, 16, 169, 10.1080/01972240050133634 · DOI
  44. Joachims, 2007, “Evaluating the Accuracy of Implicit Feedback from Clicks and Query Reformulations in Web Search”, ACM Transactions on Information Systems, 25, 7, 10.1145/1229179.1229181 · DOI · Paper page
  45. Juneja, 2021, Auditing“ E-Commerce Platforms for Algorithmically Curated Vaccine Misinformation”, Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI '21), 10.1145/3411764.3445250 · DOI
  46. Kawakami, 2020, “The Media Coverage of the 2020 US Presidential Election Candidates through the Lens of Google’s Top Stories”, Proceedings of the International AAAI Conference on Web and Social Media, 14, 868, 10.1609/icwsm.v14i1.7352 · DOI
  47. Kay, 2015, “Unequal Representation and Gender Stereotypes in Image Search Results for Occupations”, Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15), 3819, 10.1145/2702123.2702520 · DOI · Paper page
  48. Keyes, 2019, “A Mulching Proposal: Analysing and Improving an Algorithmic System for Turning the Elderly into High-Nutrient Slurry”, Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (CHI EA '19), 10.1145/3290607.3310433 · DOI
  49. Kliman-Silver, 2015, “Location, Location, Location: The Impact of Geolocation on Web Search Personalization”, Proceedings of the 2015 ACM Internet Measurement Conference (IMC '15), 121, 10.1145/2815675.2815714 · DOI
  50. Kugelmass, 2016, ““Sorry, I'm Not Accepting New Patients”: An Audit Study of Access to Mental Health Care”, Journal of Health and Social Behavior, 57, 168, 10.1177/0022146516647098 · DOI
  51. Kulshrestha, 2019, Search Bias Quantification: Investigating Political Bias in Social Media and Web Search, Information Retrieval Journal, 22, 188, 10.1007/s10791-018-9341-2 · DOI
  52. Lawrence, 1999, “Accessibility of Information on the Web”, Nature, 400, 107, 10.1038/21987 · DOI · Paper page
  53. Lazer, 2014, “The Parable of Google Flu: Traps in Big Data Analysis”, Science, 343, 1203, 10.1126/science.1248506 · DOI · Paper page
  54. Le, 2019, “Measuring Political Personalization of Google News Search”, The World Wide Web Conference (WWW ’19), 2957, 10.1145/3308558.3313682 · DOI
  55. Lurie, 2018, “ Investigating the Effects of Google’s Search Engine Result Page in Evaluating the Credibility of Online News Sources”, Proceedings of the 10th ACM Conference on Web Science (WebSci '18), 107, 10.1145/3201064.3201095 · DOI
  56. McCown, 2007, –Agreeing to Disagree: Search Engines and Their Public Interfaces–, Proceedings of the 2007 Conference on Digital Libraries (JCDL'07), 309, 10.1145/1255175.1255237 · DOI
  57. McMahon, 2017, –The Substantial Interdependence of Wikipedia and Google: A Case Study on the Relationship between Peer Production Communities and Information Technologies–, Proceedings of the Eleventh International AAAI Conference on Web and Social Media (ICWSM 2018), 10
  58. Mehrotra, 2017, –Auditing Search Engines for Differential Satisfaction Across Demographics–, Proceedings of the 26th International Conference on World Wide Web Companion (WWW '17 Companion), 626, 10.1145/3041021.3054197 · DOI
  59. Metaxa, 2019, –Search Media and Elections: A Longitudinal Investigation of Political Search Results in the 2018 U.S. Elections–, Proceedings of the 22nd ACM Conference on Computer-Supported Cooperative Work and Social Computing
  60. Metaxas, 2009, –The Battle for the 2008 US Congressional Elections on the Web–, Proceedings of the 2009 WebScience: Society On-Line Conference
  61. Metaxas, 2017, –Manipulation of Search Engine Results during the 2016 US Congressional Elections–, Proceedings of the ICIW 2017, 6
  62. Mikians, 2012, –Detecting price and search discrimination on the internet–, Proceedings of the 11th ACM Workshop on Hot Topics in Networks, 79
  63. Milan, 2016, "The alternative epistemologies of data activism", Digital Culture & Society, 2, 57, 10.14361/dcs-2016-0205 · DOI · Paper page
  64. Mowshowitz, 2002, "Assessing Bias in Search Engines, Information Processing & Management", 38, 141, 10.1016/S0306-4573(01)00020-6 · DOI
  65. Mowshowitz, 2002, "Bias on the Web", Communications of the ACM, 45, 10.1145/567498.567527 · DOI
  66. Mullainathan S. Noeth M. Schoar A. 2012 "The market for financial advice: An audit study" National Bureau of Economic Research Tech. Report · DOI · Paper page
  67. Mustafaraj, 2020, "The Case for Voter-Centered Audits of Search Engines During Political Elections", Conference on Fairness, Accountability, and Transparency (FAT* ’20), 11
  68. Noble, 2013, "Google search: Hyper-visibility as a means of rendering black women and girls invisible", InVisible Culture
  69. Noble, 2018, "Google Has a Striking History of Bias Against Black Girls", Time Magazine
  70. Obermeyer, 2019, "Dissecting Racial Bias in an Algorithm Used to Manage the Health of Populations", Science, 366, 447, 10.1126/science.aax2342 · DOI · Paper page
  71. Oreopoulos, 2011, "Why do skilled immigrants struggle in the labor market? A field experiment with thirteen thousand resumes", American Economic Journal: Economic Policy, 3, 148
  72. Pan, 2007, "In Google We Trust: Users’ Decisions on Rank, Position, and Relevance", Journal of Computer-Mediated Communication, 12, 801, 10.1111/j.1083-6101.2007.00351.x · DOI · Paper page
  73. Pitoura, 2018, "On Measuring Bias in Online Information", ACM SIGMOD Record, 46, 16, 10.1145/3186549.3186553 · DOI
  74. Prosperi, 2019, "Is it time to rethink institutional review boards for the era of big data?", Nature Machine Intelligence, 1, 260, 10.1038/s42256-019-0059-7 · doi:10.1038/s42256-019-0059-7
  75. Robertson, 2018, "Auditing Partisan Audience Bias within Google Search", Proceedings of the ACM on Human-Computer Interaction, 2, 1, 10.1145/3274417 · DOI · Paper page
  76. Robertson, 2019, "Auditing Autocomplete: Suggestion Networks and Recursive Algorithm Interrogation", Proceedings of the 10th ACM Conference on Web Science (WebSci '19), 235, 10.1145/3292522.3326047 · DOI
  77. Robertson, 2018, "Auditing the Personalization and Composition of Politically-Related Search Engine Results Pages", 955, 10.1145/3178876.3186143 · DOI
  78. Rothshild, 2019, "How the Interplay of Google and Wikipedia Affects Perceptions of Online News Sources"
  79. Sapiezynski, 2019, "Quantifying the Impact of User Attention on Fair Group Representation in Ranked Lists", 553, 10.1145/3308560.3317595 · DOI · Paper page
  80. Sharma, 2015, "Insurance, race/ethnicity, and sex in the search for a new physician", Economics Letters, 137, 150, 10.1016/j.econlet.2015.11.005 · DOI
  81. Speicher, 2018, "Potential for discrimination in online targeted advertising", Conference on Fairness, Accountability, and Transparency (FAT '20)
  82. Sweeney L. 2013 "Discrimination in Online Ad Delivery" https://dataprivacylab.org/projects/onlineads/index.html Accessed 10/26/2020 · DOI
  83. "Google’s Algorithm – Seek and You Shall Find", 2019, The Economist, 81
  84. Trielli, 2019, "Search as News Curator: The Role of Google in Shaping Attention to News Information", Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19), 1, 10.1145/3290605.3300683 · DOI
  85. Trielli, 2020, "Partisan Search Behavior and Google Results in the 2018 U.S. Midterm Elections", Information, Communication & Society, 1, 10.1080/1369118X.2020.1764605 · DOI
  86. Vincent, 2021, "A Deeper Investigation of the Importance of Wikipedia Links to Search Engine Results", Proceedings of the ACM on Human-Computer Interaction, 5, 10.1145/3449078 · DOI
  87. Vincent, 2019, "Measuring the Importance of User-Generated Content to Search Engines", Proceedings of the International AAAI Conference on Web and Social Media, 13, 10.1609/icwsm.v13i01.3248 · DOI
  88. Xing, 2014, "Exposing Inconsistent Web Search Results with Bobble", Passive and Active Measurement, 131, 10.1007/978-3-319-04918-2_13 · DOI
  89. Yinger, 1998, "Evidence on discrimination in consumer markets", Journal of Economic Perspectives, 12, 23, 10.1257/jep.12.2.23 · DOI · Paper page

Cited by

Metadata

Authors
Metaxa, D., Park, J. S., Robertson, R. E., Karahalios, K., Wilson, C., Hancock, J., & Sandvig, C.
Affiliations
  • Stanford University
  • Northeastern University
  • University of Illinois at Urbana-Champaign
  • University of Michigan
Venue
Foundations and Trends® in Human–Computer Interaction
Publisher
Emerald
Published
2021-11-25
Volume
14
Issue
4
Pages
272-344
ISSN
  • 1551-3955
  • 1551-3963
Cited by
140
References
15
Language
en

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.