The ethnographer and the algorithm: beyond the black box
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Modern AI tools, such as deep-learning algorithms, often appear as 'black boxes' to users because it may be very difficult or even impossible to examine how the algorithm arrived at a particular output.
“Modern AI tools, such as deep-learning algorithms, often appear as 'black boxes' to users since it may be very difficult or even impossible to examine how the algorithm arrived at a particular output (Christin 2020, Diakopoulos 2020, Pasquale 2015).”
From To Engage or Not to Engage with AI for Critical Judgments: How Professionals Deal with Opacity When Using AI for Medical Diagnosis (2022) · cites this paper as Christin 2020
Cites
The Discovery of Grounded Theory; Strategies for Qualitative Research
Glaser, B. G., Strauss, A. L., & Strutzel, E. (1968) · Nursing Research · doi:10.1097/00006199-196807000-00014 · Cited by 39,449 · see citing works
Orlikowski, W. J. (2000) · Organization Science · doi:10.1287/orsc.11.4.404.14600 · Cited by 4,760 · see citing works
How the machine “thinks”: Understanding opacity in machine learning algorithms
Burrell, J. (2016) · Big Data & Society · doi:10.1177/2053951715622512 · Cited by 3,196 · cited 1× · see citing works
“In Burrell ’ s words, “ When a computer learns and consequently builds its own representation of a classification decision, it does so without regard for human comprehension ” (Burrell 2016, p. 10).”
BackgroundBarocas, S., & Selbst, A. D. (2014) · doi:10.2139/ssrn.2477899 · Cited by 3,048 · see citing works
Burawoy, M. (1998) · Sociological Theory · doi:10.1111/0735-2751.00040 · Cited by 2,916 · see citing works
Barley, S. R. (1986) · Administrative Science Quarterly · doi:10.2307/2392767 · Cited by 2,315 · see citing works
Haggerty, K. D., & Ericson, R. V. (2000) · The British Journal of Sociology · doi:10.1080/00071310020015280 · Cited by 2,244 · see citing works
Ananny, M., & Crawford, K. (2018) · New Media & Society · doi:10.1177/1461444816676645 · Cited by 1,836 · see citing works
Terranova, T. (2000) · Social Text · doi:10.1215/01642472-18-2_63-33 · Cited by 1,749 · cited 1× · see citing works
“Existing work on data extraction examines the cultural and political institutionalization of the current economic model, in which online users share their data and labor for free with platforms that make enormous profits based on this behavioral surplus (Terranova 2000; Scholz 2013).”
Background(Not) Getting Paid to Do What You Love: Gender, Social Media, and Aspirational Work
Duffy, B. E. (2017) · Yale University Press · doi:10.12987/yale/9780300218176.001.0001 · Cited by 931 · cited 2× · see citing works
“…example, one can take the case of content creators and “ influencers ” producing and sharing online content such as videos, pictures, and blog posts on YouTube, Instagram, Twitter, TikTok, and so on (Marwick 2013; Duffy and Hund 2015; Duffy 2017; Bishop 2019; Stuart 2020; Burgess and Green 2018).”
BackgroundDiscrimination in online ad delivery
Sweeney, L. (2013) · Communications of the ACM · doi:10.1145/2447976.2447990 · Cited by 783 · see citing works
Managing visibility on YouTube through algorithmic gossip
Bishop, S. (2019) · New Media & Society · doi:10.1177/1461444819854731 · Cited by 535 · cited 2× · see citing works
“…example, one can take the case of content creators and “ influencers ” producing and sharing online content such as videos, pictures, and blog posts on YouTube, Instagram, Twitter, TikTok, and so on (Marwick 2013; Duffy and Hund 2015; Duffy 2017; Bishop 2019; Stuart 2020; Burgess and Green 2018).”
BackgroundAlgorithms in practice: Comparing web journalism and criminal justice
Christin, A. (2017) · Big Data & Society · doi:10.1177/2053951717718855 · Cited by 379 · see citing works
Self-citationAlgorithmic governance: Developing a research agenda through the power of collective intelligence
Danaher, J., Hogan, M. J., Noone, C., Kennedy, R., Behan, A., De Paor, A., Felzmann, H., Haklay, M., Khoo, S.-M., Morison, J., et al. (2017) · Big Data & Society · doi:10.1177/2053951717726554 · Cited by 369 · see citing works
Measuring price discrimination and steering on e-commerce web sites
Hannak, A., Soeller, G., Lazer, D., Mislove, A., & Wilson, C. (2014) · Proceedings of the 2014 Conference on Internet Measurement Conference · doi:10.1145/2663716.2663744 · Cited by 368 · cited 1× · see citing works
“Existing instances of research using these methods have found important discriminatory features in algorithmically mediated platforms, including racial discrimination in online advertising delivery (Sweeney 2013) and price discrimination on e-commerce websites (Hannak et al. 2014; Diakopoulos 2013).”
BackgroundBehind the Screen: Content Moderation in the Shadows of Social Media
ROBERTS, S. T. (2019) · Yale University Press · doi:10.2307/j.ctvhrcz0v · Cited by 339 · cited 1× · see citing works
“The human workers performing algorithmic piecework for technology companies — data annotators, content moderators, and other “ ghost workers ”— in turn experience precarious employment statuses and grueling work conditions (Robert 2019; Gray and Suri 2019).”
BackgroundAlgorithms, Governance, and Governmentality: On Governing Academic Writing
Introna, L. D. (2016) · Science, Technology, & Human Values · doi:10.1177/0162243915587360 · Cited by 339 · see citing works
The Field Site as a Network: A Strategy for Locating Ethnographic Research
Burrell, J. (2009) · Field Methods · doi:10.1177/1525822x08329699 · Cited by 334 · see citing works
Moral crumple zones: cautionary tales in human–robot interaction
Elish, M. C. (2019) · Engaging Science, Technology, and Society · doi:10.17351/ests2019.260 · Cited by 321 · cited 1× · see citing works
“Yet even in these cases, ethnographers find people and institutions creeping through the cracks of automated systems, making essential choices in the design, maintenance, and “ repair ” of the algorithms (Elish 2019; Elish and Watkins 2020).”
BackgroundTechnologies of Crime Prediction: The Reception of Algorithms in Policing and Criminal Courts
Brayne, S., & Christin, A. (2021) · Social Problems · doi:10.1093/socpro/spaa004 · Cited by 279 · see citing works
Self-citationWorking Algorithms: Software Automation and the Future of Work
Shestakofsky, B. (2017) · Work and Occupations · doi:10.1177/0730888417726119 · Cited by 186 · see citing works
Through a Glass, Darkly: Artificial Intelligence and the Problem of Opacity
Chesterman, S. (2021) · The American Journal of Comparative Law · doi:10.1093/ajcl/avab012 · Cited by 63 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Lebovitz, S., Lifshitz-Assaf, H., & Levina, N. (2022) · Organization Science · doi:10.1287/orsc.2021.1549 · Cited by 632 · see citing works
Lebovitz, S., Levina, N., & Lifshitz-Assaf, H. (2021) · MIS Quarterly · doi:10.25300/misq/2021/16564 · Cited by 49 · see citing works
Anthony, C. (2021) · Administrative Science Quarterly · doi:10.1177/00018392211016755 · Cited by 191 · see citing works
Waardenburg, L., Huysman, M., & Sergeeva, A. V. (2022) · Organization Science · doi:10.1287/orsc.2021.1544 · Cited by 165 · see citing works
Auditing of AI: Legal, Ethical and Technical Approaches
Mökander, J. (2023) · Digital Society · doi:10.1007/s44206-023-00074-y · Cited by 152 · see citing works
Yang, J., Luo, B., Zhao, C., & Zhang, H. (2022) · DIGITAL HEALTH · doi:10.1177/20552076221126034 · Cited by 81 · see citing works
Street-Level Algorithms and AI in Bureaucratic Decision-Making
Ammitzbøll Flügge, A., Hildebrandt, T., & Møller, N. H. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449114 · Cited by 78 · see citing works
The Drama of Metrics: Status, Spectacle, and Resistance Among YouTube Drama Creators
Christin, A., & Lewis, R. (2021) · Social Media + Society · doi:10.1177/2056305121999660 · Cited by 63 · see citing works
Self-citationRamaul, L., Ritala, P., Kostis, A., & Aaltonen, P. (2026) · Journal of Management Studies · doi:10.1111/joms.13246 · Cited by 41 · see citing works
- Authors
- Christin, A.
- Affiliations
- Stanford University (US)
- Corresponding author
- Christin, Angèle
- ORCID
- Christin, Angèle
- Venue
- Theory and Society
- Published
- 2020-08-15
- Volume
- 49
- Issue
- 5-6
- Pages
- 897-918
- ISSN
- 0304-2421 (ISSN-L)
- 1573-7853
- OpenAlex
- W3049046209
- Cited by
- 327
- References
- 22
- Citations by year
- 2026: 40 · 2025: 79 · 2024: 86 · 2023: 50 · 2022: 38 · 2021: 32 · 2020: 2
- FWCI
- 41.57
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Theory and Society (publishedVersion)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Social Sciences › Sociology and Political Science › Qualitative Research Methods and Ethics
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Cybernetics and Technology in Society
- Keywords
- algorithmic opacity, black box, qualitative data collection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
