The Algorithmic Imprint
Open accessThe DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
Abstract
When algorithmic harms emerge, a reasonable response is to stop using the algorithm to resolve concerns related to fairness, accountability, transparency, and ethics (FATE). However, just because an algorithm is removed does not imply its FATE-related issues cease to exist. In this paper, we introduce the notion of the “algorithmic imprint” to illustrate how merely removing an algorithm does not necessarily undo or mitigate its consequences. We operationalize this concept and its implications through the 2020 events surrounding the algorithmic grading of the General Certificate of Education (GCE) Advanced (A) Level exams, an internationally recognized UK-based high school diploma exam administered in over 160 countries. While the algorithmic standardization was ultimately removed due to global protests, we show how the removal failed to undo the algorithmic imprint on the sociotechnical infrastructures that shape students’, teachers’, and parents’ lives. These events provide a rare chance to analyze the state of the world both with and without algorithmic mediation. We situate our case study in Bangladesh to illustrate how algorithms made in the Global North disproportionately impact stakeholders in the Global South. Chronicling more than a year-long community engagement consisting of 47 interviews, we present the first coherent timeline of “what” happened in Bangladesh, contextualizing “why” and “how” they happened through the lenses of the algorithmic imprint and situated algorithmic fairness. Analyzing these events, we highlight how the contours of the algorithmic imprints can be inferred at the infrastructural, social, and individual levels. We share conceptual and practical implications around how imprint-awareness can (a) broaden the boundaries of how we think about algorithmic impact, (b) inform how we design algorithms, and (c) guide us in AI governance. The imprint-aware design mindset can make the algorithmic development process more human-centered and sociotechnically-informed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
Situated Knowledges: The Science Question in Feminism and the Privilege of Partial Perspective
Haraway, D. (1988) · Feminist Studies · doi:10.2307/3178066 · Cited by 18,364
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137 · cited 1×
“Exemplars of the technical perspective include developing technical documentation and organizational accountability practices [39, 73, 86], audits of the outputs of algorithmic systems [17, 80], demonstrations of the perils of large datasets [11, 48, 56], and formalized models of algorithmic fairness [10, 77].”
BackgroundOn the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
The Ethnography of Infrastructure
STAR, S. L. (1999) · American Behavioral Scientist · doi:10.1177/00027649921955326 · Cited by 4,486
What is AI Literacy? Competencies and Design Considerations
Long, D., & Magerko, B. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376727 · Cited by 3,241 · cited 1×
“Organizationally, imprint-awareness can be incorporated into existing AI ethics literacy programs [66] similar to recent work demonstrating that literacy of dark patterns can promote self-reflection and mitigate harms [68].”
BackgroundSteps toward an ecology of infrastructure: Design and access for large information spaces
Star, S. L., & Ruhleder, K. (1996) · Information Systems Research · doi:10.1287/isre.7.1.111 · Cited by 3,238 · cited 3×
“Weaving these threads together, we chart the otherwise invisible underlying infrastructural conditions and practices that make algorithmic deployments possible and contribute to their potentially harmful impacts [103, 104].”
BackgroundModel Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé, H., & Crawford, K. (2018) · arXiv · doi:10.48550/arxiv.1803.09010 · Cited by 2,474
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., III, H. D., & Crawford, K. (2021) · Communications of the ACM · doi:10.1145/3458723 · Cited by 2,469 · cited 1×
“Exemplars of the technical perspective include developing technical documentation and organizational accountability practices [39, 73, 86], audits of the outputs of algorithmic systems [17, 80], demonstrations of the perils of large datasets [11, 48, 56], and formalized models of algorithmic fairness [10, 77].”
BackgroundRaji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372873 · Cited by 2,245
People as infrastructure: Intersecting fragments in johannesburg
Simone, A. (2004) · Public Culture · doi:10.1215/08992363-16-3-407 · Cited by 2,234 · cited 1×
“Below we share three thematic aspects of imprint-aware participants’ critical reflections: (1) the frequent invisibility of people as infrastructure [60, 98] and how the imprint can be a resource to account for invisible labor ; (2) struggles with a lack of agency and voice and how the imprint can act as an anchor for…”
BackgroundLeonardi, P. M. (2011) · MIS Quarterly · doi:10.2307/23043493 · Cited by 1,889 · cited 1×
“Grounded in the insight that infrastructures work for some at the expense of others [27], the infrastructural lens focuses either on the unevenness of the imbrication [24, 63, 83, 100] or on such breakdowns and harmful consequences [33, 41, 59, 104].”
BackgroundA Pragmatic View of Thematic Analysis
Aronson, J. (1995) · The Qualitative Report · doi:10.46743/2160-3715/1995.2069 · Cited by 1,758
Fairness and Abstraction in Sociotechnical Systems
Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., & Vertesi, J. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287598 · Cited by 1,631 · cited 1×
“Indeed, strictly abstract and technical accounts of algorithmic un/fairness often fail to capture the full range of potential algorithmic harms and may provide cover for deeper issues of injustice [44, 74, 77, 90, 92].”
BackgroundData colonialism: rethinking big data’s relation to the contemporary subject
Couldry, N., & Mejias, U. A. (2019) · Television & New Media · doi:10.1177/1527476418796632 · Cited by 1,617
Lee, M. K. (2018) · Big Data & Society · doi:10.1177/2053951718756684 · Cited by 1,338 · cited 1×
“Research focusing on their perceptions often relies on building speculative cases for appropriation of algorithmic systems to encourage data subjects to articulate their understanding of fairness and thereby create boundaries around what an algorithmic system should or should not do [12, 43, 61, 109, 110].”
BackgroundSengers, P., Boehner, K., David, S., & Kaye, J. '. (2005) · Proceedings of the 4th decennial conference on Critical computing: between sense and sensibility · doi:10.1145/1094562.1094569 · Cited by 1,030
Algorithms as culture: Some tactics for the ethnography of algorithmic systems
Seaver, N. (2017) · Big Data & Society · doi:10.1177/2053951717738104 · Cited by 1,010
Code/Space: Software and Everyday Life
Kitchin, R., & Dodge, M. (2011) · doi:10.7551/mitpress/9780262042482.001.0001 · Cited by 964
Questioning the AI: Informing Design Practices for Explainable AI User Experiences
Liao, Q. V., Gruen, D., & Miller, S. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376590 · Cited by 925
Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems
(2024) · doi:10.1145/3613904 · Cited by 896
Mitigating bias in algorithmic hiring: evaluating claims and practices
Raghavan, M., Barocas, S., Kleinberg, J., & Levy, K. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372828 · Cited by 845
Algorithmic injustice: A relational ethics approach
Birhane, A. (2021) · Patterns · doi:10.1016/j.patter.2021.100205 · Cited by 707 · cited 1×
“Exemplars includes accounts of organizational roles and values in AI/ML ethics [1, 5, 69], studies of how data technologies redistribute social power [35, 85, 93], how algorithmic systems can reinforce extant social and political biases [79, 90], and recapitulate a worldview with harmful and undesirable histories [7,…”
Background'It's Reducing a Human Being to a Percentage': Perceptions of Justice in Algorithmic Decisions
Binns, R., Van Kleek, M., Veale, M., Lyngs, U., Zhao, J., & Shadbolt, N. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173951 · Cited by 691
Algorithmic Fairness: Choices, Assumptions, and Definitions
Mitchell, S., Potash, E., Barocas, S., D'Amour, A., & Lum, K. (2021) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-042720-125902 · Cited by 687
Prediction-Based Decisions and Fairness: A Catalogue of Choices, Assumptions, and Definitions
Mitchell, S., Potash, E., Barocas, S., D'Amour, A., & Lum, K. (2018) · arXiv · doi:10.48550/arxiv.1811.07867 · Cited by 687
Expanding Explainability: Towards Social Transparency in AI systems
Ehsan, U., Liao, Q. V., Muller, M., Riedl, M. O., & Weisz, J. D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445188 · Cited by 538 · cited 2×
“Studies have suggested new ways for data subjects to ‘know’ algorithms through improved measures for accountability and fairness, including disclosure requirements, regulatory oversight, and explainability [28, 57, 65, 89, 97].”
BackgroundSelf-citationProgrammed Visions: Software and Memory
Chun, W. H. K. (2011) · doi:10.7551/mitpress/9780262015424.001.0001 · Cited by 475
Lai, V., & Tan, C. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287590 · Cited by 377
The human infrastructure of cyberinfrastructure
Lee, C. P., Dourish, P., & Mark, G. (2006) · Proceedings of the 2006 20th anniversary conference on Computer supported cooperative work · doi:10.1145/1180875.1180950 · Cited by 369
Jacobs, A. Z., & Wallach, H. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445901 · Cited by 362 · cited 2×
“Similarly, Jacobs & Wallach [50] argue that many contestations about algorithmic fairness are actually conflicts over the validity of inferring unobservable theoretical constructs (e.g., worker effectiveness) from observable features that are readily available in datasets.”
BackgroundHutchinson, B., Smart, A., Hanna, A., Denton, R., Greer, C., Kjartansson, O., Barnes, P., & Mitchell, M. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445918 · Cited by 279
Human-Centered Explainable AI: Towards a Reflective Sociotechnical Approach
Ehsan, U., & Riedl, M. O. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-60117-1_33 · Cited by 274
Self-citationA Qualitative Exploration of Perceptions of Algorithmic Fairness
Woodruff, A., Fox, S. E., Rousso-Schindler, S., & Warshaw, J. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174230 · Cited by 256
Automated rationale generation: a technique for explainable AI and its effects on human perceptions
Ehsan, U., Tambwekar, P., Chan, L., Harrison, B., & Riedl, M. O. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302316 · Cited by 210
Self-citationGrgic-Hlaca, N., Redmiles, E. M., Gummadi, K. P., & Weller, A. (2018) · Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW '18 · doi:10.1145/3178876.3186138 · Cited by 207
"The human body is a black box": supporting clinical decision-making with deep learning
Sendak, M., Elish, M. C., Gao, M., Futoma, J., Ratliff, W., Nichols, M., Bedoya, A., Balu, S., & O'Brien, C. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372827 · Cited by 196
Algorithmic Impact Assessments and Accountability: The Co-construction of Impacts
Metcalf, J., Moss, E., Watkins, E. A., Singh, R., & Elish, M. C. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445935 · Cited by 195
Self-citationRaji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · arXiv · doi:10.48550/arxiv.2001.00973 · Cited by 179
Shen, H., DeVos, A., Eslami, M., & Holstein, K. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479577 · Cited by 171
Algorithmic realism: expanding the boundaries of algorithmic thought
Green, B., & Viljoen, S. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372840 · Cited by 161
Toward situated interventions for algorithmic equity: lessons from the field
Katell, M., Young, M., Dailey, D., Herman, B., Guetler, V., Tam, A., Bintz, C., Raz, D., & Krafft, P. M. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372874 · Cited by 153
A Relational Theory of Data Governance
Viljoen, S. (2020) · doi:10.2139/ssrn.3727562 · Cited by 109
Grgić-Hlača, N., Redmiles, E. M., Gummadi, K. P., & Weller, A. (2018) · arXiv · doi:10.48550/arxiv.1802.09548 · Cited by 75
Seeing Like an Infrastructure: Low-resolution Citizens and the Aadhaar Identification Project
Singh, R., & Jackson, S. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3476056 · Cited by 60
Self-citation
Cited by
Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction
Shelby, R., Rismani, S., Henne, K., Moon, A., Rostamzadeh, N., Nicholas, P., Yilla-Akbari, N., Gallegos, J., Smart, A., Garcia, E., et al. (2023) · Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3600211.3604673 · Cited by 279
“For instance, privacy violations may arise from algorithmic systems making predictive inference beyond what users openly disclose [222] or when data collected and algorithmic inferences made about people in one context is applied to another without the person’s knowledge or consent through big data flows [138], even a…”
BackgroundHarms from Increasingly Agentic Algorithmic Systems
Chan, A., Salganik, R., Markelius, A., Pang, C., Rajkumar, N., Krasheninnikov, D., Langosco, L., He, Z., Duan, Y., Carroll, M., et al. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594033 · Cited by 149
“Examples include the perpetuation of existing, unjust power relations [Barabas et al. 2018; Buolamwini and Gebru 2018; Ehsan et al. 2022; Kasy and Abebe 2021; steven t. piantadosi [@spiantado] 2022; Wolfe and Caliskan 2022], the generation of toxic language [Abid et al. 2021; Gehman et al. 2020],…”
BackgroundBuilding, Shifting, & Employing Power: A Taxonomy of Responses From Below to Algorithmic Harm
DeVrio, A., Eslami, M., & Holstein, K. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3658958 · Cited by 21
Tang, X., Lima, G., Jiang, L., Simko, L., & Zou, Y. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3710935 · Cited by 17
“However, there is a notable gap in longitudinal studies that examine both the long-term impacts of deploying AI systems and the consequences of their removal [58].”
Agent-Supported Foresight for AI Systemic Risks: AI Agents for Breadth, Experts for Judgment
Fröhling, L., Giaconia, A., Bogucka, E. P., & Quercia, D. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790712 · Cited by 2
Metadata
- Authors
- Ehsan, U., Singh, R., Metcalf, J., & Riedl, M.
- Venue
- 2022 ACM Conference on Fairness Accountability and Transparency
- Publisher
- ACM
- Published
- 2022-06-20
- Pages
- 1305-1317
- OpenAlex
- W4281643290
- Cited by
- 40
- References
- 47
- FWCI
- 7.45
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.