Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction
Open accessThe DOI matches its Crossref record; 204 of the 207 references it cites were found in a registry or an index, and 3 are not checked yet.
Abstract
Understanding the landscape of potential harms from algorithmic systems enables practitioners to better anticipate consequences of the systems they build. It also supports the prospect of incorporating controls to help minimize harms that emerge from the interplay of technologies and social and cultural dynamics. A growing body of scholarship has identified a wide range of harms across different algorithmic technologies. However, computing research and practitioners lack a high level and synthesized overview of harms from algorithmic systems. Based on a scoping review of computing research (n=172), we present an applied taxonomy of sociotechnical harms to support a more systematic surfacing of potential harms in algorithmic systems. The final taxonomy builds on and refers to existing taxonomies, classifications, and terminologies. Five major themes related to sociotechnical harms — representational, allocative, quality-of-service, interpersonal harms, and social system/societal harms — and sub-themes are presented along with a description of these categories. We conclude with a discussion of challenges and opportunities for future research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“However, the literature on harms from LLMs and algorithmic systems more broadly has shown that many of the harms arising from such models emerge from their embedding in wider sociotechnical systems (Weidinger et al., 2021; Bender et al., 2021; Shelby et al., 2023).”
From Helpful, harmless, honest? Sociotechnical limits of AI alignment and safety through Reinforcement Learning from Human Feedback (2025) · cites this paper as Shelby et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“The rapid pace of adoption of GenAI tools, when it comes to similar concerns, have significantly exacerbated the many socio-technical issues researchers have connected to education technologies in general [160, 311] and algorithmic harms related to GenAI [317] more specifically.”
From Navigating the Ethical and Societal Impacts of Generative AI in Higher Computing Education (2025) · cites this paper as [317]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Recent work has expressed concern around anthropomorphism leading to over-trust [1, 91], a potential path for uncertainty expression to backfire.”
From "I'm Not Sure, But...": Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trust (2024) · cites this paper as [91]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation
Tricco, A. C., Lillie, E., Zarin, W., O'Brien, K. K., Colquhoun, H., Levac, D., Moher, D., Peters, M. D. J., Horsley, T., Weeks, L., et al. (2018) · Annals of Internal Medicine · doi:10.7326/m18-0850 · Cited by 43,565
Scoping studies: towards a methodological framework
Arksey, H., & O'Malley, L. (2005) · International Journal of Social Research Methodology · doi:10.1080/1364557032000119616 · Cited by 36,742
Black Feminist Thought: Knowledge, Consciousness, and the Politics of Empowerment.
Brewer, R. M., & Collins, P. H. (1992) · Contemporary Sociology · doi:10.2307/2074808 · Cited by 19,445
Situated Knowledges: The Science Question in Feminism and the Privilege of Partial Perspective
Haraway, D. (1988) · Feminist Studies · doi:10.2307/3178066 · Cited by 18,364
Scoping studies: advancing the methodology
Levac, D., Colquhoun, H., & O'Brien, K. K. (2010) · Implementation Science · doi:10.1186/1748-5908-5-69 · Cited by 15,268
Saturation in qualitative research: exploring its conceptualization and operationalization
Saunders, B., Sim, J., Kingstone, T., Baker, S., Waterfield, J., Bartlam, B., Burroughs, H., & Jinks, C. (2018) · Quality & Quantity · doi:10.1007/s11135-017-0574-8 · Cited by 12,033
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00136 · Cited by 10,349
Guidance for conducting systematic scoping reviews
Peters, M. D. J., Godfrey, C. M., Khalil, H., McInerney, P., Parker, D., & Soares, C. B. (2015) · International Journal of Evidence-Based Healthcare · doi:10.1097/xeb.0000000000000050 · Cited by 7,472
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137 · cited 2×
“systems make incorrect health inferences [158].”
Background
Research works printed but not linked: 42
- 10.1093/cybsec · doi:10.1093/cybsec
- 10.18653/v1 · doi:10.18653/v1
- Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2016. Machine Bias. In Ethics of Data and Analytics. Auerbach Publications, Boca Raton, FL, USA, 254–264.
- Jane Bailey, Jacquelyn Burkell, Suzie Dunn, Chandell Gosse, and Valerie Steeves. 2021. AI and Technology-Facilitated Violence and Abuse. In Artificial Intelligence and the Law in Canada, Florian Martin-Bariteau and Teresa Scassa (Eds.). LexisNexis, Toronto, Canada, Chapter 10, 1–15.
- Jane Bailey, Nicola Henry, and Asher Flynn. 2021. Technology-Facilitated Violence and Abuse: International Perspectives and Experiences. In The Emerald International Handbook of Technology-Facilitated Violence and Abuse, Bailey Jane, Flynn Asher, and Henry Nicola (Eds.). Emerald Publishing Limited, Bingley, UK, 1–17.
- 10.18653/v1 · doi:10.18653/v1
- Solon Barocas, Kate Crawford, Aaron Shapiro, and Hanna Wallach. 2017. The Problem with Bias: Allocative Versus Representational Harms in Machine Learning. In 9th Annual Conference of the Special Interest Group for Computing, Information and Society. Society for the History of Technology, Philadelphia, PA, USA.
- Samy Bengio Alina Beygelzimer Kate Crawford Jeanne Fromer Iason Gabriel Amanda Levendowski Deborah Raji and Marc’Aurelio Ranzato. 2022. Provisional Draft of the NeurIPS Code of Ethics. https://openreview.net/pdf?id=zVoy8kAFKPr
- 10.1177/205630511667248 · doi:10.1177/205630511667248
- 10.18653/v1 · doi:10.18653/v1
- 10.5555/380681 · doi:10.5555/380681
- Virginia Braun and Victoria Clarke. 2021. One Size Fits All? What Counts as Quality Practice in (Reflexive) Thematic Analysis?Qualitative research in psychology 18, 3 (2021), 328–352.
- Joy Buolamwini and Timnit Gebru. 2018. Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency(Proceedings of Machine Learning Research, Vol. 81), Sorelle A. Friedler and Christo Wilson (Eds.). PMLR, New York, NY, USA, 77–91. https://proceedings.mlr.press/v81/buolamwini18a.html
- Jacquelyn Burkell and Jane Bailey. 2018. Unlawful Distinctions? Canadian Human Rights Law and Algorithmic Bias. Vol. 2016/2018. Human Rights Research and Education Centre Ottawa, Ottawa, Canada, 217–230.
- Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017. Semantics Derived Automatically from Language Corpora Contain Human-like Biases. Science 356, 6334 (2017), 183–186.
- Frances Corry, Hamsini Sridharan, Alexandra Sasha Luccioni, Mike Ananny, Jason Schultz, and Kate Crawford. 2021. The Problem of Zombie Datasets: A Framework for Deprecating Datasets. arXiv preprint arXiv:2111.04424 abs/2111.04424 (2021), 1–18. · arXiv
- Kate Crawford. 2017. Keynote: The Trouble with Bias. 2017 Conference on Neural Information Processing Systems, invited speaker.
- Catherine D’Ignazio and Lauren F Klein. 2020. Data Feminism. MIT Press, Boston, MA, USA.
- Batya Friedman and David G Hendry. 2019. Value Sensitive Design: Shaping Technology with Moral Imagination. MIT Press, Boston, MA, USA.
- Steve Fuller. 1993. Helen E. Longino Science As Social Knowledge: Values and Objectivity in Scientific Inquiry. Princeton: Princeton University Press (1990), xi+ 262 pp., 19.95. Philosophy of Science 60, 2 (1993), 360–362.
- Jan Hadja. 1961. Alienation and Integration of Student Intellectualss. American Sociological Review 26, 5 (1961), 19.
- 10.1177/104973231666534 · doi:10.1177/104973231666534
- 10.1177/146144482095872 · doi:10.1177/146144482095872
- Shawn Janzen, Caroline Orr, and Sara-Jayne Terp. 2022. Cognitive Security and Resilience: A Social Ecological Model of Disinformation and Other Harms with Applications to COVID-19 Vaccine Information Behaviors. In 2nd Workshop Reducing Online Misinformation through Credible Information Retrieval. ROMCIR 2022, Stavanger, Norway, 48–88.
- Jared Katzman, Solon Barocas, Su Lin Blodgett, Kristen Laird, Morgan Klaus Scheuerman, and Hanna Wallach. 2021. Representational Harms in Image Tagging. Beyond Fair Computer Vision Workshop at CVPR 2021 (2021).
- Hank Liao, Golan Pundak, Olivier Siohan, Melissa Carroll, Noah Coccaro, Qi-Ming Jiang, Tara N Sainath, Andrew Senior, Françoise Beaufays, and Michiel Bacchiani. 2015. Large Vocabulary Automatic Speech Recognition for Children. In Interspeech 2015. International Speech Communication Association, Dresden, Germany, 1–5.
- 10.1093/ojls · doi:10.1093/ojls
- Emanuel Moss and Jacob Metcalf. 2020. Ethics Owners: A New Model of Organizational Responsibility in Data-Driven Technology Companies. https://datasociety.net/wp-content/uploads/2020/09/Ethics-Owners_20200923-DataSociety.pdf
- Emmanuel Moss and Jacob Metcalf. 2022. Data and Society Workshop: The Social Life of Algorithmic Harms. (March 2022).
- Aik Kwang Ng, David YF Ho, Shyh Shin Wong, and Ian Smith. 2003. In Search of the Good Life: A Cultural Odyssey in the East and West. Genetic, Social, and General Psychology Monographs 129, 4 (2003), 317.
- Safiya Umoja Noble. 2016. A Future for Intersectional Black Feminist Technology Studies. Scholar & Feminist Online 13, 3 (2016), 1–8.
- Cathy O’Neil. 2016. Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. Broadway Books, New York, NY, USA.
- Eva PenzeyMoog and Danielle C Slakoff. 2021. As Technology Evolves, So Does Domestic Violence: Modern-Day Tech Abuse and Possible Solutions. In The Emerald International Handbook of Technology-Facilitated Violence and Abuse. Emerald Publishing Limited, Bingley, UK, 643–662.
- Jasbir K Puar. 2020. “I Would Rather be a Cyborg than a Boddess”: Becoming Intersectional in Assemblage Theory. In Feminist Theory Reader. Routledge, New York, NY, USA, 405–415.
- Henrik Skaug Sætra. 2021. Artificial Intelligence and its Contexts. Springer, Cham, Switzerland, Chapter A Typology of AI Applications in Politics, 27–43.
- Katie Shilton. 2015. "That’s Not An Architecture Problem!": Techniques and Challenges for Practicing Anticipatory Technology Ethics. In iConference 2015 Proceedings. iSchools Urbana-Champaign Illinois USA 1–7.
- Rebecca Kelly Slaughter, Janice Kopec, and Mohamad Batal. 2020. Algorithms and Economic Justice: A Taxonomy of Harms and a Path Forward for the Federal Trade Commission. Yale Journal of Law & Tech. 23 (2020), 1.
- 10.1177/0002716222108470 · doi:10.1177/0002716222108470
- 10.18653/v1 · doi:10.18653/v1
- Thi Tran, Rohit Valecha, Paul Rad, and H Raghav Rao. 2019. An Investigation of Misinformation Harms Related to Social Media During Humanitarian Crises. In International Conference on Secure Knowledge Management in Artificial Intelligence Era. Springer, Goa, India, 167–181.
- Zeynep Tufekci. 2015. Algorithmic Harms Beyond Facebook and Google: Emergent Challenges of Computational Agency. Colo. Tech. LJ 13 (2015), 203.
- Claire Wardle and Hossein Derakhshan. 2017. Information Disorder: Toward an Interdisciplinary Framework for Research and Policymaking.
Cited by
Scaling Instruction-Finetuned Language Models
Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2210.11416 · Cited by 1,441
Gemini: A Family of Highly Capable Multimodal Models
Gemini Team, Anil, R., Borgeaud, S., Alayrac, J.-B., Yu, J., Soricut, R., Schalkwyk, J., Dai, A. M., Hauth, A., Millican, K., et al. (2023) · arXiv · doi:10.48550/arxiv.2312.11805 · Cited by 990
Anil, R., Dai, A. M., Firat, O., Johnson, M., Lepikhin, D., Passos, A., Shakeri, S., Taropa, E., Bailey, P., Chen, Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.10403 · Cited by 834
Foundational Challenges in Assuring Alignment and Safety of Large Language Models
Anwar, U., Saparov, A., Ramírez, J. R., Paleka, D., Turpin, M., Hase, P., Lubana, E. S., Jenner, E., Casper, S., Sourbut, O., et al. (2024) · arXiv (Cornell University) · doi:10.3929/ethz-b-000706962 · Cited by 17
Evaluating the Social Impact of Generative AI Systems in Systems and Society
Solaiman, I., Talat, Z., Agnew, W. S., Ahmad, L., Baker, D., Blodgett, S. L., Chen, C., Daumé, H., Dodge, J., Duan, I., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.05949 · Cited by 42
Metadata
- Authors
- Shelby, R., Rismani, S., Henne, K., Moon, A., Rostamzadeh, N., Nicholas, P., Yilla-Akbari, N., Gallegos, J., Smart, A., Garcia, E., & Virk, G.
- Affiliations
- Google (United States) (US)
- McGill University (CA)
- Australian National University (AU)
- Google, USA
- ORCID
- Venue
- Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society
- Publisher
- ACM
- Published
- 2023-08-08
- Pages
- 723-741
- ISSN
- 3065-8365 (ISSN-L)
- OpenAlex
- W4386249234
- Cited by
- 279
- References
- 191
- Citations by year
- 2026: 97 · 2025: 101 · 2024: 65 · 2023: 15
- FWCI
- 69.56
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Proceedings of the AAAI/ACM Conference on AI Ethics and Society (publishedVersion)
- HAL (submittedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Law › Artificial Intelligence in Law
- Keywords
- algorithmic harms, harm reduction, algorithmic systems, representational harm, interpersonal harm, social harm, sociocultural dynamics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction. https://app.cochat.ai/papers/10.1145/3600211.3604673
- The paper
- Shelby, R., Rismani, S., Henne, K., Moon, A., Rostamzadeh, N., Nicholas, P., Yilla-Akbari, N., Gallegos, J., Smart, A., Garcia, E., & Virk, G. (2023). Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction. Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society. https://doi.org/10.1145/3600211.3604673
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.