CoChat

Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction

Open access
Shelby, Renee, Rismani, Shalaleh, Henne, Kathryn, Moon, AJung, Rostamzadeh, Negar, Nicholas, Paul, Yilla-Akbari, N'Mah, Gallegos, Jess, Smart, Andrew, Garcia, Emilio, Virk, Gurleen · 2023 · Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society

The DOI matches its Crossref record; 204 of the 207 references it cites were found in a registry or an index, and 3 are not checked yet.

Abstract

Understanding the landscape of potential harms from algorithmic systems enables practitioners to better anticipate consequences of the systems they build. It also supports the prospect of incorporating controls to help minimize harms that emerge from the interplay of technologies and social and cultural dynamics. A growing body of scholarship has identified a wide range of harms across different algorithmic technologies. However, computing research and practitioners lack a high level and synthesized overview of harms from algorithmic systems. Based on a scoping review of computing research (n=172), we present an applied taxonomy of sociotechnical harms to support a more systematic surfacing of potential harms in algorithmic systems. The final taxonomy builds on and refers to existing taxonomies, classifications, and terminologies. Five major themes related to sociotechnical harms — representational, allocative, quality-of-service, interpersonal harms, and social system/societal harms — and sub-themes are presented along with a description of these categories. We conclude with a discussion of challenges and opportunities for future research.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3600211.3604673

Cites

204 of 207 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 207 references

Research works printed but not linked: 42

  1. 10.1093/cybsec · doi:10.1093/cybsec
  2. 10.18653/v1 · doi:10.18653/v1
  3. Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2016. Machine Bias. In Ethics of Data and Analytics. Auerbach Publications, Boca Raton, FL, USA, 254–264.
  4. Jane Bailey, Jacquelyn Burkell, Suzie Dunn, Chandell Gosse, and Valerie Steeves. 2021. AI and Technology-Facilitated Violence and Abuse. In Artificial Intelligence and the Law in Canada, Florian Martin-Bariteau and Teresa Scassa (Eds.). LexisNexis, Toronto, Canada, Chapter 10, 1–15.
  5. Jane Bailey, Nicola Henry, and Asher Flynn. 2021. Technology-Facilitated Violence and Abuse: International Perspectives and Experiences. In The Emerald International Handbook of Technology-Facilitated Violence and Abuse, Bailey Jane, Flynn Asher, and Henry Nicola (Eds.). Emerald Publishing Limited, Bingley, UK, 1–17.
  6. 10.18653/v1 · doi:10.18653/v1
  7. Solon Barocas, Kate Crawford, Aaron Shapiro, and Hanna Wallach. 2017. The Problem with Bias: Allocative Versus Representational Harms in Machine Learning. In 9th Annual Conference of the Special Interest Group for Computing, Information and Society. Society for the History of Technology, Philadelphia, PA, USA.
  8. Samy Bengio Alina Beygelzimer Kate Crawford Jeanne Fromer Iason Gabriel Amanda Levendowski Deborah Raji and Marc’Aurelio Ranzato. 2022. Provisional Draft of the NeurIPS Code of Ethics. https://openreview.net/pdf?id=zVoy8kAFKPr
  9. 10.1177/205630511667248 · doi:10.1177/205630511667248
  10. 10.18653/v1 · doi:10.18653/v1
  11. 10.5555/380681 · doi:10.5555/380681
  12. Virginia Braun and Victoria Clarke. 2021. One Size Fits All? What Counts as Quality Practice in (Reflexive) Thematic Analysis?Qualitative research in psychology 18, 3 (2021), 328–352.
  13. Joy Buolamwini and Timnit Gebru. 2018. Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency(Proceedings of Machine Learning Research, Vol. 81), Sorelle A. Friedler and Christo Wilson (Eds.). PMLR, New York, NY, USA, 77–91. https://proceedings.mlr.press/v81/buolamwini18a.html
  14. Jacquelyn Burkell and Jane Bailey. 2018. Unlawful Distinctions? Canadian Human Rights Law and Algorithmic Bias. Vol. 2016/2018. Human Rights Research and Education Centre Ottawa, Ottawa, Canada, 217–230.
  15. Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017. Semantics Derived Automatically from Language Corpora Contain Human-like Biases. Science 356, 6334 (2017), 183–186.
  16. Frances Corry, Hamsini Sridharan, Alexandra Sasha Luccioni, Mike Ananny, Jason Schultz, and Kate Crawford. 2021. The Problem of Zombie Datasets: A Framework for Deprecating Datasets. arXiv preprint arXiv:2111.04424 abs/2111.04424 (2021), 1–18. · arXiv
  17. Kate Crawford. 2017. Keynote: The Trouble with Bias. 2017 Conference on Neural Information Processing Systems, invited speaker.
  18. Catherine D’Ignazio and Lauren F Klein. 2020. Data Feminism. MIT Press, Boston, MA, USA.
  19. Batya Friedman and David G Hendry. 2019. Value Sensitive Design: Shaping Technology with Moral Imagination. MIT Press, Boston, MA, USA.
  20. Steve Fuller. 1993. Helen E. Longino Science As Social Knowledge: Values and Objectivity in Scientific Inquiry. Princeton: Princeton University Press (1990), xi+ 262 pp., 19.95. Philosophy of Science 60, 2 (1993), 360–362.
  21. Jan Hadja. 1961. Alienation and Integration of Student Intellectualss. American Sociological Review 26, 5 (1961), 19.
  22. 10.1177/104973231666534 · doi:10.1177/104973231666534
  23. 10.1177/146144482095872 · doi:10.1177/146144482095872
  24. Shawn Janzen, Caroline Orr, and Sara-Jayne Terp. 2022. Cognitive Security and Resilience: A Social Ecological Model of Disinformation and Other Harms with Applications to COVID-19 Vaccine Information Behaviors. In 2nd Workshop Reducing Online Misinformation through Credible Information Retrieval. ROMCIR 2022, Stavanger, Norway, 48–88.
  25. Jared Katzman, Solon Barocas, Su Lin Blodgett, Kristen Laird, Morgan Klaus Scheuerman, and Hanna Wallach. 2021. Representational Harms in Image Tagging. Beyond Fair Computer Vision Workshop at CVPR 2021 (2021).
  26. Hank Liao, Golan Pundak, Olivier Siohan, Melissa Carroll, Noah Coccaro, Qi-Ming Jiang, Tara N Sainath, Andrew Senior, Françoise Beaufays, and Michiel Bacchiani. 2015. Large Vocabulary Automatic Speech Recognition for Children. In Interspeech 2015. International Speech Communication Association, Dresden, Germany, 1–5.
  27. 10.1093/ojls · doi:10.1093/ojls
  28. Emanuel Moss and Jacob Metcalf. 2020. Ethics Owners: A New Model of Organizational Responsibility in Data-Driven Technology Companies. https://datasociety.net/wp-content/uploads/2020/09/Ethics-Owners_20200923-DataSociety.pdf
  29. Emmanuel Moss and Jacob Metcalf. 2022. Data and Society Workshop: The Social Life of Algorithmic Harms. (March 2022).
  30. Aik Kwang Ng, David YF Ho, Shyh Shin Wong, and Ian Smith. 2003. In Search of the Good Life: A Cultural Odyssey in the East and West. Genetic, Social, and General Psychology Monographs 129, 4 (2003), 317.
  31. Safiya Umoja Noble. 2016. A Future for Intersectional Black Feminist Technology Studies. Scholar & Feminist Online 13, 3 (2016), 1–8.
  32. Cathy O’Neil. 2016. Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. Broadway Books, New York, NY, USA.
  33. Eva PenzeyMoog and Danielle C Slakoff. 2021. As Technology Evolves, So Does Domestic Violence: Modern-Day Tech Abuse and Possible Solutions. In The Emerald International Handbook of Technology-Facilitated Violence and Abuse. Emerald Publishing Limited, Bingley, UK, 643–662.
  34. Jasbir K Puar. 2020. “I Would Rather be a Cyborg than a Boddess”: Becoming Intersectional in Assemblage Theory. In Feminist Theory Reader. Routledge, New York, NY, USA, 405–415.
  35. Henrik Skaug Sætra. 2021. Artificial Intelligence and its Contexts. Springer, Cham, Switzerland, Chapter A Typology of AI Applications in Politics, 27–43.
  36. Katie Shilton. 2015. "That’s Not An Architecture Problem!": Techniques and Challenges for Practicing Anticipatory Technology Ethics. In iConference 2015 Proceedings. iSchools Urbana-Champaign Illinois USA 1–7.
  37. Rebecca Kelly Slaughter, Janice Kopec, and Mohamad Batal. 2020. Algorithms and Economic Justice: A Taxonomy of Harms and a Path Forward for the Federal Trade Commission. Yale Journal of Law & Tech. 23 (2020), 1.
  38. 10.1177/0002716222108470 · doi:10.1177/0002716222108470
  39. 10.18653/v1 · doi:10.18653/v1
  40. Thi Tran, Rohit Valecha, Paul Rad, and H Raghav Rao. 2019. An Investigation of Misinformation Harms Related to Social Media During Humanitarian Crises. In International Conference on Secure Knowledge Management in Artificial Intelligence Era. Springer, Goa, India, 167–181.
  41. Zeynep Tufekci. 2015. Algorithmic Harms Beyond Facebook and Google: Emergent Challenges of Computational Agency. Colo. Tech. LJ 13 (2015), 203.
  42. Claire Wardle and Hossein Derakhshan. 2017. Information Disorder: Toward an Interdisciplinary Framework for Research and Policymaking.

Cited by

See all 623

Metadata

Authors
Shelby, R., Rismani, S., Henne, K., Moon, A., Rostamzadeh, N., Nicholas, P., Yilla-Akbari, N., Gallegos, J., Smart, A., Garcia, E., & Virk, G.
Affiliations
  • Google (United States) (US)
  • McGill University (CA)
  • Australian National University (AU)
  • Google, USA
Venue
Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society
Publisher
ACM
Published
2023-08-08
Pages
723-741
ISSN
3065-8365 (ISSN-L)
OpenAlex
W4386249234
Cited by
279
References
191
Citations by year
2026: 97 · 2025: 101 · 2024: 65 · 2023: 15
FWCI
69.56
Type
conference paper
Language
en
Open access
Locations
  • Proceedings of the AAAI/ACM Conference on AI Ethics and Society (publishedVersion)
  • HAL (submittedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Social Sciences › Law › Artificial Intelligence in Law
Keywords
algorithmic harms, harm reduction, algorithmic systems, representational harm, interpersonal harm, social harm, sociocultural dynamics

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction. https://app.cochat.ai/papers/10.1145/3600211.3604673
The paper
Shelby, R., Rismani, S., Henne, K., Moon, A., Rostamzadeh, N., Nicholas, P., Yilla-Akbari, N., Gallegos, J., Smart, A., Garcia, E., & Virk, G. (2023). Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction. Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society. https://doi.org/10.1145/3600211.3604673

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.