Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction
Open accessThe DOI matches its Crossref record; 205 of the 208 references it cites were found in a registry or an index, and 3 are not checked yet.
Abstract
Understanding the landscape of potential harms from algorithmic systems enables practitioners to better anticipate consequences of the systems they build. It also supports the prospect of incorporating controls to help minimize harms that emerge from the interplay of technologies and social and cultural dynamics. A growing body of scholarship has identified a wide range of harms across different algorithmic technologies. However, computing research and practitioners lack a high level and synthesized overview of harms from algorithmic systems. Based on a scoping review of computing research (n=172), we present an applied taxonomy of sociotechnical harms to support a more systematic surfacing of potential harms in algorithmic systems. The final taxonomy builds on and refers to existing taxonomies, classifications, and terminologies. Five major themes related to sociotechnical harms — representational, allocative, quality-of-service, interpersonal harms, and social system/societal harms — and sub-themes are presented along with a description of these categories. We conclude with a discussion of challenges and opportunities for future research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“However, the literature on harms from LLMs and algorithmic systems more broadly has shown that many of the harms arising from such models emerge from their embedding in wider sociotechnical systems (Weidinger et al., 2021; Bender et al., 2021; Shelby et al., 2023).”
From Helpful, harmless, honest? Sociotechnical limits of AI alignment and safety through Reinforcement Learning from Human Feedback (2025) · cites this paper as Shelby et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“The rapid pace of adoption of GenAI tools, when it comes to similar concerns, have significantly exacerbated the many socio-technical issues researchers have connected to education technologies in general [160, 311] and algorithmic harms related to GenAI [317] more specifically.”
From Navigating the Ethical and Societal Impacts of Generative AI in Higher Computing Education (2025) · cites this paper as [317]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Recent work has expressed concern around anthropomorphism leading to over-trust [1, 91], a potential path for uncertainty expression to backfire.”
From "I'm Not Sure, But...": Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trust (2024) · cites this paper as [91]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and Explanation
Tricco, A. C., Lillie, E., Zarin, W., O'Brien, K. K., Colquhoun, H., Levac, D., Moher, D., Peters, M. D. J., Horsley, T., Weeks, L., et al. (2018) · Annals of Internal Medicine · doi:10.7326/m18-0850 · Cited by 43,565 · cited 1×
“Our approach followed prior scoping reviews in HCI literature [74, 223], in alignment with the extension of the PRISMA checklist [133], the PRISMA-ScR (Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews) [220].”
MethodScoping studies: towards a methodological framework
Arksey, H., & O'Malley, L. (2005) · International Journal of Social Research Methodology · doi:10.1080/1364557032000119616 · Cited by 36,742 · cited 5×
“We employed a descriptive-analytical method for charting data – a process of “synthesizing and interpreting qualitative data by sifting, charting and sorting material according to key issues and themes" [11, p. 26].”
BackgroundMethodInfluentialBlack Feminist Thought: Knowledge, Consciousness, and the Politics of Empowerment.
Brewer, R. M., & Collins, P. H. (1992) · Contemporary Sociology · doi:10.2307/2074808 · Cited by 19,445
Situated Knowledges: The Science Question in Feminism and the Privilege of Partial Perspective
Haraway, D. (1988) · Feminist Studies · doi:10.2307/3178066 · Cited by 18,364
Scoping studies: advancing the methodology
Levac, D., Colquhoun, H., & O'Brien, K. K. (2010) · Implementation Science · doi:10.1186/1748-5908-5-69 · Cited by 15,268
Saturation in qualitative research: exploring its conceptualization and operationalization
Saunders, B., Sim, J., Kingstone, T., Baker, S., Waterfield, J., Bartlam, B., Burroughs, H., & Jinks, C. (2018) · Quality & Quantity · doi:10.1007/s11135-017-0574-8 · Cited by 12,033
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00136 · Cited by 10,349
Guidance for conducting systematic scoping reviews
Peters, M. D. J., Godfrey, C. M., Khalil, H., McInerney, P., Parker, D., & Soares, C. B. (2015) · International Journal of Evidence-Based Healthcare · doi:10.1097/xeb.0000000000000050 · Cited by 7,472 · cited 2×
“Thus, relying on scoping reviews’ iterative characteristic [168], we used theoretical saturation as a signal to stop sourcing new papers.”
MethodDissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137 · cited 2×
“systems make incorrect health inferences [158].”
BackgroundOrlikowski, W. J. (2000) · Organization Science · doi:10.1287/orsc.11.4.404.14600 · Cited by 4,760
Epistemic Cultures: How the Sciences Make Knowledge
Knorr Cetina, K. (1999) · Harvard University Press eBooks · doi:10.4159/9780674039681 · Cited by 3,956
Pinch, T. J., & Bijker, W. E. (1984) · Social Studies of Science · doi:10.1177/030631284014003004 · Cited by 3,890 · cited 1×
“This work requires thinking critically about the distribution of benefits and harms of algorithmic systems [33, 169] and existing social hierarchies [35].”
BackgroundThe science question in feminism
Waters, K. (1987) · Women's Studies International Forum · doi:10.1016/0277-5395(87)90077-x · Cited by 3,804
Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor
Lebovits, H. (2019) · Public Integrity · doi:10.1080/10999922.2018.1511671 · Cited by 3,279 · cited 5×
“In the government or social services domain, screening tools to identify children at-risk for maltreatment can amplify already-existing biases against poor parents [84, 241].”
BackgroundInfluentialBarocas, S., & Selbst, A. D. (2014) · doi:10.2139/ssrn.2477899 · Cited by 3,048
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
A scoping review on the conduct and reporting of scoping reviews
Tricco, A. C., Lillie, E., Zarin, W., O’Brien, K., Colquhoun, H., Kastner, M., Levac, D., Ng, C., Sharpe, J. P., Wilson, K., et al. (2016) · BMC Medical Research Methodology · doi:10.1186/s12874-016-0116-4 · Cited by 2,250 · cited 1×
“They are especially appropriate when distilling and sharing research findings with practitioners [221], and are suited to identifying evidence on a topic and presenting it visually.”
Background(2020) · doi:10.7551/mitpress/12255.001.0001 · Cited by 1,706
Fairness and Abstraction in Sociotechnical Systems
Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., & Vertesi, J. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287598 · Cited by 1,631 · cited 1×
“With increased awareness of the need to anticipate harms early in product development [195], designers and researchers are central actors in pursuing harm reduction [40, 58, 97].”
BackgroundEngineering a Safer World: Systems Thinking Applied to Safety
(2012) · Choice Reviews Online · doi:10.5860/choice.49-6305 · Cited by 1,566 · cited 1×
“Harm to social systems is thus about how algorithmic systems adversely shape the emergent properties [129] of social systems [162].”
BackgroundReview of “Race After Technology: Abolitionist Tools for the New Jim Code”
Jipguep-Akhtar, M. (2020) · Social Forces · doi:10.1093/sf/soz162 · Cited by 1,473
Friedman, B., & Nissenbaum, H. (1996) · ACM Transactions on Information Systems · doi:10.1145/230538.230561 · Cited by 1,203 · cited 2×
“Computing research has traced how marginalized communities — referring to communities that face structural forms of social exclusion [130] — disproportionately experience sociotechnical harms from algorithmic systems [90, 96].”
BackgroundJain, A. (2017) · Business Economics · doi:10.1057/s11369-017-0027-3 · Cited by 1,058
Hilton, J. L., & von Hippel, W. (1996) · Annual Review of Psychology · doi:10.1146/annurev.psych.47.1.237 · Cited by 977
The nature of cyberbullying, and strategies for prevention
Slonje, R., Smith, P. K., & Frisén, A. (2013) · Computers in Human Behavior · doi:10.1016/j.chb.2012.05.024 · Cited by 940
Decolonial AI: Decolonial Theory as Sociotechnical Foresight in Artificial Intelligence
Mohamed, S., Png, M.-T., & Isaac, W. (2020) · Philosophy & Technology · doi:10.1007/s13347-020-00405-8 · Cited by 912
Not yet human: Implicit knowledge, historical dehumanization, and contemporary consequences.
Goff, P. A., Eberhardt, J. L., Williams, M. J., & Jackson, M. C. (2008) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.94.2.292 · Cited by 880 · cited 1×
“Controlling images include forms of human-animal confusion in image tagging systems [225], which reflect dehuman-izing gendered and racialized discourses used to socially exclude and control Black, Indigenous, and other people of color [95].”
BackgroundThe Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation
Brundage, M., Avin, S., Clark, J., Toner, H., Eckersley, P., Garfinkel, B., Dafoe, A., Scharre, P., Zeitzoff, T., Filar, B., et al. (2018) · arXiv · doi:10.48550/arxiv.1802.07228 · Cited by 849 · cited 1×
“cious uses of algorithmic systems [45], including cyber attacks [3] and cyberbullying [12].”
BackgroundThe Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation
Brundage, M., Avin, S., Clark, J., Toner, H., Eckersley, P., Garfinkel, B., Dafoe, A., Scharre, P., Zeitzoff, T., Filar, B., et al. (2018) · Apollo - University of Cambridge Repository · doi:10.17863/cam.22520 · Cited by 849
Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints
Zhao, J., Wang, T., Yatskar, M., Ordonez, V., & Chang, K.-W. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1323 · Cited by 829 · cited 3×
“Research identifies narrow stereotypes about masculinity and femininity represented and expressed in natural language processing and computer vision systems, particularly in relation to professions [122], cooking and shopping [244], and sport [48].”
BackgroundBourtoule, L., Chandrasekaran, V., Choquette-Choo, C. A., Jia, H., Travers, A., Zhang, B., Lie, D., & Papernot, N. (2021) · 2021 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp40001.2021.00019 · Cited by 810 · cited 1×
“Furthermore, we acknowledge that assessing the overall social impacts of an algorithmic system using only harm as a framing device can miss other negative implications of technology – for instance, inconveniences that arise from exercising the right to data portability or the right to be forgotten [30, 38].”
BackgroundPostcolonial computing: a lens on design and development
Irani, L., Vertesi, J., Dourish, P., Philip, K., & Grinter, R. E. (2010) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1753326.1753522 · Cited by 780 · cited 2×
“vides and entrenching systemic inequalities [114, 230].”
Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need?
Holstein, K., Wortman Vaughan, J., Daumé, H., Dudik, M., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300830 · Cited by 777 · cited 1×
“tices often draw on fields of auditing, impact assessment, risk management, and safety engineering where a clear understanding of harm is essential [112].”
BackgroundPollock, D., Davies, E. L., Peters, M. D. J., Tricco, A. C., Alexander, L., McInerney, P., Godfrey, C. M., Khalil, H., & Munn, Z. (2021) · Journal of Advanced Nursing · doi:10.1111/jan.14743 · Cited by 769 · cited 2×
“A scoping review is an effective method for surfacing current “pri-orities for research, clarifying concepts and definitions, providing research frameworks or providing background, or contextual information on phenomena or concepts" [65, p. 2104].”
BackgroundMethodRacial disparities in automated speech recognition
Koenecke, A., Nam, A., Lake, E., Nudell, J., Quartey, M., Mengesha, Z., Toups, C., Rickford, J. R., Jurafsky, D., & Goel, S. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1915768117 · Cited by 765
Taxonomy of Risks posed by Language Models
Weidinger, L., Uesato, J., Rauh, M., Griffin, C., Huang, P.-S., Mellor, J., Glaese, A., Cheng, M., Balle, B., Kasirzadeh, A., et al. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533088 · Cited by 764
(2016) · doi:10.22459/caepr38.11.2016 · Cited by 696 · cited 1×
“It can be strengthened by bringing in different standpoints and epistemologies, such as feminism [73, 155], value-based design [17, 88, 116], design justice perspectives [60], and post-colonial theories [126, 148].”
BackgroundTechnology-facilitated sexual violence: A literature review of empirical research
Henry, N., & Powell, A. (2018) · Trauma, Violence, & Abuse · doi:10.1177/1524838016650189 · Cited by 675 · cited 3×
“This includes work to taxonomize harms in HCI on digital safety [3, 193, 216], in sociolegal studies on technology-facilitated violence [109, 231], and canonical responsible ML research on representational, allocative [20], and quality-of-service harms [36] that have significantly shaped the…”
BackgroundMacLeod, K. (2021) · Pathfinder A Canadian Journal for Information Science Students and Early Career Professionals · doi:10.29173/pathfinder28 · Cited by 597 · cited 3×
“This harm sub-type was also popularized by Safiya Noble in Algorithms of Oppression [156].”
BackgroundMethodFairness of Exposure in Rankings
Singh, A., & Joachims, T. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3220088 · Cited by 591 · cited 2×
“In the employment domain, recommender or ranking systems that match employers and potential candidates may prioritize the resumes of men over other genders [201, 224].”
BackgroundA Framework of Potential Sources of Harm Throughout the Machine Learning Life Cycle
Suresh, H., & Guttag, J. V. (2021) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 576
Data and its (dis)contents: A survey of dataset development and use in machine learning research
Paullada, A., Raji, I. D., Bender, E. M., Denton, E., & Hanna, A. (2021) · Patterns · doi:10.1016/j.patter.2021.100336 · Cited by 548
Towards a standard for identifying and managing bias in artificial intelligence
Schwartz, R., Vassilev, A., Greene, K., Perine, L., Burt, A., & Hall, P. (2022) · doi:10.6028/nist.sp.1270 · Cited by 548 · cited 1×
“Accord-ingly, individual scholarly analyses rarely capture the wide scope of harms that can arise from a given algorithmic system, which is a kind of anticipation work required for developing trustworthy and Responsible AI (see: [194]).”
BackgroundUnequal Representation and Gender Stereotypes in Image Search Results for Occupations
Kay, M., Matuszek, C., & Munson, S. A. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702520 · Cited by 509 · cited 1×
“Research identifies narrow stereotypes about masculinity and femininity represented and expressed in natural language processing and computer vision systems, particularly in relation to professions [122], cooking and shopping [244], and sport [48].”
BackgroundThe Misgendering Machines: Trans/HCI Implications of Automatic Gender Recognition
Keyes, O. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3274357 · Cited by 505
Artificial intelligence, systemic risks, and sustainability
Galaz, V., Centeno, M. A., Callahan, P. W., Causevic, A., Patterson, T., Brass, I., Baum, S., Farber, D., Fischer, J., Garcia, D., et al. (2021) · Technology in Society · doi:10.1016/j.techsoc.2021.101741 · Cited by 497
Graham, S. D. N. (2005) · Progress in Human Geography · doi:10.1191/0309132505ph568oa · Cited by 489 · cited 1×
“Computing research has traced how marginalized communities — referring to communities that face structural forms of social exclusion [130] — disproportionately experience sociotechnical harms from algorithmic systems [90, 96].”
BackgroundSweeney, L. (2013) · Queue · doi:10.1145/2460276.2460278 · Cited by 477
Persistent Anti-Muslim Bias in Large Language Models
Abid, A., Farooqi, M., & Zou, J. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462624 · Cited by 476 · cited 1×
“Similarly, patterns of demeaning imagery have been found in hateful natural language predictions about Muslim people [1], and toxicity and sentiment classifiers that are more likely to classify descriptions or mentions of disabilities [113, 213] and LGBTQ identities [215, 237] as toxic or negative.”
BackgroundOn Measuring Social Biases in Sentence Encoders
May, C., Wang, A., Bordia, S., Bowman, S. R., & Rudinger, R. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1063 · Cited by 461 · cited 1×
“social factors [35, 90] and can encode systemic inequalities [36, 141, 143, 213].”
BackgroundOnline misinformation about climate change
Treen, K. M. D., Williams, H. T. P., & O'Neill, S. J. (2020) · WIREs Climate Change · doi:10.1002/wcc.665 · Cited by 441
Madaio, M. A., Stark, L., Wortman Vaughan, J., & Wallach, H. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376445 · Cited by 436 · cited 4×
“Researchers have also developed “ethics methods” [150] for practitioners to identify and mitigate sociotechnical harms, including statistical assessment [135, 145], software”
BackgroundInfluentialFrom a ‘race to AI’ to a ‘race to AI regulation’: regulatory competition for artificial intelligence
Smuha, N. A. (2021) · Law, Innovation and Technology · doi:10.1080/17579961.2021.1898300 · Cited by 380 · cited 1×
“Alongside broader movements towards regulation and standardization [208], harm reduction practices often draw on fields of auditing, impact assessment, risk management, and safety engineering where a clear understanding of harm is essential [112].”
BackgroundKarizat, N., Delmonaco, D., Eslami, M., & Andalibi, N. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3476046 · Cited by 339 · cited 2×
“As algorithms increasingly curate the flows of information in digital spaces (i.e., recommender systems), Karizat, Delmonaco, Eslami, and Andal-ibi [118, p. 20] describe how the presentation of content may lead to “algorithmically informed identity change... including [promotion of] harmful person…”
BackgroundFeminist Theory Reader: Local and Global Perspectives
R. McCann, C., Kim, S.-K., & Ergun, E. (2020) · Routledge eBooks · doi:10.4324/9781003001201 · Cited by 332
Rakova, B., Yang, J., Cramer, H., & Chowdhury, R. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449081 · Cited by 329
The Values Encoded in Machine Learning Research
Birhane, A., Kalluri, P., Card, D., Agnew, W., Dotan, R., & Bao, M. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533083 · Cited by 317 · cited 1×
“This work requires thinking critically about the distribution of benefits and harms of algorithmic systems [33, 169] and existing social hierarchies [35].”
BackgroundThe Values Encoded in Machine Learning Research
Birhane, A., Kalluri, P., Card, D., Agnew, W., Dotan, R., & Bao, M. (2021) · arXiv · doi:10.48550/arxiv.2106.15590 · Cited by 317 · cited 1×
“This work requires thinking critically about the distribution of benefits and harms of algorithmic systems [33, 169] and existing social hierarchies [35].”
BackgroundDuffy, B. E., & Meisner, C. (2023) · Media, Culture & Society · doi:10.1177/01634437221111923 · Cited by 305 · cited 1×
“It may arise through different technologies, including demonetization algorithms that parse content titles, metadata, and text, and it may penalize words with multiple meanings [51, 81], disproportionately impacting queer, trans, and creators of color [81].”
BackgroundAnticipatory Ethics for Emerging Technologies
Brey, P. A. E. (2012) · NanoEthics · doi:10.1007/s11569-012-0141-7 · Cited by 299 · cited 1×
“It can be done in relation to the technology holistically or with a focus on certain features of the technology and its use by different groups [43].”
BackgroundSaving Face: Investigating the Ethical Concerns of Facial Recognition Auditing
Raji, I. D., Gebru, T., Mitchell, M., Buolamwini, J., Lee, J., & Denton, R. (2020) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3375627.3375820 · Cited by 299 · cited 1×
“, facial features, skin tone, or voice), such as computer vision [46, 176], natural language processing [113, 131, 173], and speech recognition systems [124, 143].”
BackgroundAn Empirical Analysis of Algorithmic Pricing on Amazon Marketplace
Chen, L., Mislove, A., & Wilson, C. (2016) · Proceedings of the 25th International Conference on World Wide Web · doi:10.1145/2872427.2883089 · Cited by 297 · cited 1×
“Differential pricing algorithms, where people are systematically shown different prices for the same products, also leads to economic loss [55].”
BackgroundTowards a critical race methodology in algorithmic fairness
Hanna, A., Denton, R., Smart, A., & Smith-Loud, J. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372826 · Cited by 295 · cited 2×
“When such classification relies on phenotypes, this dimension of representational harm essentializes historically contingent identities [85] through which classification systems entrench and produce meaning about what they represent [100, 111].”
BackgroundSelf-citation“A Stalker's Paradise”: How Intimate Partner Abusers Exploit Technology
Freed, D., Palmer, J., Minchala, D., Levy, K., Ristenpart, T., & Dell, N. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3174241 · Cited by 290 · cited 1×
“Gender violence scholars have uncovered how algorithmic technologies can become conduits for stalking [87, 108], online sexual harassment and assault (e.g., sharing images of sexual coercion and violence, sex-tortion) [44], and coercive control backed by the threat of violence (e.g., accessing…”
BackgroundCaplan, R., & Gillespie, T. (2020) · Social Media + Society · doi:10.1177/2056305120936636 · Cited by 283 · cited 1×
“It may arise through different technologies, including demonetization algorithms that parse content titles, metadata, and text, and it may penalize words with multiple meanings [51, 81], disproportionately impacting queer, trans, and creators of color [81].”
BackgroundDragiewicz, M., Burgess, J., Matamoros-Fernández, A., Salter, M., Suzor, N. P., Woodlock, D., & Harris, B. (2018) · Feminist Media Studies · doi:10.1080/14680777.2018.1447341 · Cited by 282 · cited 4×
“…become conduits for stalking [87, 108], online sexual harassment and assault (e.g., sharing images of sexual coercion and violence, sex-tortion) [44], and coercive control backed by the threat of violence (e.g., accessing accounts, impersonating a partner, doxxing, sharing sexualized content) [80].”
BackgroundInfluentialOwning Ethics: Corporate Logics, Silicon Valley, and the Institutionalization of Ethics
Metcalf, J., Moss, E., & boyd, D. (2019) · Social Research: An International Quarterly · doi:10.1353/sor.2019.0022 · Cited by 278 · cited 1×
“Practitioners have sought to develop practices that better identify and minimize sociotechnical harms from algorithmic systems (e.g., [25, 62, 135, 144]).”
BackgroundWho Audits the Auditors? Recommendations from a field scan of the algorithmic auditing ecosystem
Costanza-Chock, S., Raji, I. D., & Buolamwini, J. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533213 · Cited by 277 · cited 1×
“Moreover, algorithmic systems may exacerbate social inequalities and reduction of civil liberties within legal systems [139, 181], such as unreasonable searches [152], wrongful arrest [61, 61, 124], or court transcription errors [124].”
BackgroundChallenges in Detoxifying Language Models
Welbl, J., Glaese, A., Uesato, J., Dathathri, S., Mellor, J. F. J., Hendricks, L. A., Anderson, K., Kohli, P., Coppin, B., & Huang, P.-S. (2021) · Conference on Empirical Methods in Natural Language Processing · doi:10.48550/arxiv.2109.07445 · Cited by 257 · cited 3×
“Similarly, patterns of demeaning imagery have been found in hateful natural language predictions about Muslim people [1], and toxicity and sentiment classifiers that are more likely to classify descriptions or mentions of disabilities [113, 213] and LGBTQ identities [215, 237] as toxic or negative.”
BackgroundThe Fallacy of AI Functionality
Raji, I. D., Kumar, I. E., Horowitz, A., & Selbst, A. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533158 · Cited by 251 · cited 1×
“Scholars have developed harm taxonomies related to system misbehaviors and failures [18, 198], particularly to aid algorithmic auditing [177].”
BackgroundWhose science? whose knowledge?: thinking from women's lives
(1992) · Choice Reviews Online · doi:10.5860/choice.29-3277 · Cited by 247
Ethics of Data and Analytics: Concepts and Cases
Martin, K. (2022) · Auerbach Publications eBooks · doi:10.1201/9781003278290 · Cited by 246
Fordyce, S. (2020) · Design and Culture · doi:10.1080/17547075.2019.1684698 · Cited by 243 · cited 1×
“It can be strengthened by bringing in different standpoints and epistemologies, such as feminism [73, 155], value-based design [17, 88, 116], design justice perspectives [60], and post-colonial theories [126, 148].”
BackgroundEmbracing Four Tensions in Human-Computer Interaction Research with Marginalized People
Liang, C. A., Munson, S. A., & Kientz, J. A. (2021) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3443686 · Cited by 238 · cited 1×
“Computing research has traced how marginalized communities — referring to communities that face structural forms of social exclusion [130] — disproportionately experience sociotechnical harms from algorithmic systems [90, 96].”
BackgroundTechnology-Facilitated Domestic and Sexual Violence: A Review
Henry, N., Flynn, A., & Powell, A. (2020) · Violence Against Women · doi:10.1177/1077801219875821 · Cited by 235 · cited 4×
“Gender violence scholars have uncovered how algorithmic technologies can become conduits for stalking [87, 108], online sexual harassment and assault (e.g., sharing images of sexual coercion and violence, sex-tortion) [44], and coercive control backed by the threat of violence (e.g., accessing…”
BackgroundInfluentialRe-imagining Algorithmic Fairness in India and Beyond
Sambasivan, N., Arnesen, E., Hutchinson, B., Doshi, T., & Prabhakaran, V. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445896 · Cited by 228 · cited 3×
“proliferating false ideas about cultural groups [78, 189].”
BackgroundInfluentialPassi, S., & Jackson, S. J. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3274405 · Cited by 225
Scheuerman, M. K., Branham, S. M., & Hamidi, F. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3274424 · Cited by 219
Outsider Oversight: Designing a Third Party Audit Ecosystem for AI Governance
Raji, I. D., Xu, P., Honigsberg, C., & Ho, D. (2022) · Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3514094.3534181 · Cited by 209 · cited 2×
“As such, it presents navigational challenges for practitioners seeking to comprehensively evaluate a system for potential harms [178, 179, 182], particularly for large generative models that perform different tasks across many use cases.”
BackgroundDias Oliva, T., Antonialli, D. M., & Gomes, A. (2021) · Sexuality & Culture · doi:10.1007/s12119-020-09790-w · Cited by 199 · cited 2×
“Similarly, patterns of demeaning imagery have been found in hateful natural language predictions about Muslim people [1], and toxicity and sentiment classifiers that are more likely to classify descriptions or mentions of disabilities [113, 213] and LGBTQ identities [215, 237] as toxic or negative.”
BackgroundZajko, M. (2022) · Sociology Compass · doi:10.1111/soc4.12962 · Cited by 199 · cited 1×
“Allocative harms “arc towards existing patterns of power” [66, p. 2] as they entrench material divisions between social groups [243].”
BackgroundAlgorithmic Impact Assessments and Accountability: The Co-construction of Impacts
Metcalf, J., Moss, E., Watkins, E. A., Singh, R., & Elish, M. C. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445935 · Cited by 195 · cited 2×
“…have also developed “ethics methods” [150] for practitioners to identify and mitigate sociotechnical harms, including statistical assessment [135, 145], software toolkits [32], and algorithmic impact assessments and audits [178], providing notable benefit in how harms are anticipated and…”
BackgroundProblematic Machine Behavior: A Systematic Literature Review of Algorithm Audits
Bandy, J. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449148 · Cited by 193 · cited 3×
“Bandy [18] summarizes four problematic behaviors of algorithmic systems — discrimination, distortion, exploitation, and misjudgement.”
BackgroundDouglas, D. M. (2016) · Ethics and Information Technology · doi:10.1007/s10676-016-9406-0 · Cited by 191 · cited 3×
“Beyond gender violence, other facets of technology-facilitated violence, include doxxing [79], trolling [14], cyberstalking [14], cyberbullying [14, 98, 204], monitoring and control [44], and online harassment and intimidation [98, 192, 199, 226], under the broader banner of online toxicity [98,…”
BackgroundVitak, J., Chadha, K., Steiner, L., & Ashktorab, Z. (2017) · Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing · doi:10.1145/2998181.2998337 · Cited by 180 · cited 1×
“Beyond gender violence, other facets of technology-facilitated violence, include doxxing [79], trolling [14], cyberstalking [14], cyberbullying [14, 98, 204], monitoring and control [44], and online harassment and intimidation [98, 192, 199, 226], under the broader banner of online toxicity [98, 136].”
BackgroundDefining and detecting toxicity on social media: context and knowledge are key
Sheth, A., Shalin, V. L., & Kursuncu, U. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.11.095 · Cited by 176 · cited 1×
“…gender violence, other facets of technology-facilitated violence, include doxxing [79], trolling [14], cyberstalking [14], cyberbullying [14, 98, 204], monitoring and control [44], and online harassment and intimidation [98, 192, 199, 226], under the broader banner of online toxicity [98, 136].”
BackgroundChancellor, S., Baumer, E. P. S., & De Choudhury, M. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359249 · Cited by 172 · cited 1×
“and classified automatically, and often at the cost of autonomy loss [54, 165].”
BackgroundShen, H., DeVos, A., Eslami, M., & Holstein, K. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479577 · Cited by 171 · cited 1×
“Scholars have developed harm taxonomies related to system misbehaviors and failures [18, 198], particularly to aid algorithmic auditing [177].”
BackgroundAlgorithmic realism: expanding the boundaries of algorithmic thought
Green, B., & Viljoen, S. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372840 · Cited by 161 · cited 3×
“With increased awareness of the need to anticipate harms early in product development [195], designers and researchers are central actors in pursuing harm reduction [40, 58, 97].”
BackgroundToward situated interventions for algorithmic equity: lessons from the field
Katell, M., Young, M., Dailey, D., Herman, B., Guetler, V., Tam, A., Bintz, C., Raz, D., & Krafft, P. M. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372874 · Cited by 153 · cited 1×
“…and focus on how algorithmic systems govern through individualized nudges or micro-directives [187], that may destabi-lize governance systems, erode human rights, be used as weapons of war [188], and enact surveillant regimes that disproportionately target and harm people of color [120].”
BackgroundSoK: Hate, Harassment, and the Changing Landscape of Online Abuse
Thomas, K., Akhawe, D., Bailey, M., Boneh, D., Bursztein, E., Consolvo, S., Dell, N., Durumeric, Z., Kelley, P. G., Kumar, D., et al. (2021) · 2021 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp40001.2021.00028 · Cited by 148 · cited 2×
“Taxonomies have been developed related to online content [19, 193, 227], social media [157, 217, 218], users “at-risk" for experiencing online abuse [216, 234], and mali-”
Background2022 ACM Conference on Fairness Accountability and Transparency
(2022) · doi:10.1145/3531146 · Cited by 146
Beyond the individual: Governing AI’s societal harm
Smuha, N. A. (2021) · Internet Policy Review · doi:10.14763/2021.3.1574 · Cited by 143 · cited 5×
“More generally, these harms may erode democracy [97], through election interference or censorship [207].”
BackgroundInfluentialDavis, J. L., Williams, A., & Yang, M. W. (2021) · Big Data & Society · doi:10.1177/20539517211044808 · Cited by 143 · cited 1×
“, [66]), while others focus on mid-range”
Mengesha, Z., Heldreth, C., Lahav, M., Sublewski, J., & Tuennerman, E. (2021) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2021.725911 · Cited by 140 · cited 7×
“Such experiences include, but are not limited to, the inequitable distribution of resources [63], hierarchical representations of people and communities [156, 242], disparate performance based on identity categories [23, 143], and the entrenchment of social and economic inequalities [27, 84].”
BackgroundInfluentialFairness in Information Access Systems
Ekstrand, M. D., Das, A., Burke, R., & Diaz, F. (2022) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000079 · Cited by 133
Cooper, A. F., Moss, E., Laufer, B., & Nissenbaum, H. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533150 · Cited by 129 · cited 1×
“With increased awareness of the need to anticipate harms early in product development [195], designers and researchers are central actors in pursuing harm reduction [40, 58, 97].”
BackgroundIntimate Futures: Staying with the Trouble of Digital Personal Assistants through Design Fiction
Søndergaard, M. L. J., & Hansen, L. K. (2018) · Proceedings of the 2018 Designing Interactive Systems Conference · doi:10.1145/3196709.3196766 · Cited by 128 · cited 1×
“Stereotyping often reflects repeated patterns of over- and under-representation — for instance, how gendered beliefs about women’s submissiveness are reflected in digital assistants [50, 210, 231].”
BackgroundBennett, C. L., Gleason, C., Scheuerman, M. K., Bigham, J. P., Guo, A., & To, A. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445498 · Cited by 128 · cited 4×
“As classification systems areusedacrossmany consequential domains,denying opportunities to self-identify can materially impact marginalized communities, ranging from nonconsensual inclusion in datasets to surveillance and wrongful arrest [28].”
BackgroundInfluentialRethinking Fairness: An Interdisciplinary Survey of Critiques of Hegemonic ML Fairness Approaches
Weinberg, L. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13196 · Cited by 127 · cited 2×
“Thus, in anticipating harms, practitioners need to account for computational harms as well as those arising through contextual use [40, 164, 191, 236].”
BackgroundPandey, A., & Caliskan, A. (2021) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3461702.3462561 · Cited by 126 · cited 1×
“These algorithms may be especially sensitive to feedback loops from existing inequities related to education level, income, and race, as these inequalities are likely reflected in the criteria algorithms use to make decisions [22, 163].”
BackgroundHarms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologies
Dev, S., Monajatipoor, M., Ovalle, A., Subramonian, A., Phillips, J., & Chang, K.-W. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.150 · Cited by 123 · cited 3×
“We expect and hope it will evolve as research and engagement progress, particularly in terms of participatory and community-driven research methodologies (e.g., [68, 71, 89]).”
BackgroundMen Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level ConstraintsPreprint
Zhao, J., Wang, T., Yatskar, M., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.09457 · Cited by 122 · cited 3×
“Research identifies narrow stereotypes about masculinity and femininity represented and expressed in natural language processing and computer vision systems, particularly in relation to professions [122], cooking and shopping [244], and sport [48].”
BackgroundCambre, J., & Kulkarni, C. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359325 · Cited by 119 · cited 1×
“Stereotyping often reflects repeated patterns of over- and under-representation — for instance, how gendered beliefs about women’s submissiveness are reflected in digital assistants [50, 210, 231].”
BackgroundBeyond a Human Rights-Based Approach to AI Governance: Promise, Pitfalls, Plea
Smuha, N. A. (2020) · Philosophy & Technology · doi:10.1007/s13347-020-00403-w · Cited by 118 · cited 1×
“Environmental harms entail ecological concerns, such as the depletion or contamination of natural resources [24, 92, 139, 148, 206, 207, 235], and damage to built environments [139].”
BackgroundA Human-Centered Review of Algorithms used within the U.S. Child Welfare System
Saxena, D., Badillo-Urquiola, K., Wisniewski, P. J., & Guha, S. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376229 · Cited by 116 · cited 2×
“Thus, in anticipating harms, practitioners need to account for computational harms as well as those arising through contextual use [40, 164, 191, 236].”
BackgroundGandy, O. H. (2010) · Ethics and Information Technology · doi:10.1007/s10676-009-9198-6 · Cited by 115 · cited 1×
“As they retain social and technical elements, they cannot be reme-died by technical fixes [94] alone.”
BackgroundFrom Treatment to Healing:Envisioning a Decolonial Digital Mental Health
Pendse, S. R., Nkemelu, D., Bidwell, N. J., Jadhav, S., Pathare, S., De Choudhury, M., & Kumar, N. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501982 · Cited by 115 · cited 1×
“Thus, builders of algorithmic systems must be attentive to forms of distress falling outside Eurocentric and Western care models [166].”
BackgroundA framework of severity for harmful content online
Scheuerman, M. K., Jiang, J. A., Fiesler, C., & Brubaker, J. R. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479512 · Cited by 114 · cited 4×
“Scholars in HCI, machine learning, Science and Technology Studies (STS), and related disciplines have identified various harms from digital technologies (e.g., [9, 121, 192, 193, 207]).”
BackgroundInfluentialChallenges in Detoxifying Language Models
Welbl, J., Glaese, A., Uesato, J., Dathathri, S., Mellor, J., Hendricks, L. A., Anderson, K., Kohli, P., Coppin, B., & Huang, P.-S. (2021) · Findings of the Association for Computational Linguistics: EMNLP 2021 · doi:10.18653/v1/2021.findings-emnlp.210 · Cited by 112
Sætra, H. S. (2020) · Technology in Society · doi:10.1016/j.techsoc.2020.101283 · Cited by 111
A Framework for Fairness: A Systematic Review of Existing Fair AI Solutions
Richardson, B., & Gilbert, J. E. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.05700 · Cited by 111 · cited 1×
“Our corpus included 11 thematic codes for allocative harms, which encompass problems arising from how algorithmic decisions are distributed unevenly to different groups of people [22, 183].”
Background(2020) · doi:10.1145/3386296.3386301 · Cited by 107
Re-contextualizing Fairness in NLP: The Case of India
Bhatt, S., Dev, S., Talukdar, P. P., Dave, S., & Prabhakaran, V. (2022) · Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing (AACL-IJCNLP) · doi:10.18653/v1/2022.aacl-main.55 · Cited by 107
Women also Snowboard: Overcoming Bias in Captioning Models
Burns, K., Hendricks, L. A., Saenko, K., Darrell, T. J., & Rohrbach, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.09797 · Cited by 100 · cited 3×
“While computing literature often describes stereotyping along single-axis dimensions of identity [48, 244], an intersectional approach draws attention to how harms play out for people whose lives are shaped by interlocking forms of oppression [46, 229] – for example, when a search for the term “unprofessional hairstyl…”
BackgroundBallard, S., Chappell, K. M., & Kennedy, K. (2019) · Proceedings of the 2019 on Designing Interactive Systems Conference · doi:10.1145/3322276.3323697 · Cited by 98 · cited 1×
“It can be strengthened by bringing in different standpoints and epistemolo-gies, such as feminism [73, 155], value-based design [17, 88, 116], de-sign justice perspectives [60], and post-colonial theories [126, 148].”
BackgroundDeVos, A., Dhabalia, A., Shen, H., Holstein, K., & Eslami, M. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517441 · Cited by 98 · cited 4×
“We expect and hope it will evolve as research and engagement progress, particularly in terms of participatory and community-driven research methodologies (e.g., [68, 71, 89]).”
BackgroundInfluentialDoes Object Recognition Work for Everyone?
DeVries, T., Misra, I., Wang, C., & van der Maaten, L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.02659 · Cited by 95 · cited 1×
“outline that these harms reflect how system training data are optimized for dominant groups [72].”
BackgroundAssembling Accountability: Algorithmic Impact Assessment for the Public InterestPreprint
Singh, R., Metcalf, J., Elish, M. C., et al. (2021) · Social Science Research Network · doi:10.2139/ssrn.3877437 · Cited by 93 · cited 2×
“bate social inequalities and reduction of civil liberties within legal systems [139, 181], such as unreasonable searches [152], wrongful”
BackgroundNumbers will not save us: Agonistic data practices
Crooks, R., & Currie, M. (2021) · The Information Society · doi:10.1080/01972243.2021.1920081 · Cited by 91
Trust in Human-AI Interaction: Scoping Out Models, Measures, and Methods
Ueno, T., Sawa, Y., Kim, Y., Urakami, J., Oura, H., & Seaborn, K. (2022) · Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491101.3519772 · Cited by 91 · cited 1×
“Our approach followed prior scoping reviews in HCI literature [74, 223], in alignment with the extension of the PRISMA checklist [133], the PRISMA-ScR (Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews) [220].”
Method”Because AI is 100% right and safe”: User Attitudes and Sources of AI Authority in India
Kapania, S., Siy, O., Clapper, G., SP, A. M., & Sambasivan, N. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517533 · Cited by 90 · cited 3×
“watched or surveilled [181], and the collection of data without explicit and informed consent [117].”
BackgroundMann, M., & Matzner, T. (2019) · Big Data & Society · doi:10.1177/2053951719895805 · Cited by 88 · cited 3×
“One dimension of agency loss is algorithmic profiling [138], through which people are subject to social sorting and discriminatory outcomes to access basic services [189].”
BackgroundJoint Multisided Exposure Fairness for Recommendation
Wu, H., Mitra, B., Ma, C., Díaz, F., & Liu, X. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3532007 · Cited by 88 · cited 2×
“In the government or social services domain, screening tools to identify children at-risk for maltreatment can amplify already-existing biases against poor parents [84, 241].”
BackgroundWardle, C., & Singerman, E. (2021) · BMJ · doi:10.1136/bmj.n26 · Cited by 87 · cited 1×
“Information harms are often accompanied by co-occurring impacts, including physical, psychological or emotional, financial, and reputational harms [3, 218, 233, 240], which scale into broader societal harm.”
BackgroundWang, A., Ramaswamy, V. V., & Russakovsky, O. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533101 · Cited by 87 · cited 2×
“…244], an intersectional approach draws attention to how harms play out for people whose lives are shaped by interlocking forms of oppression [46, 229] – for aim(s)tocapturethemor-phological sexual differences between male and female faces by comparing their shape differences to a defined face…”
BackgroundAt the Tensions of South and North: Critical Roles of Global South Stakeholders in AI Governance
Png, M.-T. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533200 · Cited by 85 · cited 3×
“The existing literature privileges Western perspectives [148, 170], leaving a dearth of perspectives from the majority of the world.”
BackgroundAI’s Regimes of Representation: A Community-centered Study of Text-to-Image Models in South Asia
Qadri, R., Shelby, R., Bennett, C. L., & Denton, R. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594016 · Cited by 85 · cited 1×
“Recognizing the sociotechnical character of harms from algorithmic systems draws attention to how the development and experience of digital technologies cannot be separated from cultural and social dynamics [7, 60, 172, 175].”
BackgroundSelf-citationAI Ethics – Too Principled to Fail?
Mittelstadt, B. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3391293 · Cited by 82 · cited 1×
“They encourage more rigorous analysis of social and ethical considerations [135], especially when operationalizing principles seems opaque [147].”
A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle
Suresh, H., & Guttag, J. (2021) · Equity and Access in Algorithms, Mechanisms, and Optimization · doi:10.1145/3465416.3483305 · Cited by 82 · cited 2×
“Design choices [143] and training data [212] influence which people and experiences are legible to an algorithmic system.”
BackgroundA Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle
Suresh, H., & Guttag, J. V. (2019) · arXiv · doi:10.48550/arxiv.1901.10002 · Cited by 82 · cited 2×
“Design choices [143] and training data [212] influence which people and experiences are legible to an algorithmic system.”
BackgroundTo Live in Their Utopia: Why Algorithmic Systems Create Absurd Outcomes
Alkhatib, A. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445740 · Cited by 81 · cited 2×
“Recognizing the sociotechnical character of harms from algorithmic systems draws attention to how the development and experience of digital technologies cannot be separated from cultural and social dynamics [7, 60, 172, 175].”
BackgroundTowards Debiasing Sentence RepresentationsPreprint
Liang, P. P., Li, I., Zheng, E., Lim, Y. C., Salakhutdinov, R., & Morency, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.08100 · Cited by 81 · cited 1×
“Prior work has described how quality-of-service harms are especially likely when system inputs rely on biometric data (e.g., facial features, skin tone, or voice), such as computer vision [46, 176], natural language processing [113, 131, 173], and speech recognition systems [124, 143].”
Background"'More gay' fits in better": Intracommunity Power Dynamics and Harms in Online LGBTQ+ Spaces
Walker, A. M., & DeVito, M. A. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376497 · Cited by 80 · cited 1×
“Taxonomies have been developed related to online content [19, 193, 227], social media [157, 217, 218], users “at-risk" for experiencing online abuse [216, 234], and malicious uses of algorithmic systems [45], including cyber attacks [3] and cyberbullying [12].”
BackgroundEthical Foresight Analysis: What it is and Why it is Needed?
Floridi, L., & Strait, A. (2020) · Minds and Machines · doi:10.1007/s11023-020-09521-y · Cited by 78 · cited 2×
“Anticipating harms is best conducted when grounded in a use case [86, 150, 200], because it provides deeper consideration of the domain of use, impacted downstream communities, and technological affordances [40].”
BackgroundTechnology-facilitated abuse in relationships: Victimisation patterns and impact in young people
Brown, C., Sanci, L. A., & Hegarty, K. L. (2021) · Computers in Human Behavior · doi:10.1016/j.chb.2021.106897 · Cited by 78 · cited 6×
“They can lead to both physical harms [71, 79, 139, 181] emotional harms, such as distress [235], dignity loss [44, 139], misgendering [123], and reputational harms [152, 181].”
BackgroundInfluentialMarkl, N. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533117 · Cited by 78 · cited 1×
“However, when technologies with performance disparities are used in consequential domains — such as in job application videos — degraded service can not only stigmatize users but also lead to other types of harm, such as allocative harms [140].”
BackgroundBridging near- and long-term concerns about AI
Cave, S., & hÉigeartaigh, S. Ó. (2018) · Nature Machine Intelligence · doi:10.1038/s42256-018-0003-2 · Cited by 76
Informing Age-Appropriate AI: Examining Principles and Practices of AI for Children
Wang, G., Zhao, J., Van Kleek, M., & Shadbolt, N. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502057 · Cited by 76 · cited 1×
“Algorithmic systems can increase “power imbalances in socio-economic relations” at the societal level [4, 137, p. 182], including through exacerbating digital divides and entrenching systemic inequalities [114, 230].”
BackgroundA Survey of Race, Racism, and Anti-Racism in NLP
Field, A., Blodgett, S. L., Waseem, Z., & Tsvetkov, Y. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.149 · Cited by 74 · cited 1×
“When such classification relies on phenotypes, this dimension of representational harm essentializes historically contingent identities [85] through which classification systems entrench and produce meaning about what they represent [100, 111].”
BackgroundThe Nature, Patterns and Consequences of Technology-Facilitated Domestic Abuse: A Scoping Review
Afrouz, R. (2023) · Trauma, Violence, & Abuse · doi:10.1177/15248380211046752 · Cited by 73
Designing Disaggregated Evaluations of AI Systems: Choices, Considerations, and Tradeoffs
Barocas, S., Guo, A., Kamar, E., Krones, J., Morris, M. R., Vaughan, J. W., Wadsworth, W. D., & Wallach, H. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462610 · Cited by 72
Andalibi, N., & Garcia, P. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449201 · Cited by 71 · cited 4×
“…may accompany other identified sociotechnical harms – for example, experiences of representational harms, including algorithmic anni-hilation [8] or the internalization of stereotypes may spark other emotional or psychological effects, such as “epistemic doubt” [231], which affects overall…”
BackgroundInfluentialSystem Safety and Artificial Intelligence
Dobbe, R. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533215 · Cited by 69 · cited 1×
“An opportunity thus arises to consider known harms and explore their interdependence [75], which may be useful when anticipating harms for novel systems.”
BackgroundSaving Face: Investigating the Ethical Concerns of Facial Recognition Auditing
Raji, I. D., Gebru, T., Mitchell, M., Buolamwini, J., Lee, J., & Denton, E. (2020) · arXiv · doi:10.48550/arxiv.2001.00964 · Cited by 69 · cited 1×
“, facial features, skin tone, or voice), such as computer vision [46, 176], natural language processing [113, 131, 173], and speech recognition systems [124, 143].”
BackgroundDecolonizing Tactics as Collective Resilience: Identity Work of AAPI Communities on Reddit
Dosono, B., & Semaan, B. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3392881 · Cited by 65 · cited 3×
“Erasure, as Dosono and Semaan [78, p. 1] describe, reflects a kind of algorithmic hegemony, as the dominant “system of ideas, practices, and social relations that permeate the institutional and private domains of society” become normalized in sociotechnical systems.”
BackgroundHard choices in artificial intelligence
Dobbe, R., Krendl Gilbert, T., & Mintz, Y. (2021) · Artificial Intelligence · doi:10.1016/j.artint.2021.103555 · Cited by 65
SoK: A Framework for Unifying At-Risk User Research
Warford, N., Matthews, T., Yang, K., Akgul, O., Consolvo, S., Kelley, P. G., Malkin, N., Mazurek, M. L., Sleeper, M., & Thomas, K. (2022) · 2022 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp46214.2022.9833643 · Cited by 62 · cited 1×
“Taxonomies have been developed related to online content [19, 193, 227], social media [157, 217, 218], users “at-risk" for experiencing online abuse [216, 234], and malicious uses of algorithmic systems [45], including cyber attacks [3] and cyberbullying [12].”
BackgroundOpinion Conflicts: An Effective Route to Detect Incivility in Twitter
Maity, S., Chakraborty, A., Goyal, P., & Mukherjee, A. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3274386 · Cited by 61 · cited 1×
“…gender violence, other facets of technology-facilitated violence, include doxxing [79], trolling [14], cyberstalking [14], cyberbullying [14, 98, 204], monitoring and control [44], and online harassment and intimidation [98, 192, 199, 226], under the broader banner of online toxicity [98, 136].”
BackgroundEvaluating Human-Language Model Interaction
Lee, M., Srivastava, M., Hardy, A., Thickstun, J., Durmus, E., Paranjape, A., Gerard-Ursin, I., Li, X. L., Ladhak, F., Rong, F., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.09746 · Cited by 61
Assembling Accountability: Algorithmic Impact Assessment for the Public Interest
Moss, E., Watkins, E. A., Singh, R., Elish, M. C., & Metcalf, J. (2021) · doi:10.69985/rswq7227 · Cited by 61
Yee, K., Tantipongpipat, U., & Mishra, S. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479594 · Cited by 59 · cited 2×
“Representational harms include instances where certain social groups experience both over- and under-exposure [27], leading to unequal visibility [242].”
BackgroundValues (Mis)alignment: Exploring Tensions Between Platform and LGBTQ+ Community Design Values
DeVito, M. A., Walker, A. M., & Fernandez, J. R. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449162 · Cited by 58 · cited 1×
“underscores harm as a relational outcome of entangled dynamics between design decisions, norms, and power [8, 27, 70, 148, 238], particularly along intersecting axes of gender [34, 197], race [106,”
BackgroundUsers' conceptions of risks and harms on the web: a comparative study
Friedman, B., Hurley, D. A., Howe, D. C., Nissenbaum, H. F., & Felten, E. W. (2002) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/506443.506510 · Cited by 57
Towards Fairness in Practice: A Practitioner-Oriented Rubric for Evaluating Fair ML Toolkits
Richardson, B., Garcia-Gathright, J., Way, S. F., Thom, J., & Cramer, H. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445604 · Cited by 56 · cited 1×
“As such, it presents navigational challenges for practitioners seeking to comprehensively evaluate a system for potential harms [178, 179, 182], particularly for large generative models that perform different tasks across many use cases.”
BackgroundFive Years of the Right to be Forgotten
Bertram, T., Bursztein, E., Caro, S., Chao, H., Chin Feman, R., Fleischer, P., Gustafsson, A., Hemerly, J., Hibbert, C., Invernizzi, L., et al. (2019) · Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3319535.3354208 · Cited by 55 · cited 1×
“Furthermore, we acknowledge that assessing the overall social impacts of an algorithmic system using only harm as a framing device can miss other negative implications of technology – for instance, inconveniences that arise from exercising the right to data portability or the right to be forgotten [30, 38].”
BackgroundSelf-citationAssessing and addressing algorithmic bias in practice
Cramer, H., Garcia-Gathright, J., Springer, A., & Reddy, S. (2018) · Interactions · doi:10.1145/3278156 · Cited by 52 · cited 1×
“Practitioners have sought to develop practices that better identify and minimize sociotechnical harms from algorithmic systems (e.g., [25, 62, 135, 144]).”
BackgroundActivism by the AI Community: Analysing Recent Achievements and Future Prospects
Belfield, H. (2020) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3375627.3375814 · Cited by 52
Nova, F. F., DeVito, M. A., Saha, P., Rashid, K. S., Roy Turzo, S., Afrin, S., & Guha, S. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449231 · Cited by 51 · cited 1×
“Taxonomies have been developed related to online content [19, 193, 227], social media [157, 217, 218], users “at-risk" for experiencing online abuse [216, 234], and mali-”
Background(Re)Politicizing Digital Well-Being: Beyond User Engagements
Docherty, N., & Biega, A. J. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501857 · Cited by 50 · cited 3×
“As constructs of well-being are culturally relative [154], health ideologies operationalized into machine learning models may “not be relevant, or potentially even harmful, to users living differently to the ways assumed by situated designers” [77, p. 5].”
BackgroundUnintended machine learning biases as social barriers for persons with disabilitiess
Hutchinson, B., Prabhakaran, V., Denton, R., Webster, K., Zhong, Y., & Denuyl, S. (2020) · ACM SIGACCESS Accessibility and Computing · doi:10.1145/3386296.3386305 · Cited by 49 · cited 2×
“, facial features, skin tone, or voice), such as computer vision [46, 176], natural language processing [113, 131, 173], and speech recognition systems [124, 143].”
BackgroundTheories of ”gender” in NLP bias research
Devinney, H., Björklund, J., & Björklund, H. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3534627 · Cited by 49 · cited 2×
“Reify-ing essentialist categories can contribute to “existential harm” in which people are “portrayed in overly reductive terms” [186, p. 162], often from a Western or Eurocentric perspective [69].”
BackgroundWright, C., Williams, P., Elizarova, O., Dahne, J., Bian, J., Zhao, Y., & Tan, A. S. L. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-045445 · Cited by 48 · cited 1×
“Information harms are often accompanied by co-occurring impacts, including physical, psychological or emotional, financial, and reputational harms [3, 218, 233, 240], which scale into broader societal harm.”
BackgroundThe Scoping Review Method: Mapping the Literature in “Structural Change” Public Health Interventions
Hanneke, R., Asada, Y., Lieberman, L., Neubauer, L. C., & Fagen, M. (2017) · doi:10.4135/9781473999008 · Cited by 47 · cited 2×
“” Within scoping review methodology, keyword search strategies should be devised to surface relevant literature [11] and include terms common to the field [101].”
BackgroundResponsible Language Technologies: Foreseeing and Mitigating Harms
Blodgett, S. L., Liao, Q. V., Olteanu, A., Mihalcea, R. F., Muller, M. J., Scheuerman, M. K., Tan, C., & Yang, Q. (2022) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3491101.3516502 · Cited by 47
Ramesh, D., Kameswaran, V., Wang, D., & Sambasivan, N. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533237 · Cited by 47 · cited 2×
“Here, privacy violations may reflect more traditional conceptualizations of privacy attacks or security violations [79, 105] and privacy elements beyond what may be protected by regulations or under the traditional purview of a privacy officer [138, 180].”
BackgroundThe Datafication of #MeToo: Whiteness, Racial Capitalism, and Anti-Violence Technologies
Henne, K., Shelby, R. M., & Harb, J. I. (2021) · Big Data & Society · doi:10.1177/20539517211055898 · Cited by 45 · cited 2×
“In this way, algorithmic systems’ enactment of power dynamics [106, 162] can function as a minoritizing practice [64] through which unjust social hierarchies are reinforced.”
BackgroundSelf-citationNeumann, T., De‐Arteaga, M., & Fazelpour, S. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533205 · Cited by 44 · cited 1×
“Misinformation refers to the spread of mis-leading information whether or not there is intention to deceive and disinformation is deliberately false information [153, 211, 219, 232].”
BackgroundQuantifying Toxicity and Verbal Violence on Twitter
Guberman, J., Schmitz, C., & Hemphill, L. (2016) · Proceedings of the 19th ACM Conference on Computer Supported Cooperative Work and Social Computing Companion · doi:10.1145/2818052.2869107 · Cited by 42 · cited 2×
“Beyond gender violence, other facets of technology-facilitated violence, include doxxing [79], trolling [14], cyberstalking [14], cyberbullying [14, 98, 204], monitoring and control [44], and online harassment and intimidation [98, 192, 199, 226], under the broader banner of online toxicity [98,…”
BackgroundCultural Incongruencies in Artificial Intelligence
Prabhakaran, V., Qadri, R., & Hutchinson, B. (2022) · arXiv · doi:10.48550/arxiv.2211.13069 · Cited by 42 · cited 1×
“ence of digital technologies cannot be separated from cultural and social dynamics [7, 60, 172, 175].”
BackgroundBedford, L., Mann, M., Foth, M., & Walters, R. (2022) · International Journal for Crime Justice and Social Democracy · doi:10.5204/ijcjsd.2191 · Cited by 41 · cited 4×
“Similar to other sociotechnical harms, the “benefits and burdens of extractivism are unevenly distributed around the planet” whereby consumption in the economic core are contingent on extraction from the economic periphery [24, p. 170].”
BackgroundInfluentialEhsan, U., Singh, R., Metcalf, J., & Riedl, M. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533186 · Cited by 40 · cited 1×
“For instance, privacy violations may arise from algorithmic systems making predictive inference beyond what users openly disclose [222] or when data collected and algorithmic inferences made about people in one context is applied to another without the person’s knowledge or consent through big data flows [138], even a…”
BackgroundMeasuring Representational Harms in Image Captioning
Wang, A., Barocas, S., Laird, K., & Wallach, H. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533099 · Cited by 40 · cited 4×
“Three of these harm types — allocative, representational, and quality-of-service — reflect where there was strong definitional and terminological consensus in pioneering responsible ML literature (see: [20, 63, 228]).”
BackgroundInfluentialCreating a taxonomic tool for technocracy and applying it to Silicon Valley
Sadowski, J., & Selinger, E. (2014) · Technology in Society · doi:10.1016/j.techsoc.2014.05.001 · Cited by 38 · cited 4×
“Reify-ing essentialist categories can contribute to “existential harm” in which people are “portrayed in overly reductive terms” [186, p. 162], often from a Western or Eurocentric perspective [69].”
BackgroundInfluentialRedistribution and Rekognition
West, S. M. (2020) · Catalyst Feminism Theory Technoscience · doi:10.28968/cftt.v6i2.33043 · Cited by 38
Governance, Risk, and Artificial Intelligence
Mannes, A. (2020) · AI Magazine · doi:10.1609/aimag.v41i1.5200 · Cited by 37
Dynamics of Social Harms in an Algorithmic Context
Malik, H. M., Viljanen, M., Lepinkäinen, N., & Alvesalo‐Kuusi, A. (2022) · International Journal for Crime Justice and Social Democracy · doi:10.5204/ijcjsd.2141 · Cited by 34 · cited 4×
“Loss of agency occurs when the use [123, 137] or abuse [142] of algorithmic systems reduces autonomy.”
BackgroundInfluentialKorman, D. Z., Mack, E. A., Jett, J., & Renear, A. H. (2018) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24007 · Cited by 33 · cited 1×
“This taxonomy enables what linguists call semantic “entailment” through which clear relationships are established between concepts [125], even though it draws together insights from different epistemological standpoints [103, 104].”
BackgroundSpeaking from Experience: Trans/Non-Binary Requirements for Voice-Activated AI
Rincón, C., Keyes, O., & Cath, C. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449206 · Cited by 33
Shelby, R. (2023) · Science, Technology, & Human Values · doi:10.1177/01622439211046047 · Cited by 33
Self-citationDesigning for the Bittersweet: Improving Sensitive Experiences with Recommender Systems
Lustig, C., Konrad, A., & Brubaker, J. R. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502049 · Cited by 33
Rismani, S., Shelby, R., Smart, A., Jatho, E., Kroll, J., Moon, A., & Rostamzadeh, N. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581407 · Cited by 33
Self-citationThe Three Harms of Gendered Technology
Wang, L. (2020) · AJIS. Australasian journal of information systems/AJIS. Australian journal of information systems/Australian journal of information systems · doi:10.3127/ajis.v24i0.2799 · Cited by 32 · cited 4×
“This includes work to taxonomize harms in HCI on digital safety [3, 193, 216], in sociolegal studies on technology-facilitated violence [109, 231], and canonical responsible ML research on representational, allocative [20], and quality-of-service harms [36] that have significantly shaped the…”
BackgroundInfluentialBeyond Checklist Approaches to Ethics in Design
Wong, R. Y., Boyd, K., Metcalf, J., & Shilton, K. (2020) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3406865.3418590 · Cited by 31 · cited 2×
“Structured frameworks can aid practitioners’ anticipation of harms throughout the product lifecycle [135, 239].”
BackgroundMisinformation Harms: A Tale of Two Humanitarian Crises
Tran, T., Valecha, R., Rad, P., & Rao, H. R. (2020) · IEEE Transactions on Professional Communication · doi:10.1109/tpc.2020.3029685 · Cited by 31 · cited 3×
“Taxonomies have been developed related to online content [19, 193, 227], social media [157, 217, 218], users “at-risk" for experiencing online abuse [216, 234], and malicious uses of algorithmic systems [45], including cyber attacks [3] and cyberbullying [12].”
BackgroundPruksachatkun, Y., Krishna, S., Dhamala, J., Gupta, R., & Chang, K.-W. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.294 · Cited by 31
Monroe‐White, T. (2021) · ACM SIGCPR Conference on Computer Personnel Research (SIGCPR) · doi:10.1145/3458026.3462161 · Cited by 24 · cited 1×
“been framed as “data harms” [149], which encompass the adverse effects of data that “impair, injure, or set back a person, entity, or”
BackgroundArtificial Intelligence and Its Contexts: Security, Business and Governance
(2021) · Advanced sciences and technologies for security applications · doi:10.1007/978-3-030-88972-2 · Cited by 23 · cited 1×
“of war [188], and enact surveillant regimes that disproportionately target and harm people of color [120].”
BackgroundCyber Harm: Concepts, Taxonomy and MeasurementPreprint
Agrafiotis, I., Bada, M., Cornish, P., Creese, S., Goldsmith, M. H., Ignatuschtschenko, E., Roberts, T., & Upton, D. M. (2016) · SSRN Electronic Journal · doi:10.2139/ssrn.2828646 · Cited by 22 · cited 2×
“Cultural harm has been described as the development or use of algorithmic systems that affects cultural stability and safety, such as “loss of communication means, loss of cultural property, and harm to social values” [4, p. 30].”
BackgroundAxes for Sociotechnical Inquiry in AI Research
Dean, S., Gilbert, T. K., Lambert, N., & Zick, T. (2021) · IEEE Transactions on Technology and Society · doi:10.1109/tts.2021.3074097 · Cited by 22 · cited 2×
“…the myriad sources of this harm including from algorithmic behavioral exploitation [18, 209], emotional manipulation [202] whereby algorithmic designs exploit user behavior, safety failures involving algorithms (e.g., collisions) [67], and when systems make incorrect health inferences [158].”
BackgroundA Framework for Deprecating Datasets: Standardizing Documentation, Identification, and Communication
Luccioni, A. S., Corry, F., Sridharan, H., Ananny, M., Schultz, J., & Crawford, K. (2021) · arXiv · doi:10.48550/arxiv.2111.04424 · Cited by 22 · cited 2×
“of their identity on their own terms [59].”
BackgroundTransgender People’s Technology Needs to Support Health and Transition
Chuanromanee, T., & Metoyer, R. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445276 · Cited by 20 · cited 1×
“marginalized and excluded from technology development [56].”
Overcoming Failures of Imagination in AI Infused System Development and Deployment
Boyarskaya, M., Olteanu, A., & Crawford, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.13416 · Cited by 20 · cited 8×
“In alignment with prior calls to anticipate computational and contextual harms [40, 151], we synthesize insights on harms from”
BackgroundInfluentialGendered language and employment Web sites: How search algorithms can cause allocative harm
van Es, K. F., Everts, D., & Muis, I. (2021) · First Monday · doi:10.5210/fm.v26i8.11717 · Cited by 19 · cited 2×
“In the employment domain, recommender or ranking systems that match employers and potential candidates may prioritize the resumes of men over other genders [201, 224].”
BackgroundHow Colonial Continuities Underlie Algorithmic Injustices in Education
Karumbaiah, S., & Brooks, J. (2021) · Research on Equity and Sustained Participation in Engineering, Computing, and Technology (RESPECT) · doi:10.1109/respect51740.2021.9620605 · Cited by 17 · cited 1×
“These harms occur when a sys-tem withholds information, opportunities, or resources [22] from historically marginalized groups in domains that affect material well-being [146], such as housing [47], employment [201], social services [15, 201], finance [117], education [119], and healthcare [158].”
BackgroundSkirpan, M. W., & Fiesler, C. L. (2017) · International ACM SIGGROUP Conference on Supporting Group Work (GROUP) · doi:10.1145/3148330.3149407 · Cited by 16 · cited 1×
“corpus attribute the myriad sources of this harm including from algorithmic behavioral exploitation [18, 209], emotional manipulation [202] whereby algorithmic designs exploit user behavior, safety failures involving algorithms (e.”
Background“The Continuum of Harm” Taxonomy of Cyberbullying Mitigation and Prevention
Ashktorab, Z. (2018) · Human-computer interaction series · doi:10.1007/978-3-319-78583-7_9 · Cited by 15 · cited 1×
“cious uses of algorithmic systems [45], including cyber attacks [3] and cyberbullying [12].”
BackgroundPassword Management: How Secure Is Your Login Process?
Hatzivasilis, G. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-62433-0_10 · Cited by 13
Research works printed but not linked: 41
- 10.1093/cybsec · doi:10.1093/cybsec
- 10.18653/v1 · doi:10.18653/v1
- Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2016. Machine Bias. In Ethics of Data and Analytics. Auerbach Publications, Boca Raton, FL, USA, 254–264.
- Jane Bailey, Jacquelyn Burkell, Suzie Dunn, Chandell Gosse, and Valerie Steeves. 2021. AI and Technology-Facilitated Violence and Abuse. In Artificial Intelligence and the Law in Canada, Florian Martin-Bariteau and Teresa Scassa (Eds.). LexisNexis, Toronto, Canada, Chapter 10, 1–15.
- Jane Bailey, Nicola Henry, and Asher Flynn. 2021. Technology-Facilitated Violence and Abuse: International Perspectives and Experiences. In The Emerald International Handbook of Technology-Facilitated Violence and Abuse, Bailey Jane, Flynn Asher, and Henry Nicola (Eds.). Emerald Publishing Limited, Bingley, UK, 1–17.
- 10.18653/v1 · doi:10.18653/v1
- Solon Barocas, Kate Crawford, Aaron Shapiro, and Hanna Wallach. 2017. The Problem with Bias: Allocative Versus Representational Harms in Machine Learning. In 9th Annual Conference of the Special Interest Group for Computing, Information and Society. Society for the History of Technology, Philadelphia, PA, USA.
- Samy Bengio Alina Beygelzimer Kate Crawford Jeanne Fromer Iason Gabriel Amanda Levendowski Deborah Raji and Marc’Aurelio Ranzato. 2022. Provisional Draft of the NeurIPS Code of Ethics. https://openreview.net/pdf?id=zVoy8kAFKPr
- 10.1177/205630511667248 · doi:10.1177/205630511667248
- 10.18653/v1 · doi:10.18653/v1
- 10.5555/380681 · doi:10.5555/380681
- Virginia Braun and Victoria Clarke. 2021. One Size Fits All? What Counts as Quality Practice in (Reflexive) Thematic Analysis?Qualitative research in psychology 18, 3 (2021), 328–352.
- Joy Buolamwini and Timnit Gebru. 2018. Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification. In Proceedings of the 1st Conference on Fairness, Accountability and Transparency(Proceedings of Machine Learning Research, Vol. 81), Sorelle A. Friedler and Christo Wilson (Eds.). PMLR, New York, NY, USA, 77–91. https://proceedings.mlr.press/v81/buolamwini18a.html
- Jacquelyn Burkell and Jane Bailey. 2018. Unlawful Distinctions? Canadian Human Rights Law and Algorithmic Bias. Vol. 2016/2018. Human Rights Research and Education Centre Ottawa, Ottawa, Canada, 217–230.
- Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017. Semantics Derived Automatically from Language Corpora Contain Human-like Biases. Science 356, 6334 (2017), 183–186.
- Kate Crawford. 2017. Keynote: The Trouble with Bias. 2017 Conference on Neural Information Processing Systems, invited speaker.
- Catherine D’Ignazio and Lauren F Klein. 2020. Data Feminism. MIT Press, Boston, MA, USA.
- Batya Friedman and David G Hendry. 2019. Value Sensitive Design: Shaping Technology with Moral Imagination. MIT Press, Boston, MA, USA.
- Steve Fuller. 1993. Helen E. Longino Science As Social Knowledge: Values and Objectivity in Scientific Inquiry. Princeton: Princeton University Press (1990), xi+ 262 pp., 19.95. Philosophy of Science 60, 2 (1993), 360–362.
- Jan Hadja. 1961. Alienation and Integration of Student Intellectualss. American Sociological Review 26, 5 (1961), 19.
- 10.1177/104973231666534 · doi:10.1177/104973231666534
- 10.1177/146144482095872 · doi:10.1177/146144482095872
- Shawn Janzen, Caroline Orr, and Sara-Jayne Terp. 2022. Cognitive Security and Resilience: A Social Ecological Model of Disinformation and Other Harms with Applications to COVID-19 Vaccine Information Behaviors. In 2nd Workshop Reducing Online Misinformation through Credible Information Retrieval. ROMCIR 2022, Stavanger, Norway, 48–88.
- Jared Katzman, Solon Barocas, Su Lin Blodgett, Kristen Laird, Morgan Klaus Scheuerman, and Hanna Wallach. 2021. Representational Harms in Image Tagging. Beyond Fair Computer Vision Workshop at CVPR 2021 (2021).
- Hank Liao, Golan Pundak, Olivier Siohan, Melissa Carroll, Noah Coccaro, Qi-Ming Jiang, Tara N Sainath, Andrew Senior, Françoise Beaufays, and Michiel Bacchiani. 2015. Large Vocabulary Automatic Speech Recognition for Children. In Interspeech 2015. International Speech Communication Association, Dresden, Germany, 1–5.
- 10.1093/ojls · doi:10.1093/ojls
- Emanuel Moss and Jacob Metcalf. 2020. Ethics Owners: A New Model of Organizational Responsibility in Data-Driven Technology Companies. https://datasociety.net/wp-content/uploads/2020/09/Ethics-Owners_20200923-DataSociety.pdf
- Emmanuel Moss and Jacob Metcalf. 2022. Data and Society Workshop: The Social Life of Algorithmic Harms. (March 2022).
- Aik Kwang Ng, David YF Ho, Shyh Shin Wong, and Ian Smith. 2003. In Search of the Good Life: A Cultural Odyssey in the East and West. Genetic, Social, and General Psychology Monographs 129, 4 (2003), 317.
- Safiya Umoja Noble. 2016. A Future for Intersectional Black Feminist Technology Studies. Scholar & Feminist Online 13, 3 (2016), 1–8.
- Cathy O’Neil. 2016. Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy. Broadway Books, New York, NY, USA.
- Eva PenzeyMoog and Danielle C Slakoff. 2021. As Technology Evolves, So Does Domestic Violence: Modern-Day Tech Abuse and Possible Solutions. In The Emerald International Handbook of Technology-Facilitated Violence and Abuse. Emerald Publishing Limited, Bingley, UK, 643–662.
- Jasbir K Puar. 2020. “I Would Rather be a Cyborg than a Boddess”: Becoming Intersectional in Assemblage Theory. In Feminist Theory Reader. Routledge, New York, NY, USA, 405–415.
- Henrik Skaug Sætra. 2021. Artificial Intelligence and its Contexts. Springer, Cham, Switzerland, Chapter A Typology of AI Applications in Politics, 27–43.
- Katie Shilton. 2015. "That’s Not An Architecture Problem!": Techniques and Challenges for Practicing Anticipatory Technology Ethics. In iConference 2015 Proceedings. iSchools Urbana-Champaign Illinois USA 1–7.
- Rebecca Kelly Slaughter, Janice Kopec, and Mohamad Batal. 2020. Algorithms and Economic Justice: A Taxonomy of Harms and a Path Forward for the Federal Trade Commission. Yale Journal of Law & Tech. 23 (2020), 1.
- 10.1177/0002716222108470 · doi:10.1177/0002716222108470
- 10.18653/v1 · doi:10.18653/v1
- Thi Tran, Rohit Valecha, Paul Rad, and H Raghav Rao. 2019. An Investigation of Misinformation Harms Related to Social Media During Humanitarian Crises. In International Conference on Secure Knowledge Management in Artificial Intelligence Era. Springer, Goa, India, 167–181.
- Zeynep Tufekci. 2015. Algorithmic Harms Beyond Facebook and Google: Emergent Challenges of Computational Agency. Colo. Tech. LJ 13 (2015), 203.
- Claire Wardle and Hossein Derakhshan. 2017. Information Disorder: Toward an Interdisciplinary Framework for Research and Policymaking.
Cited by
Scaling Instruction-Finetuned Language Models
Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang, X., Dehghani, M., Brahma, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2210.11416 · Cited by 1,441
Gemini: A Family of Highly Capable Multimodal Models
Gemini Team, Anil, R., Borgeaud, S., Alayrac, J.-B., Yu, J., Soricut, R., Schalkwyk, J., Dai, A. M., Hauth, A., Millican, K., et al. (2023) · arXiv · doi:10.48550/arxiv.2312.11805 · Cited by 990
Anil, R., Dai, A. M., Firat, O., Johnson, M., Lepikhin, D., Passos, A., Shakeri, S., Taropa, E., Bailey, P., Chen, Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.10403 · Cited by 834
Foundational Challenges in Assuring Alignment and Safety of Large Language Models
Anwar, U., Saparov, A., Ramírez, J. R., Paleka, D., Turpin, M., Hase, P., Lubana, E. S., Jenner, E., Casper, S., Sourbut, O., et al. (2024) · arXiv (Cornell University) · doi:10.3929/ethz-b-000706962 · Cited by 17
Evaluating the Social Impact of Generative AI Systems in Systems and Society
Solaiman, I., Talat, Z., Agnew, W. S., Ahmad, L., Baker, D., Blodgett, S. L., Chen, C., Daumé, H., Dodge, J., Duan, I., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.05949 · Cited by 42
Metadata
- Authors
- Shelby, R., Rismani, S., Henne, K., Moon, A., Rostamzadeh, N., Nicholas, P., Yilla-Akbari, N., Gallegos, J., Smart, A., Garcia, E., & Virk, G.
- Affiliations
- Google (United States) (US)
- McGill University (CA)
- Australian National University (AU)
- Google, USA
- ORCID
- Venue
- Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society
- Publisher
- ACM
- Published
- 2023-08-08
- Pages
- 723-741
- ISSN
- 3065-8365 (ISSN-L)
- OpenAlex
- W4386249234
- Cited by
- 279
- References
- 191
- Citations by year
- 2026: 97 · 2025: 101 · 2024: 65 · 2023: 15
- FWCI
- 69.56
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Proceedings of the AAAI/ACM Conference on AI Ethics and Society (publishedVersion)
- HAL (submittedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Law › Artificial Intelligence in Law
- Keywords
- algorithmic harms, harm reduction, algorithmic systems, representational harm, interpersonal harm, social harm, sociocultural dynamics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.