Cited by 330Ethics and Social Impacts of AINatural Language Processing TechniquesArtificial Intelligence in Law

The DOI matches its Crossref record; 53 of the 53 references it cites were found in a registry or an index.

The Social Impact of Natural Language Processing

Open access

Hovy, Dirk, Spruit, Shannon L.·2016·Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)

DOI 10.18653/v1/p16-2096

Open this paper in CoChat — free account Open PDF

Abstract

Medical sciences have long since established an ethics code for experiments, to minimize the risk of harm to subjects.Natural language processing (NLP) used to involve mostly anonymous corpora, with the goal of enriching linguistic analysis, and was therefore unlikely to raise ethical concerns.As NLP becomes increasingly wide-spread and uses more data from social media, however, the situation has changed: the outcome of NLP experiments and applications can now have a direct effect on individual users' lives.Until now, the discourse on this topic in the field has not followed the technological development, while public discourse was often focused on exaggerated dangers.This position paper tries to take back the initiative and start a discussion.We identify a number of social implications of NLP and discuss their ethical significance, as well as ways to address them.

Integrity signals

Registry

Crossref · 10.18653/v1/p16-2096

Cites

53 of 53 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Principles of Biomedical Ethics
(2012) · Thieves of Virtue · doi:10.7551/mitpress/9079.003.0009 · Cited by 13,453
Availability: A heuristic for judging frequency and probability
Tversky, A., & Kahneman, D. (1973) · Cognitive Psychology · doi:10.1016/0010-0285(73)90033-9 · Cited by 9,984 · cited 1×
“Such topic overexposure may lead to a psychological effect called availability heuristic (Tversky and Kahneman, 1973): if people can recall a certain event, or have knowledge about specific things, they infer it must be more important.”
Background
Reproduction in Education, Society and Culture
Erben, M., Bourdieu, P., & Passeron, J.-C. (1979) · British Journal of Sociology · doi:10.2307/589547 · Cited by 8,141
Indexical order and the dialectics of sociolinguistic life
Silverstein, M. C. (2003) · Language & Communication · doi:10.1016/s0271-5309(03)00013-2 · Cited by 3,737
Experimental evidence of massive-scale emotional contagion through social networks
Kramer, A. D. I., Guillory, J. E., & Hancock, J. T. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1320040111 · Cited by 3,687
The Affect Heuristic
Slovic, P., Finucane, M., Peters, E., & MacGregor, D. G. (2002) · Heuristics and Biases · doi:10.1017/cbo9780511808098.025 · Cited by 2,349
The Normative Structure of Science
Merton, R. Κ. (1973) · Medical Entomology and Zoology · Cited by 1,625
Models of ecological rationality: The recognition heuristic.
Goldstein, D. G., & Gigerenzer, G. (2002) · Psychological Review · doi:10.1037/0033-295x.109.1.75 · Cited by 1,352
Voice, Footing, Enregisterment
Agha, A. (2005) · Journal of Linguistic Anthropology · doi:10.1525/jlin.2005.15.1.38 · Cited by 1,326
Finding Deceptive Opinion Spam by Any Stretch of the Imagination
Ott, M., Choi, Y., Cardie, C., & Hancock, J. T. (2011) · arXiv · doi:10.48550/arxiv.1107.4557 · Cited by 1,008

Show all 53 references

Sorted by citations

Language (Technology) is Power: A Critical Survey of “Bias” in NLP
Blodgett, S. L., Barocas, S., Daumé III, H., & Wallach, H. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.485 · Cited by 1,050
“A.2 Full categorization: Motivations Allocational harms Hovy and Spruit (2016); Caliskan et al. (2017); Madnani et al. (2017); Dixon et al. (2018); Kiritchenko and Mohammad (2018); Shen et al. (2018); Zhao et al. (2018b); Bhaskaran and Bhallamudi (2019); Bordia and Bowman (2019); Brunet et al.…”
BackgroundInfluential
Data Statements for Natural Language Processing: Toward Mitigating System Bias and Enabling Better Science
Bender, E. M., & Friedman, B. (2018) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00041 · Cited by 854
“, 2017) and papers (Hovy and Spruit, 2016) in NLP, as well as workshops in related fields, notably the FATML series (http://www.”
BackgroundInfluential
Social Data: Biases, Methodological Pitfalls, and Ethical Boundaries
Olteanu, A., Castillo, C., Diaz, F., & Kıcıman, E. (2019) · Frontiers in Big Data · doi:10.3389/fdata.2019.00013 · Cited by 763
“Thus, data, tools, and inferences obtained for one purpose can be used for another purpose (Hovy and Spruit, 2016; Benton et al., 2017); yet, the risks associated with secondary uses may not be well understood.”
Background
Taxonomy of Risks posed by Language Models
Weidinger, L., Uesato, J., Rauh, M., Griffin, C., Huang, P.-S., Mellor, J., Glaese, A., Cheng, M., Balle, B., Kasirzadeh, A., et al. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533088 · Cited by 764
“Choices on model architecture may also have an impact, as it has been proposed that the architecture of current LMs and particularly tokenisation is well-suited to English, but not morphologically more complex languages [14, 79, 162].”
Background
Measuring and Mitigating Unintended Bias in Text Classification
Dixon, L., Li, J., Sorensen, J., Thain, N., & Vasserman, L. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278729 · Cited by 674
Mitigating Gender Bias in Natural Language Processing: Literature Review
Sun, T., Gaut, A., Tang, S., Huang, Y., ElSherief, M., Zhao, J., Mirza, D., Belding, E., Chang, K.-W., & Wang, W. Y. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1159 · Cited by 492
Ethical and social risks of harm from Language Models
Weidinger, L., Mellor, J., Rauh, M., Griffin, C., Uesato, J., Huang, P.-S., Cheng, M., Glaese, M., Balle, B., Kasirzadeh, A., et al. (2021) · arXiv · doi:10.48550/arxiv.2112.04359 · Cited by 477
Gender Bias in Coreference Resolution
Rudinger, R., Naradowsky, J., Leonard, B., & Van Durme, B. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2002 · Cited by 443
Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systems
Kiritchenko, S., & Mohammad, S. (2018) · Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics · doi:10.18653/v1/s18-2005 · Cited by 384
Depression and Self-Harm Risk Assessment in Online Forums
Yates, A., Cohan, A., & Goharian, N. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1322 · Cited by 390
Gender and Dialect Bias in YouTube’s Automatic Captions
Tatman, R. (2017) · Proceedings of the First ACL Workshop on Ethics in Natural Language Processing · doi:10.18653/v1/w17-1606 · Cited by 338
Beneath the Tip of the Iceberg: Current Challenges and New Directions in Sentiment Analysis Research
Poria, S., Hazarika, D., Majumder, N., & Mihalcea, R. F. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.3038167 · Cited by 362
Five sources of bias in natural language processing
Hovy, D., & Prabhumoye, S. (2021) · Language and Linguistics Compass · doi:10.1111/lnc3.12432 · Cited by 298
Self-citation
Utilizing Neural Networks and Linguistic Metadata for Early Detection of Depression Indications in Text Sequences
Trotzek, M., Koitka, S., & Friedrich, C. M. (2018) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2018.2885515 · Cited by 312
Demographic dialectal variation in social media: a case study of African-American English
Blodgett, S. L., Green, L., & O'Connor, B. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1120 · Cited by 291
WILDS: A Benchmark of in-the-Wild Distribution Shifts
Koh, P. W., Sagawa, S., Marklund, H., Xie, S. M., Zhang, M., Balsubramani, A., Hu, W., Yasunaga, M., Phillips, R. L., Gao, I., et al. (2020) · arXiv · doi:10.48550/arxiv.2012.07421 · Cited by 518
“Such demographic disparities have been widely documented in natural language and speech processing tasks (Hovy and Spruit, 2016), among other areas.”
Social Biases in NLP Models as Barriers for Persons with Disabilities
Hutchinson, B., Prabhakaran, V., Denton, E., Webster, K., Zhong, Y., & Denuyl, S. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.487 · Cited by 274
Predictive biases in natural language processing models: A conceptual framework and overview
Shah, D. S., Schwartz, H. A., & Hovy, D. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.468 · Cited by 241
Self-citation
Towards generalisable hate speech detection: a review on obstacles and solutions
Yin, W., & Zubiaga, A. (2021) · PeerJ Computer Science · doi:10.7717/peerj-cs.598 · Cited by 238
Multitask Learning for Mental Health Conditions with Limited Social Media Data
Benton, A., Mitchell, M., & Hovy, D. (2017) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/e17-1015 · Cited by 176
Self-citation
Deep Learning for Text Style Transfer: A Survey
Jin, D., Jin, Z., Hu, Z., Vechtomova, O., & Mihalcea, R. F. (2021) · Computational Linguistics · doi:10.1162/coli_a_00426 · Cited by 217
Who is the "Human" in Human-Centered Machine Learning
Chancellor, S., Baumer, E. P. S., & Choudhury, M. D. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359249 · Cited by 192
Gender Bias in Machine Translation
Savoldi, B., Gaido, M., Bentivogli, L., Negri, M., & Turchi, M. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00401 · Cited by 229
Detection of Fake News Text Classification on COVID-19 Using Deep Learning Approaches
Bangyal, W. H., Qasim, R., Rehman, N. U., Ahmad, Z., Dar, H. S., Rukhsar, L., Aman, Z., & Ahmad, J. (2021) · Computational and Mathematical Methods in Medicine · doi:10.1155/2021/5514220 · Cited by 149
The Importance of Modeling Social Factors of Language: Theory and Practice
Hovy, D., & Yang, D. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.49 · Cited by 151
Self-citation
Content Analysis of Textbooks via Natural Language Processing: Findings on Gender, Race, and Ethnicity in Texas U.S. History Textbooks
Lucy, L., Demszky, D., Bromley, P., & Jurafsky, D. (2020) · AERA Open · doi:10.1177/2332858420940312 · Cited by 119
A Time-Aware Transformer Based Model for Suicide Ideation Detection on Social Media
Sawhney, R., Joshi, H., Gandhi, S., & Shah, R. R. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.619 · Cited by 122
It’s Morphin’ Time! Combating Linguistic Discrimination with Inflectional Perturbations
Tan, S., Joty, S., Kan, M., & Socher, R. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.263 · Cited by 111
Reducing Gender Bias in Neural Machine Translation as a Domain Adaptation Problem
Saunders, D., & Byrne, B. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.690 · Cited by 100
Racial Disparity in Natural Language Processing: A Case Study of Social Media African-American English
Blodgett, S. L., & O'Connor, B. (2017) · arXiv · doi:10.48550/arxiv.1707.00061 · Cited by 153
SMHD: A Large-Scale Resource for Exploring Online Language Usage for Multiple Mental Health ConditionsPreprint
Cohan, A., Desmet, B., Yates, A., Soldaini, L., MacAvaney, S., & Goharian, N. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.05258 · Cited by 142
Social Bias in Elicited Natural Language Inferences
Rudinger, R., May, C., & Durme, B. V. (2017) · doi:10.18653/v1/w17-1609 · Cited by 98
Privacy-preserving Neural Representations of Text
Coavoux, M., Narayan, S., & Cohen, S. B. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1001 · Cited by 126
Stereotypical Bias Removal for Hate Speech Detection Task using Knowledge-based Generalizations
Badjatiya, P., Gupta, M., & Varma, V. (2019) · International World Wide Web Conference (WWW) · doi:10.1145/3308558.3313504 · Cited by 97
HONEST: Measuring Hurtful Sentence Completion in Language Models
Nozza, D., Bianchi, F., & Hovy, D. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.191 · Cited by 95
Self-citation
The Gradient of Generative AI Release: Methods and Considerations
Solaiman, I. (2023) · 2023 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3593013.3593981 · Cited by 90
Fair Is Better than Sensational: Man Is to Doctor as Woman Is to Doctor
Nissim, M., van Noord, R., & van der Goot, R. (2020) · Computational Linguistics · doi:10.1162/coli_a_00379 · Cited by 103
Multi-Dimensional Gender Bias Classification
Dinan, E., Fan, A., Wu, L., Weston, J., Kiela, D., & Williams, A. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.23 · Cited by 117
Obfuscating Gender in Social Media Writing
Reddy, S., & Knight, K. K. (2016) · Workshop on Natural Language Processing and Computational Social Science (NLP+CSS) · doi:10.18653/v1/w16-5603 · Cited by 86
Multi-Task Learning for Mental Health using Social Media TextPreprint
Benton, A., Mitchell, M., & Hovy, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.03538 · Cited by 132
Self-citation
SoulChat: Improving LLMs’ Empathy, Listening, and Comfort Abilities through Fine-tuning with Multi-turn Empathy Conversations
Chen, Y., Xing, X., Lin, J., Zheng, H., Wang, Z., Liu, Q., & Xu, X. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.83 · Cited by 118
GAIA: A Fine-grained Multimedia Knowledge Extraction System
Li, M., Zareian, A., Lin, Y., Pan, X., Whitehead, S., Chen, B., Wu, B., Ji, H., Chang, S., Voss, C. R., et al. (2020) · Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) · doi:10.18653/v1/2020.acl-demos.11 · Cited by 81
Uneven success: automatic speech recognition and ethnicity-related dialects
Wassink, A. B., Gansen, C., & Bartholomew, I. (2022) · Speech Communication · doi:10.1016/j.specom.2022.03.009 · Cited by 77
Incorporating Dialectal Variability for Socially Equitable Language Identification
Jurgens, D., Tsvetkov, Y., & Jurafsky, D. (2017) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p17-2009 · Cited by 108
Abstractive text summarization: State of the art, challenges, and improvements
Shakil, H., Farooq, A., & Kalita, J. K. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128255 · Cited by 100
Ethical by Design: Ethics Best Practices for Natural Language Processing
Leidner, J. L., & Plachouras, V. (2017) · doi:10.18653/v1/w17-1604 · Cited by 75
PsyQA: A Chinese Dataset for Generating Long Counseling Text for Mental Health Support
Sun, H., Lin, Z., Zheng, C., Liu, S., & Huang, M. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.130 · Cited by 78
Exposing the limits of Zero-shot Cross-lingual Hate Speech Detection
Nozza, D. (2021) · doi:10.18653/v1/2021.acl-short.114 · Cited by 74
Glottocodes: Identifiers linking families, languages and dialects to comprehensive reference information
Forkel, R., & Hammarström, H. (2022) · Semantic Web · doi:10.3233/sw-212843 · Cited by 67
Evaluation Gaps in Machine Learning Practice
Hutchinson, B., Rostamzadeh, N., Greer, C., Heller, K., & Prabhakaran, V. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533233 · Cited by 97
Bias in Automatic Speech Recognition: The Case of African American Language
Martin, J., & Wright, K. E. (2022) · Applied Linguistics · doi:10.1093/applin/amac066 · Cited by 71
Unmasking Contextual Stereotypes: Measuring and Mitigating BERT's Gender Bias
Bartl, M., Nissim, M., & Gatt, A. (2020) · arXiv · doi:10.48550/arxiv.2010.14534 · Cited by 84
Beyond Fair Pay: Ethical Implications of NLP Crowdsourcing
Shmueli, B., Fell, J., Ray, S., & Ku, L. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.295 · Cited by 67
The many dimensions of algorithmic fairness in educational applications
Loukina, A., Madnani, N., & Zechner, K. (2019) · Workshop on Innovative Use of NLP for Building Educational Applications · doi:10.18653/v1/w19-4401 · Cited by 64
Detecting Hate Speech with GPT-3Preprint
Chiu, K.-L., Collins, A., & Alexander, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.12407 · Cited by 148
Equity Beyond Bias in Language Technologies for Education
Mayfield, E., Madaio, M., Prabhumoye, S., Gerritsen, D., McLaughlin, B., Dixon-Román, E., & Black, A. W. (2019) · Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications · doi:10.18653/v1/w19-4446 · Cited by 62
Automated Evaluation of Writing – 50 Years and Counting
Klebanov, B. B., & Madnani, N. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.697 · Cited by 60
A Transparent Framework for Evaluating Unintended Demographic Bias in Word Embeddings
Sweeney, C., & Najafian, M. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1162 · Cited by 58
Finding the traces of behavioral and cognitive processes in big data and naturally occurring datasets
Paxton, A., & Griffiths, T. L. (2017) · Behavior Research Methods · doi:10.3758/s13428-017-0874-x · Cited by 57
PHASE: Learning Emotional Phase-aware Representations for Suicide Ideation Detection on Social Media
Sawhney, R., Joshi, H., Flek, L., & Shah, R. R. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.205 · Cited by 59
Returning the N to NLP: Towards Contextually Personalized Classification Models
Flek, L. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.700 · Cited by 59
Towards Ordinal Suicide Ideation Detection on Social Media
Sawhney, R., Joshi, H., Gandhi, S., & Shah, R. R. (2021) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3437963.3441805 · Cited by 58
What to do about non-standard (or non-canonical) language in NLPPreprint
Plank, B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.07836 · Cited by 59
The subtle language of exclusion: Identifying the Toxic Speech of Trans-exclusionary Radical Feminists
Lu, C., & Jurgens, D. (2022) · Workshop on Online Abuse and Harms (WOAH) · doi:10.18653/v1/2022.woah-1.8 · Cited by 53
It’s Just a Matter of Time: Detecting Depression with Time-Enriched Multimodal Transformers
Bucur, A.-M., Cosma, A., Rosso, P., & Dinu, L. P. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-28244-7_13 · Cited by 58
Agreeing to Disagree: Annotating Offensive Language Datasets with Annotators’ Disagreement
Leonardelli, E., Menini, S., Aprosio, A. P., Guerini, M., & Tonelli, S. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.822 · Cited by 51
How AI tools can—and cannot—help organizations become more ethical
Cremer, D. D., & Narayanan, D. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1093712 · Cited by 64
TextHide: Tackling Data Privacy in Language Understanding Tasks
Huang, Y., Song, Z., Chen, D., Li, K., & Arora, S. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.123 · Cited by 55
Suicide Ideation Detection via Social and Temporal User Representations using Hyperbolic Learning
Sawhney, R., Joshi, H., Shah, R. R., & Flek, L. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.176 · Cited by 51
Measuring Harmful Sentence Completion in Language Models for LGBTQIA+ Individuals
Nozza, D., Bianchi, F., Lauscher, A., & Hovy, D. (2022) · Workshop on Language Technology for Equality, Diversity, Inclusion (LT-EDI) · doi:10.18653/v1/2022.ltedi-1.4 · Cited by 52
Self-citation
“You Sound Just Like Your Father” Commercial Machine Translation Systems Include Stylistic Biases
Hovy, D., Bianchi, F., & Fornaciari, T. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.154 · Cited by 52
Self-citation
Empirical Evaluation of Pre-trained Transformers for Human-Level NLP: The Role of Sample Size and Dimensionality
Ganesan, A. V., Matero, M., Ravula, A. R., Vu, H., & Schwartz, H. A. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.357 · Cited by 50
Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling
Dinan, E., Abercrombie, G., Bergman, A. S., Spruit, S., Hovy, D., Boureau, Y.-L., & Rieser, V. (2021) · arXiv · doi:10.48550/arxiv.2107.03451 · Cited by 44
Self-citation
Ethics and Good Practice in Computational Paralinguistics
Batliner, A., Hantke, S., & Schuller, B. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.3021015 · Cited by 30
Embeddings in Natural Language Processing: Theory and Advances in Vector Representations of Meaning
Pilehvar, M. T., & Camacho-Collados, J. (2020) · Synthesis lectures on human language technologies · doi:10.2200/s01057ed1v01y202009hlt047 · Cited by 46
Annotating Online Misogyny
Zeinert, P., Inie, N., & Derczynski, L. (2021) · doi:10.18653/v1/2021.acl-long.247 · Cited by 49
Generative Artificial Intelligence: Fundamentals
Corchado, J. M., Sebastian López, F., V., J., Raul Garcia, S., & Chamoso, P. (2023) · ADCAIJ ADVANCES IN DISTRIBUTED COMPUTING AND ARTIFICIAL INTELLIGENCE JOURNAL · doi:10.14201/adcaij.31704 · Cited by 53
Women’s Syntactic Resilience and Men’s Grammatical Luck: Gender-Bias in Part-of-Speech Tagging and Dependency Parsing
Garimella, A., Banea, C., Hovy, D., & Mihalcea, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1339 · Cited by 41
Self-citation
Dual-use implications of AI text generation
Koplin, J. J. (2023) · Ethics and Information Technology · doi:10.1007/s10676-023-09703-z · Cited by 49
Identifying Depressive Symptoms from Tweets: Figurative Language Enabled Multitask Learning Framework
Yadav, S., Chauhan, J., Sain, J. P., Thirunarayan, K., Sheth, A., & Schumm, J. A. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.61 · Cited by 48
Reducing sentiment polarity for demographic attributes in word embeddings using adversarial learning
Sweeney, C., & Najafian, M. (2020) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3351095.3372837 · Cited by 38
Gender and Racial Fairness in Depression Research using Social Media
Aguirre, C., Harrigian, K., & Dredze, M. H. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.256 · Cited by 41
French CrowS-Pairs: Extending a challenge dataset for measuring social bias in masked language models to a language other than English
Névéol, A., Dupont, Y., Bezançon, J., & Fort, K. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.583 · Cited by 38
Examining Gender and Race Bias in Two Hundred Sentiment Analysis SystemsPreprint
Kiritchenko, S., & Mohammad, S. M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.04508 · Cited by 137
Give Me Convenience and Give Her Death: Who Should Decide What Uses of NLP are Appropriate, and on What Basis?
Leins, K., Lau, J. H., & Baldwin, T. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.261 · Cited by 45
From text to talk: Harnessing conversational corpora for humane and diversity-aware language technology
Dingemanse, M., & Liesenfeld, A. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.385 · Cited by 37
Mitigating Gender Bias in Machine Translation with Target Gender Annotations
Stafanovičs, A., Bergmanis, T., & Pinnis, M. (2020) · Conference on Machine Translation (WMT) · doi:10.18653/v1/2020.wmt-1.73 · Cited by 35
Artificial Intelligence and Bias Towards Marginalised Groups: Theoretical Roots and Challenges
Mergen, A., Çetin-Kiliç, N., & Özbilgin, M. F. (2025) · International perspectives on equality, diversity and inclusion · doi:10.1108/s2051-233320250000012004 · Cited by 38
Mind Your Inflections! Improving NLP for Non-Standard Englishes with Base-Inflection Encoding
Tan, S., Joty, S., Varshney, L. R., & Kan, M. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.455 · Cited by 36
Mitigating Gender Bias in Natural Language Processing: Literature ReviewPreprint
Sun, T., Gaut, A., Tang, S., Huang, Y., ElSherief, M., Zhao, J., Mirza, D., Belding, E., Chang, K.-W., & Wang, W. Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.08976 · Cited by 109
ASTRAEA: Grammar-based Fairness Testing
Soremekun, E., Udeshi, S., & Chattopadhyay, S. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3141758 · Cited by 41
Differential Tweetment
Ball-Burack, A., Lee, M. S. A., Cobbe, J., & Singh, J. P. (2021) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3442188.3445875 · Cited by 38
SafetyKit: First Aid for Measuring Safety in Open-domain Conversational Systems
Dinan, E., Abercrombie, G., Bergman, A., Spruit, S., Hovy, D., Boureau, Y.-L., & Rieser, V. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.284 · Cited by 32
Self-citation
Automatic Fairness Testing of Neural Classifiers Through Adversarial Sampling
Zhang, P., Wang, J., Sun, J., Wang, X., Dong, G., Wang, X., Dai, T., & Dong, J. S. (2021) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2021.3101478 · Cited by 33
Exploratory Data Analysis and Classification of a New Arabic Online Extremism Dataset
Aldera, S., Emam, A. Z., Al‐Qurishi, M., Alrubaian, M., & Alothaim, A. (2021) · IEEE Access · doi:10.1109/access.2021.3132651 · Cited by 35
An Automated Toxicity Classification on Social Media Using LSTM and Word Embedding
Alsharef, A., Aggarwal, K., Sonia, S., Koundal, D., Alyami, H., & Ameyed, D. (2022) · Computational Intelligence and Neuroscience · doi:10.1155/2022/8467349 · Cited by 32
Retracted
Toward Practical Usage of the Attention Mechanism as a Tool for Interpretability
Tutek, M., & Šnajder, J. (2022) · IEEE Access · doi:10.1109/access.2022.3169772 · Cited by 35
Case Study: Deontological Ethics in NLP
Prabhumoye, S., Boldt, B., Salakhutdinov, R., & Black, A. W. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.297 · Cited by 35
Conceptor Debiasing of Word Representations Evaluated on WEAT
Karve, S., Ungar, L., & Sedoc, J. (2019) · Workshop on Gender Bias in Natural Language Processing (GeBNLP) · doi:10.18653/v1/w19-3806 · Cited by 29
On the Machine Learning of Ethical Judgments from Natural Language
Talat, Z., Blix, H., Valvoda, J., Ganesh, M. I., Cotterell, R., & Williams, A. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.56 · Cited by 33

Authors: Hovy, D., & Spruit, S. L.
Affiliations: University of Copenhagen (DK)
ORCID: Hovy, Dirk
Spruit, Shannon L.
Venue: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
Publisher: Association for Computational Linguistics
Published: 2016
Pages: 591-598
ISSN: 0736-587X (ISSN-L)
DOI: 10.18653/v1/p16-2096
OpenAlex: W2511234952
Cited by: 329
References: 53
Citations by year: 2026: 31 · 2025: 39 · 2024: 18 · 2023: 47 · 2022: 45 · 2021: 58 · 2020: 40 · 2019: 16 · 2018: 17 · 2017: 15 · 2016: 3
FWCI: 71.55
Type: conference paper
Language: en
Open access: green
hdl.handle.net
Licence: cc-by
Locations: Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion)
Archivio istituzionale della Ricerca - Bocconi (submittedVersion, cc-by)
Research at the University of Copenhagen (University of Copenhagen) (submittedVersion)
Research Repository (Delft University of Technology) (submittedVersion)
Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Social Sciences › Law › Artificial Intelligence in Law
Keywords: social media data, ethical risks, public discourse on AI

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

The Social Impact of Natural Language Processing

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks