Toward Gender-Inclusive Coreference Resolution: An Analysis of Gender and Bias Throughout the Machine Learning Lifecycle
Open accessThe DOI matches its Crossref record; 329 of the 329 references it cites were found in a registry or an index.
Abstract
Correctly resolving textual mentions of people fundamentally entails making inferences about those people. Such inferences raise the risk of systematic biases in coreference resolution systems, including biases that can harm binary and non-binary trans and cis stakeholders. To better understand such biases, we foreground nuanced conceptualizations of gender from sociology and sociolinguistics, and investigate where in the machine learning pipeline such biases can enter a coreference resolution system. We inspect many existing data sets for trans-exclusionary biases, and develop two new data sets for interrogating bias in both crowd annotations and in existing coreference resolution systems. Through these studies, conducted on English text, we confirm that without acknowledging and building systems that recognize the complexity of gender, we will build systems that fail for: quality of service, stereotyping, and over- or under-representation, especially for binary and non-binary trans users.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“and processes of creating resources and technologies may further entrench such disparities [25, 38, 144].”
From Data Governance in the Age of Large-Scale Data-Driven Language Technology (2022) · cites this paper as [25]
Citations
Cites
Gender Trouble: Feminism and the Subversion of Identity
McIntosh, M. (1991) · Feminist Review · doi:10.1057/fr.1991.33 · Cited by 29,379 · cited 3×
“Following (Butler 1989), in our annotation we considered drag performance as a form of genderfluidity; as such, we annotate the performer and the drag persona as coreferent with each other (as well as the relevant pronouns), akin to how we believed a reasonable model for handling stage names (e.g.,…”
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · cited 3×
“In the case of machine translation, one can ask whether standard evaluation metrics like Bleu (Papineni et al. 2002) are sensitive to misgendering.”
BackgroundInfluentialWEST, C., & ZIMMERMAN, D. H. (1987) · Gender & Society · doi:10.1177/0891243287001002002 · Cited by 8,954
Gender trouble: feminism and the subversion of identity
(1990) · Choice Reviews Online · doi:10.5860/choice.28-1264 · Cited by 7,707
Bleu: a method for automatic evaluation of machine translation
Papineni, K. (2024) · doi:10.57702/ljteqmrb · Cited by 3,672
(2018) · Cited by 3,510
Lakoff, R. (1973) · Language in Society · doi:10.1017/s0047404500000051 · Cited by 3,138
(2005) · Cited by 3,125
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887
The World Atlas of Language Structures Online
Dryer, M. S., Haspelmath, M., & Forkel, R. (2022) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.7135714 · Cited by 2,542
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé, H., & Crawford, K. (2018) · arXiv · doi:10.48550/arxiv.1803.09010 · Cited by 2,474 · cited 2×
“com/TristaCao/into_inclusivecoref with corresponding datasheets (Gebru et al. 2018).”
BackgroundSelf-citationGebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., III, H. D., & Crawford, K. (2021) · Communications of the ACM · doi:10.1145/3458723 · Cited by 2,469 · cited 2×
“com/TristaCao/into_inclusivecoref with corresponding datasheets (Gebru et al. 2018).”
BackgroundSelf-citationA Call for Clarity in Reporting BLEU Scores
Post, M. (2018) · Proceedings of the Third Conference on Machine Translation: Research Papers · doi:10.18653/v1/w18-6319 · Cited by 2,112
A Call for Clarity in Reporting BLEU Scores
Post, M. (2018) · arXiv · doi:10.48550/arxiv.1804.08771 · Cited by 2,111
Jagose, A. R. (1996) · Cited by 1,745
Proceedings of the 40th Annual Meeting on Association for Computational Linguistics
Isabelle, P. (2002) · Cited by 1,635
Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
Bolukbasi, T., Chang, K.-W., Zou, J., Saligrama, V., & Kalai, A. (2016) · arXiv · doi:10.48550/arxiv.1607.06520 · Cited by 1,615 · cited 1×
“…than just in coreference resolution, including, for instance, natural language inference (Rudinger, May, and Van Durme 2017), word embeddings (e.g., Bolukbasi et al. 2016; Romanov et al. 2019; Gonen and Goldberg 2019), sentiment analysis (Kiritchenko and Mohammad 2018), machine translation (Font…”
BackgroundParallel Data, Tools and Interfaces in OPUS
Tiedemann, J. (2012) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/59wvhpiwbv7y · Cited by 1,599 · cited 1×
“Even limiting to just she and he pronouns, the bias is significant: four times as many he than she in Europarl (Koehn 2005) and the Common Crawl (Smith et al. 2013), six times as many in News Commentaries (Tiedemann 2012), and fifty times as many in Hong Kong Laws corpus.”
BackgroundWhipping Girl: A Transsexual Woman on Sexism and the Scapegoating of Femininity
Serano, J. M. (2007) · Cited by 1,537
Parallel Data, Tools and Interfaces in OPUS
Tiedemann, J. (2012) · Language Resources and Evaluation · Cited by 1,421
Scott, A., & Lakoff, R. (1976) · The English Journal · doi:10.2307/814504 · Cited by 1,394
Corbett, G. G. (1991) · doi:10.1017/cbo9781139166119 · Cited by 1,385
AllenNLP: A Deep Semantic Natural Language Processing Platform
Gardner, M., Grus, J., Neumann, M., Tafjord, O., Dasigi, P., Liu, N. F., Peters, M., Schmitz, M., & Zettlemoyer, L. (2018) · Proceedings of Workshop for NLP Open Source Software (NLP-OSS) · doi:10.18653/v1/w18-2501 · Cited by 1,252 · cited 2×
“In particular, we run five coreference resolution systems on our ablated data: the AI2 system (AI2; Gardner et al. 2017), hugging face (HF; Wolf 2017), which is a neural system based on spacy (Honnibal and Montani 2017), and the Stanford deterministic (SfdD; Raghunathan et al. 2010), statistical…”
MethodFriedman, B., & Nissenbaum, H. (1996) · ACM Transactions on Information Systems · doi:10.1145/230538.230561 · Cited by 1,203 · cited 1×
“This can lead to significant biases in coreference resolution systems: cases where systems “systematically and unfairly discriminate against certain individuals or groups of individuals in favor of others” (Friedman and Nissenbaum 1996, p. 332).”
BackgroundBirch, A., & Osborne, M. (2011) · The Association for Computational Linguistics · Cited by 1,200
Mariani, J. (2018) · HAL · Cited by 1,151
Gender: An Ethnomethodological Approach
Kessler, S. J., & McKenna, W. (1978) · Medical Entomology and Zoology · Cited by 1,007
Gender: An Ethnomethodological Approach
Stoneall, L., Kessler, S., & McKenna, W. (1979) · Contemporary Sociology · doi:10.2307/2064613 · Cited by 912 · cited 1×
“…the regressive “female” / “male” binary 8 ; additionally, that one’s gender may shift by time or context (often “genderfluid”), and some people do not experience gender at all (often “agender”) (Kessler and McKenna 1978; Schilt and Westbrook 2009; Darwin 2017; Richards, Bouman, and Barker 2017).”
BackgroundProceedings of the 49th Annual Meeting of the Association for Computational Linguistics
Gimpel, K., Schneider, N., O’Connor, B., Das, D., Mills, D. Q., Eisenstein, J., Heilman, M., Yogatama, D., Flanigan, J., & Smith, N. A. (2011) · The Association for Computational Linguistics · Cited by 890
Levesque, H. J., Davis, E., & Morgenstern, L. (2011) · Cited by 885
Predicting pragmatic reasoning in language games
Frank, M. C., & Goodman, N. D. (2012) · Science · doi:10.1126/science.1218633 · Cited by 868
Bender, E. M., & Friedman, B. (2018) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00041 · Cited by 854
Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints
Zhao, J., Wang, T., Yatskar, M., Ordonez, V., & Chang, K.-W. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1323 · Cited by 829 · cited 1×
“These results echo and extend previous observations made by Zhao et al. (2018a),”
ResultSchilt, K., & Westbrook, L. (2009) · Gender & Society · doi:10.1177/0891243209340034 · Cited by 788 · cited 2×
“Trans-inclusive views further typically recognize that gender exists beyond the regressive “female” / “male” binary 8 ; additionally, that one’s gender may shift by time or context (often “genderfluid”), and some people do not experience gender at all (often “agender”) (Kessler and McKenna 1978; Schilt and Westbrook 20…”
BackgroundDiscrimination in online ad delivery
Sweeney, L. (2013) · Communications of the ACM · doi:10.1145/2447976.2447990 · Cited by 783 · cited 1×
“5 In all cases, we focus largely on harms due to over-and under-representation (Kay, Matuszek, and Munson 2015), replicating stereotypes (Sweeney 2013; Caliskan, Bryson, and Narayanan 2017) (particular those Significant limitations..”
BackgroundSocial Data: Biases, Methodological Pitfalls, and Ethical Boundaries
Olteanu, A., Castillo, C., Diaz, F., & Kıcıman, E. (2019) · Frontiers in Big Data · doi:10.3389/fdata.2019.00013 · Cited by 763 · cited 2×
“This echoes a related finding in annotation of hate-speech that annotator positionality matters (Olteanu et al. 2019).”
BackgroundSocial Data: Biases, Methodological Pitfalls, and Ethical Boundaries
Olteanu, A., Castillo, C., Diaz, F., & Kiciman, E. (2016) · doi:10.2139/ssrn.2886526 · Cited by 763
SemEval-2018 Task 1: Affect in Tweets
Mohammad, S., Bravo-Marquez, F., Salameh, M., & Kiritchenko, S. (2018) · Proceedings of The 12th International Workshop on Semantic Evaluation · doi:10.18653/v1/s18-1001 · Cited by 746
Matsumoto, Y., & Prasad, R. (2016) · International Conference on Computational Linguistics · Cited by 725
Discriminating Gender on Twitter
Burger, J. D., Henderson, J. C., Kim, G., & Zarrella, G. (2011) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · Cited by 720
Event-Related Brain Potentials Elicited by Failure to Agree
Osterhout, L., & Mobley, L. A. (1995) · Journal of Memory and Language · doi:10.1006/jmla.1995.1033 · Cited by 705 · cited 1×
“In neurolinguistic studies, Osterhout and Mobley (1995) and Hagoort and Brown (1999) looked at event-related potential (ERP) violations for reflexive pronouns and antecedent in English, finding similar e ff ects to violations of number agreement, but di ff erent e ff ects from semantic violations.”
BackgroundVanderwende, L., Daumé, H., & Kirchhoff, K. (2013) · North American Chapter of the Association for Computational Linguistics · Cited by 685
Self-citationGender bias in coreference resolution: Evaluation and debiasing methods
Zhao, J., Wang, T., Yatskar, M., Ordonez, V., & Chang, K.-W. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2003 · Cited by 684 · cited 8×
“In comparison to (Zhao et al. 2018a) and (Rudinger et al. 2018) (as well as in contrast to our MAP dataset), we focused on naturally occurring data, but sampled specifically to surface more gender-related phenomena than may be found in, say, the Wall Street Journal.”
BackgroundMethodResultInfluentialA model-theoretic coreference scoring scheme
Vilain, M., Burger, J., Aberdeen, J., Connolly, D., & Hirschman, L. (1995) · Proceedings of the 6th conference on Message understanding - MUC6 '95 · doi:10.3115/1072399.1072405 · Cited by 683 · cited 3×
“In terms of metrics, most intrinsic evaluation is carried out using metrics like MUC (Vilain et al. 1995), ACE (Mitchell et al.”
MethodSchilt, K., & Westbrook, L. (2009) · Language arts journal of Michigan · Cited by 595
Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing
Lapata, M., & Ng, H. T. (2008) · Cited by 567
Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC-2014)
Calzolari, N., Choukri, K., Declerck, T., Loftsson, H., Mægaard, B., Mariani, J., Moreno, A., Odijk, J., & Piperidis, S. (2014) · Language Resources and Evaluation · Cited by 561
On coreference resolution performance metrics
Luo, X. (2005) · Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing - HLT '05 · doi:10.3115/1220575.1220579 · Cited by 536
Silberer, C., & Lapata, M. (2012) · The Association for Computational Linguistics · Cited by 520
Unequal Representation and Gender Stereotypes in Image Search Results for Occupations
Kay, M., Matuszek, C., & Munson, S. A. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702520 · Cited by 509 · cited 1×
“5 In all cases, we focus largely on harms due to over-and under-representation (Kay, Matuszek, and Munson 2015), replicating stereotypes (Sweeney 2013; Caliskan, Bryson, and Narayanan 2017) (particular those Significant limitations..”
BackgroundThe Misgendering Machines: Trans/HCI Implications of Automatic Gender Recognition
Keyes, O. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3274357 · Cited by 505
Sweeney, L. (2013) · Queue · doi:10.1145/2460276.2460278 · Cited by 477
Gonen, H., & Goldberg, Y. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1061 · Cited by 454
Providing a unified account of definite noun phrases in discourse
Grosz, B. J., Joshi, A. K., & Weinstein, S. (1983) · Proceedings of the 21st annual meeting on Association for Computational Linguistics - · doi:10.3115/981311.981320 · Cited by 450
Lyons, J. (1977) · Cambridge University Press eBooks · doi:10.1017/cbo9781139165693 · Cited by 448
Gender Bias in Coreference Resolution
Rudinger, R., Naradowsky, J., Leonard, B., & Van Durme, B. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2002 · Cited by 443 · cited 9×
“In comparison to (Zhao et al. 2018a) and (Rudinger et al. 2018) (as well as in contrast to our MAP dataset), we focused on naturally occurring data, but sampled specifically to surface more gender-related phenomena than may be found in, say, the Wall Street Journal.”
BackgroundMethodResultInfluentialHate Speech Dataset from a White Supremacy Forum
de Gibert, O., Perez, N., García-Pablos, A., & Cuadros, M. (2018) · Proceedings of the 2nd Workshop on Abusive Language Online (ALW2) · doi:10.18653/v1/w18-5102 · Cited by 425 · cited 1×
“(2018) N N Y N − Y Y − Karlekar, Niu, and Bansal (2018) N N Y Y − Y Y N de Gibert et al. (2018) N N N Y − − − − Mickus, Bonami, and Paperno (2019) N Y N N − − − −”
Assessing gender bias in machine translation: a case study with Google Translate
Prates, M. O. R., Avelar, P. H., & Lamb, L. C. (2020) · Neural Computing and Applications · doi:10.1007/s00521-019-04144-6 · Cited by 410 · cited 1×
“…and Van Durme 2017), word embeddings (e.g., Bolukbasi et al. 2016; Romanov et al. 2019; Gonen and Goldberg 2019), sentiment analysis (Kiritchenko and Mohammad 2018), machine translation (Font and Costa-jussà 2019; Prates, Avelar, and Lamb 2019), among many others (Blodgett et al. 2020, inter alia).”
BackgroundAssessing Gender Bias in Machine Translation -- A Case Study with Google Translate
Prates, M. O. R., Avelar, P. H. C., & Lamb, L. (2018) · arXiv · doi:10.48550/arxiv.1809.02208 · Cited by 409
From Doing To Undoing: Gender as We Know It
Risman, B. J. (2009) · Gender & Society · doi:10.1177/0891243208326874 · Cited by 400 · cited 1×
“The latter two notions are captured by the “doing gender” model from social constructionism, which views gender as something that one does and to which one is socially accountable (West and Zimmerman 1987; Butler 1989; Risman 2009).”
BackgroundExamining Gender and Race Bias in Two Hundred Sentiment Analysis Systems
Kiritchenko, S., & Mohammad, S. (2018) · Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics · doi:10.18653/v1/s18-2005 · Cited by 384 · cited 1×
“…May, and Van Durme 2017), word embeddings (e.g., Bolukbasi et al. 2016; Romanov et al. 2019; Gonen and Goldberg 2019), sentiment analysis (Kiritchenko and Mohammad 2018), machine translation (Font and Costa-jussà 2019; Prates, Avelar, and Lamb 2019), among many others (Blodgett et al.…”
BackgroundTsujii, J., & Hajič, J. (2014) · International Conference on Computational Linguistics · Cited by 375
Learning Gender-Neutral Word Embeddings
Zhao, J., Zhou, Y., Li, Z., Wang, W., & Chang, K.-W. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1521 · Cited by 345 · cited 8×
“In comparison to (Zhao et al. 2018a) and (Rudinger et al. 2018) (as well as in contrast to our MAP dataset), we focused on naturally occurring data, but sampled specifically to surface more gender-related phenomena than may be found in, say, the Wall Street Journal.”
BackgroundMethodResultInfluentialGender and Dialect Bias in YouTube’s Automatic Captions
Tatman, R. (2017) · Proceedings of the First ACL Workshop on Ethics in Natural Language Processing · doi:10.18653/v1/w17-1606 · Cited by 338
A Multi-Pass Sieve for Coreference Resolution
Raghunathan, K., Lee, H., Rangarajan, S., Chambers, N., Surdeanu, M., Jurafsky, D., & Manning, C. D. (2010) · Cited by 335
Deep Reinforcement Learning for Mention-Ranking Coreference Models
Clark, K., & Manning, C. D. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1245 · Cited by 328 · cited 1×
“…the AI2 system (AI2; Gardner et al. 2017), hugging face (HF; Wolf 2017), which is a neural system based on spacy (Honnibal and Montani 2017), and the Stanford deterministic (SfdD; Raghunathan et al. 2010), statistical (SfdS; Clark and Manning 2015) and neural (SfdN; Clark and Manning 2016) systems.”
BackgroundComputational Generation of Referring Expressions: A Survey
Krahmer, E., & van Deemter, K. (2012) · Computational Linguistics · doi:10.1162/coli_a_00088 · Cited by 325
In Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC
Grover, C., & Tobin, R. (2006) · Cited by 324
Reducing Gender Bias in Abusive Language Detection
Park, J. H., Shin, J., & Fung, P. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1302 · Cited by 323
Supervised Noun Phrase Coreference Research: The First Fifteen Years
Ng, V. (2010) · Cited by 318
Maxwell, K. T., Oberlander, J., & Croft, W. B. (2013) · The Association for Computational Linguistics · Cited by 294
Brain potentials reflect violations of gender stereotypes
Osterhout, L., Bersick, M., & Mclaughlin, J. (1997) · Memory & Cognition · doi:10.3758/bf03211283 · Cited by 267 · cited 1×
“Osterhout, Bersick, and McLaughlin (1997) found ERP violations of the P600 type for violations of social gender stereotypes.”
BackgroundImproving Gender Classification of Blog Authors
Mukherjee, A., & Liu, B. (2010) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · Cited by 256
Handbook of gender in archaeology
(2007) · Choice Reviews Online · doi:10.5860/choice.44-4537 · Cited by 252
Reference production: Production-internal and addressee-oriented processes
Arnold, J. E. (2008) · Language and Cognitive Processes · doi:10.1080/01690960801920099 · Cited by 251
The Balancing Act: Combining Symbolic and Statistical Approaches to Language
Klavans, J. L., & Resnik, P. (1996) · Cited by 249
Mooney, R. J. (2005) · Cited by 236
Fraser, N. (2008) · Critical Inquiry · doi:10.1086/589478 · Cited by 236 · cited 1×
“…generally treat all errors similarly, regardless of whether the error compounds societal injustices (e.g., ignoring an instance of xyr ) or not (e.g., ignore an instance of he ), despite the fact that these have vastly di ff erent implications from the perspective of justice (Fraser 2008, e.g.,).”
BackgroundCentering: A Parametric Theory and Its Instantiations
Poesio, M., Stevenson, R., Eugenio, B. D., & Hitzeman, J. (2004) · Computational Linguistics · doi:10.1162/0891201041850911 · Cited by 235 · cited 2×
“On the other hand, in the Z ero case, there is no such information available, and a reader must rely, perhaps, on parallel syntactic structure or centering (Joshi and Weinstein 1981; Grosz, Joshi, and Weinstein 1983; Poesio et al. 2004) if ey are to correctly identify that the referent is Booth.”
BackgroundDeveloping Age and Gender Predictive Lexica over Social Media
Sap, M., Park, G., Eichstaedt, J., Kern, M., Stillwell, D., Kosinski, M., Ungar, L., & Schwartz, H. A. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1121 · Cited by 230
HCI as heterodoxy: Technologies of identity and the queering of interaction with computers
Light, A. (2011) · Interacting with Computers · doi:10.1016/j.intcom.2011.02.002 · Cited by 228
Gonen, H., & Goldberg, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.03862 · Cited by 225
Mind the GAP: A balanced corpus of gendered ambiguous pronouns
Webster, K., Recasens, M., Axelrod, V., & Baldridge, J. (2018) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00240 · Cited by 223 · cited 18×
“We then conduct annotation experiments using crowdworkers on Amazon Mechanical Turk following the methodology by which the original GAP corpus was created 21 .”
InfluentialHamidi, F., Scheuerman, M. K., & Branham, S. M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173582 · Cited by 222 · cited 1×
“Gender is also an object of study in gender recognition systems (Hamidi, Scheuerman, and Branham 2018).”
BackgroundSennrich, R. (2012) · The Association for Computational Linguistics · Cited by 219
Carreiras, M., Garnham, A., Oakhill, J., & Cain, K. (1996) · The Quarterly Journal of Experimental Psychology A · doi:10.1080/027249896392531 · Cited by 215 · cited 3×
“(1995) and Carreiras et al. (1996) use reading speed tests for gender-ambiguous contexts, and observe faster reading when the reference was “obvious” in Spanish. Relatedly, Esaulova, Reali, and von Stockhausen (2014) and Reali, Esaulova, and Von Stockhausen (2015) conduct eye movement studies around anaphor resolution”
BackgroundCounterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphology
Zmigrod, R., Mielke, S. J., Wallach, H., & Cotterell, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1161 · Cited by 214 · cited 1×
“We then generate “counterfactual augmentations” of this dataset by ablating the various notions of linguistic gender described in §3.2, similar to Zmigrod et al. (2019) and Zhao et al. (2018a).”
MethodInfluentialProceedings of the 2006 Conference on Empirical Methods in Natural Language Processing
Jurafsky, D., & Gaussier, É. (2006) · Cited by 200
Gender Electrified: ERP Evidence on the Syntactic Nature of Gender Processing
Hagoort, P., & Brown, C. M. (1999) · Journal of Psycholinguistic Research · doi:10.1023/a:1023277213129 · Cited by 182 · cited 1×
“In neurolinguistic studies, Osterhout and Mobley (1995) and Hagoort and Brown (1999) looked at event-related potential (ERP) violations for reflexive pronouns and antecedent in English, finding similar e ff ects to violations of number agreement, but di ff erent e ff ects from semantic violations.”
BackgroundExploring Demographic Language Variations to Improve Multilingual Sentiment Analysis in Social Media
Volkova, S., Wilson, T., & Yarowsky, D. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1187 · Cited by 182
Genderqueer and Non-Binary Genders
(2017) · Palgrave Macmillan UK eBooks · doi:10.1057/978-1-137-51053-2 · Cited by 182 · cited 2×
“Trans-inclusive views further typically recognize that gender exists beyond the regressive “female” / “male” binary 8 ; additionally, that one’s gender may shift by time or context (often “genderfluid”), and some people do not experience gender at all (often “agender”) (Kessler and McKenna 1978; Schilt and Westbrook 20…”
BackgroundTracking Sentiment in Mail: How Genders Differ on Emotional Axes
Mohammad, S. M., Tony, & Yang (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1309.6347 · Cited by 171
Proceedings of the Sixth Workshop on Statistical Machine Translation.
Callison-Burch, C., Koehn, P., Monz, C., & Zaidan, O. F. (2011) · Cited by 169
The role of personality, age, and gender in tweeting about mental illness
Preoţiuc-Pietro, D., Eichstaedt, J., Park, G., Sap, M., Smith, L., Tobolsky, V., Schwartz, H. A., & Ungar, L. (2015) · Proceedings of the 2nd Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality · doi:10.3115/v1/w15-1203 · Cited by 161
Entity-Centric Coreference Resolution with Model Stacking
Clark, K., & Manning, C. D. (2015) · Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.3115/v1/p15-1136 · Cited by 159 · cited 1×
“…the AI2 system (AI2; Gardner et al. 2017), hugging face (HF; Wolf 2017), which is a neural system based on spacy (Honnibal and Montani 2017), and the Stanford deterministic (SfdD; Raghunathan et al. 2010), statistical (SfdS; Clark and Manning 2015) and neural (SfdN; Clark and Manning 2016) systems.”
BackgroundDoing Gender Beyond the Binary: A Virtual Ethnography
Darwin, H. (2017) · Symbolic Interaction · doi:10.1002/symb.316 · Cited by 159 · cited 2×
“Trans-inclusive views further typically recognize that gender exists beyond the regressive “female” / “male” binary 8 ; additionally, that one’s gender may shift by time or context (often “genderfluid”), and some people do not experience gender at all (often “agender”) (Kessler and McKenna 1978; Schilt and Westbrook 20…”
BackgroundGonen, H., & Goldberg, Y. (2019) · arXiv (Cornell University) · Cited by 156
Getting Gender Right in Neural Machine Translation
Vanmassenhove, E., Hardmeier, C., & Way, A. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1334 · Cited by 153
Coreference Resolution with World Knowledge
Rahman, A., & Ng, V. (2011) · Cited by 150
Demographic Factors Improve Classification Performance
Hovy, D. (2015) · Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.3115/v1/p15-1073 · Cited by 148
Animacy and the notion of semantic gender
Dahl, Ö. (2000) · Mouton eBooks · doi:10.1515/9783110802603.99 · Cited by 144
A property-sharing constraint in Centering
Kameyama, M. (1986) · Annual Meeting of the Association for Computational Linguistics (ACL) · doi:10.3115/981131.981159 · Cited by 140
Proceedings of the 12th international joint conference on Artificial intelligence - Volume 1
Mylopoulos, J., & Reiter, R. (1991) · Cited by 139
Gender Attribution: Tracing Stylometric Evidence Beyond Topic and Genre
Sarawgi, R., Gajulapalli, K., & Choi, Y. (2011) · Conference on Computational Natural Language Learning · Cited by 137
Improving Automatic Emotion Recognition from Speech via Gender Differentiaion
Vogt, T., & André, E. (2006) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/32rkhvcgrj6f · Cited by 137
McConnell‐Ginet, S., Kramarae, C., & Treichler, P. A. (1987) · Tulsa Studies in Women s Literature · doi:10.2307/464171 · Cited by 131 · cited 6×
“(2018) N N Y Y − Y − − Wang and Jurgens (2018) N Y Y Y Y N N N Kraus et al. (2018) N N Y Y − Y − − Martinc and Pollak (2018) N N Y Y − Y Y N Chan and Fyshe (2018) N N Y Y − Y Y N Durmus and Cardie (2018) N N N Y − − − − Zaghouani and Charfi (2018) N Y Y N N Y Y − Plank (2018) N N Y Y − Y Y N Wood-Doughty et al. (2018)…”
BackgroundInfluentialGender Inference of Twitter Users in Non-English Contexts
Ciot, M., Sonderegger, M., & Ruths, D. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1114 · Cited by 131
Bootstrapping path-based pronoun resolution
Bergsma, S., & Lin, D. (2006) · Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the ACL - ACL '06 · doi:10.3115/1220175.1220180 · Cited by 129 · cited 2×
“…B + L), in which a large quantity of text was ), a confusion matrix between the pronoun(s) they use (rows) and the inferred gender of their name (based on (Bergsma and Lin 2006)) in columns (where Masc = “he”, Fem = “she”, Neut = “it” and Plur = “they”; “Unk” means the name was not found).”
BackgroundMethodConundrums in noun phrase coreference resolution: making sense of the state-of-the-art
Stoyanov, V., Gilbert, N., Cardie, C., & Riloff, E. (2009) · Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2 - ACL-IJCNLP '09 · doi:10.3115/1690219.1690238 · Cited by 129 · cited 1×
“Interms of metrics, most intrinsic evaluation is carried out using metrics like MUC (Vilain et al. 1995), ACE (Mitchell et al. 2005), B 3 (Bagga and Baldwin 1998; Stoyanov et al. 2009), or CEAF (Luo 2005) (see also (Cai and Strube 2010) for additional discussion and variants).”
MethodControl of inference: role of some aspects of discourse structure-centering
Joshi, A. K., & Weinstein, S. A. (1981) · Cited by 127
Which Coreference Evaluation Metric Do You Trust? A Proposal for a Link-based Entity Aware Metric
Moosavi, N. S., & Strube, M. (2016) · Annual Meeting of the Association for Computational Linguistics (ACL) · doi:10.18653/v1/p16-1060 · Cited by 124
Stylometric Analysis of Scientific Articles
Bergsma, S., Post, M., & Yarowsky, D. (2012) · North American Chapter of the Association for Computational Linguistics · Cited by 116
Function Words in Authorship Attribution. From Black Magic to Theory?
Kestemont, M. (2014) · Proceedings of the 3rd Workshop on Computational Linguistics for Literature (CLFL) · doi:10.3115/v1/w14-0908 · Cited by 114
Goldwater, S. J., & McClosky, D. (2005) · The Association for Computational Linguistics · Cited by 113
Modelling Pronominal Anaphora in Statistical Machine Translation
Hardmeier, C., & Federico, M. (2010) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 113
Detecting Linguistic Characteristics of Alzheimer’s Dementia by Interpreting Neural Models
Karlekar, S., Niu, T., & Bansal, M. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2110 · Cited by 112
Gender as a Variable in Natural-Language Processing: Ethical Considerations
Larson, B. (2017) · Proceedings of the First ACL Workshop on Ethics in Natural Language Processing · doi:10.18653/v1/w17-1601 · Cited by 111 · cited 1×
“Much of this work has focused on gender bias with a (usually implicit) binary lens, an issue which was also called out recently by Larson (2017).”
BackgroundFocusing for interpretation of pronouns
Sidner, C. L. (1981) · doi:10.5555/972911.972912 · Cited by 104
Dirt cheap web-scale parallel text from the Common Crawl
Smith, J., Saint-Amand, H., Plamadă, M., Koehn, P., Callison-Burch, C., & Lopez, A. (2013) · Zurich Open Repository and Archive (University of Zurich) · doi:10.5167/uzh-80038 · Cited by 102 · cited 2×
“times as many HE than SHE in Europarl (Koehn 2005) and the Common Crawl (Smith et al. 2013), six times as many in News Commentaries (Tiedemann 2012), and fifty times”
BackgroundLi, H., Lin, C.-Y., & Osborne, M. (2012) · Cited by 102
Aiding Pronoun Translation with Co-Reference Resolution
Nagard, R. L., & Koehn, P. (2010) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 100
Social Bias in Elicited Natural Language Inferences
Rudinger, R., May, C., & Van Durme, B. (2017) · Proceedings of the First ACL Workshop on Ethics in Natural Language Processing · doi:10.18653/v1/w17-1609 · Cited by 97 · cited 10×
“This confirms the findings of Rudinger et al. (2018) and Zhao et al.”
BackgroundMethodResultInfluentialEqualizing Gender Bias in Neural Machine Translation with Word Embeddings Techniques
Escudé Font, J., & Costa-jussà, M. R. (2019) · Proceedings of the First Workshop on Gender Bias in Natural Language Processing · doi:10.18653/v1/w19-3821 · Cited by 93
Modeling latent biographic attributes in conversational genres
Garera, N., & Yarowsky, D. (2009) · Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP: Volume 2 - ACL-IJCNLP '09 · doi:10.3115/1690219.1690245 · Cited by 92
Personalized Machine Translation: Preserving Original Author Traits
Rabinovich, E., Patel, R. N., Mirkin, S., Specia, L., & Wintner, S. (2017) · Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers · doi:10.18653/v1/e17-1101 · Cited by 91
Garnham, A., Oakhill, J. V., Ehrlich, M., & Carreiras, M. (1995) · Journal of Memory and Language · doi:10.1006/jmla.1995.1003 · Cited by 87 · cited 1×
“For instance, Garnham et al. (1995) and Carreiras et al. (1996) use reading speed tests for gender-ambiguous contexts, and observe faster reading when the reference was “obvious” in Spanish.”
BackgroundDaumé, H., & Marcu, D. (2005) · doi:10.3115/1220575.1220588 · Cited by 86
Self-citationObfuscating Gender in Social Media Writing
Reddy, S., & Knight, K. (2016) · Proceedings of the First Workshop on NLP and Computational Social Science · doi:10.18653/v1/w16-5603 · Cited by 86
Cross-lingual syntactic variation over age and gender
Johannsen, A., Hovy, D., & Søgaard, A. (2015) · Proceedings of the Nineteenth Conference on Computational Natural Language Learning · doi:10.18653/v1/k15-1011 · Cited by 84
Implementing systemic classification by unification
Mellish, C. (1988) · OLAC - Open Language Archives Community · Cited by 83
Singular they and the syntactic representation of gender in English
Bjorkman, B. M. (2017) · Glossa: a journal of general linguistics · doi:10.5334/gjgl.374 · Cited by 83
It’s going to be okay: Measuring Access to Support in Online Communities
Wang, Z., & Jurgens, D. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1004 · Cited by 82
Dependency Parsing of Modern Standard Arabic with Lexical and Inflectional Features
Marton, Y., Habash, N. Y., & Rambow, O. (2012) · Computational Linguistics · doi:10.1162/coli_a_00138 · Cited by 81
Connotation Frames of Power and Agency in Modern Films
Sap, M., Prasettio, M. C., Holtzman, A., Rashkin, H., & Choi, Y. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1247 · Cited by 80
Syntactic and cognitive issues in investigating gendered coreference
Ackerman, L. (2019) · Glossa: a journal of general linguistics · doi:10.5334/gjgl.721 · Cited by 80
Esaulova, Y., Reali, C., & von Stockhausen, L. (2013) · Language Cognition and Neuroscience · doi:10.1080/01690965.2013.794295 · Cited by 79
A Database of Age and Gender Annotated Telephone Speech
Burkhardt, F., Eckert, M., Johannsen, W., & Stegmann, J. (2010) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/2mgazc5z3h3u · Cited by 78 · cited 2×
“Ng (2010) Y Y Y Y N Y Y N Burkhardt et al. (2010) N N Y N − Y Y − Marton, Habash, and Rambow (2010) N Y N N − − − − Le Nagard and Koehn (2010) Y Y Y Y N Y Y N Rojas-Barahona et al.”
BackgroundA corpus-based view of lexical gender in written Business English
Fuertes-Olivera, P. A. (2007) · English for Specific Purposes · doi:10.1016/j.esp.2006.07.001 · Cited by 77
Proceedings of the Fifteenth Conference on Computational Natural Language Learning
Goldwater, S. J., & Manning, C. D. (2011) · Cited by 77
Singular ‘they’ and novel pronouns: gender-neutral, nonbinary, or both?
BRADLEY, E. D., Salkind, J., Moore, A., & Teitsort, S. (2019) · Proceedings of the Linguistic Society of America · doi:10.3765/plsa.v4i1.4542 · Cited by 77 · cited 2×
“, ZE/HIR, XEY/XEM), which have been in usage since at least the 1970s (Spivak 1997; Bustillos 2011; Merriam-Webster 2016; Bradley et al. 2019; Hord 2016).”
BackgroundBleaching Text: Abstract Features for Cross-lingual Gender Prediction
van der Goot, R., Ljubešić, N., Matroos, I., Nissim, M., & Plank, B. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-2061 · Cited by 74
Proceedings of the SIGDIAL 2011 Conference
Moore, J. D., Traum, D. R., Chai, J. Y., & Passonneau, R. J. (2011) · Cited by 73
User Demographics and Language in an Implicit Social Network
Filippova, K. (2012) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · Cited by 72
Young, M., Magassa, L., & Friedman, B. (2019) · Ethics and Information Technology · doi:10.1007/s10676-019-09497-z · Cited by 71 · cited 1×
“It may be the case that to truly build gender inclusive datasets and systems, we need to hire or consult experiential experts (Patton et al. 2019; Young, Magassa, and Friedman 2019).”
BackgroundA quantitative analysis of lexical differences between genders in telephone conversations
Boulis, C., & Mari Ostendorf, E. (2005) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/1219840.1219894 · Cited by 70
Proceedings of the 12th Conference of the European Chapter of the ACL (EACL 2009)
Lopez, A. (2009) · The Association for Computational Linguistics · Cited by 70
Key Female Characters in Film Have More to Talk About Besides Men: Automating the Bechdel Test
Agarwal, A., Zheng, J., Kamath, S., Balasubramanian, S., & Dey, S. A. (2015) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.3115/v1/n15-1084 · Cited by 68 · cited 2×
“As observed by Agarwal et al. (2019), most of these metrics are rather”
BackgroundSegmentation for English-to-Arabic statistical machine translation
Badr, I., Zbib, R., & Glass, J. R. (2008) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/1557690.1557732 · Cited by 67
Improving Pronoun Translation for Statistical Machine Translation
Guillou, L. (2012) · Cited by 66
Context-based morphological disambiguation with random fields
Smith, N. A., Smith, D. A., & Tromble, R. W. (2005) · doi:10.3115/1220575.1220635 · Cited by 64
Bucking the Linguistic Binary: Gender Neutral Language in English, Swedish, French, and German
Hord, L. C. R. (2016) · Scholarship@Western (Western University) · Cited by 60
Referential form, word duration, and modelling the listener in spoken dialogue
Bard, E. G., & Aylett, M. P. (2004) · Proceedings of the Annual Meeting of the Cognitive Science Society · Cited by 59
Linguistic Cues to Deception and Perceived Deception in Interview Dialogues
Levitan, S. I., Maredia, A., & Hirschberg, J. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1176 · Cited by 58
The impact of parse quality on syntactically-informed statistical machine translation
Quirk, C., & Corston-Oliver, S. H. (2006) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/1610075.1610085 · Cited by 57
Evaluation Metrics For End-to-End Coreference Resolution Systems
Cai, J. L., & Strube, M. (2010) · Cited by 56
Building the Croatian morphological lexicon
Tadić, M., & Fulgosi, S. (2003) · doi:10.3115/1613200.1613206 · Cited by 53
Analyzing Biases in Human Perception of User Age and Gender from Text
Flekova, L., Carpenter, J., Giorgi, S., Ungar, L., & Preoţiuc-Pietro, D. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1080 · Cited by 52 · cited 4×
“(2016) N N Y Y − Y Y N Flekova et al. (2016) N N Y Y − Y Y N Tran and Ostendorf (2016) N N N Y − − − − Qian, Qiu, and Huang (2016) N Y N Y − − − − Li et al. (2016) N N Y Y − Y Y N Zhang et al. (2016) N N Y Y − Y Y N Garimella and Mihalcea (2016) N N Y Y − Y Y N Reddy and Knight (2016) N N Y Y − Y Y N Li and Dickinson…”
InfluentialA Corpus for Modeling Morpho-Syntactic Agreement in Arabic: Gender, Number and Rationality
Alkuhlani, S., & Habash, N. Y. (2011) · Cited by 51
Proceedings of the Joint Fifth Workshop on Statistical Machine Translation and MetricsMATR
Callison-Burch, C., Koehn, P., Monz, C., Peterson, K., & Zaidan, O. F. (2010) · Cited by 51
A reanalysis of english grammatical gender
Baron, N. S. (1971) · Lingua · doi:10.1016/0024-3841(71)90082-9 · Cited by 50
Proceedings of the Eighth Workshop on Statistical Machine Translation
Nădejde, M., Williams, P. J., & Koehn, P. (2013) · The Association for Computational Linguistics · Cited by 48
He Said, She Said: Gender in the ACL Anthology
Vogel, A. P., & Jurafsky, D. (2012) · Cited by 47
Minimally supervised induction of grammatical gender
Cucerzan, S., & Yarowsky, D. (2003) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.3115/1073445.1073451 · Cited by 47
On the Relevance of Syntactic and Discourse Features for Author Profiling and Identification
Juan Soler-Company, & Wanner, L. (2017) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/e17-2108 · Cited by 47
These are not the Stereotypes You are Looking For: Bias and Fairness in Authorial Gender Attribution
Koolen, C., & van Cranenburgh, A. (2017) · Proceedings of the First ACL Workshop on Ethics in Natural Language Processing · doi:10.18653/v1/w17-1602 · Cited by 46
Gender Across Languages: Volume 4
(2015) · Impact · doi:10.1075/impact.36 · Cited by 44 · cited 2×
“For instance, a common example is the German term “Mädchen”, meaning “girl” (e.g., Hellinger and Motschenbacher 2015).”
BackgroundEffects of Gender Stereotypes on Trust and Likability in Spoken Human-Robot Interaction
Kraus, M., Kraus, J., Baumann, M. R. K., & Minker, W. (2018) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/3eafaj49u3f7 · Cited by 44
Improving Arabic Dependency Parsing with Lexical and Inflectional Morphological Features
Marton, Y., Habash, N. Y., & Rambow, O. (2010) · Cited by 43
Bootstrapping coreference classifiers with multiple machine learning algorithms
Ng, V., & Cardie, C. (2003) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/1119355.1119370 · Cited by 42 · cited 1×
“Considering the approaches used in dominant papers, the active learning algorithms used are not statistically unbiased (Ng and Cardie 2003; Laws, Heimerl, and Schütze 2012; Miller, Dligach, and Savova 2012; Sachan, Hovy, and Xing 2015; Guha et al. 2015).”
MethodWhy discourse affects speakers' choice of referring expressions
Orita, N., Vornov, E., Feldman, N. H., & Daumé, H. (2015) · doi:10.3115/v1/p15-1158 · Cited by 42 · cited 3×
“This speaker may use this assumption of the listener to determine that “her” is su ffi ciently unambiguous in this case as to be an acceptable reference (trading o ff brevity and specificity; see, for instance (Arnold 2008; Frank and Goodman 2012; Orita et al. 2015a)).”
BackgroundMethodSelf-citationInvestigating Language Universal and Specific Properties in Word Embeddings
Qian, P., Qiu, X., & Huang, X. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1140 · Cited by 42
Characterizing the Language of Online Communities and its Relation to Community Reception
Tran, T., & Ostendorf, M. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1108 · Cited by 42
Hellinger, M. (2004) · English and American Studies in German · doi:10.1515/9783484431034.29 · Cited by 42
Predicting Twitter User Demographics from Names Alone
Wood-Doughty, Z., Andrews, N., Marvin, R., & Dredze, M. (2018) · Proceedings of the Second Workshop on Computational Modeling of People’s Opinions, Personality, and Emotions in Social Media · doi:10.18653/v1/w18-1114 · Cited by 41
Arap-Tweet: A Large Multi-Dialect Twitter Corpus for Gender, Age and Language Variety Identification
Zaghouani, W., & Charfi, A. (2018) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/32ycwnmhmp3n · Cited by 41
Coreference Semantics from Web Features
Bansal, M., & Klein, D. (2012) · Cited by 40
Balahur, A., Boldrini, E., Montoyo, A., & Martínez-Barco, P. (2011) · Cited by 40
Cavalli‐Sforza, V., & Zitouni, I. (2007) · Cited by 39
Sprachen, vergleichbar und unvergleichlich
MEYER, H. H. D., & Wandruszka, M. (1970) · Books Abroad · doi:10.2307/40124670 · Cited by 38
Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task
Pradhan, S. (2011) · Cited by 38
Identifying Unknown Proper Names in Newswire Text
Mani, I., MacMillan, T. R., LuperFoy, S., Lusher, E. P., & Laskowski, S. J. (1993) · Cited by 38
IwC Special Issue “Feminism and HCI: New Perspectives” Special Issue Editors’ Introduction
Bardzell, S., & Churchill, E. F. (2011) · Interacting with Computers · doi:10.1016/s0953-5438(11)00089-0 · Cited by 37
Using Stem-Templates to Improve Arabic POS and Gender/Number Tagging
Darwish, K. M., Abdelalí, A., & Mubarak, H. (2014) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/5m5e2fcsu8jd · Cited by 37
Learning Morpho-Lexical Probabilities from an Untagged Corpus with an Application to Hebrew
Levinger, M., Ornan, U., & Itai, A. (1995) · Cited by 34
The origins and role of same-sex relations in human societies
(2009) · Choice Reviews Online · doi:10.5860/choice.47-1734 · Cited by 34
Nissen, U. K. (2002) · Linguistik Online · doi:10.13092/lo.11.914 · Cited by 33 · cited 1×
“Some languages have no grammatical gender at all; English is generally seen as one (viewing that referential agreement of personal pronouns does not count as a form of grammatical agreement, a view which we follow, but one that is contested (Nissen 2002; Baron 1971; Bjorkman 2017)).”
BackgroundAnaphora in natural language processing and information retrieval
Liddy, E. D. (1990) · Information Processing & Management · doi:10.1016/0306-4573(90)90008-p · Cited by 33 · cited 1×
“For instance, information retrieval can use coreference resolution to help accurately rank documents by up-weighting the importance of entities that are referred to frequently (Du and Liddy 1990; Pirkola and Järvelin 1996; Edens et al. 2003).”
BackgroundIsolating stereotypical gender in a grammatical gender language: Evidence from eye movements
Reali, C., Esaulova, Y., & von Stockhausen, L. (2014) · Applied Psycholinguistics · doi:10.1017/s0142716414000010 · Cited by 33
Research works printed but not linked: 18
- Andrews, 2017, The singular, gender-neutral ‘they’ added to the Associated Press Stylebook
- Bagga, 1998, Algorithms for scoring coreference chains, 563
- Barocas, 2017, The problem with bias: Allocative versus representational harms in machine learning
- Bender, 2019, A typology of ethical risks in language technology with an eye towards where transparent documentation can help, The Future of Artificial Intelligence: Language, Ethics, Technology
- Blodgett, 2020, Language (technology) is power: The need to be explicit about NLP harms
- Burton, 1883, Kama Sutra, Translation
- Conrod, 2018, Changes in singular they
- Conrod, 2018, What does it mean to agree? Coreference with singular they
- Corbett, 2013, Number of genders
- Craig, 1994, Classifier languages, The Encyclopedia of Language and Linguistics, 2, 565
- Dryer, 2013, Expression of pronominal subjects
- HaNasi, 189, Mishnah bikkurim
- Mitchell, 2005, Ace 2004 multilingual training corpus, Linguistic Data Consortium, Philadelphia, 1, 1
- Ochs, 1992, Indexing gender, Rethinking Context: Language as an Interactive Phenomenon, 11, 335
- Orita, 2015, Why discourse affects speakers’ choice of referring expressions, 10.3115/v1/P15-1158 · DOI
- Silverstein, 1979, Language structure and linguistic ideology, The Elements: A Parasession on Linguistic Units and Levels, 193
- Stryker, 2008, Transgender History
- Weischedel, 2011, OntoNotes: A Large Training Corpus for Enhanced Processing
Cited by
Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologies
Dev, S., Monajatipoor, M., Ovalle, A., Subramonian, A., Phillips, J., & Chang, K.-W. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.150 · Cited by 123
Ethics Sheet for Automatic Emotion Recognition and Sentiment Analysis
Mohammad, S. M. (2022) · Computational Linguistics · doi:10.1162/coli_a_00433 · Cited by 107
“See Cao and Daumé (2021) for a useful discussion on linguistic vs. social gender and also for a great example to create more inclusive data for research.”
BackgroundLee, J., Wang, J., Brown, E., Chu, L., Rodríguez, S., & Froehlich, J. E. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642230 · Cited by 166
“Below, we describe GazePointAR’s design and implementation, starting with a taxonomy of pronoun usage drawn from linguistics literature.”
A Systematic Investigation of Commonsense Knowledge in Large Language Models
Li, X. L., Kuncoro, A., Hoffmann, J., d’Autume, C. D. M., Blunsom, P., & Nematzadeh, A. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.812 · Cited by 133
Data Governance in the Age of Large-Scale Data-Driven Language Technology
Jernite, Y., Nguyen, H., Biderman, S., Rogers, A., Masoud, M., Danchev, V., Tan, S., Luccioni, A. S., Subramani, N., Johnson, I., et al. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3534637 · Cited by 75
“Unsurprisingly, gendered and racial disparities have been documented in a number of language technologies [37, 79, 141], and processes of creating resources and technologies may further entrench such disparities [25, 38, 144].”
Background
Metadata
- Authors
- Cao, Y. T., & Daumé, H.
- Affiliations
- University of Maryland, College Park (US)
- Microsoft Research, University of Maryland. me@hal3.name
- Corresponding author
- Cao, Yang Trista
- Daumé, Hal
- Venue
- Computational Linguistics
- Publisher
- MIT Press - Journals
- Published
- 2021-11
- Volume
- 47
- Issue
- 3
- Pages
- 615-661
- ISSN
- 0891-2017 (ISSN-L)
- 1530-9312
- OpenAlex
- W3178522238
- Cited by
- 47
- References
- 319
- Citations by year
- 2026: 3 · 2025: 10 · 2024: 7 · 2023: 14 · 2022: 12 · 2021: 1
- FWCI
- 10.64
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Computational Linguistics (publishedVersion, cc-by-nc-nd)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Social Sciences › Gender Studies › Gender Studies in Language
- Keywords
- coreference resolution, gender bias, trans and non-binary people, stereotyping, machine learning pipelines
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.