Toward Gender-Inclusive Coreference Resolution: An Analysis of Gender and Bias Throughout the Machine Learning Lifecycle
Open accessThe DOI matches its Crossref record; 329 of the 329 references it cites were found in a registry or an index.
Abstract
Correctly resolving textual mentions of people fundamentally entails making inferences about those people. Such inferences raise the risk of systematic biases in coreference resolution systems, including biases that can harm binary and non-binary trans and cis stakeholders. To better understand such biases, we foreground nuanced conceptualizations of gender from sociology and sociolinguistics, and investigate where in the machine learning pipeline such biases can enter a coreference resolution system. We inspect many existing data sets for trans-exclusionary biases, and develop two new data sets for interrogating bias in both crowd annotations and in existing coreference resolution systems. Through these studies, conducted on English text, we confirm that without acknowledging and building systems that recognize the complexity of gender, we will build systems that fail for: quality of service, stereotyping, and over- or under-representation, especially for binary and non-binary trans users.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“and processes of creating resources and technologies may further entrench such disparities [25, 38, 144].”
From Data Governance in the Age of Large-Scale Data-Driven Language Technology (2022) · cites this paper as [25]
Citations
Cites
Gender Trouble: Feminism and the Subversion of Identity
McIntosh, M. (1991) · Feminist Review · doi:10.1057/fr.1991.33 · Cited by 29,379 · cited 3×
“Following (Butler 1989), in our annotation we considered drag performance as a form of genderfluidity; as such, we annotate the performer and the drag persona as coreferent with each other (as well as the relevant pronouns), akin to how we believed a reasonable model for handling stage names (e.g.,…”
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · cited 3×
“In the case of machine translation, one can ask whether standard evaluation metrics like Bleu (Papineni et al. 2002) are sensitive to misgendering.”
BackgroundInfluentialWEST, C., & ZIMMERMAN, D. H. (1987) · Gender & Society · doi:10.1177/0891243287001002002 · Cited by 8,954
Gender trouble: feminism and the subversion of identity
(1990) · Choice Reviews Online · doi:10.5860/choice.28-1264 · Cited by 7,707
Bleu: a method for automatic evaluation of machine translation
Papineni, K. (2024) · doi:10.57702/ljteqmrb · Cited by 3,672
(2018) · Cited by 3,510
Lakoff, R. (1973) · Language in Society · doi:10.1017/s0047404500000051 · Cited by 3,138
(2005) · Cited by 3,125
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887
The World Atlas of Language Structures Online
Dryer, M. S., Haspelmath, M., & Forkel, R. (2022) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.7135714 · Cited by 2,542
Research works printed but not linked: 18
- Andrews, 2017, The singular, gender-neutral ‘they’ added to the Associated Press Stylebook
- Bagga, 1998, Algorithms for scoring coreference chains, 563
- Barocas, 2017, The problem with bias: Allocative versus representational harms in machine learning
- Bender, 2019, A typology of ethical risks in language technology with an eye towards where transparent documentation can help, The Future of Artificial Intelligence: Language, Ethics, Technology
- Blodgett, 2020, Language (technology) is power: The need to be explicit about NLP harms
- Burton, 1883, Kama Sutra, Translation
- Conrod, 2018, Changes in singular they
- Conrod, 2018, What does it mean to agree? Coreference with singular they
- Corbett, 2013, Number of genders
- Craig, 1994, Classifier languages, The Encyclopedia of Language and Linguistics, 2, 565
- Dryer, 2013, Expression of pronominal subjects
- HaNasi, 189, Mishnah bikkurim
- Mitchell, 2005, Ace 2004 multilingual training corpus, Linguistic Data Consortium, Philadelphia, 1, 1
- Ochs, 1992, Indexing gender, Rethinking Context: Language as an Interactive Phenomenon, 11, 335
- Orita, 2015, Why discourse affects speakers’ choice of referring expressions, 10.3115/v1/P15-1158 · DOI
- Silverstein, 1979, Language structure and linguistic ideology, The Elements: A Parasession on Linguistic Units and Levels, 193
- Stryker, 2008, Transgender History
- Weischedel, 2011, OntoNotes: A Large Training Corpus for Enhanced Processing
Cited by
Harms of Gender Exclusivity and Challenges in Non-Binary Representation in Language Technologies
Dev, S., Monajatipoor, M., Ovalle, A., Subramonian, A., Phillips, J., & Chang, K.-W. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.150 · Cited by 123
Ethics Sheet for Automatic Emotion Recognition and Sentiment Analysis
Mohammad, S. M. (2022) · Computational Linguistics · doi:10.1162/coli_a_00433 · Cited by 107
“See Cao and Daumé (2021) for a useful discussion on linguistic vs. social gender and also for a great example to create more inclusive data for research.”
BackgroundLee, J., Wang, J., Brown, E., Chu, L., Rodríguez, S., & Froehlich, J. E. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642230 · Cited by 166
“Below, we describe GazePointAR’s design and implementation, starting with a taxonomy of pronoun usage drawn from linguistics literature.”
A Systematic Investigation of Commonsense Knowledge in Large Language Models
Li, X. L., Kuncoro, A., Hoffmann, J., d’Autume, C. D. M., Blunsom, P., & Nematzadeh, A. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.812 · Cited by 133
Data Governance in the Age of Large-Scale Data-Driven Language Technology
Jernite, Y., Nguyen, H., Biderman, S., Rogers, A., Masoud, M., Danchev, V., Tan, S., Luccioni, A. S., Subramani, N., Johnson, I., et al. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3534637 · Cited by 75
“Unsurprisingly, gendered and racial disparities have been documented in a number of language technologies [37, 79, 141], and processes of creating resources and technologies may further entrench such disparities [25, 38, 144].”
Background
Metadata
- Authors
- Cao, Y. T., & Daumé, H.
- Affiliations
- University of Maryland, College Park (US)
- Microsoft Research, University of Maryland. me@hal3.name
- Corresponding author
- Cao, Yang Trista
- Daumé, Hal
- Venue
- Computational Linguistics
- Publisher
- MIT Press - Journals
- Published
- 2021-11
- Volume
- 47
- Issue
- 3
- Pages
- 615-661
- ISSN
- 0891-2017 (ISSN-L)
- 1530-9312
- OpenAlex
- W3178522238
- Cited by
- 47
- References
- 319
- Citations by year
- 2026: 3 · 2025: 10 · 2024: 7 · 2023: 14 · 2022: 12 · 2021: 1
- FWCI
- 10.64
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Computational Linguistics (publishedVersion, cc-by-nc-nd)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Social Sciences › Gender Studies › Gender Studies in Language
- Keywords
- coreference resolution, gender bias, trans and non-binary people, stereotyping, machine learning pipelines
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Toward Gender-Inclusive Coreference Resolution: An Analysis of Gender and Bias Throughout the Machine Learning Lifecycle. https://app.cochat.ai/papers/10.1162/coli_a_00413
- The paper
- Cao, Y. T., & Daumé, H. (2021). Toward Gender-Inclusive Coreference Resolution: An Analysis of Gender and Bias Throughout the Machine Learning Lifecycle. Computational Linguistics. https://doi.org/10.1162/coli_a_00413
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.