The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
NeuTral Rewriter: A Rule-Based and Neural Approach to Automatic Rewriting into Gender Neutral Alternatives
Open accessAbstract
Recent years have seen an increasing need for gender-neutral and inclusive language.Within the field of NLP, there are various mono-and bilingual use cases where gender inclusive language is appropriate, if not preferred due to ambiguity or uncertainty in terms of the gender of referents.In this work, we present a rulebased and a neural approach to gender-neutral rewriting for English along with manually curated synthetic data (WinoBias+) and natural data (OpenSubtitles and Reddit) benchmarks.A detailed manual and automatic evaluation highlights how our NeuTral Rewriter, trained on data generated by the rule-based approach, obtains word error rates (WER) below 0.18% on synthetic, in-domain and out-domain test sets.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
WinoBias+ augments WinoBias with gender-neutral alternatives, with 3,167 total instances.
“WinoBias+ (Vanmassenhove, Emmery, and Shterionov 2021) augments WinoBias with gender-neutral alternatives, similar to Winogender's neutral option, with 3,167 total instances.”
From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Vanmassenhove, Emmery, and Shterionov 2021
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Moses: open source toolkit for statistical machine translation
Koehn, P., Zens, R., Dyer, C., Bojar, O., Constantin, A., Herbst, E., Hoang, H., Birch, A., Callison-Burch, C., Federico, M., et al. (2007) · Proceedings of the 45th Annual Meeting of the ACL on Interactive Poster and Demonstration Sessions - ACL '07 · doi:10.3115/1557769.1557821 · Cited by 4,735 · cited 1×
“All data is preprocessed using the Moses (de)tokenizer (Koehn et al., 2007).”
MethodStanza: A Python Natural Language Processing Toolkit for Many Human Languages
Qi, P., Zhang, Y., Zhang, Y., Bolton, J., & Manning, C. D. (2020) · arXiv · doi:10.48550/arxiv.2003.07082 · cited 1×
“To disambi-guate these forms, the POS tagger and dependency parser from Stanza (Qi et al., 2020) were used.”
Method
Flagged first, then by citations
Research works printed but not linked: 24
- Bashar Alhafni, Nizar Habash, and Houda Bouamor. 2020. Gender-aware reinflection using linguistically enhanced neural models. In Proceedings of the Second Workshop on Gender Bias in Natural Lan- guage Processing, pages 139-150.
- Christine Basta, Marta R Costa-jussà, and Noe Casas. 2020. Extensive study on the underlying gender bias in contextualized word embeddings. Neural Compu- ting and Applications, pages 1-14.
- Anthony Bau, Yonatan Belinkov, Hassan Sajjad, Nadir Durrani, Fahim Dalvi, and James Glass. 2019. Iden- tifying and Controlling Important Neurons in Neu- ral Machine Translation. In Proceedings of the Se- venth International Conference on Learning Repre- sentations (ICLR), New Orleans, USA.
- Jason Baumgartner, Savvas Zannettou, Brian Keegan, Megan Squire, and Jeremy Blackburn. 2020. The pushshift reddit dataset. In Proceedings of the Inter- national AAAI Conference on Web and Social Media, volume 14, pages 830-839.
- Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Ven- katesh Saligrama, and Adam T Kalai. 2016. Man is to Computer Programmer as Woman is to Home- maker ? Debiasing Word Embeddings. In Procee- dings of Thirtieth Conference on Neural Information Processing Systems (NIPS), pages 4349-4357, Bar- celona, Spain.
- Chris Emmery, Ákos Kádár, and Grzegorz Chrupała. 2021. Adversarial stylometry in the wild : Transfe- rable lexical substitution attacks on author profiling. In Proceedings of the 16th Conference of the Euro- pean Chapter of the Association for Computational Linguistics : Main Volume, pages 2388-2402.
- Joel Escudé Font and Marta R Costa-jussà. 2019. Equa- lizing Gender Biases in Neural Machine Translation with Word Embeddings Techniques. In To appear in Proceedings of the 1st ACL Workshop on Gen- der Bias for Natural Language Processing, Florence, Italy.
- Hila Gonen and Yoav Goldberg. 2019. Lipstick on a pig : Debiasing methods cover up systematic gender biases in word embeddings but do not remove them. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computa- tional Linguistics : Human Language Technologies, Volume 1 (Long and Short Papers), pages 609-614.
- Nizar Habash, Houda Bouamor, and Christine Chung. 2019. Automatic gender identification and reinflec- tion in arabic. In Proceedings of the First Work- shop on Gender Bias in Natural Language Proces- sing, pages 155-165.
- Yitong Li, Timothy Baldwin, and Trevor Cohn. 2018. Towards robust and privacy-preserving text represen- tations. In Proceedings of the 56th Annual Mee- ting of the Association for Computational Linguis- tics (Volume 2 : Short Papers), pages 25-30.
- Pierre Lison and Jörg Tiedemann. 2016. Opensub- titles2016 : Extracting large parallel corpora from movie and tv subtitles. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16), pages 923-929.
- Kaiji Lu, Piotr Mardziel, Fangjing Wu, Preetam Aman- charla, and Anupam Datta. 2020. Gender bias in neural natural language processing. In Logic, Lan- guage, and Security, pages 189-202. Springer.
- Amit Moryossef, Roee Aharoni, and Yoav Goldberg. 2019. Filling Gender & Number Gaps in Neural Machine Translation with Black-box Context Injec- tion. In TO APPEAR IN 1st ACL Workshop on Gen- der Bias for Natural Language Processing, Florence, Italy.
- Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Mi- chael Auli. 2019. fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demons- trations), pages 48-53, Minneapolis, Minnesota. As- sociation for Computational Linguistics. · DOI
- Sravana Reddy and Kevin Knight. 2016. Obfuscating gender in social media writing. In Proceedings of the First Workshop on NLP and Computational So- cial Science, pages 17-26.
- Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Lin- guistics, ACL 2016, August 7-12, 2016, Berlin, Ger- many, Volume 1 : Long Papers. The Association for Computer Linguistics. · DOI · Paper page
- Rakshith Shetty, Bernt Schiele, and Mario Fritz. 2018. A4nt : author attribute anonymity by adversarial training of neural machine translation. In 27th {USENIX} Security Symposium ({USENIX} Secu- rity 18), pages 1633-1650.
- Tony Sun, Kellie Webster, Apu Shah, William Yang Wang, and Melvin Johnson. 2021. They, them, theirs : Rewriting with gender-neutral english. arXiv preprint arXiv :2102.06788. · arXiv
- Eva Vanmassenhove and Christian Hardmeier. 2018. Europarl datasets with demographic speaker infor- mation. In Proceedings of the 21st Annual Confe- rence of the European Associations for Machine Translation (EAMT), Alicante, Spain.
- Eva Vanmassenhove, Christian Hardmeier, and Andy Way. 2019. Getting gender right in neural machine translation. Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 3003-3008.
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Pro- cessing Systems 30 : Annual Conference on Neural Information Processing Systems 2017, December 4- 9, 2017, Long Beach, CA, USA, pages 5998-6008.
- Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordo- nez, and Kai-Wei Chang. 2018a. Gender bias in co- reference resolution : Evaluation and debiasing me- thods. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Com- putational Linguistics : Human Language Technolo- gies, Volume 2 (Short Papers), pages 15-20.
- Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordo- nez, and Kai-Wei Chang. 2018b. Gender bias in co- reference resolution: Evaluation and debiasing me- thods. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Com- putational Linguistics : Human Language Technolo- gies, Volume 2 (Short Papers), pages 15-20, New Orleans, Louisiana. Association for Computational Linguistics. · DOI · Paper page
- Jieyu Zhao, Yichao Zhou, Zeyu Li, Wei Wang, and Kai- Wei Chang. 2018c. Learning Gender-Neutral Word Embeddings. In Proceedings of the 2018 Confe- rence on Empirical Methods in Natural Language Processing (EMNLP), pages 4847-4853, Brussels, Belgium.
Cited by
Sorted by citations
Bias and Fairness in Large Language Models: A Survey
Gallegos, I. O., Rossi, R. A., Barrow, J., Tanjim, M. M., Kim, S., Dernoncourt, F., Yu, T., Zhang, R., & Ahmed, N. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00524 · Cited by 1,245
Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models
Lin, Z., Guan, S., Zhang, W., Zhang, H., Li, Y., & Zhang, H. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10896-y · Cited by 141
Metadata
- Authors
- Vanmassenhove, E., Emmery, C., & Shterionov, D.
- Affiliations
- Tilburg University (NL)
- Venue
- Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing
- Published
- 2021
- Pages
- 8940-8948
- OpenAlex
- W3201522485
- Cited by
- 35
- References
- 3
- Citations by year
- 2025: 11 · 2024: 9 · 2023: 11 · 2022: 4
- FWCI
- 2.68
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- Research portal (Tilburg University) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by-sa)
- Data Archiving and Networked Services (DANS) (submittedVersion, cc-by)
- arXiv (Cornell University)
- Conference on Empirical Methods in Natural Language Processing (EMNLP)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Social Sciences › Gender Studies › Gender Studies in Language
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- gender-inclusive language, synthetic data, out-of-domain evaluation, word error rate, English language
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
