The DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
Incorporating Subjectivity into Gendered Ambiguous Pronoun (GAP) Resolution using Style Transfer
Open accessAbstract
The GAP dataset is a Wikipedia-based evaluation dataset for gender bias detection in coreference resolution, containing mostly objective sentences.Since subjectivity is ubiquitous in our daily texts, it becomes necessary to evaluate models for both subjective and objective instances.In this work, we present a new evaluation dataset for gender bias in coreference resolution, GAP-Subjective, which increases the coverage of the original GAP dataset by including subjective sentences.We outline the methodology used to create this dataset.Firstly, we detect objective sentences and transfer them into their subjective variants using a sequenceto-sequence model.Secondly, we outline the thresholding techniques based on fluency and content preservation to maintain the quality of the sentences.Thirdly, we perform automated and human-based analysis of the style transfer and infer that the transferred sentences are of high quality.Finally, we benchmark both GAP and GAP-Subjective datasets using a BERTbased model and analyze its predictive performance and gender bias.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“thus, GAP-Subjective is the same size as GAP, with 8,908 instances.”
From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Pant and Dadu 2022
The cited source says what the claim attributes to it.
In the citing paper
“GAP-Subjective (Pant and Dadu 2022) expands on GAP to include more subjective sentences expressing opinions and viewpoints.”
From Bias and Fairness in Large Language Models: A Survey (2024) · cites this paper as Pant and Dadu 2022
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Word embeddings quantify 100 years of gender and ethnic stereotypes
Garg, N., Schiebinger, L., Jurafsky, D., & Zou, J. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1720347115 · Cited by 1,065
Massively Multilingual Neural Machine Translation
Aharoni, R., Johnson, M., & Firat, O. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1388 · Cited by 402
Style transfer through back-translation
Prabhumoye, S., Tsvetkov, Y., Salakhutdinov, R., & Black, A. W. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1080 · Cited by 348
Reducing Gender Bias in Abusive Language Detection
Park, J. H., Shin, J., & Fung, P. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1302 · Cited by 323 · cited 1×
“Many recent studies aim to detect, analyze, and mitigate gender bias in different NLP tools and applications (Bolukbasi et al., 2016; Rudinger et al., 2018; Park et al., 2018).”
Background
Flagged first, then by citations
Research works printed but not linked: 20
- Abolfazl Asudeh, Zhongjun Jin, and H. V. Jagadish. 2019. Assessing and remedying coverage for a given dataset. In 35th IEEE International Conference on Data Engineering, ICDE 2019, Macao, China, April 8-11, 2019, pages 554-565. IEEE. · DOI
- Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. Cite arxiv:1409.0473Comment: Accepted at ICLR 2015 as oral presentation.
- Fei Gao, Jinhua Zhu, Lijun Wu, Yingce Xia, Tao Qin, Xueqi Cheng, Wengang Zhou, and Tie-Yan Liu. 2019. Soft contextual data augmentation for neural machine translation. In Proceedings of the 57th Annual Meet- ing of the Association for Computational Linguistics, pages 5539-5544, Florence, Italy. Association for Computational Linguistics. · DOI
- Jiatao Gu, Zhengdong Lu, Hang Li, and Victor O.K. Li. 2016. Incorporating copying mechanism in sequence- to-sequence learning. In Proceedings of the 54th An- nual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1631- 1640, Berlin, Germany. Association for Computa- tional Linguistics. · DOI
- Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P. Xing. 2017. Toward con- trolled generation of text. In Proceedings of the 34th International Conference on Machine Learning, vol- ume 70 of Proceedings of Machine Learning Re- search, pages 1587-1596, International Convention Centre, Sydney, Australia. PMLR.
- Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senel- lart, and Alexander Rush. 2017. OpenNMT: Open- source toolkit for neural machine translation. In Pro- ceedings of ACL 2017, System Demonstrations, pages 67-72, Vancouver, Canada. Association for Compu- tational Linguistics.
- Juncen Li, Robin Jia, He He, and Percy Liang. 2018. Delete, retrieve, generate: a simple approach to senti- ment and style transfer. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1865-1874, New Orleans, Louisiana. Associa- tion for Computational Linguistics. · DOI
- Andrew Mutton, Mark Dras, Stephen Wan, and Robert Dale. 2007. GLEU: Automatic evaluation of sentence-level fluency. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 344-351, Prague, Czech Republic. Association for Computational Linguistics.
- Sameer Pradhan, Alessandro Moschitti, Nianwen Xue, Olga Uryupina, and Yuchen Zhang. 2012. CoNLL- 2012 shared task: Modeling multilingual unrestricted coreference in OntoNotes. In Joint Conference on EMNLP and CoNLL -Shared Task, pages 1-40, Jeju Island, Korea. Association for Computational Lin- guistics.
- Marta Recasens, Cristian Danescu-Niculescu-Mizil, and Dan Jurafsky. 2013. Linguistic models for analyz- ing and detecting biased language. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1650-1659, Sofia, Bulgaria. Association for Computational Linguistics.
- Ellen Riloff, Janyce Wiebe, and William Phillips. 2005. Exploiting subjectivity classification to improve in- formation extraction. In Proceedings of the 20th Na- tional Conference on Artificial Intelligence -Volume 3, AAAI'05, page 1106-1111. AAAI Press.
- Rachel Rudinger, Jason Naradowsky, Brian Leonard, and Benjamin Durme. 2018. Gender bias in corefer- ence resolution. pages 8-14. · DOI · Paper page
- Abigail See, Peter J. Liu, and Christopher D. Manning. 2017. Get to the point: Summarization with pointer- generator networks. In Proceedings of the 55th An- nual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1073- 1083, Vancouver, Canada. Association for Computa- tional Linguistics. · DOI · Paper page
- Akhilesh Sudhakar, Bhargav Upadhyay, and Arjun Ma- heswaran. 2019. Transforming delete, retrieve, gen- erate approach for controlled text style transfer. In EMNLP/IJCNLP.
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, undefine- dukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the 31st Interna- tional Conference on Neural Information Processing Systems, NIPS'17, page 6000-6010, Red Hook, NY, USA.
- Kellie Webster, Marta Recasens, Vera Axelrod, and Jason Baldridge. 2018. Mind the gap: A balanced corpus of gendered ambiguou. In Transactions of the ACL, page to appear.
- Janyce Wiebe, Theresa Wilson, Rebecca Bruce, Matthew Bell, and Melanie Martin. 2004. Learn- ing subjective language. Comput. Linguist., 30(3):277-308. · DOI · Paper page
- Yunzhao Wu, Yunli Wang, and Shujie Liu. 2020. A dataset for low-resource stylized sequence-to- sequence generation. In AAAI 2020.
- Kai-Chou Yang, Timothy Niven, Tzu Hsuan Chou, and Hung-Yu Kao. 2019. Fill the GAP: Exploiting BERT for pronoun resolution. In Proceedings of the First Workshop on Gender Bias in Natural Language Pro- cessing, pages 102-106, Florence, Italy. Association for Computational Linguistics. · DOI
- Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Or- donez, and Kai-Wei Chang. 2018. Gender bias in coreference resolution: Evaluation and debiasing methods. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume 2 (Short Papers), pages 15-20, New Orleans, Louisiana. Association for Computational Linguistics. · DOI · Paper page
Cited by
Sorted by citations
Bias and Fairness in Large Language Models: A Survey
Gallegos, I. O., Rossi, R. A., Barrow, J., Tanjim, M. M., Kim, S., Dernoncourt, F., Yu, T., Zhang, R., & Ahmed, N. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00524 · Cited by 1,245
Metadata
- Authors
- Pant, K., & Dadu, T.
- Venue
- Proceedings of the 4th Workshop on Gender Bias in Natural Language Processing (GeBNLP)
- Published
- 2022
- OpenAlex
- W4287887516
- Cited by
- 3
- References
- 4
- Citations by year
- 2025: 1 · 2024: 2
- FWCI
- 0.41
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Workshop on Gender Bias in Natural Language Processing (GeBNLP) (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Social Sciences › Gender Studies › Gender Studies in Language
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- coreference resolution, gender bias, subjectivity, style transfer, content preservation, BERT-based models
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
