The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Adversarial Removal of Demographic Attributes from Text Data
Open accessAbstract
Recent advances in Representation Learning and Adversarial Training seem to succeed in removing unwanted features from the learned representation.We show that demographic information of authors is encoded in-and can be recovered from-the intermediate representations learned by text-based neural classifiers.The implication is that decisions of classifiers trained on textual data are not agnostic to-and likely condition on-demographic attributes.When attempting to remove such demographic information using adversarial training, we find that while the adversarial component achieves chance-level development-set accuracy during training, a post-hoc classifier, trained on the encoded sentences from the first part, still manages to reach substantially higher classification accuracies on the same data.This behavior is consistent across several tasks, demographic properties and datasets.We explore several techniques to improve the effectiveness of the adversarial component.Our main conclusion is a cautionary one: do not rely on the adversarial training to achieve invariant representation to sensitive features.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
Improving neural networks by preventing co-adaptation of feature detectors
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2012) · arXiv · doi:10.48550/arxiv.1207.0580 · Cited by 6,382
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536 · cited 2×
“Many other definitions of fairness exist, including demographic parity , equality of odds and equality of opportunity (see e.g. discussion in (Hardt et al., 2016; Beutel et al., 2017)).”
BackgroundMethodMan is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
Bolukbasi, T., Chang, K.-W., Zou, J., Saligrama, V., & Kalai, A. (2016) · arXiv · doi:10.48550/arxiv.1607.06520 · Cited by 1,615 · cited 1×
“Several works document biases and stereotypes that are captured by unsupervised word embeddings (Bolukbasi et al., 2016; Caliskan et al., 2017) and ways of mitigating them (Bolukbasi et al., 2016; Zhang et al., 2018).”
BackgroundMitigating Unwanted Biases with Adversarial Learning
Zhang, B. H., Lemoine, B., & Mitchell, M. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278779 · Cited by 1,387
Kiros, R., Zhu, Y., Salakhutdinov, R., Zemel, R. S., Torralba, A., Urtasun, R., & Fidler, S. (2015) · arXiv · doi:10.48550/arxiv.1506.06726 · Cited by 1,293 · cited 1×
“…is not surprising: it is at the core of the success of NLP methods for deriving “generic” word and sentence representations (e.g. Word2vec (Mikolov et al., 2013), Skip-thought vectors (Kiros et al., 2015), Contextual-ized Word Representations (Melamud et al., 2016; Peters et al., 2018) etc.).”
BackgroundMen Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints
Zhao, J., Wang, T., Yatskar, M., Ordonez, V., & Chang, K.-W. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1323 · Cited by 829 · cited 1×
“By using the raw text as is, a discrimination issue might arise, as textual information can be predictive of some demographic factors (Hovy et al., 2015) and author’s attributes might correlate with target variables (Zhao et al., 2017).”
BackgroundDiscrimination-aware data mining
Pedreshi, D., Ruggieri, S., & Turini, F. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1401959 · Cited by 676 · cited 1×
“The naive solution of not including protected attributes in the features to a Machine Learning system is insufficient: other features may be highly correlated with—and thus predictive of—the protected attributes (Pedreshi et al., 2008).”
BackgroundAutomatically Categorizing Written Texts by Author Gender
Koppel, M. (2002) · Literary and Linguistic Computing · doi:10.1093/llc/17.4.401 · Cited by 623 · cited 1×
“It is well established that textual information can be predictive of age, race, gender, and many other social factors of the author (Koppel et al., 2002; Burger et al., 2011; Nguyen et al., 2013; Weren et al., 2014; Verhoeven and Daelemans, 2014; Rangel et al., 2016; Verhoeven et al., 2016; Blodgett et al., 2016), or…”
Background
Cited by
WinoGrande: An Adversarial Winograd Schema Challenge at Scale
Sakaguchi, K., Le Bras, R., Bhagavatula, C., & Choi, Y. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6399 · Cited by 579
Plug and Play Language Models: A Simple Approach to Controlled Text Generation
Dathathri, S., Madotto, A., Lan, J., Hung, J., Frank, E., Molino, P., Yosinski, J., & Liu, R. (2019) · arXiv · doi:10.48550/arxiv.1912.02164 · Cited by 508
Mitigating Gender Bias in Natural Language Processing: Literature Review
Sun, T., Gaut, A., Tang, S., Huang, Y., ElSherief, M., Zhao, J., Mirza, D., Belding, E., Chang, K.-W., & Wang, W. Y. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1159 · Cited by 492
WinoGrande: an adversarial winograd schema challenge at scale
Sakaguchi, K., Bras, R. L., Bhagavatula, C., & Choi, Y. (2021) · Communications of the ACM · doi:10.1145/3474381 · Cited by 356
Wang, T., Zhao, J., Yatskar, M., Chang, K.-W., & Ordonez, V. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00541 · Cited by 347
Nuanced Metrics for Measuring Unintended Bias with Real Data for Text Classification
Borkan, D., Dixon, L., Sorensen, J., Thain, N., & Vasserman, L. (2019) · Companion Proceedings of The 2019 World Wide Web Conference · doi:10.1145/3308560.3317593 · Cited by 309
Evaluating Gender Bias in Machine Translation
Stanovsky, G., Smith, N. A., & Zettlemoyer, L. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1164 · Cited by 292
Predictive biases in natural language processing models: A conceptual framework and overview
Shah, D. S., Schwartz, H. A., & Hovy, D. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.468 · Cited by 241
Differential Privacy Has Disparate Impact on Model Accuracy
Bagdasaryan, E., & Shmatikov, V. (2019) · arXiv · doi:10.48550/arxiv.1905.12101 · Cited by 219
“To simplify the task and speed up convergence, we used a technique inspired by [15] and with probability 90% appended to each tweet a special emoji associated with the tweet’s class and subgroup.”
MethodMeasuring and Reducing Gendered Correlations in Pre-trained Models
Webster, K., Wang, X., Tenney, I., Beutel, A., Pitler, E., Pavlick, E., Chen, J., Chi, E., & Petrov, S. (2020) · arXiv · doi:10.48550/arxiv.2010.06032 · Cited by 195
Information Leakage in Embedding Models
Song, C., & Raghunathan, A. (2020) · Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3372297.3417270 · Cited by 175
(2020) · neural information processing systems · Cited by 142
Towards trustworthy LLMs: a review on debiasing and dehallucinating in large language models
Lin, Z., Guan, S., Zhang, W., Zhang, H., Li, Y., & Zhang, H. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10896-y · Cited by 141
Fairness GAN: Generating datasets with fairness properties using a generative adversarial network
Sattigeri, P., Hoffman, S. C., Chenthamarakshan, V., & Varshney, K. R. (2019) · IBM Journal of Research and Development · doi:10.1147/jrd.2019.2945519 · Cited by 134
Probing the Probing Paradigm: Does Probing Accuracy Entail Task Relevance?
Ravichander, A., Belinkov, Y., & Hovy, E. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · doi:10.18653/v1/2021.eacl-main.295 · Cited by 132
Amnesic Probing: Behavioral Explanation with Amnesic Counterfactuals
Elazar, Y., Ravfogel, S., Jacovi, A., & Goldberg, Y. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00359 · Cited by 126
Self-citationIn-Processing Modeling Techniques for Machine Learning Fairness: A Survey
Wan, M., Zha, D., Liu, N., & Zou, N. (2023) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3551390 · Cited by 99
“Another work has explored similar methods for fair word embedding [32].”
BackgroundAnonymisation Models for Text Data: State of the art, Challenges and Future Directions
Lison, P., Pilán, I., Sanchez, D., Batet, M., & Øvrelid, L. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.323 · Cited by 80
Don’t Take the Premise for Granted: Mitigating Artifacts in Natural Language Inference
Belinkov, Y., Poliak, A., Shieber, S., Van Durme, B., & Rush, A. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1084 · Cited by 73
Do neural ranking models intensify gender bias?
Rekabsaz, N., & Schedl, M. (2020) · Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3397271.3401280 · Cited by 64
Poulain, R., Bin Tarek, M. F., & Beheshti, R. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594102 · Cited by 63
PANDORA Talks: Personality and Demographics on Reddit
Gjurković, M., Karan, M., Vukojević, I., Bošnjak, M., & Snajder, J. (2021) · Proceedings of the Ninth International Workshop on Natural Language Processing for Social Media · doi:10.18653/v1/2021.socialnlp-1.12 · Cited by 58
Lyu, L., He, X., & Li, Y. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.213 · Cited by 51
Unlearning Protected User Attributes in Recommendations with Adversarial Training
Ganhör, C., Penz, D., Rekabsaz, N., Lesota, O., & Schedl, M. (2022) · Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3477495.3531820 · Cited by 47
Hurtful Words: Quantifying Biases in Clinical Contextual Word Embeddings
Zhang, H., Lu, A. X., Abdalla, M., McDermott, M., & Ghassemi, M. (2020) · arXiv · doi:10.48550/arxiv.2003.11515 · Cited by 37
Debiasing pretrained text encoders by paying attention to paying attention
Gaci, Y., Benatallah, B., Casati, F., & Benabdeslem, K. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.651 · Cited by 14
Metadata
- Authors
- Elazar, Y., & Goldberg, Y.
- Venue
- Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing
- Published
- 2018
- Pages
- 11-21
- OpenAlex
- W2888161220
- Cited by
- 237
- References
- 13
- FWCI
- 57.03
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.