Removing Spurious Features can Hurt Accuracy and Affect Groups Disproportionately
Open accessThe DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
Spurious features interfere with the goal of obtaining robust models that perform well across many groups within the population. A natural remedy is to remove such features from the model. However, in this work, we show that removing spurious features can surprisingly decrease accuracy due to the inductive biases of overparameterized models. In noiseless overparameterized linear regression, we completely characterize how the removal of spurious features affects accuracy across different groups (more generally, test distributions). In addition, we show that removal of spurious features can decrease the accuracy even on balanced datasets (where each target co-occurs equally with each spurious feature); and it can inadvertently make the model more susceptible to other spurious features. Finally, we show that robust self-training produces models that no longer depend on spurious features without affecting their overall accuracy. The empirical results on the Toxic-Comment-Detection and CelebA datasets show that our results hold in non-linear models.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536 · cited 1×
“(i) The statistical notions which measure how much a model loss is different among groups according to the sensitive features (Hardt et al., 2016; Khani et al., 2019; Agarwal et al., 2018; Woodworth et al., 2017).”
BackgroundMan is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
Bolukbasi, T., Chang, K.-W., Zou, J., Saligrama, V., & Kalai, A. (2016) · arXiv · doi:10.48550/arxiv.1607.06520 · Cited by 1,615 · cited 1×
“, 2019; Bolukbasi et al., 2016; Zhao et al., 2018a;b). There is a common trend in all of these works: accuracy drops when we train the same model using the new representation. Khani and Liang (2020) show that when the core features are noisy or incomplete, the model can obtain better accuracy by using sensitive featur…”
MethodTheoretically principled trade-off between robustness and accuracy
Zhang, H., Yu, Y., Jiao, J., Xing, E. P., Ghaoui, L. E., & Jordan, M. I. (2019) · arXiv · doi:10.48550/arxiv.1901.08573 · Cited by 1,560
A Reductions Approach to Fair Classification
Agarwal, A., Beygelzimer, A., Dudík, M., Langford, J., & Wallach, H. (2018) · arXiv · doi:10.48550/arxiv.1803.02453 · Cited by 1,288
Kusner, M. J., Loftus, J. R., Russell, C., & Silva, R. (2017) · arXiv · doi:10.48550/arxiv.1703.06856 · Cited by 995 · cited 1×
“(ii) Counterfactual notion of fairness which measure how much two “similar” individuals are incurred different losses because of their sensitive feature (Kusner et al., 2017; Chiappa, 2019; Loftus et al., 2018; Khani and Liang, 2020; Kilbertus et al., 2017; Garg et al., 2019).”
BackgroundGender bias in coreference resolution: Evaluation and debiasing methods
Zhao, J., Wang, T., Yatskar, M., Ordonez, V., & Chang, K.-W. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers) · doi:10.18653/v1/n18-2003 · Cited by 684 · cited 1×
“However, there are works that try to learn a new representation entirely uncorrelated to the sensitive feature in categorical data [29, 30, 32, 51], in vision [8, 38, 46] and in natural language processing [5, 41, 54, 55].”
BackgroundMeasuring and Mitigating Unintended Bias in Text Classification
Dixon, L., Li, J., Sorensen, J., Thain, N., & Vasserman, L. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278729 · Cited by 674 · cited 4×
“Dixon et al. (2018) provide 50 identity terms2 that a model should be robust against them.”
BackgroundInfluentialDeep double descent: where bigger models and more data hurt*
Nakkiran, P., Kaplun, G., Bansal, Y., Yang, T., Barak, B., & Sutskever, I. (2021) · Journal of Statistical Mechanics: Theory and Experiment · doi:10.1088/1742-5468/ac3a74 · Cited by 565 · cited 2×
“The recent line of work on “double descent” [2, 3, 14, 33, 36] has shed some lights on this trade-off and show that unlike conventional under-parameterized regime in the new over-parameterized regime, more data and fewer parameters might result in a worse error.”
BackgroundThe generalization error of random features regression: Precise asymptotics and double descent curve
Mei, S., & Montanari, A. (2019) · arXiv · doi:10.48550/arxiv.1908.05355 · Cited by 537
Cited by
Data collection and quality challenges in deep learning: a data-centric AI perspective
Whang, S. E., Roh, Y., Song, H., & Lee, J.-G. (2023) · The VLDB Journal · doi:10.1007/s00778-022-00775-9 · Cited by 573
Data and its (dis)contents: A survey of dataset development and use in machine learning research
Paullada, A., Raji, I. D., Bender, E. M., Denton, E., & Hanna, A. (2021) · Patterns · doi:10.1016/j.patter.2021.100336 · Cited by 548
“removing spurious cues from training data or ‘‘de-biasing’’ models, recent work has shown that this can be damaging for model accuracy.(88) In contrast to a focus on statistical properties of datasets as a site for addressing andmitigating harms, Denton et al.”
BackgroundPicking on the Same Person: Does Algorithmic Monoculture lead to Outcome Homogenization?
Bommasani, R., Creel, K. A., Kumar, A., Jurafsky, D., & Liang, P. (2022) · arXiv · doi:10.48550/arxiv.2211.13972 · Cited by 60
“Attribute prediction in CelebA has been studied previously in work on fairness and robustness [Sagawa* et al., 2020, Khani and Liang, 2021, Wang et al., 2021].”
BackgroundSelf-citationUnderstanding the Failure Modes of Out-of-Distribution Generalization
Nagarajan, V., Andreassen, A., & Neyshabur, B. (2020) · arXiv · doi:10.48550/arxiv.2010.15775 · Cited by 55
“We would also like to point the reader to concurrent work of Khani & Liang (2021) that has proposed a different model addressing the above points.”
BackgroundEnhancing Fairness in Face Detection in Computer Vision Systems by Demographic Bias Mitigation
Yang, Y., Gupta, A., Feng, J., Singhal, P., Yadav, V., Wu, Y., Natarajan, P., Hedau, V., & Joo, J. (2022) · Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3514094.3534153 · Cited by 24
“Recent studies have also shown that removing spurious features can decrease the model accuracy [25].”
Background
Metadata
- Authors
- Khani, F., & Liang, P.
- Venue
- Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency
- Publisher
- ACM
- Published
- 2021-03
- Pages
- 196-205
- OpenAlex
- W3134374554
- Cited by
- 33
- References
- 15
- FWCI
- 5.73
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Removing Spurious Features can Hurt Accuracy and Affect Groups Disproportionately. https://app.cochat.ai/papers/10.1145/3442188.3445883
- The paper
- Khani, F., & Liang, P. (2021). Removing Spurious Features can Hurt Accuracy and Affect Groups Disproportionately. Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency. https://doi.org/10.1145/3442188.3445883
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.