The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Measuring and Mitigating Unintended Bias in Text Classification
Open accessAbstract
We introduce and illustrate a new approach to measuring and mitigating unintended bias in machine learning models. Our definition of unintended bias is parameterized by a test set and a subset of input features. We illustrate how this can be used to evaluate text classifiers using a synthetic test set and a public corpus of comments annotated for toxicity from Wikipedia Talk pages. We also demonstrate how imbalances in training data can lead to unintended bias in the resulting models, and therefore potentially unfair applications. We use a set of common demographic identity terms as the subset of input features on which we measure bias. This technique permits analysis in the common scenario where demographic information on authors and readers is unavailable, so that bias mitigation must focus on the content of the text itself. The mitigation method we introduce is an unsupervised approach based on balancing the training dataset. We demonstrate that this approach reduces the unintended bias without compromising overall model quality.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Synthetic data is valuable here because [11] shows that real data often has disproportionate amounts of toxicity directed at specific groups.”
From Model Cards for Model Reporting (2019) · cites this paper as [11]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Prior work has shown that toxicity classifiers can pick up on biases in the training data and spuriously associate toxicity with the mention of certain demographics (Park et al., 2018; Dixon et al., 2018).”
From WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020) · cites this paper as Dixon et al., 2018
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
An introduction to ROC analysis
Fawcett, T. (2006) · Pattern Recognition Letters · doi:10.1016/j.patrec.2005.10.010 · Cited by 21,879
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536
Certifying and Removing Disparate Impact
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783311 · Cited by 1,827
Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
Bolukbasi, T., Chang, K.-W., Zou, J., Saligrama, V., & Kalai, A. (2016) · arXiv · doi:10.48550/arxiv.1607.06520 · Cited by 1,615
Inherent Trade-Offs in the Fair Determination of Risk Scores
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2016) · arXiv · doi:10.48550/arxiv.1609.05807 · Cited by 941
Ex Machina: Personal Attacks Seen at Scale
Wulczyn, E., Thain, N., & Dixon, L. (2017) · Proceedings of the 26th International Conference on World Wide Web · doi:10.1145/3038912.3052591 · Cited by 636
Self-citationInherent Tradeoffs in the Fair Determination of Risk Scores
Raghavan, M. (2023) · ACM eBooks · doi:10.1145/3603195.3603197 · Cited by 350
Gender and Dialect Bias in YouTube’s Automatic Captions
Tatman, R. (2017) · Proceedings of the First ACL Workshop on Ethics in Natural Language Processing · doi:10.18653/v1/w17-1606 · Cited by 338
The social impact of natural language processing
Hovy, D., & Spruit, S. L. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p16-2096 · Cited by 329
Data Decisions and Theoretical Implications when Adversarially Learning Fair RepresentationsPreprint
Beutel, A., Chen, J., Zhao, Z., & Chi, E. H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.00075 · Cited by 282
Flagged first, then by citations
Cited by
Sorted by citations
Items 101–200 of 664
Artifical Intelligence and Bias: Challenges, Implications, and Remedies
Min, A. (2023) · Journal Of Social Research · doi:10.55324/josr.v2i11.1477 · Cited by 87
Cultural Bias in Large Language Models: A Comprehensive Analysis and Mitigation Strategies
Liu, Z. (2025) · Journal of Transcultural Communication · doi:10.1515/jtc-2023-0019 · Cited by 77
Is Your Toxicity My Toxicity? Exploring the Impact of Rater Identity on Toxicity Annotation
Goyal, N., Kivlichan, I. D., Rosen, R., & Vasserman, L. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555088 · Cited by 76
Self-citationNbias: A natural language processing framework for BIAS identification in text
Raza, S., Garg, M., Reji, D. J., Bashir, S. R., & Ding, C. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121542 · Cited by 76
FairFil: Contrastive Neural Debiasing Method for Pretrained Text EncodersPreprint
Cheng, P., Hao, W., Yuan, S., Si, S., & Carin, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.06413 · Cited by 93
AMI @ EVALITA2020: Automatic Misogyny Identification
Fersini, E., Nozza, D., & Rosso, P. (2020) · Accademia University Press eBooks · doi:10.4000/books.aaccademia.6764 · Cited by 75
Biswas, S., & Rajan, H. (2021) · Iowa State University Digital Repository (Iowa State University) · Cited by 75
Probing Toxic Content in Large Pre-Trained Language Models
Ousidhoum, N. D., Zhao, X., Fang, T., Song, Y., & Yeung, D. (2021) · doi:10.18653/v1/2021.acl-long.329 · Cited by 89
Delphi: Towards Machine Ethics and NormsPreprint
Jiang, L., Hwang, J. D., Bhagavatula, C., Bras, R. J. L., Forbes, M., Borchardt, J., Liang, J., Etzioni, O., Sap, M., & Choi, Y. (2021) · arXiv (Cornell University) · Cited by 74
Addressing Unintended Bias in Toxicity Detection: An LSTM and Attention-Based Approach
Dai, W., Tao, J., Xu, Y., Feng, Z., & Chen, J. (2023) · IEEE International Conference on Artificial Intelligence and Computer Applications (ICAICA) · doi:10.1109/icaica58456.2023.10405429 · Cited by 75
Czarnowska, P., Vyas, Y., & Shah, K. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00425 · Cited by 72
GeDi: Generative Discriminator Guided Sequence GenerationPreprint
Krause, B., Gotmare, A. D., McCann, B., Keskar, N. S., Joty, S., Socher, R., & Rajani, N. F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.06367 · Cited by 164
Thomas, K., Kelley, P. G., Consolvo, S., Samermit, P., & Bursztein, E. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501879 · Cited by 70
The Role of Explainable AI in the Research Field of AI Ethics
Vainio-Pekka, H., Agbese, M., Jantunen, M., Vakkuri, V., Mikkonen, T., Rousi, R. A., & Abrahamsson, P. (2023) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3599974 · Cited by 85
BiasFinder: Metamorphic Test Generation to Uncover Bias for Sentiment Analysis Systems
Asyrofi, M. H., Yang, Z., Yusuf, I. N. B., Kang, H. J., Thung, F., & Lo, D. (2021) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2021.3136169 · Cited by 81
X under Musk’s leadership: Substantial hate and no reduction in inauthentic activity
Hickey, D., Fessler, D. M. T., Lerman, K., & Burghardt, K. (2025) · PLoS ONE · doi:10.1371/journal.pone.0313293 · Cited by 88
Antisocial online behavior detection using deep learning
Zinovyeva, E., Härdle, W. K., & Lessmann, S. (2020) · Decision Support Systems · doi:10.1016/j.dss.2020.113362 · Cited by 71
Multilingual Twitter Corpus and Baselines for Evaluating Demographic Bias in Hate Speech RecognitionPreprint
Huang, X., Xing, L., Dernoncourt, F., & Paul, M. J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.10361 · Cited by 75
LIRME: Locally Interpretable Ranking Model Explanation
Verma, M., & Ganguly, D. (2019) · Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3331184.3331377 · Cited by 64
Bot-Adversarial Dialogue for Safe Conversational Agents
Xu, J., Ju, D., Li, M., Boureau, Y.-L., Weston, J., & Dinan, E. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.235 · Cited by 64
Hate Speech Classifiers Learn Normative Social Stereotypes
Davani, A. M., Atari, M., Kennedy, B., & Dehghani, M. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00550 · Cited by 73
Feature-Based Explanations Don't Help People Detect Misclassifications of Online Toxicity
Carton, S., Mei, Q., & Resnick, P. (2020) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v14i1.7282 · Cited by 58
Li, C., Xing, W., & Leite, W. L. (2022) · British Journal of Educational Technology · doi:10.1111/bjet.13227 · Cited by 59
Deep learning for religious and continent-based toxic content detection and classification
Abbasi, A., Javed, A. R., Iqbal, F., Kryvinska, N., & Jalil, Z. (2022) · Scientific Reports · doi:10.1038/s41598-022-22523-3 · Cited by 58
Semantics Derived Automatically from Language Corpora Contain Human-like Moral Choices
Jentzsch, S. F., Schramowski, P., Rothkopf, C. A., & Kersting, K. (2019) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3306618.3314267 · Cited by 58
Zhang, G., Bai, B., Zhang, J., Bai, K., Zhu, C., & Zhao, T. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.380 · Cited by 65
Large Language Models for Text Classification: From Zero-Shot Learning to Instruction-Tuning
Chae, Y., & Davidson, T. (2025) · Sociological Methods & Research · doi:10.1177/00491241251325243 · Cited by 72
Understanding the Failure Modes of Out-of-Distribution Generalization
Nagarajan, V., Andreassen, A., & Neyshabur, B. (2020) · arXiv · doi:10.48550/arxiv.2010.15775 · Cited by 55
Impact of politically biased data on hate speech classification
Wich, M., Bauer, J., & Groh, G. (2020) · Workshop on Abusive Language Online · doi:10.18653/v1/2020.alw-1.7 · Cited by 54
Mitigating Bias in Algorithmic Systems—A Fish-eye View
Orphanou, K., Otterbacher, J., Kleanthous, S., Batsuren, K., Giunchiglia, F., Bogina, V., Tal, A. S., Hartman, A., & Kuflik, T. (2022) · ACM Computing Surveys · doi:10.1145/3527152 · Cited by 58
Mitigating Gender Bias in Machine Learning Data Sets
Leavy, S., Meaney, G., Wade, K., & Greene, D. (2020) · Communications in computer and information science · doi:10.1007/978-3-030-52485-2_2 · Cited by 54
Cyberbullying Detection With Fairness Constraints
Gencoglu, O. (2021) · IEEE Internet Computing · doi:10.1109/mic.2020.3032461 · Cited by 53
KOLD: Korean Offensive Language Dataset
Jeong, Y., Oh, J., Lee, J., Ahn, J., Moon, J., Park, S., & Oh, A. H. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.744 · Cited by 53
Chaloner, K., & Maldonado, A. (2019) · Workshop on Gender Bias in Natural Language Processing (GeBNLP) · doi:10.18653/v1/w19-3804 · Cited by 52
Just Say No: Analyzing the Stance of Neural Dialogue Generation in Offensive Contexts
Baheti, A., Sap, M., Ritter, A., & Riedl, M. O. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.397 · Cited by 60
Proceedings of the Third Workshop on Abusive Language Online
(2019) · doi:10.18653/v1/w19-35 · Cited by 52
On Transferability of Bias Mitigation Effects in Language Model Fine-Tuning
Jin, X., Barbieri, F., Kennedy, B., Davani, A. M., Neves, L., & Ren, X. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.296 · Cited by 63
An evaluation of synthetic data augmentation for mitigating covariate bias in health data
Juwara, L., El‐Hussuna, A., & Emam, K. E. (2024) · Patterns · doi:10.1016/j.patter.2024.100946 · Cited by 58
Reducing Sentiment Bias in Language Models via Counterfactual EvaluationPreprint
Huang, P.-S., Zhang, H., Jiang, R., Stanforth, R., Welbl, J., Rae, J. W., Maini, V., Yogatama, D., & Kohli, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.03064 · Cited by 98
Model Positionality and Computational Reflexivity: Promoting Reflexivity in Data Science
Cambo, S. A., & Gergle, D. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501998 · Cited by 49
Unintended machine learning biases as social barriers for persons with disabilitiess
Hutchinson, B., Prabhakaran, V., Denton, R., Webster, K., Zhong, Y., & Denuyl, S. (2020) · ACM SIGACCESS Accessibility and Computing · doi:10.1145/3386296.3386305 · Cited by 49
STELA: a community-centred approach to norm elicitation for AI alignment
Bergman, S., Marchal, N., Mellor, J., Mohamed, S., Gabriel, I., & Isaac, W. (2024) · Scientific Reports · doi:10.1038/s41598-024-56648-4 · Cited by 48
Zheng, Y., Wang, S., & Zhao, J. (2021) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2021.103410 · Cited by 51
On Modality Bias Recognition and Reduction
Guo, Y., Nie, L., Cheng, H. H., Cheng, Z., Kankanhalli, M., & Bimbo, A. D. (2022) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3565266 · Cited by 50
Debiasing Embeddings for Reduced Gender Bias in Text Classification
Prost, F., Thain, N., & Bolukbasi, T. (2019) · Workshop on Gender Bias in Natural Language Processing (GeBNLP) · doi:10.18653/v1/w19-3810 · Cited by 48
Self-citationMultilingual HateCheck: Functional Tests for Multilingual Hate Speech Detection Models
Röttger, P., Seelawi, H., Nozza, D., Talat, Z., & Vidgen, B. (2022) · Workshop on Online Abuse and Harms (WOAH) · doi:10.18653/v1/2022.woah-1.15 · Cited by 61
Simbeck, K. (2019) · IBM Journal of Research and Development · doi:10.1147/jrd.2019.2915067 · Cited by 53
Detection and Evaluation of Machine Learning Bias
Alelyani, S. (2021) · Applied Sciences · doi:10.3390/app11146271 · Cited by 53
Exploring accidental triggers of smart speakers
Schönherr, L., Golla, M., Eisenhofer, T., Wiele, J., Kolossa, D., & Holz, T. (2021) · Computer Speech & Language · doi:10.1016/j.csl.2021.101328 · Cited by 46
Nascimento, F. R. S., Cavalcanti, G. D. C., & Costa‐Abreu, M. D. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.117032 · Cited by 49
We have to talk about emotional AI and crime
Podoletz, L. (2022) · AI & Society · doi:10.1007/s00146-022-01435-w · Cited by 45
Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling
Dinan, E., Abercrombie, G., Bergman, A. S., Spruit, S., Hovy, D., Boureau, Y.-L., & Rieser, V. (2021) · arXiv · doi:10.48550/arxiv.2107.03451 · Cited by 44
Respectful or Toxic? Using Zero-Shot Learning with Language Models to Detect Hate Speech
Plaza-del-Arco, F. M., Nozza, D., & Hovy, D. (2023) · doi:10.18653/v1/2023.woah-1.6 · Cited by 47
Lu, J., Xu, B., Zhang, X., Min, C., Yang, L., & Lin, H. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.898 · Cited by 44
Zeinert, P., Inie, N., & Derczynski, L. (2021) · doi:10.18653/v1/2021.acl-long.247 · Cited by 49
Kirk, H. R., Vidgen, B., Röttger, P., Thrush, T., & Hale, S. A. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.97 · Cited by 47
Explaining The Efficacy of Counterfactually Augmented DataPreprint
Kaushik, D., Setlur, A., Hovy, E. H., & Lipton, Z. C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.02114 · Cited by 58
Hassan, S., Huenerfauth, M., & Alm, C. O. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.267 · Cited by 54
From Learning to Relearning: A Framework for Diminishing Bias in Social Robot Navigation
Hurtado, J. V., Londoño, L., & Valada, A. (2021) · Frontiers in Robotics and AI · doi:10.3389/frobt.2021.650325 · Cited by 39
Towards a multi-stakeholder value-based assessment framework for algorithmic systems
Yurrita, M., Murray-Rust, D., Balayn, A., & Bozzon, A. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3533118 · Cited by 50
They, Them, Theirs: Rewriting with Gender-Neutral EnglishPreprint
Sun, T., Webster, K., Shah, A., Wang, W. Y., & Johnson, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.06788 · Cited by 65
Data Representativeness in Accessibility Datasets: A Meta-Analysis
Kamikubo, R., Wang, L., Marte, C., Mahmood, A., & Kacorri, H. (2022) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3517428.3544826 · Cited by 46
Transfer of Machine Learning Fairness across DomainsPreprint
Schumann, C., Wang, X., Beutel, A., Chen, J., Qian, H., & Ed, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.09688 · Cited by 41
Investigating machine moral judgement through the Delphi experiment
Jiang, L., Hwang, J. D., Bhagavatula, C., Bras, R. J. L., Liang, J. T., Levine, S., Dodge, J., Sakaguchi, K., Forbes, M., Hessel, J., et al. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-024-00969-6 · Cited by 58
Textual variations in social media text processing applications: challenges, solutions, and trends
Khan, J., Ahmad, K., Jagatheesaperumal, S. K., & Sohn, K.-A. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-024-11071-z · Cited by 39
Everything Has a Cause: Leveraging Causal Inference in Legal Text Analysis
Liu, X., Yin, D., Feng, Y., Wu, Y., & Zhao, D. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.155 · Cited by 42
A study of UX practitioners roles in designing real-world, enterprise ML systems
Zdanowska, S., & Taylor, A. S. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517607 · Cited by 39
Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey
Kumar, S., Balachandran, V., Njoo, L., Anastasopoulos, A., & Tsvetkov, Y. (2023) · Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.241 · Cited by 38
Adaptive fairness improvement based on causality analysis
Zhang, M., & Sun, J. (2022) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3540250.3549103 · Cited by 38
Reducing sentiment polarity for demographic attributes in word embeddings using adversarial learning
Sweeney, C., & Najafian, M. (2020) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3351095.3372837 · Cited by 38
Removing Spurious Features can Hurt Accuracy and Affect Groups Disproportionately
Khani, F., & Liang, P. (2021) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3442188.3445883 · Cited by 45
Towards Understanding Fairness and its Composition in Ensemble Machine Learning
Gohar, U., Biswas, S., & Rajan, H. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00133 · Cited by 47
Bastings, J., Ebert, S., Zablotskaia, P., Sandholm, A., & Filippova, K. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.64 · Cited by 65
Intersectional Bias in Hate Speech and Abusive Language DatasetsPreprint
Kim, J., Ortiz, C., Nam, S., Santiago, S., & Datta, V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.05921 · Cited by 49
FIND: Human-in-the-Loop Debugging Deep Text Classifiers
Lertvittayakumjorn, P., Specia, L., & Toni, F. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.24 · Cited by 39
Wang, Y., Chen, M., Zhou, W., Cai, Y., Liang, Y., Liu, D., Yang, B., Liu, J., & Hooi, B. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.224 · Cited by 39
Stereotypes in ChatGPT: an empirical study
Busker, T., Choenni, S., & Bargh, M. S. (2023) · International Conference on Theory and Practice of Electronic Governance · doi:10.1145/3614321.3614325 · Cited by 40
Constantinides, M., Bogucka, E. P., Quercia, D., Kallio, S., & Tahaei, M. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3686927 · Cited by 38
Personal Bias in Prediction of Emotions Elicited by Textual Opinions
Miłkowski, P., Gruza, M., Kanclerz, K., Kazienko, P., Grimling, D., & Kocoń, J. (2021) · doi:10.18653/v1/2021.acl-srw.26 · Cited by 34
Beyond "Fairness:" Structural (In)justice Lenses on AI for EducationPreprint
Madaio, M., Blodgett, S. L., Mayfield, E., & Dixon‐Román, E. J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.08847 · Cited by 39
Schramowski, P., Turan, C., Jentzsch, S. F., Rothkopf, C. A., & Kersting, K. (2020) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2020.00036 · Cited by 33
Intrinsic bias metrics do not correlate with application bias
Goldfarb-Tarrant, S., Marchant, R., Muñoz Sánchez, R., Pandya, M., & Lopez, A. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.150 · Cited by 33
Why Attention is Not Explanation: Surgical Intervention and Causal Reasoning about Neural Models.
Grimsley, C., Mayfield, E., & Bursten, J. R. S. (2020) · PhilPapers (PhilPapers Foundation) · Cited by 32
Six Attributes of Unhealthy Conversations
Price, I., Gifford-Moore, J., Flemming, J., Musker, S., Roichman, M., Sylvain, G., Thain, N., Dixon, L., & Sorensen, J. S. (2020) · Workshop on Online Abuse and Harms (WOAH) · doi:10.18653/v1/2020.alw-1.15 · Cited by 34
Self-citationInvestigating Annotator Bias with a Graph-Based Approach
Wich, M., Kuwatly, H. A., & Groh, G. (2020) · Workshop on Online Abuse and Harms (WOAH) · doi:10.18653/v1/2020.alw-1.22 · Cited by 32
Ball-Burack, A., Lee, M. S. A., Cobbe, J., & Singh, J. P. (2021) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3442188.3445875 · Cited by 38
Automatic Fairness Testing of Neural Classifiers Through Adversarial Sampling
Zhang, P., Wang, J., Sun, J., Wang, X., Dong, G., Wang, X., Dai, T., & Dong, J. S. (2021) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2021.3101478 · Cited by 33
Entropy-based attention regularization frees unintended bias mitigation from lists
Attanasio, G., Nozza, D., Hovy, D., & Baralis, E. (2022) · Findings of the Association for Computational Linguistics: ACL 2022 · doi:10.18653/v1/2022.findings-acl.88 · Cited by 56
Towards Robust and Secure Embodied AI: A Survey on Vulnerabilities and Attacks
Xing, W., Li, M., Li, M., & Han, M. (2026) · ACM Computing Surveys · doi:10.1145/3806048 · Cited by 67
Violence Rating Prediction from Movie Scripts
Martínez, V. R., Somandepalli, K., Singla, K., Ramakrishna, A., Uhls, Y. T., & Narayanan, S. S. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.3301671 · Cited by 33
Mitigating Bias in Session-based Cyberbullying Detection: A Non-Compromising Approach
Cheng, L., Mosallanezhad, A., Silva, Y. N., Hall, D. L., & Liu, H. (2021) · doi:10.18653/v1/2021.acl-long.168 · Cited by 31
Mitigating Racial Biases in Toxic Language Detection with an Equity-Based Ensemble Framework
Halevy, M., Harris, C., Bruckman, A., Yang, D., & Howard, A. (2021) · ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO) · doi:10.1145/3465416.3483299 · Cited by 36
Ruddit: Norms of Offensiveness for English Reddit Comments
Hada, R., Sudhir, S., Mishra, P., Yannakoudakis, H., Mohammad, S. M., & Shutova, E. (2021) · doi:10.18653/v1/2021.acl-long.210 · Cited by 30
Designing Up with Value-Sensitive Design: Building a Field Guide for Ethical ML Development
Boyd, K. (2022) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3531146.3534626 · Cited by 30
Towards fairness-aware multi-objective optimization
Yu, G., Ma, L., Wang, X., Du, W., Du, W., & Jin, Y. (2024) · Complex & Intelligent Systems · doi:10.1007/s40747-024-01668-w · Cited by 32
Conceptor Debiasing of Word Representations Evaluated on WEAT
Karve, S., Ungar, L., & Sedoc, J. (2019) · Workshop on Gender Bias in Natural Language Processing (GeBNLP) · doi:10.18653/v1/w19-3806 · Cited by 29
Detecting Cross-Geographic Biases in Toxicity Modeling on Social Media
Ghosh, S., Baker, D., Jurgens, D., & Prabhakaran, V. (2021) · Workshop on Noisy User-generated Text (W-NUT) · doi:10.18653/v1/2021.wnut-1.35 · Cited by 35
Reward modeling for mitigating toxicity in transformer-based language models
Faal, F., Schmitt, K., & Yu, J. Y. (2022) · Applied Intelligence · doi:10.1007/s10489-022-03944-z · Cited by 35
Pipelines for Social Bias Testing of Large Language Models
Nozza, D., Bianchi, F., & Hovy, D. (2022) · doi:10.18653/v1/2022.bigscience-1.6 · Cited by 33
Metadata
- Authors
- Dixon, L., Li, J., Sorensen, J., Thain, N., & Vasserman, L.
- Affiliations
- Google (United States) (US)
- Venue
- Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society
- Publisher
- ACM
- Published
- 2018-12-27
- Pages
- 67-73
- ISSN
- 3065-8365 (ISSN-L)
- OpenAlex
- W2791170418
- Cited by
- 674
- References
- 11
- Citations by year
- 2026: 43 · 2025: 77 · 2024: 70 · 2023: 103 · 2022: 120 · 2021: 132 · 2020: 76 · 2019: 50 · 2018: 3
- FWCI
- 115.48
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the AAAI/ACM Conference on AI Ethics and Society (publishedVersion, cc-by-nc-nd)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Imbalanced Data Classification Techniques
- Keywords
- text classification, bias mitigation, demographic bias, toxicity detection, Wikipedia talk pages, imbalanced training data, algorithmic fairness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
