CoChat
Sign Up
Cited by 99Ethics and Social Impacts of AIExplainable Artificial Intelligence (XAI)Discrimination and Equality Law

In-Processing Modeling Techniques for Machine Learning Fairness: A Survey

Open access
Wan, Mingyang, Zha, Daochen, Liu, Ninghao, Zou, Na·2023·ACM Transactions on Knowledge Discovery from Data
DOI 10.1145/3551390
Open this paper in CoChat — free accountOpen PDF

Abstract

Machine learning models are becoming pervasive in high-stakes applications. Despite their clear benefits in terms of performance, the models could show discrimination against minority groups and result in fairness issues in a decision-making process, leading to severe negative impacts on the individuals and the society. In recent years, various techniques have been developed to mitigate the unfairness for machine learning models. Among them, in-processing methods have drawn increasing attention from the community, where fairness is directly taken into consideration during model design to induce intrinsically fair models and fundamentally mitigate fairness issues in outputs and representations. In this survey, we review the current progress of in-processing fairness mitigation techniques. Based on where the fairness is achieved in the model, we categorize them into explicit and implicit methods, where the former directly incorporates fairness metrics in training objectives, and the latter focuses on refining latent representation learning. Finally, we conclude the survey with a discussion of the research challenges in this community to motivate future exploration.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3551390

Cites

8 of 8 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Studies such as Wan et al. [4] and Caton and Haas [5] provide comprehensive reviews of processing techniques and fairness in ML but often lack a specific focus on how these techniques impact underserved populations.

    “Studies such as Wan et al. [4] and Caton and Haas [5] provide comprehensive reviews of processing techniques and fairness in ML but often lack a specific focus on how these techniques impact underserved populations.”

    From Advancing Health Equity through Multi-Level Fairness in Health Informatics (2026) · cites this paper as [4]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 80

  1. Bilge Acun, Matthew Murphy, Xiaodong Wang, Jade Nie, Carole-Jean Wu, and Kim Hazelwood. 2021. Understanding training efficiency of deep learning recommendation models at scale. In Proceedings of the 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA). IEEE, 802-814.
  2. Tameem Adel, Zoubin Ghahramani, and Adrian Weller. 2018. Discovering interpretable representations for both deep generative and discriminative models. In Proceedings of the International Conference on Machine Learning. PMLR, 50-59.
  3. Alekh Agarwal, Miroslav Dudík, and Zhiwei Steven Wu. 2019. Fair regression: Quantitative definitions and reduction- based algorithms. In Proceedings of the International Conference on Machine Learning. PMLR, 120-129.
  4. Sina Aghaei, Mohammad Javad Azizi, and Phebe Vayanos. 2019. Learning optimal and fair decision trees for non- discriminative decision-making. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 1418-1426.
  5. Julia Angwin, Jeff Larson, Surya Mattu, and Lauren Kirchner. 2016. Machine bias. ProPublica, May 23, 2016 (2016), 139-159.
  6. Arturs Backurs, Piotr Indyk, Krzysztof Onak, Baruch Schieber, Ali Vakilian, and Tal Wagner. 2019. Scalable fair clus- tering. In Proceedings of the International Conference on Machine Learning. PMLR, 405-413.
  7. Yoshua Bengio, Aaron Courville, and Pascal Vincent. 2013. Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 8 (2013), 1798-1828.
  8. Richard Berk, Hoda Heidari, Shahin Jabbari, Matthew Joseph, Michael Kearns, Jamie Morgenstern, Seth Neel, and Aaron Roth. 2017. A convex framework for fair regression. arXiv:1706.02409. Retrieved from https://arxiv.org/abs/ 1706.02409. · arXiv
  9. Alex Beutel, Jilin Chen, Tulsee Doshi, Hai Qian, Li Wei, Yi Wu, Lukasz Heldt, Zhe Zhao, Lichan Hong, Ed H. Chi, et al. 2019. Fairness in recommendation ranking through pairwise comparisons. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 2212-2220.
  10. Reuben Binns. 2020. On the apparent conflict between individual and group fairness. In Proceedings of the 2020 Con- ference on Fairness, Accountability, and Transparency. 514-524.
  11. Sumon Biswas and Hridesh Rajan. 2021. Fair preprocessing: Towards understanding compositional fairness of data transformers in machine learning pipeline. In Proceedings of the ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. ACM.
  12. Brandon M. Booth, Louis Hickman, Shree Krishna Subburaj, Louis Tay, Sang Eun Woo, and Sidney K. D'Mello. 2021. Integrating psychometrics and computing perspectives on bias and fairness in affective computing: A case study of automated video interviews. IEEE Signal Processing Magazine 38, 6 (2021), 84-95.
  13. Avishek Bose and William Hamilton. 2019. Compositional fairness constraints for graph embeddings. In Proceedings of the International Conference on Machine Learning. PMLR, 715-724.
  14. Sabri Boughorbel, Fethi Jarray, and Abdou Kadri. 2021. Fairness in TabNet model by disentangled representation for the prediction of hospital no-show. arXiv:2103.04048. Retrieved from https://arxiv.org/abs/2103.04048. · arXiv
  15. Ruichu Cai, Zijian Li, Pengfei Wei, Jie Qiao, Kun Zhang, and Zhifeng Hao. 2019. Learning disentangled semantic representation for domain adaptation. In IJCAI: Proceedings of the Conference, Vol. 2019. NIH Public Access, 2060.
  16. Simon Caton and Christian Haas. 2020. Fairness in machine learning: A survey. arXiv:2010.04053. Retrieved from https://arxiv.org/abs/2010.04053. · arXiv
  17. Pengyu Cheng, Weituo Hao, Siyang Yuan, Shijing Si, and Lawrence Carin. 2021. FairFil: Contrastive neural debiasing method for pretrained text encoders. In Proceedings of the International Conference on Learning Representations.
  18. Flavio Chierichetti, Ravi Kumar, Silvio Lattanzi, and Sergei Vassilvitskii. 2017. Fair clustering through fairlets. Ad- vances in Neural Information Processing Systems 30 (2017).
  19. Diane J. Cook and Lawrence B. Holder. 2006. Mining Graph Data. John Wiley & Sons.
  20. Sam Corbett-Davies, Emma Pierson, Avi Feller, Sharad Goel, and Aziz Huq. 2017. Algorithmic decision making and the cost of fairness. In Proceedings of the 23rd ACM Sigkdd International Conference on Knowledge Discovery and Data Mining. 797-806.
  21. Anupam Datta, Matt Fredrikson, Gihyuk Ko, Piotr Mardziel, and Shayak Sen. 2017. Proxy non-discrimination in data- driven systems. arXiv:1707.08120. Retrieved from https://arxiv.org/abs/1707.08120. · arXiv
  22. Pietro G. Di Stefano, James M. Hickey, and Vlasios Vasileiou. 2020. Counterfactual fairness: Removing direct effects through regularization. arXiv:2002.10774. Retrieved from https://arxiv.org/abs/2002.10774. · arXiv
  23. Mengnan Du, Ninghao Liu, and Xia Hu. 2019. Techniques for interpretable machine learning. Communications of the ACM 63, 1 (2019), 68-77.
  24. Mengnan Du, Subhabrata Mukherjee, Guanchu Wang, Ruixiang Tang, Ahmed Awadallah, and Xia Hu. 2021. Fairness via representation neutralization. Advances in Neural Information Processing Systems 34 (2021).
  25. Mengnan Du, Fan Yang, Na Zou, and Xia Hu. 2020. Fairness in deep learning: A computational perspective. IEEE Intelligent Systems (2020).
  26. Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. 2012. Fairness through awareness. In Proceedings of the Innovations in Theoretical Computer Science Conference. 214-226.
  27. Harrison Edwards and Amos Storkey. 2015. Censoring representations with an adversary. arXiv:1511.05897. Retrieved from https://arxiv.org/abs/1511.05897. · arXiv
  28. Yanai Elazar and Yoav Goldberg. 2018. Adversarial removal of demographic attributes from text data. arXiv:1808.06640. Retrieved from https://arxiv.org/abs/1808.06640. · arXiv
  29. Michael Feldman, Sorelle A. Friedler, John Moeller, Carlos Scheidegger, and Suresh Venkatasubramanian. 2015. Cer- tifying and removing disparate impact. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 259-268.
  30. David Garcia-Soriano and Francesco Bonchi. 2021. Maxmin-fair ranking: Individual fairness under group-fairness constraints. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.
  31. Sahaj Garg, Vincent Perot, Nicole Limtiaco, Ankur Taly, Ed H. Chi, and Alex Beutel. 2019. Counterfactual fairness in text classification through robustness. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society. 219-226.
  32. Naman Goel, Mohammad Yaghini, and Boi Faltings. 2021. Counterfactual fairness with disentangled causal effect variational autoencoder. In Proceedings of the AAAI Conference on Artificial Intelligence.
  33. Maya Gupta, Andrew Cotter, Mahdi Milani Fard, and Serena Wang. 2018. Proxy fairness. arXiv:1806.11212. Retrieved from https://arxiv.org/abs/1806.11212. · arXiv
  34. Moritz Hardt, Eric Price, and Nati Srebro. 2016. Equality of opportunity in supervised learning. Advances in Neural Information Processing Systems 29 (2016).
  35. Tatsunori Hashimoto, Megha Srivastava, Hongseok Namkoong, and Percy Liang. 2018. Fairness without demo- graphics in repeated loss minimization. In Proceedings of the International Conference on Machine Learning. PMLR, 1929-1938.
  36. Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In Proceedings of the International Conference on World Wide Web. 173-182.
  37. Ray Jiang, Aldo Pacchiano, Tom Stepleton, Heinrich Jiang, and Silvia Chiappa. 2020. Wasserstein fair classification. In Uncertainty in Artificial Intelligence. PMLR, 862-872.
  38. Zhimeng Jiang, Xiaotian Han, Chao Fan, Zirui Liu, Na Zou, Ali Mostafavi, and Xia Hu. 2022. FMP: Toward fair graph message passing against topology bias. arXiv:2202.04187. Retrieved from https://arxiv.org/abs/2202.04187. · arXiv
  39. Christopher Jung, Sampath Kannan, Changhwa Lee, Mallesh Pai, Aaron Roth, and Rakesh Vohra. 2020. Fair prediction with endogenous behavior. In Proceedings of the ACM Conference on Economics and Computation. 677-678.
  40. Toshihiro Kamishima, Shotaro Akaho, and Jun Sakuma. 2011. Fairness-aware learning through regularization ap- proach. In Proceedings of the IEEE International Conference on Data Mining Workshops. IEEE, 643-650.
  41. Michael Kearns, Seth Neel, Aaron Roth, and Zhiwei Steven Wu. 2018. Preventing fairness gerrymandering: Auditing and learning for subgroup fairness. In Proceedings of the International Conference on Machine Learning. PMLR, 2564- 2572.
  42. Michael Kearns, Seth Neel, Aaron Roth, and Zhiwei Steven Wu. 2019. An empirical study of rich subgroup fairness for machine learning. In Proceedings of the Conference on Fairness, Accountability, and Transparency. 100-109.
  43. Michael P. Kim, Omer Reingold, and Guy N. Rothblum. 2018. Fairness through computationally-bounded awareness. In Proceedings of the International Conference on Neural Information Processing Systems.
  44. Svetlana Kiritchenko and Saif M. Mohammad. 2018. Examining gender and race bias in two hundred sentiment anal- ysis systems. arXiv:1805.04508. Retrieved from https://arxiv.org/abs/1805.04508. · arXiv
  45. Ari Kobren, Barna Saha, and Andrew McCallum. 2019. Paper matching with local fairness constraints. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1247-1257.
  46. Öykü Deniz Köse and Yanning Shen. 2021. Fairness-aware node representation learning. arXiv:2106.05391. Retrieved from https://arxiv.org/abs/2106.05391. · arXiv
  47. Matt J. Kusner, Joshua R. Loftus, Chris Russell, and Ricardo Silva. 2017. Counterfactual fairness. arXiv:1703.06856. Retrieved from https://arxiv.org/abs/1703.06856. · arXiv
  48. Preethi Lahoti, Alex Beutel, Jilin Chen, Kang Lee, Flavien Prost, Nithum Thain, Xuezhi Wang, and Ed H. Chi. 2020. Fair- ness without demographics through adversarially reweighted learning. In Proceedings of the International Conference on Neural Information Processing Systems.
  49. Peizhao Li, Han Zhao, and Hongfu Liu. 2020. Deep fair clustering for visual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 9070-9079.
  50. Frederick Liu and Besim Avci. 2019. Incorporating priors with feature attribution on text classification. In Proceedings of the Annual Meeting of the Association for Computational Linguistics. 6274-6283.
  51. Francesco Locatello, Gabriele Abbati, Thomas Rainforth, Stefan Bauer, Bernhard Schölkopf, and Olivier Bachem. 2019. On the fairness of disentangled representations. In Proceedings of the Advances in Neural Information Processing Sys- tems, Vol. 32. 14611-14624.
  52. Binh Thanh Luong, Salvatore Ruggieri, and Franco Turini. 2011. k-NN as an implementation of situation testing for discrimination discovery and prevention. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 502-510.
  53. David Madras, Elliot Creager, Toniann Pitassi, and Richard Zemel. 2018. Learning adversarially fair and transferable representations. In Proceedings of the International Conference on Machine Learning. PMLR, 3384-3393.
  54. Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. 2021. A survey on bias and fairness in machine learning. ACM Computing Surveys (CSUR) 54, 6 (2021), 1-35.
  55. Amitabha Mukerjee, Rita Biswas, Kalyanmoy Deb, and Amrit P. Mathur. 2002. Multi-objective evolutionary algo- rithms for the risk-return tradeoff in bank loan management. International Transactions in Operational Research 9, 5 (2002), 583-597.
  56. Razieh Nabi and Ilya Shpitser. 2018. Fair inference on outcomes. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 32.
  57. Alejandro Noriega-Campero, Michiel A. Bakker, Bernardo Garcia-Bulle, and Alex'Sandy' Pentland. 2019. Active fair- ness in algorithmic decision making. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society. 77-83.
  58. Sungho Park, Sunhee Hwang, Dohyung Kim, and Hyeran Byun. 2021. Learning disentangled representation for fair facial attribute classification via fairness-aware information alignment. In Proceedings of the AAAI Conference on Ar- tificial Intelligence, Vol. 35. 2403-2411.
  59. Judea Pearl. 2009. Causality. Cambridge University Press.
  60. Dana Pessach and Erez Shmueli. 2020. Algorithmic fairness. arXiv:2001.09784. Retrieved from https://arxiv.org/abs/ 2001.09784. · arXiv
  61. Geoff Pleiss, Manish Raghavan, Felix Wu, Jon Kleinberg, and Kilian Q. Weinberger. 2017. On fairness and calibration. In Proceedings of the International Conference on Neural Information Processing Systems.
  62. John Rawls. 2001. Justice as Fairness: A Restatement. Harvard University Press.
  63. Yuji Roh, Kangwook Lee, Steven Euijong Whang, and Changho Suh. 2020. Fairbatch: Batch selection for model fairness. arXiv:2012.01696. Retrieved from https://arxiv.org/abs/2012.01696. · arXiv
  64. Andrew Slavin Ross, Michael C. Hughes, and Finale Doshi-Velez. 2007. Right for the right reasons: Training differ- entiable models by constraining their explanations. In Proceedings of the International Joint Conference on Artificial Intelligence.
  65. Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto, and Percy Liang. 2019. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. arXiv:1911.08731. Retrieved from https://arxiv.org/abs/1911.0873. · arXiv
  66. Melanie Schmidt, Chris Schwiegelshohn, and Christian Sohler. 2018. Fair coresets and streaming algorithms for fair k-means clustering. arXiv:1812.10854. Retrieved from https://arxiv.org/abs/1812.10854. · arXiv
  67. Xinyue Shen, Steven Diamond, Yuantao Gu, and Stephen Boyd. 2016. Disciplined convex-concave programming. In Proceedings of the IEEE Conference on Decision and Control. IEEE, 1009-1014.
  68. Yujun Shen, Ceyuan Yang, Xiaoou Tang, and Bolei Zhou. 2020. Interfacegan: Interpreting the disentangled face rep- resentation learned by gans. IEEE Transactions on Pattern Analysis and Machine Intelligence (2020).
  69. Chris Sweeney and Maryam Najafian. 2020. Reducing sentiment polarity for demographic attributes in word em- beddings using adversarial learning. In Proceedings of the Conference on Fairness, Accountability, and Transparency. 359-368.
  70. Yao-Hung Hubert Tsai, Martin Q. Ma, Han Zhao, Kun Zhang, Louis-Philippe Morency, and Ruslan Salakhutdi- nov. 2021. Conditional contrastive learning: Removing undesirable information in self-supervised representations. arXiv:2106.02866. Retrieved from https://arxiv.org/abs/2106.02866. · arXiv
  71. Christina Wadsworth, Francesca Vera, and Chris Piech. 2018. Achieving fairness through adversarial learning: An application to recidivism prediction. arXiv:1807.00199. Retrieved from https://arxiv.org/abs/1807.00199. · arXiv
  72. Angelina Wang and Olga Russakovsky. 2021. Directional bias amplification. arXiv:2102.12594. Retrieved from https:// arxiv.org/abs/2102.12594. · arXiv
  73. Tianlu Wang, Jieyu Zhao, Mark Yatskar, Kai-Wei Chang, and Vicente Ordonez. 2019. Balanced datasets are not enough: Estimating and mitigating gender bias in deep image representations. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 5310-5319.
  74. Michael Wick, Swetasudha Panda, and Jean-Baptiste Tristan. 2019. Unlocking fairness: A trade-off revisited. In Pro- ceedings of the International Conference on Neural Information Processing Systems. 8783-8792.
  75. Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rodriguez, and Krishna P. Gummadi. 2017. Fairness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment. In Proceedings of the International Conference on World Wide Web. 1171-1180.
  76. Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez-Rodriguez, and Krishna P. Gummadi. 2019. Fairness constraints: A flexible approach for fair classification. The Journal of Machine Learning Research 20, 1 (2019), 2737-2778.
  77. Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rogriguez, and Krishna P. Gummadi. 2017. Fairness constraints: Mechanisms for fair classification. In Proceedings of the Artificial Intelligence and Statistics. PMLR, 962-970.
  78. Brian Hu Zhang, Blake Lemoine, and Margaret Mitchell. 2018. Mitigating unwanted biases with adversarial learning. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society. 335-340.
  79. Tao Zhang, Jing Li, Mengde Han, Wanlei Zhou, Philip Yu, et al. 2020. Fairness in semi-supervised learning: Unlabeled data help to reduce discrimination. IEEE Transactions on Knowledge and Data Engineering (2020).
  80. Chang Zhou, Jianxin Ma, Jianwei Zhang, Jingren Zhou, and Hongxia Yang. 2021. Contrastive learning for debiased candidate generation in large-scale recommender systems. In Proceedings of the ACM SIGKDD Conference on Knowl- edge Discovery & Data Mining. 3985-3995.

Cited by

Sorted by citations

Authors
Wan, M., Zha, D., Liu, N., & Zou, N.
Affiliations
  • Texas A&M University (US)
  • Rice University (US)
  • University of Georgia (US)
ORCID
Venue
ACM Transactions on Knowledge Discovery from Data
Publisher
Association for Computing Machinery (ACM)
Published
2023-03-20
Volume
17
Issue
3
Pages
1-27
ISSN
  • 1556-4681 (ISSN-L)
  • 1556-472X
DOI
10.1145/3551390
OpenAlex
W4288758404
Cited by
99
References
8
Citations by year
2026: 20 · 2025: 31 · 2024: 33 · 2023: 14 · 2022: 1
FWCI
22.88
Type
journal article
Language
en
Open access
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations
ACM Transactions on Knowledge Discovery from Data (publishedVersion)
Funders
NSF (IIS-1939716, and IIS-1900990)
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Social Sciences › Law › Discrimination and Equality Law
Keywords
algorithmic fairness, algorithmic discrimination, representation learning, fairness metrics

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review