Fairer machine learning in the real world: Mitigating discrimination without collecting sensitive data
Open accessPreprint — published as 10.1177/2053951717743530 · Paper page
The DOI matches its Crossref record; 93 of the 93 references it cites were found in a registry or an index.
Abstract
Cite as: Veale, Michael and Binns, Reuben (2017) Fairer machine learning in the real world: Mitigating discrimination without collecting sensitive data. Big Data & Society 4(2). doi:10.1177/2053951717743530Decisions based on algorithmic, machine learning models can be unfair, reproducing biases in historical data used to train them. While computational techniques are emerging to address aspects of these concerns through communities such as discrimination-aware data mining (DADM) and fair, accountable and transparent machine learning (FATML), their practical implementation faces real-world challenges. For legal, institutional or commercial reasons, organisations might not hold the data on sensitive attributes such as gender, ethnicity, sexuality or disability needed to diagnose and mitigate emergent indirect discrimination-by-proxy, such as redlining. Such organisations might also lack the knowledge and capacity to identify and manage fairness issues that are emergent properties of complex sociotechnical systems.This paper presents and discusses three potential approaches to deal with such knowledge and information deficits in the context of fairer machine learning. Trusted third parties could selectively store data necessary for performing discrimination discovery and incorporating fairness constraints into model-building in a privacy-preserving manner. Collaborative online platforms would allow diverse organisations to record, share and access contextual and experiential knowledge to promote fairness in machine learning systems. Finally, unsupervised learning and pedagogically interpretable algorithms might allow fairness hypotheses to be built for further selective testing and exploration.Real-world fairness challenges in machine learning are not abstract, constrained optimisation problems, but are institutionally and contextually grounded. Computational fairness tools are useful, but must be researched and developed in and with the messy contexts that will shape their deployment, rather than just for imagined situations. Not doing so risks real, near-term algorithmic harm.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Citations
Cites
DiMaggio, P. J., & Powell, W. W. (1983) · American Sociological Review · doi:10.2307/2095101 · Cited by 38,435
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
The Logic of Collective Action: Public Goods and the Theory of Groups.
Sufrin, S. C., & Olson, M. (1966) · Industrial and Labor Relations Review · doi:10.2307/2521228 · Cited by 11,518
Sorting Things Out: Classification and Its Consequences
Bowker, G. C., & Star, S. L. (1999) · doi:10.7551/mitpress/6352.001.0001 · Cited by 6,188
boyd, D., & Crawford, K. (2012) · Information, Communication & Society · doi:10.1080/1369118x.2012.678878 · Cited by 5,479
Sorting Things Out: Classification and Its Consequences
(2000) · The Journal of Academic Librarianship · doi:10.1016/s0099-1333(00)00155-5 · Cited by 5,210
Pandora's Hope: Essays on the Reality of Science Studies
Casper, M. J., & Latour, B. (2000) · Contemporary Sociology · doi:10.2307/2655272 · Cited by 5,049
A survey on concept drift adaptation
Gama, J., Žliobaitė, I., Bifet, A., Pechenizkiy, M., & Bouchachia, A. (2014) · ACM Computing Surveys · doi:10.1145/2523813 · Cited by 3,735
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501
Wikidata: a free collaborative knowledgebase
Vrandečić, D., & Krötzsch, M. (2014) · Communications of the ACM · doi:10.1145/2629489 · Cited by 3,342
Cited by
Items 501–590 of 590
Social Justice With AI: Addressing Bias and Promoting Fairness in AI Systems
Bhargava, A., & Chouhan, R. S. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-6598-5.ch002
Socio-Institutional Issues and Policy Directions for Advanced Sports Policy in the AX Era
Kim, M., & Kwak, M. (2025) · 한국여성체육학회지 · doi:10.16915/jkapesgw.2025.12.39.4.1
Putra, A. S. (2025) · Journal of Cyber Law · doi:10.63913/jcl.v1i4.2
AI Ethics in Financial Decision-Making: Accountability, Transparency, and Bias
Fatimah, N., & Jayashree, K. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-2597-2.ch013
Digital Divide: Equity and Access in AI-Driven Transnational Learning
Pandey, P. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-5403-3.ch011
The Role of Machine Learning Adoption in Enterprises: Literature Review and European Insights
Pallérová, D. (2026) · Inproforum ... · doi:10.32725/978-80-7694-143-4.24
Challenges and Future Directions: Transforming OBE with Machine Learning
Magotra, S., Awasthi, H., Kaur, S., Wasson, V., & Kaur, B. (2026) · Springer eBooks · doi:10.1007/978-981-95-4925-2_9
Algorithmic Contracting, Discrimination, and Human Dignity in Transport
Gheoculescu, A.-V. (2026) · Bulletin of the Transilvania University of Braşov Series VII Social Sciences • Law · doi:10.31926/but.ssl.2025.18.67.3.28
The role of artificial intelligence in forensic analysis of health care data
Sankhla, K., & Sharma, S. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-38299-4.00008-4
Ensuring Equity and Mitigating Harm in AI (Fairness and Bias)
Bhardwaj, R., & Sangal, S. (2026) · IGI Global eBooks · doi:10.4018/979-8-3373-6935-8.ch007
Grădinaru, C., Barbaros, M. C., & Grădinaru, I.-A. (2026) · Postdisciplinary studies in discourse · doi:10.1007/978-3-032-07761-5_10
Olugbade, D., Ajao, S. O., Abuhassna, H., Ifedayo, A. E., & Adelana, O. P. (2026) · IGI Global eBooks · doi:10.4018/979-8-3373-6546-6.ch014 · Cited by 1
Remolina, J. P., Mancipe, Y. C. B., Zárate, A. F. C., Moreno, S. M. G., & Niño, N. F. M. (2025) · Universidad Industrial de Santander
Jovanovic, T. (2026) · Serbian Journal of Engineering Management · doi:10.5937/sjem2600092j
When Robots Should Break the Rules
Ramnauth, R., & Scassellati, B. M. (2026) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/3757279.3788815 · Cited by 1
The CRM Data Quality Advantage - Driving Pharma Performance Through Data Engineering and AI
Nadella, S., Chinthala, S., & Chemuru, V. (2025) · International Conference on Engineering and Emerging Technologies (ICEET) · doi:10.1109/iceet67911.2025.11424103
Asif, S., Mathur, A., & Minz, N. K. (2026) · IGI Global eBooks · doi:10.4018/979-8-3373-3867-5.ch003
Identifying Patterns of Regional Polarization in Russia: A Machine Learning Approach
Balungu, D. M., Rozanova, A. V., Andreeva, K. A., Solod, A. V., & Chen, Y. (2026) · Journal of Applied Economic Research · doi:10.15826/vestnik.2026.25.1.005
Poechhacker, N. (2024) · Directory of Open access Books (OAPEN Foundation)
Sawlani, C., & Dusi, P. (2026) · IGI Global eBooks · doi:10.4018/979-8-3373-2140-0.ch015
Khaustova, Y., & Lebedenko, Y. I. (2026) · Смарт-економіка підприємництво та безпека · doi:10.60022/sis.4.(01).1
Wu, H., & Xu, J. (2026) · Scientific Reports · doi:10.1038/s41598-026-47149-7
GenAI in Business: Opportunities and Challenges
Vassakis, K., Christodoulaki, D., & Perakakis, E. (2026) · Signals and communication technology · doi:10.1007/978-3-032-02705-4_9
Legal challenges of data-driven international M et As
Kurowska, I. (2021) · theses.fr (ABES) · doi:10.70675/44bc335ez24cfz4956z8dc3zd4ee5ea015d4
Fairness in recommender systems : insights from social choice
Do, V. (2023) · theses.fr (ABES) · doi:10.70675/88cc4241z934bz4b88z9c20zfd7d46a54701 · Cited by 4
Privacy and security challenges of explainable artificial intelligence in healthcare
Kandan, M., Mutharasu, M., Ramesh, S., Nagarajan, G., & Tyagi, A. K. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-44111-0.00015-9
Conception de jeux sérieux en réalité mixte : application à l’apprentissage des fractions
Touel, S. (2024) · theses.fr (ABES) · doi:10.70675/58482048z9031z4833za37ezf931d1d0d5cf
He, C., Jahangirirad, P., Kyi, L., & Biega, A. J. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791802 · Cited by 1
The Ethics of Predictive Analytics in Employment and Hiring Practice
Dordunoo, L. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-2600-1016-7.ch001
RETRACTED: She codes, she leads: the tipping point for ethical digital transformation in banking
Bentaleb, D. (2026) · Journal of Information Communication and Ethics in Society · doi:10.1108/jices-09-2025-0246
RetractedGazi, S. E., Ait-Hroch, A., & Ibrahimi, A. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/407448
THE FUTURE OF GENERAL MANAGEMENT IN THE AGE OF INTELLIGENT TECHNOLOGIES
Gupta, P., Gulyani, M., & Dahiya, P. (2026) · doi:10.65647/msp/114/2026
Privacy-Preserving AI: Designing Secure Data Science Frameworks in the Age of Surveillance
Waasi, M. B. M. M. A. (2025) · International Journal of Research & Technology · doi:10.64882/ijrt.v13.i2.1183
Polaszek, J., & Orzeszko, W. (2026) · Intelligent Data Analysis · doi:10.1177/1088467x261441796
Investigation of modern approaches for automated short answer grading
Hussain, S. A., & Abdullah, N. A. Z. (2026) · AIP conference proceedings · doi:10.1063/5.0328153
AI & Tech Challenges to Judicial Values
Žalnieriūtė, M., & Limantė, A. (2026) · Cambridge University Press eBooks · doi:10.1017/9781009744225.018
Intelligente Vertriebslösungen: KI-gestützte Use Cases im B2B-Vertrieb
Kern, K., & Herbst, D. (2026) · Springer eBooks · doi:10.1007/978-3-658-49614-2_106
Fountoukidis, I. (2026) · London School of Economics and Political Science Research Online (London School of Economics and Political Science) · doi:10.5281/zenodo.20175803
Diversity in the Digital Age: Cybervetting, Doxxing, and Employment Discrimination
Melson-Silimon, A. T., Harmata, R., LeFevre-Levy, R., Behrend, T. S., & Carter, N. T. (2022) · Emerald Group Publishing Limited eBooks · doi:10.1108/978-1-64802-826-720251007
Data Privacy in Public Employment: Impacts of the UK GDPR and Nigeria's NDPR 2023
Orisankoko, A. S. (2026) · International Journal of Research and Innovation in Social Science · doi:10.47772/ijriss.2026.100500202
Evaluating the Safety of Medical LLMs for Patient Use
Vishva, R., Fahmitha, F. F., Indhu, S., Gayathri, R., & Saranraj, P. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-7862-6.ch006
Zaga, F. R. (2026) · Revista Digital de Derecho Administrativo · doi:10.18601/21452946.n36.03
Lee, H. P., Sidnam-Mauch, E., Mendoza, S., Wane, K., Knijnenburg, B. P., McGregor, S. E., & Caine, K. E. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812246
Controllable Many-Objective Optimization for Fairness-Aware Uplift Modeling in Recommender Systems
Oladejo, S. O. (2026) · IEEE Access · doi:10.1109/access.2026.3709619
George, B. (2026) · Machine learning for computational science and engineering · doi:10.1007/s44379-026-00082-y
Khanum, B. H., & Durrani, O. K. (2026) · International Journal of Modern Science and Research Technology · doi:10.67317/ijmsrt26jul027
Boban, M., Gombar, M., & Bachal, M. P. (2026) · International Convention on Information and Communication Technology, Electronics and Microelectronics · doi:10.1109/mipro70003.2026.11591781
From Algorithmic Welfare to Participatory Data Justice
Gripsy, J. V., Praneesh, M., Senthilkumar, G., Baskar, G., Kamalam, K. S., & Sheeba, L. (2026) · IGI Global eBooks · doi:10.4018/979-8-3373-8423-8.ch005
Audit Kredit Digital Berbasis Explainable Artificial Intelligence (XAI): Tinjauan Pustaka Sistematis
Sutisna, M. A., Riadi, I., & Fadlil, A. (2026) · Jurnal Teknologi Terpadu · doi:10.54914/jtt.v12i1.2614
Humanism and AI-Driven Vulnerabilities: Customers, Partners, Employees
Fioravante, R., & Vaccaro, A. (2026) · Humanism in business series · doi:10.1007/978-3-032-17121-4_5
Emerging trends and technologies in explainable AI (XAI) for clinical practice
Nazreen, Thayanithi, C. A., Arjun, E., Singh, P., & Hetu, V. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-45374-8.00012-5
Trust by design: Human factors and trust calibration in ML-based educational systems
Anthis, Z., & Kyriakidou-Zacharoudiou, A. (2026) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2026.103893
The Sovereignty Stack - Engineering Integrity from Classical to Quantum AIPreprint
Rocha, A. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6988000
The Future of Resource Optimization with AI and Machine Learning
Mir, K. H., Bashir, A., & Dar, H. M. U. D. (2026) · BENTHAM SCIENCE PUBLISHERS eBooks · doi:10.2174/9798898815042126010013
Amenitsky, A. V., & Vorobyov, E. G. (2026) · CyberLeninK (CyberLeninka)
Compliance-Aware Feature Engineering for Predictive Analytics in Financial Cloud Data Architectures
-, A. V. (2018) · International Journal of Data Engineering and Intelligent Computing · doi:10.67228/30715717/ijdeic-2018pii4n8b
Justice, Algorithms, and Accountability: Ethical Considerations for AI in the Judiciary
Pandey, A. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-5213-8.ch015
Bridging Technology and Ethics in Dialogue Systems: A Survey of Real-World Challenges
Shah, J., & Thakkar, P. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-27154-9_37
La inteligencia artificial como herramienta de innovación de la Administración pública
González, R. N. (2025) · RIO (Universidad Pablo de Olavide)
Same System, Opposite Verdicts: Metric Discretion in AI Ethics Audits and the Limits of DisclosurePreprint
Tsaban, S. (2026) · Research Square · doi:10.21203/rs.3.rs-10576395/v1
Accessibility of Service Digital Infrastructures
Giudici, M., & Brambilla, M. (2026) · SpringerBriefs in applied sciences and technology · doi:10.1007/978-3-032-33964-5_6
Balancing Innovation and Academic Integrity
Sinha, E., Chandra, S., & Waghmare, C. D. (2026) · Emerald Group Publishing Limited eBooks · doi:10.1108/978-1-83742-179-420261006
Yapay Zekâ ve İnsan Zekâsı İş Birliğiyle Kamu Değeri Yaratımı: Hibrit Yönetişim Modellerinin Rolü
Uzun, H. (2026) · Ankara Hacı Bayram Veli Üniversitesi İktisadi ve İdari Bilimler Fakültesi Dergisi · doi:10.26745/ahbvuibfd.1763217
Racial Inclusion in AI-Powered Learning Environments
Mumtaz, M., Tayyab, M. A., Younis, N., & Oshodin, E. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-2600-3663-1.ch012
Chen, Z., Chen, B., Qian, W., & Huang, J. (2026) · Neurocomputing · doi:10.1016/j.neucom.2026.134901
Temporal Injustice in Human-Reviewed Public-Sector AIPreprint
Dimakopoulos, S. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7354124
AI Presents Both Problems and Opportunities for Minorities
Triana, M. D. C., & Upadhyay, A. (2026) · Journal of Management Studies · doi:10.1111/joms.70145 · Cited by 1
Zero-Based Budgeting and Artificial Intelligence Applied to the Brazilian Public Sector
Mourão, L., de Araújo, L. V., & Bliacheriene, A. C. (2026) · Open Journal of Social Sciences · doi:10.4236/jss.2026.148049
Potential of Artificial Intelligence in Mitigating Inclusive and Affordable Housing Crisis in IndiaPreprint
Bhattacharya, O. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7369718
Privacy, Data Sovereignty, and Consent Ethics of Agentic AI in Higher Education
Nwasra, N. M., & Alharthy, A. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-2600-4050-8.ch002
Saeudy, M., Foster, M., & Abdou, H. A. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7468006
Yadav, P., Kumari, J., Kumar, B. P., Nagaraj, D. S., Sharma, K., Mahboob, M. S., Glory, K. B., & Gupta, D. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-9164-9.ch006
Artificial intelligence in tax compliance: A bibliometric mapping and research agenda (2015–2025)
Zaim, H., & Sahbani, S. (2026) · European Journal of Sustainable Development Research · doi:10.29333/ejosdr/19260
Guan, H., Saade, R., Liu, H., Qu, G., Zhao, M., & Xu, Z. (2025) · Revista Cubana de Administración Pública y Empresarial
Sukumar, L. D. K. (2026) · The International Journal of Interdisciplinary Cultural Studies · doi:10.18848/7knjyz87
Transparansi Algoritma Kecerdasan Buatan sebagai Instrumen Perlindungan Hak Pengguna di Indonesia
Ardiansyah, K., Permatasari, I., & Rayhan, M. (2026) · Legal Note · doi:10.70716/legalnote.v2i3.340 · Cited by 1
Braeken, A., Wellner, G., Chatzimisios, P., Guo, Y., Kanaki, E., Kanta, A., Kour, R., Meishar‐Tal, H., Moremada, C., Nallur, V., et al. (2026) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2026.3740371
Customer Expectations of AI in Retail Banking: A Vignette-Based Qualitative Study
Huopalainen, J. (2026) · UTUPub (University of Turku)
Exploring How AI Can Support and Drive Inclusivity, Belonging, and Social Justice Strategies
Sharma, R., & Dar, J. A. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-2600-4800-9.ch005
Zipperling, D., Deck, L., Lanzl, J., & Kühl, N. (2026) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3858913
Ababneh, J., Al-Nsour, E., Alasasfeh, M., Aljbour, A., Ahmad, A. A., Alshaikh, A., Al-Ababneh, H. A., & Hafez, M. (2026) · International Conference on Networks, Computers and Communications (ICNCCom) · doi:10.1109/icnccom69117.2026.11736202
Hasan, R., & Mahmood, S. (2026) · Annals of Data Science · doi:10.1007/s40745-026-00733-0
Halder, S. R., Mukherjee, A., Tripathi, A., Bansal, M., Sridevi, R., & Seniasamy, R. (2026) · Research Square · doi:10.21203/rs.3.rs-11152053/v1
The Human Stakes: Fairness Across Domains and Populations
Tavakoli, H. (2026) · Apress eBooks · doi:10.1007/979-8-8688-2885-0_3
Kaplan, Y. (2026) · Kamu Yönetimi ve Politikaları Dergisi · doi:10.58658/kaypod.1884485
Linking Principles to Practice: What the FDK Adds
Tavakoli, H. (2026) · Apress eBooks · doi:10.1007/979-8-8688-2885-0_4
Filho, P. L. C., & Cid, R. (2026) · PhilPapers (PhilPapers Foundation)
Metadata
- Authors
- Veale, M., & Binns, R.
- Affiliations
- University College London (GB)
- University of Oxford (GB)
- Corresponding author
- Veale, Michael
- Venue
- Big Data & Society
- Publisher
- California Digital Library (CDL)
- Published
- 2017-10-27
- Volume
- 4
- Issue
- 2
- Pages
- 205395171774353-205395171774353
- ISSN
- 2053-9517 (ISSN-L)
- OpenAlex
- W2765146466
- Cited by
- 512
- References
- 48
- Citations by year
- 2026: 84 · 2025: 97 · 2024: 59 · 2023: 74 · 2022: 78 · 2021: 50 · 2020: 35 · 2019: 20 · 2018: 14 · 2017: 1
- FWCI
- 43.94
- Type
- journal article
- Language
- en
- Open access
- gold
- dx.doi.org
- Locations
- Big Data & Society (publishedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- UCL Discovery (University College London) (submittedVersion)
- SocArXiv (OSF Preprints) (submittedVersion, cc-by)
- OSF Preprints (OSF Preprints) (submittedVersion, cc-by)
- Funders
- Engineering and Physical Sciences Research Council (EP/J017728/2)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Keywords
- algorithmic fairness, indirect discrimination, sensitive attributes, trusted third party, collaborative platforms, unsupervised learning, interpretable algorithms, algorithmic harms, redlining, socio-technical systems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.