Fairer machine learning in the real world: Mitigating discrimination without collecting sensitive data
Open accessPreprint — published as 10.1177/2053951717743530 · Paper page
The DOI matches its Crossref record; 93 of the 93 references it cites were found in a registry or an index.
Abstract
Cite as: Veale, Michael and Binns, Reuben (2017) Fairer machine learning in the real world: Mitigating discrimination without collecting sensitive data. Big Data & Society 4(2). doi:10.1177/2053951717743530Decisions based on algorithmic, machine learning models can be unfair, reproducing biases in historical data used to train them. While computational techniques are emerging to address aspects of these concerns through communities such as discrimination-aware data mining (DADM) and fair, accountable and transparent machine learning (FATML), their practical implementation faces real-world challenges. For legal, institutional or commercial reasons, organisations might not hold the data on sensitive attributes such as gender, ethnicity, sexuality or disability needed to diagnose and mitigate emergent indirect discrimination-by-proxy, such as redlining. Such organisations might also lack the knowledge and capacity to identify and manage fairness issues that are emergent properties of complex sociotechnical systems.This paper presents and discusses three potential approaches to deal with such knowledge and information deficits in the context of fairer machine learning. Trusted third parties could selectively store data necessary for performing discrimination discovery and incorporating fairness constraints into model-building in a privacy-preserving manner. Collaborative online platforms would allow diverse organisations to record, share and access contextual and experiential knowledge to promote fairness in machine learning systems. Finally, unsupervised learning and pedagogically interpretable algorithms might allow fairness hypotheses to be built for further selective testing and exploration.Real-world fairness challenges in machine learning are not abstract, constrained optimisation problems, but are institutionally and contextually grounded. Computational fairness tools are useful, but must be researched and developed in and with the messy contexts that will shape their deployment, rather than just for imagined situations. Not doing so risks real, near-term algorithmic harm.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Citations
Cites
DiMaggio, P. J., & Powell, W. W. (1983) · American Sociological Review · doi:10.2307/2095101 · Cited by 38,435
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
The Logic of Collective Action: Public Goods and the Theory of Groups.
Sufrin, S. C., & Olson, M. (1966) · Industrial and Labor Relations Review · doi:10.2307/2521228 · Cited by 11,518
Sorting Things Out: Classification and Its Consequences
Bowker, G. C., & Star, S. L. (1999) · doi:10.7551/mitpress/6352.001.0001 · Cited by 6,188
boyd, D., & Crawford, K. (2012) · Information, Communication & Society · doi:10.1080/1369118x.2012.678878 · Cited by 5,479
Sorting Things Out: Classification and Its Consequences
(2000) · The Journal of Academic Librarianship · doi:10.1016/s0099-1333(00)00155-5 · Cited by 5,210
Pandora's Hope: Essays on the Reality of Science Studies
Casper, M. J., & Latour, B. (2000) · Contemporary Sociology · doi:10.2307/2655272 · Cited by 5,049
A survey on concept drift adaptation
Gama, J., Žliobaitė, I., Bifet, A., Pechenizkiy, M., & Bouchachia, A. (2014) · ACM Computing Surveys · doi:10.1145/2523813 · Cited by 3,735
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501
Wikidata: a free collaborative knowledgebase
Vrandečić, D., & Krötzsch, M. (2014) · Communications of the ACM · doi:10.1145/2629489 · Cited by 3,342
Cited by
Items 401–500 of 590
Ethical Considerations in AI Design and Deployment: Insights into Sensitive IndustriesPreprint
Rongali, S. K. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5220284 · Cited by 1
Requirements engineering for explainable systems
Chazette, L. (2023) · Institutional Repository of Leibniz Universität Hannover (Leibniz Universität Hannover) · doi:10.15488/13261 · Cited by 6
Adversarial mitigation to reduce unwanted biases in machine learning
Grari, V. (2022) · theses.fr (ABES) · doi:10.70675/df2cbae5zd700z4b62z845dz004fa5581c09 · Cited by 3
Auditing a Dutch Public Sector Risk Profiling Algorithm Using an Unsupervised Bias Detection Tool
Holstege, F., Jorgensen, M., Padh, K., Parie, J., Proroković, K., Persson, J., & Snoek, L. (2025) · arXiv · doi:10.48550/arxiv.2502.01713
Identifying and Mitigating Bias in Machine Learning Applications
Bald, L. (2019) · Scholars' Bank (University of Oregon) · Cited by 1
Legally grounded fairness objectivesPreprint
Holden-Sim, D., Leech, G., & Aitchison, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.11677
A Discrimination Aware Model to Predict Childhood Literacy Levels
Byrne, K. (2020) · Arrow - TU Dublin (Technological University Dublin) · doi:10.21427/cmtz-1m13
Bouchagiar, G., & Koutras, N. (2020) · Information technology and law series/Information technology & law series · doi:10.1007/978-94-6265-407-5_5
General Perspectives Toward the Impact of AI on Race and Society
Roberts, A. L., Richardson, B., Alikhademi, K., Drobina, E., & Gilbert, J. E. (2021) · Diversity and inclusion research · doi:10.1007/978-3-030-65417-7_18
Adapting by copying. Towards a sustainable machine learning
Unceta, I. (2021) · Dipòsit Digital de la Universitat de Barcelona (Universitat de Barcelona) · Cited by 1
Beretta, E. (2021) · PORTO@Iris · doi:10.5281/zenodo.7825544
Assessing Algorithmic Fairness without Sensitive Information
Noiret, S. (2021) · International Conference on Information Technology for Social Good (GoodIT) · doi:10.1145/3462203.3475894
Reflections on Algorithmic Reputation: Judgment and Equity in a Digitally Mediated Society
Katell, M. A. (2020) · ResearchWorks at the University of Washington (University of Washington)
Islamic Sunni perspectives on the ethics of precision medicine
Suleman, M. (2022) · Oxford University Press eBooks · doi:10.1093/oso/9780198863465.003.0017
Fins, J. J. (2022) · Oxford University Press eBooks · doi:10.1093/oso/9780198863465.003.0010
The revolution of personalized medicine is already upon us in rare diseases
Austin, C. P. (2022) · Oxford University Press eBooks · doi:10.1093/oso/9780198863465.003.0009
Gender and personalized medicine: Methodological and ethical pitfalls
Legato, M. J. (2022) · Oxford University Press eBooks · doi:10.1093/oso/9780198863465.003.0011
Personalized medicine and genetic newborn screening
Brusa, M., & Dickenson, D. L. (2022) · Oxford University Press eBooks · doi:10.1093/oso/9780198863465.003.0008 · Cited by 1
Genetics, genetic profiles, and Jewish law
Weissinger, Y., & Barilan, Y. M. (2022) · Oxford University Press eBooks · doi:10.1093/oso/9780198863465.003.0018
Precision and persons in medicine: An introduction
Barilan, Y. M., & Brusa, M. (2022) · Oxford University Press eBooks · doi:10.1093/oso/9780198863465.003.0001
Risk Prediction of Opioid Use Disorder (OUD) using Electronic Health Records
Cao, W. (2022) · Digital Commons - URI (University of Rhode Island) · doi:10.23860/diss-cao-wenqiu-2022 · Cited by 1
The Causes and Effects of Algorithmic Decision Making
Saluja, S., Kethar, J., Appavu, R., & Ahorta, M. (2022) · Journal of Student Research · doi:10.47611/jsrhs.v11i4.3327
Peculiarities of the application of machine learning algorithms in humanities
Chursinova, O. Y. (2023) · Izdevniecība “Baltija Publishing” eBooks · doi:10.30525/978-9934-26-319-4-18
Floridi, L. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780198883098.002.0009
Floridi, L. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780198883098.003.0001
Floridi, L. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780198883098.002.0004
Mapping the Ethics of Algorithms
Floridi, L. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780198883098.003.0007
How to Deliver a Good AI Society: Some Recommendations
Floridi, L. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780198883098.003.0010
Floridi, L. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780198883098.002.0007
Bad Practices: The Misuse of AI for Social Evil
Floridi, L. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780198883098.003.0008
List of Most Common Abbreviations and Acronyms
Floridi, L. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780198883098.002.0011
The Gambit: AI Impact on Climate Change
Floridi, L. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780198883098.003.0011
Floridi, L. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780198883098.002.0010
Good Practices: The Proper Use of AI for Social Good
Floridi, L. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780198883098.003.0009
Floridi, L. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780198883098.002.0005
Conclusion: The Green and the Blue
Floridi, L. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780198883098.003.0013
Discurso influenciado: aprendizaje automático y discurso de odio
Jaimes, F. J. (2023) · Daímon · doi:10.6018/daimon.562091
Prieto, J. V. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-42935-4_22
Wang, X., Verhoef, A., Hou, J., Jia, L., Carvalho, H. D. R., & He, H. (2026) · Soil and Tillage Research · doi:10.1016/j.still.2026.107283
A complicated picture: Media diversity in the case of Google’s video search during the pandemic
Wang, Q. (2023) · Internet Policy Review · doi:10.14763/2023.4.1728
Laveraging Machine Learning for Improved Project Inspections and Internal Control Systems
Savković, M., & Lalić, D. Ć. (2024) · Engineering cyber-physical systems and critical infrastructures · doi:10.1007/978-3-031-56322-5_20
Barilan, Y. M. (2022) · Oxford University Press eBooks · doi:10.1093/oso/9780198863465.002.0005
Barilan, Y. M., Brusa, M., & Ciechanover, A. J. (2022) · Oxford University Press eBooks · doi:10.1093/oso/9780198863465.002.0006
Barilan, Y. M., Brusa, M., & Ciechanover, A. J. (2022) · Oxford University Press eBooks · doi:10.1093/oso/9780198863465.002.0003
Uncovering Discrimination Generated by Different Machine Learning Methods Using Data VisualizationPreprint
Calcina, J. E., & Freitas, C. M. D. S. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4844651
Ethical Leadership in the Age of AI
(2024) · Advances in logistics, operations, and management science book series · doi:10.4018/979-8-3693-2695-4.ch008
Künstliche Intelligenz in der Persönlichkeitsdiagnostik
Horstmann, K. T., & Ziegler, M. D. (2024) · Springer eBooks · doi:10.1007/978-3-662-64801-8_133-1
Estimating Group Fairness Using Pairwise Similarity
Supeesun, A., Chawachat, J., Mettanant, V., & Fakcharoenphol, J. (2024) · International Joint Conference on Computer Science and Software Engineering · doi:10.1109/jcsse61278.2024.10613692
The need of E-health and literacy of cancer patients for Healthcare providers
Kalra, R., Gupta, M., & Sharma, P. (2024) · Elsevier eBooks · doi:10.1016/b978-0-443-13619-1.00010-6
Olivier, A. (2024) · Casa Leiria eBooks · doi:10.29327/5448881.1-6 · Cited by 1
Mitigating algorithmic bias in the UK civil service: Challenges and implications for practice
Beresford, H. (2026) · Big Data & Society · doi:10.1177/20539517261473357
The Role of Innovation in Social Awareness
AlRayssi, R. C. (2025) · Journal of Lifestyle and SDGs Review · doi:10.47172/2965-730x.sdgsreview.v5.n01.pe04575
Künstliche Intelligenz in der Persönlichkeitsdiagnostik
Horstmann, K. T., & Ziegler, M. D. (2025) · Springer eBooks · doi:10.1007/978-3-662-64797-4_133
A multivariate randomized response model for mixed-type data
Chu, A. M. Y., Omori, Y., So, H., & So, M. K. P. (2025) · Journal of Applied Statistics · doi:10.1080/02664763.2025.2480865
The Future of Data Governance: Ethical and Legal Considerations in AI-Driven Analytics
Cherekar, R. (2022) · International Journal of Artificial Intelligence Data Science and Machine Learning · doi:10.63282/3050-9262.ijaidsml-v3i2p103 · Cited by 1
Wright, C. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5343469
Code and Character: The Ethical Journey of Artificial Intelligence
Gündüz, A. Y. (2025) · IGI Global eBooks · doi:10.4018/979-8-3373-1702-1.ch001
Elgujja, A. A., Arimoro, A. E., Hersi, A. S., Fatimah, F., Musa, H., & Ezreqat, S. (2026) · Journal of Nature and Science of Medicine · doi:10.4103/jnsm.jnsm_239_25
Evaluating Dataset Bias in Biometric ResearchPreprint
Heleen, B. (2025) · Preprints.org · doi:10.20944/preprints202508.1479.v1
Elgujja, A. A., Alshahrani, F. S., Arimoro, A. E., Hersi, A. S., Musa, H., & Ezreqat, S. (2025) · doi:10.2196/preprints.82912
Cybersecurity Challenges in Open Data and AI Systems for Sustainable Local Development
Grönman, J., Pohjola, T., & Haukioja, T. (2025) · International Convention on Information and Communication Technology, Electronics and Microelectronics · doi:10.1109/mipro65660.2025.11132045
Giuseppe, T. D., & Barone, S. L. (2025) · Form re - Open Journal per la formazione in rete · doi:10.36253/form-17173
Solving Financial Data Imbalanced Distribution Problem Using Simple Balancing Approach
Aljawazneh, H., & Smadi, Y. (2025) · Studies in systems, decision and control · doi:10.1007/978-3-031-90271-0_54
Uloga umjetne inteligencije u liječenju psihijatrijskih poremećaja
Erić, A. P., Erić, I., Anton, S., & Kovačević, M. P. (2025) · Znanstveni sabor Slavonije i Baranje · doi:10.21857/moxpjhz31m
Ethical AI in the Supply Chain: Driving Responsible and Sustainable Flow Optimization
Bentatou, N., & Lemqeddem, H. A. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-3104-1.ch007
Advancements in Artificial Intelligence: The Role of Deep Learning in Automated Decision-Making
Shrivastava, A. K., Sushma, J., Tomar, A. S., Talreja, P., Kohli, N. K., & Mishra, A. (2025) · World Skills Conference on Universal Data Analytics and Sciences (WorldSUAS) · doi:10.1109/worldsuas66815.2025.11199223
SHAPE: Symptom-based Hallucination-driven Augmented Prompt Engineering (Preprint)Preprint
Kim, Y.-J., Shin, J., You, H.-S., Yoon, S., An, D.-J., Lee, A.-Y., Yoo, I.-S., Jeon, H. J., Lee, J. W., & Kim, K. D. (2025) · doi:10.2196/preprints.86028
Ethical Implications of AI-Based Hiring Tools
Goel, E. L. (2025) · Scientific Journal of Artificial Intelligence and Blockchain Technologies · doi:10.63345/sjaibt.v2.i3.108
Ai-Augmented Public Administration: Balancing Innovation with Democratic Values
Heri, H. (2025) · International Conference on Sustainable Environment and Innovation (ICOSEI) · doi:10.53675/icosei.v1i1.1485
Shahidi, F., MacDonald, M. E., Seitz, D. P., Barry, R., & Messier, G. G. (2025) · PLOS Digital Health · doi:10.1371/journal.pdig.0001064 · Cited by 1
Deep learning techniques in neuroimaging
Kovač, N., Simeunović, M., Farahani, H., & Watson, P. (2025) · Elsevier eBooks · doi:10.1016/b978-0-443-30248-0.00012-7
Paz, C. D. (2025) · IEEE International Symposium on Technology and Society (ISTAS) · doi:10.1109/istas65609.2025.11269665
Skiba, Ł. (2025) · Scientific Papers of Silesian University of Technology Organization and Management Series · doi:10.29119/1641-3466.2025.234.24
Von Requirements zu Privacy Explanations: Ein nutzerzentrierter Ansatz für Usable Privacy
Brunotte, W. (2024) · Institutional Repository of Leibniz Universität Hannover (Leibniz Universität Hannover) · doi:10.15488/16335
Towards a psychological science of neural network behaviour
Bell, S. J. (2022) · Apollo (University of Cambridge) · doi:10.17863/cam.101916
Research Group Critical Information Infrastructures (Cii), Sunyaev, A., Renner, M., Toussaint, P. A., Thiebes, S., & Lins, S. (2022) · Repository KITopen (Karlsruhe Institute of Technology) · doi:10.5445/ir/1000150078
Les technologies et les politiques publiques en appui à l’essor de l’intelligence artificielle
Beaudry, C., Hage, G., Piantoni, G., & Solar-Pelletier, L. (2020) · PolyPublie (École Polytechnique de Montréal)
978-3-95796-225-6-Poechhacker.pdf
Poechhacker, N. (2024) · OAPEN (The OAPEN Foundation)
Science des données, réseaux sociaux et politiques publiques
Aubert, B. A., de Marcellis-Warin, N., & Warin, T. (2020) · PolyPublie (École Polytechnique de Montréal) · Cited by 4
Bias in, bias out: A Study on Social Bias in Automated Decision-Making Algorithms
Mohammad, S. H. A. (2021) · Utrecht University Repository (Utrecht University) · Cited by 1
Exploring the Ethical Implications of AI Algorithms in Decision-Making Processes
Atul Verma, R. G. (2024) · PhilPapers (PhilPapers Foundation)
Naudts, L. (2023) · Lirias (KU Leuven)
Le Québec économique – Perspectives et enjeux de la transformation numérique
(2020) · PolyPublie (École Polytechnique de Montréal)
Paloheimo, O. (2023) · Aaltodoc (Aalto University) · Cited by 1
Evaluation of Pruning Algorithms for Activity Recognition on Embedded Machine Learning
Namazi, A. (2023) · KTH Publication Database DiVA (KTH Royal Institute of Technology)
La révolution numérique appliquée à l'agriculture au Québec
Royer, A., de Marcellis-Warin, N., Peignier, I., & Warin, T. (2020) · PolyPublie (École Polytechnique de Montréal)
Investigating algorithmic bias mitigation in the public sector
Beresford, H. (2023) · White Rose eTheses Online (University of Leeds, The University of Sheffield, University of York) · Cited by 1
Fairness in Zeiten Künstlicher Intelligenz
Brink, A. (2025) · Nomos Verlagsgesellschaft mbH & Co. KG eBooks · doi:10.5771/9783748965039
Fairness aus Prinzip - Reflexionsleitfaden für faire KI-Entscheidungen in Unternehmen
Christ, A.-S. (2025) · Nomos Verlagsgesellschaft mbH & Co. KG eBooks · doi:10.5771/9783748965039-147
Balancing Bias Mitigation and Data Protection in AI-Driven Healthcare
Doğan, F. S. (2025) · University of Vienna Law Review · doi:10.25365/vlr-2025-9-3-99
Machine Learning, Law and Ethics: The Discrimination Problem and Solution
Agnello, V., & Kumiega, A. (2025) · Business and Professional Ethics Journal · doi:10.5840/bpej20251112176 · Cited by 1
Freund, L., Diehl, F., & Brunnbauer, M. (2025) · Springer eBooks · doi:10.1007/978-3-658-48425-5_7
AI Ethics Guidelines: Is regulation a value of trust?
Almeida, A. P. P. P. B. (2025) · Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT)
Dordunoo, L. (2025) · RAIS Journal for Social Sciences · doi:10.5281/zenodo.17648737
On the “bullshit” of the intelligent explosion and singularity discourse
Ugar, E. T. (2025) · South African Journal of Philosophy · doi:10.1080/02580136.2025.2536950
Intelligente Vertriebslösungen: KI-gestützte Use Cases im B2B-Vertrieb
Kern, K., & Herbst, D. (2026) · Springer eBooks · doi:10.1007/978-3-658-46399-1_106-1
Metadata
- Authors
- Veale, M., & Binns, R.
- Affiliations
- University College London (GB)
- University of Oxford (GB)
- Corresponding author
- Veale, Michael
- Venue
- Big Data & Society
- Publisher
- California Digital Library (CDL)
- Published
- 2017-10-27
- Volume
- 4
- Issue
- 2
- Pages
- 205395171774353-205395171774353
- ISSN
- 2053-9517 (ISSN-L)
- OpenAlex
- W2765146466
- Cited by
- 512
- References
- 48
- Citations by year
- 2026: 84 · 2025: 97 · 2024: 59 · 2023: 74 · 2022: 78 · 2021: 50 · 2020: 35 · 2019: 20 · 2018: 14 · 2017: 1
- FWCI
- 43.94
- Type
- journal article
- Language
- en
- Open access
- gold
- dx.doi.org
- Locations
- Big Data & Society (publishedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- UCL Discovery (University College London) (submittedVersion)
- SocArXiv (OSF Preprints) (submittedVersion, cc-by)
- OSF Preprints (OSF Preprints) (submittedVersion, cc-by)
- Funders
- Engineering and Physical Sciences Research Council (EP/J017728/2)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Keywords
- algorithmic fairness, indirect discrimination, sensitive attributes, trusted third party, collaborative platforms, unsupervised learning, interpretable algorithms, algorithmic harms, redlining, socio-technical systems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.