The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.
Fairer machine learning in the real world: Mitigating discrimination without collecting sensitive data
Open accessPublished version of a preprint
Abstract
Decisions based on algorithmic, machine learning models can be unfair, reproducing biases in historical data used to train them. While computational techniques are emerging to address aspects of these concerns through communities such as discrimination-aware data mining (DADM) and fairness, accountability and transparency machine learning (FATML), their practical implementation faces real-world challenges. For legal, institutional or commercial reasons, organisations might not hold the data on sensitive attributes such as gender, ethnicity, sexuality or disability needed to diagnose and mitigate emergent indirect discrimination-by-proxy, such as redlining. Such organisations might also lack the knowledge and capacity to identify and manage fairness issues that are emergent properties of complex sociotechnical systems. This paper presents and discusses three potential approaches to deal with such knowledge and information deficits in the context of fairer machine learning. Trusted third parties could selectively store data necessary for performing discrimination discovery and incorporating fairness constraints into model-building in a privacy-preserving manner. Collaborative online platforms would allow diverse organisations to record, share and access contextual and experiential knowledge to promote fairness in machine learning systems. Finally, unsupervised learning and pedagogically interpretable algorithms might allow fairness hypotheses to be built for further selective testing and exploration. Real-world fairness challenges in machine learning are not abstract, constrained optimisation problems, but are institutionally and contextually grounded. Computational fairness tools are useful, but must be researched and developed in and with the messy contexts that will shape their deployment, rather than just for imagined situations. Not doing so risks real, near-term algorithmic harm.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The Article 22 prohibition is limited to decisions which produce legal or similarly significant effects concerning the data subject 41 .”
From Administrative law and the machines of government: judicial review of automated public-sector decision-making (2019) · cites this paper as [41]
In the citing paper
“Many of the methods to tackle discrimination in AI systems implicitly assume that organisations hold these sensitive data”
From Discrimination, artificial intelligence, and algorithmic decision-making (2025) · cites this paper as Veale and Binns 2017
In the citing paper
“Not considering correlated variables has been shown to increase the risk of discrimination (e.g., redlining 4 ) [54,88,89,89,196,202,231,247,286,308].”
From Fairness in machine learning: A survey (2024) · cites this paper as [286]
In the citing paper
“Computing by hand can introduce new biases without training [286], as fairness is typically not monotonic, thus assigning a threshold value may be quite arbitrary [270].”
From Fairness in machine learning: A survey (2024) · cites this paper as [286]
In the citing paper
“However, beyond this, critics of current research [50,193,266,286,287,297,297,306] argue that eforts will fail unless contextual, sociocultural, and social policy challenges are better understood.”
From Fairness in machine learning: A survey (2024) · cites this paper as [286]
4 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
DiMaggio, P. J., & Powell, W. W. (1983) · American Sociological Review · doi:10.2307/2095101 · Cited by 38,435
Sorting Things Out: Classification and Its Consequences
Bowker, G. C., & Star, S. L. (1999) · doi:10.7551/mitpress/6352.001.0001 · Cited by 6,188
boyd, D., & Crawford, K. (2012) · Information, Communication & Society · doi:10.1080/1369118x.2012.678878 · Cited by 5,479
Pandora's Hope: Essays on the Reality of Science Studies
Casper, M. J., & Latour, B. (2000) · Contemporary Sociology · doi:10.2307/2655272 · Cited by 5,049
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501
Wikidata: a free collaborative knowledgebase
Vrandečić, D., & Krötzsch, M. (2014) · Communications of the ACM · doi:10.1145/2629489 · Cited by 3,342
How the machine “thinks”: Understanding opacity in machine learning algorithms
Burrell, J. (2016) · Big Data & Society · doi:10.1177/2053951715622512 · Cited by 3,196 · cited 2×
“These are nested within a range of linked concerns, including algorithmic transparency and accountability (Burrell, 2016; Keats Citron and Pasquale, 2014; Kroll et al., 2016; Nissenbaum, 1996), in-the-wild reliability (Žliobaite_ et al.”
BackgroundBarocas, S., & Selbst, A. D. (2014) · doi:10.2139/ssrn.2477899 · Cited by 3,048
Semantics derived automatically from language corpora contain human-like biases
Caliskan, A., Bryson, J. J., & Narayanan, A. (2017) · Science · doi:10.1126/science.aal4230 · Cited by 2,887
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536
Research works printed but not linked: 10
- Angwin J, 2016, ProPublica, 23
- Barocas S, 2016, California Law Review, 104, 671
- Hacking I, 1995, Causal Cognition: A Multidisciplinary Debate, 351
- Kamiran F, 2012, Discrimination and Privacy in the Information Society, 223
- Keats Citron D, 2014, Washington Law Review, 89, 1
- Kroll JA, 2016, University of Pennsylvania Law Review, 165, 633
- Lyon D (2007) Resisting Surveillance. The Surveillance Studies Reader, London: McGraw-Hill Education, pp. 368–377.
- Pasquale F, 2010, Northwestern University Law Review, 104, 105
- Sandvig C, 2015, Media-N, 11, 35
- van der Sloot B, 2012, Discrimination and Privacy in the Information Society, 273
Cited by
The global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0088-2 · Cited by 6,220
The Ethics of AI Ethics -- An Evaluation of Guidelines
Hagendorff, T. (2019) · arXiv · doi:10.48550/arxiv.1903.03425 · Cited by 1,907
“Many of those endeavors are unified under the FAT ML or XAI community (Veale and Binns 2017; Selbst et al. 2018).”
The Ethics of AI Ethics: An Evaluation of Guidelines
Hagendorff, T. (2020) · Minds and Machines · doi:10.1007/s11023-020-09517-8 · Cited by 1,911
“Many of those endeavors are unified under the FAT ML or XAI community (Veale and Binns 2017; Selbst et al. 2018).”
Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need?
Holstein, K., Wortman Vaughan, J., Daumé, H., Dudik, M., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300830 · Cited by 777
Algorithmic fairness in artificial intelligence for medicine and healthcare
Chen, R. J., Wang, J. J., Williamson, D. F. K., Chen, T. Y., Lipkova, J., Lu, M. Y., Sahai, S., & Mahmood, F. (2023) · Nature Biomedical Engineering · doi:10.1038/s41551-023-01056-8 · Cited by 750
Metadata
- Authors
- Veale, M., & Binns, R.
- Affiliations
- University College London (GB)
- University of Oxford (GB)
- Corresponding author
- Veale, Michael
- Venue
- Big Data & Society
- Publisher
- SAGE Publications
- Published
- 2017-11-20
- Volume
- 4
- Issue
- 2
- Pages
- 205395171774353
- ISSN
- 2053-9517 (ISSN-L)
- OpenAlex
- W2765146466
- Semantic Scholar
- 61d37ec780aba86806af77b36d513624ec1b66b2
- Cited by
- 512
- References
- 48
- Citations by year
- 2026: 84 · 2025: 97 · 2024: 59 · 2023: 74 · 2022: 78 · 2021: 50 · 2020: 35 · 2019: 20 · 2018: 14 · 2017: 1
- FWCI
- 43.94
- Type
- journal article
- Language
- en
- Open access
- gold
- dx.doi.org
- Licence
- cc-by
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Big Data & Society (publishedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- UCL Discovery (University College London) (submittedVersion)
- SocArXiv (OSF Preprints) (submittedVersion, cc-by)
- OSF Preprints (OSF Preprints) (submittedVersion, cc-by)
- Funders
- Engineering and Physical Sciences Research Council (EP/J017728/2)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Keywords
- algorithmic fairness, indirect discrimination, sensitive attributes, trusted third party, collaborative platforms, unsupervised learning, interpretable algorithms, algorithmic harms, redlining, socio-technical systems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Fairer machine learning in the real world: Mitigating discrimination without collecting sensitive data. https://app.cochat.ai/papers/10.1177/2053951717743530
- The paper
- Veale, M., & Binns, R. (2017). Fairer machine learning in the real world: Mitigating discrimination without collecting sensitive data. Big Data & Society. https://doi.org/10.1177/2053951717743530
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.