The DOI matches its Crossref record; 442 of the 442 references it cites were found in a registry or an index.
A multidisciplinary survey on discrimination analysis
Abstract
The collection and analysis of observational and experimental data represent the main tools for assessing the presence, the extent, the nature, and the trend of discrimination phenomena. Data analysis techniques have been proposed in the last 50 years in the economic, legal, statistical, and, recently, in the data mining literature. This is not surprising, since discrimination analysis is a multidisciplinary problem, involving sociological causes, legal argumentations, economic models, statistical techniques, and computational issues. The objective of this survey is to provide a guidance and a glue for researchers and anti-discrimination data analysts on concepts, problems, application areas, datasets, methods, and approaches from a multidisciplinary perspective. We organize the approaches according to their method of data collection as observational, quasi-experimental, and experimental studies. A fourth line of recently blooming research on knowledge discovery based methods is also covered. Observational methods are further categorized on the basis of their application context: labor economics, social profiling, consumer markets, and others.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“the machine learning field has seen an explosion of research on fairness.”
From The Discriminating (Pricing) Actuary (2023) · cites this paper as Romei and Ruggieri (2014)
In the citing paper
“This literature provides discussions and debates on how to define fairness of predictive models and how to measure the performance in terms of discrimination.”
From The Discriminating (Pricing) Actuary (2023) · cites this paper as Romei and Ruggieri (2014)
In the citing paper
“Not considering correlated variables has been shown to increase the risk of discrimination (e.g., redlining 4 ) [54,88,89,89,196,202,231,247,286,308].”
From Fairness in machine learning: A survey (2024) · cites this paper as [247]
In the citing paper
“There is also the risk of redlining, where although the sensitive attribute is łhandledž suiciently, correlated variables are still present [54,89,231,247,286,308], amplifying instead of reducing unfairness [89].”
From Fairness in machine learning: A survey (2024) · cites this paper as [247]
Citations
Cites
Items 401–442 of 442
Approaches for Dealing with Small Sample Sizes in Employment Discrimination Litigation
Piette, M. J., & White, P. F. (1999) · Journal of Forensic Economics · doi:10.5085/0898-5510-12.1.43 · Cited by 10
Breda, T., & Ly, S. T. (2012) · RePEc: Research Papers in Economics · Cited by 9
Discovering Gender Discrimination in Project Funding
Romei, A., Ruggieri, S., & Turini, F. (2012) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw.2012.39 · Cited by 9
Self-citationDiscrimination in Recruitment: Evidence from a Field Experiment
McGinnity, F., Nelson, J., Lunn, P., et al. (2009) · RePEc: Research Papers in Economics · Cited by 9
Profiling of Customers and Consumers - Customer Loyalty Programmes and Scoring Practices
Kamp, M., Körffer, B., & Meints, M. (2008) · Springer eBooks · doi:10.1007/978-1-4020-6914-7_11 · Cited by 8
Application of Reverse Regression to Boston Federal Reserve Data Refutes Claims of Discrimination
LaCour‐Little, M. (1996) · Journal of Real Estate Research · doi:10.1080/10835547.1996.12090813 · Cited by 8
Discrimination and Mortgage Lending in Boston: The Effects of Model Uncertainty
Goenner, C. F. (2008) · The Journal of Real Estate Finance and Economics · doi:10.1007/s11146-008-9154-y · Cited by 8
Reverse Regression, Collinearity, and Employment Discrimination
Whiteside, M. M., & Narayanan, A. (1989) · Journal of Business and Economic Statistics · doi:10.1080/07350015.1989.10509749 · Cited by 8
Racial Discrimination in Mortgage Lending in Washington, D.C.: A Mixed Methods Approach1
Blank, E. C., Venkatachalam, P., McNeil, L., et al. (2005) · The Review of Black Political Economy · doi:10.1007/s12114-005-1012-z · Cited by 8
Reverse discrimination by the numbers
Lerner, R. E., & Nagai, A. K. (2000) · Academic Questions · doi:10.1007/s12129-000-1039-3 · Cited by 8
Credit Access, the Costs of Credit, and Credit Market DiscriminationPreprint
Weller, C. E. (2008) · SSRN Electronic Journal · doi:10.2139/ssrn.1161991 · Cited by 8
Why Do I Like People Like Me?Preprint
Bagues, M. F., & Villadóniga, M. J. P. (2008) · SSRN Electronic Journal · doi:10.2139/ssrn.1091247 · Cited by 8
Gender Bias in Power Relationships: Evidence from Police Traffic StopsPreprint
Blalock, G., DeVaro, J., Leventhal, S., et al. (2007) · SSRN Electronic Journal · doi:10.2139/ssrn.998464 · Cited by 8
Statistical Methods in Discrimination Litigation
Klemm, R. J., Kaye, D. H., & Aickin, M. G. (1987) · Journal of Business and Economic Statistics · doi:10.2307/1392006 · Cited by 8
Productivity, Wages, and Marriage: The Case of Major League BaseballPreprint
Cornaglia, F., & Feldman, N. E. (2011) · SSRN Electronic Journal · doi:10.2139/ssrn.1714994 · Cited by 7
The Reverse Regression Problem: Statistical Paradox or Artefact of Misspecification?
Racine, J. S., & Rilstone, P. (1995) · Canadian Journal of Economics/Revue canadienne d économique · doi:10.2307/136046 · Cited by 6
Nunley, J. M., Owens, M. F., & Jones, J. A. (2010) · RePEc: Research Papers in Economics · Cited by 6
Lifetime Network Externality and the Dynamics of Group InequalityPreprint
Kim, Y. (2009) · SSRN Electronic Journal · doi:10.2139/ssrn.1504577 · Cited by 6
Customer discrimination against black major league baseball pitchers reconsidered
Hersch, P. L. (2008) · Applied Economics Letters · doi:10.1080/13504850701720031 · Cited by 6
Policing, profiling and discrimination law: US and European approaches compared
Baker, A., & Phillipson, G. (2011) · Journal of Global Ethics · doi:10.1080/17449626.2011.556142 · Cited by 6
Decomposition of the Black-White Wage Differential in the Physician MarketPreprint
Tsao, T.-Y., & Pearlman, A. (2010) · SSRN Electronic Journal · doi:10.2139/ssrn.1585394 · Cited by 6
Applying for Home Mortgages in Immigrant Communities: The Case of Asian Applicants in Los Angeles
Zonta, M. (2012) · Environment and Planning A Economy and Space · doi:10.1068/a43419 · Cited by 6
Visual data mining for higher-level patterns: Discrimination-aware data mining and beyond
Bo, G., & Berendt, B. (2011) · Lirias · Cited by 6
Profiling in Employment Situations (Fraud)
Leopold, N., & Meints, M. (2008) · Springer eBooks · doi:10.1007/978-1-4020-6914-7_12 · Cited by 5
The economics of affirmative action
Nugent, R. A. (2002) · Routledge eBooks · doi:10.4324/9780203021149-13 · Cited by 5
Generalized discrimination discovery on semi-structured data supported by ontology
Thanh, B. L. (2011) · Cited by 5
A test of employer learning in the labour market for young Australians
Cheung, S. L. (2008) · Applied Economics Letters · doi:10.1080/13504850701719850 · Cited by 5
Sugrue, T. J., & Fairley, W. B. (1983) · bepress Legal Repository · Cited by 4
Testing the taste-based discrimination hypothesis:evidence from data on Japanese Listed firms
Sano, S. (2009) · Japan labor review · Cited by 4
Bertero, A. P. (2011) · SSRN Electronic Journal · doi:10.2139/ssrn.1878252 · Cited by 4
Whitney, M. J. (2007) · bepress Legal Repository · Cited by 3
Mortgage Lending Disparities in Metropolitan Buffalo: Implications for Community Reinvestment Policy
Silverman, R. M., & Silverman, R. M. (2008) · RePEc: Research Papers in Economics · doi:10.22004/ag.econ.132340 · Cited by 3
Statistics and employment discrimination law: An interdisciplinary review
Wingate, P. H., & Thornton, G. C. (2004) · Research in personnel and human resources management · doi:10.1016/s0742-7301(00)19008-5 · Cited by 3
Social Interactions (Empirics)
Ioannides, Y. M. (2010) · Palgrave Macmillan UK eBooks · doi:10.1057/9780230280816_32 · Cited by 3
Man Vs. Machine: An Investigation of Speeding Ticket Disparities Based on Gender and Race
Quintanar, S. M. (2019) · RePEc: Research Papers in Economics · doi:10.6084/m9.figshare.11298149 · Cited by 3
Deterrence Externalities and Racial Bias in Law Enforcement
Gardner, J. (2018) · PubMed · doi:10.1184/r1/6471197.v1 · Cited by 2
Brown, C. L., & Simpson, W. (2010) · Cited by 2
Reverse Regression and Orthogonal Regression in Employment Discrimination Analysis
Schaefer, K. C., & Visser, M. (2003) · Journal of Forensic Economics · doi:10.5085/0898-5510-16.3.283 · Cited by 2
Statistical Discrimination, Productivity and the Height of ImmigrantsPreprint
Wang, S.-Y. B. (2011) · SSRN Electronic Journal · doi:10.2139/ssrn.1851845 · Cited by 2
Facing a Dilemma: Cooperative Behavior and BeautyPreprint
Darai, D., & Grätz, S. (2012) · SSRN Electronic Journal · doi:10.2139/ssrn.2087601 · Cited by 2
Wage discrimination: The case for reverse regressionPreprint
Kapsalis, C. (2010) · RePEc: Research Papers in Economics · Cited by 1
Detecting and describing heterogeneity in meta-analysis
Hardy, R. J., & Thompson, S. G. (1998) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19980430)17:8
Research works printed but not linked: 26
- Ghosh, 2000, Hierarchical Bayesian analysis of binary matched data, Statistica Sinica, 10, 647
- Armstrong, 2006, Proceedings of World Congress on Advances in Economics and Econometrics Theory and Applications, 1
- Gastwirth, 1992, Methods for assessing the sensitivity of statistical comparisons used in Title VII cases to omitted variables, Jurimetrics Journal, 33, 19
- Ding, 2008, Neighborhood patterns of high-cost lending: the case of Atlanta, Journal of Affordable Housing, 17, 194
- Clifton C. 2003. Privacy preserving data mining: How do we mine data when we aren't allowed to see it? In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2003), Tutorial, Washington, DC, USA. http://www.cs.purdue.edu.
- Charness, 2011, Handbook of Labor Economics, 229, 10.1016/S0169-7218(11)00409-6 · DOI
- Blank, 2004, Measuring Racial Discrimination – Panel on Methods for Assessing Discrimination
- Fix, 1999, A National Report Card on Discrimination in America: The Role of Testing
- 2007, Science, Technology and Gender: An International Report
- Heckman, 1993, Clear and Convincing Evidence: Measures of discrimination in America, 187
- Greene, 2008, Econometric Analysis
- Hoeting, 1999, Bayesian model averaging: a tutorial, Statistical Science, 14, 382
- Ross, 1999, Mortgage lending discrimination: A review of existing evidence, 107
- Paetzold, 1994, The Statistics of Discrimination: Using Statistical Evidence in Discrimination Cases
- Körffer, 2008, Profiling the European Citizen: Cross-Disciplinary Perspectives, 201
- 10.1023/A:1008616203852 · DOI
- Leopold, 2008, Profiling the European Citizen: Cross-Disciplinary Perspectives, 236
- Scheinkman, 2008, The New Palgrave Dictionary of Economics
- Longhofer, 1999, Why is mortgage discrimination illegal? A fresh look at the mortgage discrimination debate, Regulation, Cato Institute, 22, 28
- Fienberg, 1995, Understanding and evaluating statistical evidence in litigation, Jurimetrics Journal of Law, Science, and Technology, 36, 1
- Lee, 2009, American Idol: evidence on same-race preferences, The Berkeley Electronic Journal of Economic Analysis and Policy, 9
- 10.1353/jhr.2011.0025 · DOI
- Rachlis, 1993, Serious flaws in statistical tests for discrimination in mortgage markets, Journal of Housing Research, 2, 315
- Ross, 1997, Mortgage lending discrimination and racial differences in loan default: a simulation approach, Journal of Housing Research, 8, 277
- Ellis, 2005, EU Anti-Discrimination Law
- Shoben, 1992, Statistical Methods in Discrimination Litigation, 55
Cited by
Sorted by citations
A Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2022) · ACM Computing Surveys · doi:10.1145/3457607 · Cited by 5,812
The ethics of algorithms: Mapping the debate
Mittelstadt, B. D., Allo, P., Taddeo, M., Wachter, S., & Floridi, L. (2016) · Big Data & Society · doi:10.1177/2053951716679679 · Cited by 4,567
“However, proxies for protected attributes are not easy to predict or detect (Romei and Ruggieri, 2014; Zarsky, 2016), particularly when algorithms access linked datasets (Barocas and Selbst, 2015).”
InfluentialFair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159
Certifying and Removing Disparate Impact
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783311 · Cited by 1,827
Algorithmic Decision Making and the Cost of Fairness
Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., & Huq, A. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098095 · Cited by 1,149
Metadata
- Authors
- Romei, A., & Ruggieri, S.
- Affiliations
- University of Pisa (IT)
- Venue
- The Knowledge Engineering Review
- Publisher
- Maximum Academic Press
- Published
- 2013-04-03
- Volume
- 29
- Issue
- 5
- Pages
- 582-638
- ISSN
- 0269-8889 (ISSN-L)
- 1469-8005
- OpenAlex
- W2166454173
- Semantic Scholar
- b0d3968e7a630d3661cd0878acc38c85f1828a9c
- Cited by
- 288
- References
- 442
- Citations by year
- 2026: 4 · 2025: 18 · 2024: 21 · 2023: 31 · 2022: 27 · 2021: 33 · 2020: 23 · 2019: 38 · 2018: 31 · 2017: 28 · 2016: 17 · 2015: 9 · 2014: 5 · 2013: 3
- FWCI
- 29.96
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- The Knowledge Engineering Review (publishedVersion)
- UnipiEprints Open Archive (Università di Pisa) (acceptedVersion)
- CINECA IRIS Institutial research information system (University of Pisa) (submittedVersion)
- ISTI Open Portal (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Sociology and Political Science › Names, Identity, and Discrimination Research
- Social Sciences › Law › Discrimination and Equality Law
- Keywords
- discrimination analysis, anti-discrimination, observational studies, quasi-experimental design, experimental study, knowledge discovery, labor economics, social profiling, consumer market
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review