Assessing Algorithmic Fairness with Unobserved Protected Class Using Data Combination
Open accessAbstract
The increasing impact of algorithmic decisions on people's lives compels us to scrutinize their fairness and, in particular, the disparate impacts that ostensibly-color-blind algorithms can have on different groups. Examples include credit decisioning, hiring, advertising, criminal justice, personalized medicine, and targeted policymaking, where in some cases legislative or regulatory frameworks for fairness exist and define specific protected classes. In this paper we study a fundamental challenge to assessing disparate impacts in practice: protected class membership is often not observed in the data. This is particularly a problem in lending and healthcare. We consider the use of an auxiliary dataset, such as the US census, to construct models that predict the protected class from proxy variables, such as surname and geolocation. We show that even with such data, a variety of common disparity measures are generally unidentifiable, providing a new perspective on the documented biases of popular proxy-based methods. We provide exact characterizations of the tightest-possible set of all possible true disparities that are consistent with the data (and possibly any assumptions). We further provide optimization-based algorithms for computing and visualizing these sets and statistical tools to assess sampling uncertainty. Together, these enable reliable and robust assessments of disparities -- an important tool when disparity assessment can have far-reaching policy implications. We demonstrate this in two case studies with real data: mortgage lending and personalized medicine dosing.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
When information on the sensitive attribute is unavailable but can be predicted from an auxiliary dataset, disparity measures are generally unidentifiable.
“When information on the sensitive attribute is unavailable (e.g., information on gender was not collected), but can be predicted from an auxiliary dataset, disparity measures are generally unidentifiable [20,63].”
From Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty (2021) · cites this paper as [63]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162 · see citing works
Justice as Fairness: A Restatement
Metz, T., Rawls, J., & Kelly, E. I. (2002) · The Philosophical Review · doi:10.2307/1556442 · Cited by 3,586 · see citing works
Programming with linear fractional functionals
Charnes, A., & Cooper, W. W. (1962) · Naval Research Logistics Quarterly · doi:10.1002/nav.3800090303 · Cited by 2,965 · see citing works
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536 · see citing works
Fair prediction with disparate impact: a study of bias in recidivism prediction instruments
Chouldechova, A. (2017) · Big Data · doi:10.1089/big.2016.0047 · Cited by 2,159 · see citing works
Certifying and Removing Disparate Impact
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., & Venkatasubramanian, S. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783311 · Cited by 1,827 · see citing works
Lectures on Stochastic Programming: Modeling and Theory
Shapiro, A., Dentcheva, D., & Ruszczyński, A. (2009) · Cited by 1,735 · see citing works
Estimation of the Warfarin Dose with Clinical and Pharmacogenetic Data
The International Warfarin Pharmacogenetics Consortium (2009) · New England Journal of Medicine · doi:10.1056/nejmoa0809329 · Cited by 1,573 · see citing works
Ensuring fairness in machine learning to advance health equity.
Rajkomar, A., Hardt, M., Howell, M. D., Corrado, G., & Chin, M. H. (2018) · Annals of Internal Medicine · doi:10.7326/m18-1990 · Cited by 1,341 · see citing works
(2013) · Cited by 1,126 · see citing works
Fairness definitions explained
Verma, S., & Rubin, J. (2018) · Proceedings of the International Workshop on Software Fairness · doi:10.1145/3194770.3194776 · Cited by 1,124 · see citing works
Adjusting for Nonignorable Drop-Out Using Semiparametric Nonresponse Models
Scharfstein, D. O., Rotnitzky, A. G., & Robins, J. M. (1999) · Journal of the American Statistical Association · doi:10.1080/01621459.1999.10473862 · Cited by 1,021 · see citing works
Inherent trade-offs in the fair determination of risk scores
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2017) · Schloss Dagstuhl – Leibniz-Zentrum für Informatik · doi:10.4230/lipics.itcs.2017.43 · Cited by 943 · cited 3× · see citing works
“Selecting fairness criteria to enforce is further complicated by the fact that some are incompatible (Chouldechova 2017, Feller et al. 2016, Kleinberg et al. 2017) and many are closely correlated (Friedler et al. 2019).”
BackgroundLambrecht, A., & Tucker, C. (2019) · Management Science · doi:10.1287/mnsc.2018.3093 · Cited by 854 · see citing works
Online Display Advertising: Targeting and Obtrusiveness
Avi Goldfarb, B., & Tucker, C. E. (2011) · Marketing Science · doi:10.1287/mksc.1100.0583 · Cited by 801 · see citing works
Equity: In Theory and Practice
Kuenne, R. E., & Young, H. P. (1995) · Southern Economic Journal · doi:10.2307/1061013 · Cited by 677 · see citing works
Automated Experiments on Ad Privacy Settings: A Tale of Opacity, Choice, and Discrimination
Datta, A., Tschantz, M. C., & Datta, A. (2015) · Proceedings on Privacy Enhancing Technologies · doi:10.1515/popets-2015-0007 · Cited by 676 · see citing works
Market Structure and Multiple Equilibria in Airline Markets
Ciliberto, F., & Tamer, E. (2009) · Econometrica · doi:10.3982/ecta5368 · Cited by 665 · see citing works
A comparative study of fairness-enhancing interventions in machine learning
Friedler, S. A., Scheidegger, C., Venkatasubramanian, S., Choudhary, S., Hamilton, E. P., & Roth, D. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287589 · Cited by 601 · see citing works
Iyer, G. V., Soberman, D. A., & Villas‐Boas, J. M. (2005) · Marketing Science · doi:10.1287/mksc.1050.0117 · Cited by 542 · see citing works
Double/Debiased Machine Learning for Treatment and Structural Parameters
Chernozhukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., Newey, W. K., & Robins, J. M. (2017) · National Bureau of Economic Research · doi:10.3386/w23564 · Cited by 537 · see citing works
Fast learning rates for plug-in classifiers
Audibert, J.-Y., & Tsybakov, A. (2005) · The Annals of Statistics · doi:10.1214/009053606000001217 · Cited by 529 · see citing works
Double/debiased machine learning for treatment and structural parameters
Chernozhukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., Newey, W. K., & Robins, J. M. (2017) · doi:10.1920/wp.cem.2017.2817 · Cited by 484 · see citing works
Sweeney, L. (2013) · Queue · doi:10.1145/2460276.2460278 · Cited by 477 · see citing works
The Measure and Mismeasure of Fairness
Corbett-Davies, S., Gaebler, J. D., Nilforoshan, H., Shroff, R., & Goel, S. (2018) · arXiv · doi:10.48550/arxiv.1808.00023 · Cited by 444 · see citing works
Rüschendorf, L. (2013) · Springer series in operations research/Springer series in operations research and financial engineering · doi:10.1007/978-3-642-33590-7 · Cited by 289 · see citing works
Chakraborty, B., & Murphy, S. A. (2013) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-022513-115553 · Cited by 288 · see citing works
Risk Assessment in Criminal Sentencing
Monahan, J., & Skeem, J. L. (2016) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-021815-092945 · Cited by 256 · see citing works
Ecological Inference and the Ecological Fallacy
Freedman, D. A. (1999) · UC Berkeley · Cited by 251 · see citing works
Improving Ecological Inference by Predicting Individual Ethnicity from Voter Registration Records
Imai, K., & Khanna, K. (2016) · Political Analysis · doi:10.1093/pan/mpw001 · Cited by 249 · see citing works
Elliott, M. N., Morrison, P. A., Fremont, A., McCaffrey, D. F., Pantoja, P., & Lurie, N. (2009) · Health Services and Outcomes Research Methodology · doi:10.1007/s10742-009-0047-1 · Cited by 224 · see citing works
Stability Theory for Systems of Inequalities. Part I: Linear Systems
Robinson, S. M. (1975) · SIAM Journal on Numerical Analysis · doi:10.1137/0712056 · Cited by 223 · see citing works
Elliott, M., Fremont, A. M., Morrison, P., Pantoja, P. M., & Lurie, N. (2008) · Health Services Research · doi:10.1111/j.1475-6773.2008.00854.x · Cited by 216 · see citing works
On the relation between accuracy and fairness in binary classificationPreprint
Žliobaitė, I. (2015) · arXiv.org · doi:10.48550/arxiv.1505.05723 · Cited by 210 · see citing works
On Equitable Resource Allocation Problems: A Lexicographic Minimax Approach
Luss, H. (1999) · Operations Research · doi:10.1287/opre.47.3.361 · Cited by 191 · see citing works
The Affordable Care Act: implications for health-care equity
Gaffney, A. W., & McCormick, D. (2017) · The Lancet · doi:10.1016/s0140-6736(17)30786-9 · Cited by 168 · see citing works
Dynamic treatment regimes: Technical challenges and applications
Laber, E. B., Lizotte, D. J., Qian, M., Pelham, W. E., & Murphy, S. A. (2014) · Electronic Journal of Statistics · doi:10.1214/14-ejs920 · Cited by 164 · see citing works
The Identification Zoo: Meanings of Identification in Econometrics
Lewbel, A. (2019) · Journal of Economic Literature · doi:10.1257/jel.20181361 · Cited by 159 · see citing works
Fairness Under Unawareness: Assessing Disparity When Protected Class Is Unobserved
Chen, J., Kallus, N., Mao, X., Svacha, G., & Udell, M. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287594 · Cited by 150 · see citing works
Self-citationDissecting Racial Bias in an Algorithm that Guides Health Decisions for 70 Million People
Obermeyer, Z., & Mullainathan, S. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287593 · Cited by 124 · see citing works
Impossibility Results for Nondifferentiable Functionals
Hirano, K., & Porter, J. R. (2012) · Econometrica · doi:10.3982/ecta8681 · Cited by 114 · see citing works
Fair Optimization and Networks: A Survey
Ogryczak, W., Luss, H., Pióro, M., Nace, D., & Tomaszewski, A. (2014) · Journal of Applied Mathematics · doi:10.1155/2014/612018 · Cited by 101 · see citing works
Online Decision-Making with High-Dimensional CovariatesPreprint
Bastani, H., & Bayati, M. (2015) · SSRN Electronic Journal · doi:10.2139/ssrn.2661896 · Cited by 97 · see citing works
Elliott, M. N., Morrison, P. A., Fremont, A. M., McCaffrey, D. F., Pantoja, P. M., & Lurie, N. (2009) · Health Services and Outcomes Research Methodology · doi:10.1007/s10742-009-0055-1 · Cited by 95 · see citing works
Ecological Inference for 2 × 2 Tables (with Discussion)
Wakefield, J. (2004) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2004.02046.x · Cited by 92 · see citing works
Targeted Learning of the Mean Outcome under an Optimal Dynamic Treatment Rule
van der Laan, M. J., & Luedtke, A. R. (2014) · Journal of Causal Inference · doi:10.1515/jci-2013-0022 · Cited by 81 · see citing works
Confounding-Robust Policy ImprovementPreprint
Kallus, N., & Zhou, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.08593 · Cited by 78 · see citing works
Self-citationZafar, M. B., Valera, I., Gomez-Rodriguez, M., & Gummadi, K. P. (2015) · MPG.PuRe (Max Planck Society) · Cited by 75 · see citing works
Assessing Fair Lending Risks Using Race/Ethnicity Proxies
Zhang, Y. (2018) · Management Science · doi:10.1287/mnsc.2016.2579 · Cited by 71 · cited 5× · see citing works
“research is still limited. Although advanced proxy methods like BISG outperform previous proxy methods, further research shows that it leads to biased disparity assessment (Baines and Courchane 2014, Zhang 2016). In particular, Chen et al. (2019) analyzed the underlying mechanism for the statistical bias of BISG’s ass…”
BackgroundMethodInfluentialUse Of Geocoding In Managed Care Settings To Identify Quality Disparities
Fremont, A. M., Bierman, A. S., Wickstrom, S. L., Bird, C. E., Shah, M., Escarce, J. J., Horstman, T., & Rector, T. S. (2005) · Health Affairs · doi:10.1377/hlthaff.24.2.516 · Cited by 71 · see citing works
Using First Name Information to Improve Race and Ethnicity Classification
Voicu, I. (2018) · Statistics and Public Policy · doi:10.1080/2330443x.2018.1427012 · Cited by 67 · see citing works
Recursive Partitioning for Personalization using Observational DataPreprint
Kallus, N. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.08925 · Cited by 63 · see citing works
Self-citationSchuessler, A. A. (1999) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.96.19.10578 · Cited by 62 · see citing works
Machine Learning, Health Disparities, and Causal Reasoning
Goodman, S. N., Goel, S., & Cullen, M. R. (2018) · Annals of Internal Medicine · doi:10.7326/m18-3297 · Cited by 57 · see citing works
Indirect estimation of race ethnicity for survey respondents who do not report race/ethnicity
Dembosky, J. W., Haviland, A. M., Haas, A., Hambarsoomian, K., Weech-Maldonado, R., Wilson-Frederick, S. M., Gaillot, S., & Elliott, M. N. (2019) · Medical Care · doi:10.1097/mlr.0000000000001011 · Cited by 45 · see citing works
Disparate Impact under Title VII: An Objective Theory of Discrimination
Rutherglen, G. A. (1987) · Virginia Law Review · doi:10.2307/1072940 · Cited by 44 · see citing works
Sharp instruments for classifying compliers and generalizing causal effects
Kennedy, E. H., Balakrishnan, S., & G’Sell, M. (2020) · The Annals of Statistics · doi:10.1214/19-aos1874 · Cited by 42 · see citing works
Sensitivity Analysis via the Proportion of Unmeasured Confounding
Bonvini, M., & Kennedy, E. H. (2020) · Journal of the American Statistical Association · doi:10.1080/01621459.2020.1864382 · Cited by 38 · see citing works
Targeted Learning of an Optimal Dynamic Treatment, and Statistical Inference for its Mean Outcome
van der Laan, M. J., & Luedtke, A. R. (2014) · Collection of Biostatistics Research Archive · Cited by 36 · see citing works
Advancing Health Care Equity through Improved Data Collection
Weissman, J. S., & Hasnain‐Wynia, R. (2011) · New England Journal of Medicine · doi:10.1056/nejmp1103069 · Cited by 35 · see citing works
D’Amour, A. (2019) · International Conference on Artificial Intelligence and Statistics · Cited by 28 · see citing works
Using Bayesian imputation to assess racial and ethnic disparities in pediatric performance measures
Brown, D. P., Knapp, C., Baker, K., & Kaufmann, M. (2016) · Health Services Research · doi:10.1111/1475-6773.12405 · Cited by 27 · see citing works
The Emerging Uniform Structure of Disparate Treatment Discrimination LitigationPreprint
Zimmer, M. J. (2009) · SSRN Electronic Journal · Cited by 13 · see citing works
When Race/Ethnicity Data Are Lacking
Fremont, A. M., Weissman, J. S., Hoch, E., & Elliott, M. N. (2016) · Cited by 6 · see citing works
Ecological Regression with Partial Identification
Jiang, W., King, G., Schmaltz, A., & Tanner, M. A. (2020) · RePEc: Research Papers in Economics · Cited by 1 · see citing works
Policy Learning with Observational Data
Athey, S., & Wager, S. (2017) · arXiv · doi:10.48550/arxiv.1702.02896
Inequalities for E k(X, Y) when the marginals are fixed
Cambanis, S., Simons, G., & Stout, W. (1976) · Zeitschrift f�r Wahrscheinlichkeitstheorie und Verwandte Gebiete · doi:10.1007/bf00532695
Discrimination in Online Ad Delivery
Sweeney, L. (2013) · arXiv · doi:10.48550/arxiv.1301.6822
Partial identification with missing data: concepts and findings
Manski, C. F. (2005) · International Journal of Approximate Reasoning · doi:10.1016/j.ijar.2004.10.006
Econometrics with Partial Identification
Molinari, F. (2019) · doi:10.1920/wp.cem.2019.2519
Sensitivity analysis via the proportion of unmeasured confounding
Bonvini, M., & Kennedy, E. H. (2019) · arXiv · doi:10.48550/arxiv.1912.02793
Ecological Regression with Partial Identification
Jiang, W., King, G., Schmaltz, A., & Tanner, M. A. (2018) · arXiv · doi:10.48550/arxiv.1804.05803
Sharp instruments for classifying compliers and generalizing causal effects
Kennedy, E. H., Balakrishnan, S., & G'Sell, M. (2018) · arXiv · doi:10.48550/arxiv.1801.03635
Flagged first, then by citations
Research works printed but not linked: 8
- Beresteanu A, Molchanov I, Molinari F (2011) Sharp identification regions in models with convex moment predictions. Econometrica 79(6):1785-1821.
- Fan Y, Sherman R, Shum M (2014) Identifying treatment effects under data combination. Econometrica 82(2):811-822.
- Kallus N, Mao X, Zhou A (2019) Interval estimation of individual-level causal effects under unobserved confounding. Proceedings of the 22nd International Conference on Artificial Intelligence and Statistics, 2281-2290.
- Manski CF (2003) Partial identification of probability distributions (Springer Science & Business Media).
- Nerenz DR, McFadden B, Ulmer C, et al. (2009) Race, ethnicity, and language data: standardization for health care quality improvement (National Academies Press).
- Rockafellar RT (2015) Convex analysis (Princeton university press).
- Van der Vaart AW (2000) Asymptotic statistics, volume 3 (Cambridge university press).
- Villani C (2008) Optimal transport: old and new, volume 338 (Springer).
Cited by
Sorted by citations
A Review on Fairness in Machine Learning
Pessach, D., & Shmueli, E. (2023) · ACM Computing Surveys · doi:10.1145/3494672 · Cited by 598 · see citing works
Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty
Bhatt, U., Antorán, J., Zhang, Y., Liao, Q. V., Sattigeri, P., Fogliato, R., Melançon, G., Krishnan, R., Stanley, J., Tickoo, O., et al. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462571 · Cited by 225 · see citing works
Fairness without Demographics through Adversarially Reweighted LearningPreprint
Lahoti, P., Beutel, A., Chen, J., Lee, K., Prost, F., Thain, N., Wang, X., & Ed, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.13114 · Cited by 135 · see citing works
Pessach, D., & Shmueli, E. (2023) · Springer eBooks · doi:10.1007/978-3-031-24628-9_37 · Cited by 177 · see citing works
Antidiscrimination Insurance Pricing: Regulations, Fairness Criteria, and Models
Xin, X., & Huang, F. (2024) · North American Actuarial Journal · doi:10.1080/10920277.2023.2190528 · Cited by 43 · see citing works
“Recently, several methods in the machine learning field have been proposed to deal with this issue; see Kallus, Mao, and Zhou (2021) and Wang et al. (2020).”
MethodMeasuring Fairness Under Unawareness of Sensitive Attributes: A Quantification-Based Approach
Fabris, A., Esuli, A., Moreo, A., & Sebastiani, F. (2023) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.14033 · Cited by 48 · see citing works
Mitigating Bias in Set Selection with Noisy Protected Attributes
Mehrotra, A., & Celis, L. E. (2021) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3442188.3445887 · Cited by 37 · see citing works
Fair Learning with Private Demographic DataPreprint
Mozannar, H., Ohannessian, M. I., & Srebro, N. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.11651 · Cited by 36 · see citing works
Fairness without Imputation: A Decision Tree Approach for Fair Prediction with Missing Values
Jeong, H., Wang, H., & Calmon, F. P. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i9.21189 · Cited by 25 · see citing works
Artificial Intelligence, Trust, and Perceptions of Agency
Puranam, P., & Vanneste, B. (2021) · Academy of Management Review · doi:10.5465/amr.2022.0041 · Cited by 188 · see citing works
Improving Fairness and Privacy in Selection Problems
Khalili, M. M., Zhang, X., Abroshan, M., & Sojoudi, S. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i9.16986 · Cited by 14 · see citing works
Gender and Racial Stereotype Detection in Legal Opinion Word Embeddings
Matthews, S., Hudzina, J., & Sepehr, D. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i11.21461 · Cited by 15 · see citing works
Probabilistic Fair ClusteringPreprint
Esmaeili, S. A. H., Brubach, B., Tsepenekas, L., & Dickerson, J. P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.10916 · Cited by 10 · see citing works
Fair Sequential Selection Using Supervised Learning ModelsPreprint
Khalili, M. M., Zhang, X., & Abroshan, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.13986 · Cited by 10 · see citing works
Li, C., Wang, H., Jiang, S., & Gu, B. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4562266 · Cited by 8 · see citing works
Fairness without Demographics through Shared Latent Space-Based Debiasing
Islam, R., Chen, H., & Cai, Y. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i11.29167 · Cited by 6 · see citing works
Propose and Review: Practical Bias Mitigation for Machine ClassificationPreprint
Li, T., Tang, Z., Lu, T., & Zhang, X. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4139244 · Cited by 4 · see citing works
An Anti-Subordination Approach to Fair ClassificationPreprint
Keswani, V., & Celis, L. E. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4116835 · Cited by 3 · see citing works
Regulating Discriminatory Pricing in the Presence of Tacit CollusionPreprint
Yang, Z., Xiao, L., & Gao, P. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4633784 · Cited by 3 · see citing works
Robust Classification Via Support Vector MachinesPreprint
Asimit, A. V., Kyriakou, I., Santoni, S., Scognamiglio, S., & Zhu, R. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4074846 · Cited by 2 · see citing works
Chubba, E., Missaouib, S., Concannonc, S., Maloneyb, L., & Walker, J. A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.13076 · Cited by 1 · see citing works
Estimating Racial Disparities When Race is Not ObservedPreprint
McCartan, C., Fisher, R., Goldin, J. S., Ho, D. E., & Imai, K. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4810588 · Cited by 1 · see citing works
Algorithmic Bias and Historical Injustice: Race and Digital ProfilingPreprint
Matthew, A., Miller, A. R., & Tucker, C. E. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4812943 · Cited by 1 · see citing works
Kursuncu, U., Baird, A. M., & Xia, Y. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.5035722 · Cited by 1 · see citing works
Fairness-Aware Unsupervised Feature SelectionPreprint
Xing, X., Liu, H., Chen, C., & Li, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.02216
Quantifying the Uncertainty of Imputed Demographic Disparity Estimates: The Dual-BootstrapPreprint
Lu, B., Wan, J., Ouyang, D., Goldin, J. S., & Ho, D. E. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4787403
Yuan, M., Bardhan, I. R., & Wen, W. (2026) · Management Science · doi:10.1287/mnsc.2024.05617
The Informativeness of Combined Experimental And Observational Data Under Dynamic SelectionPreprint
Park, Y., & Sasaki, Y. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4812306
Hu, X., Huang, Y. H., Li, B., & Lu, T. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5395464
- Authors
- Kallus, N., Mao, X., & Zhou, A.
- Affiliations
- Cornell University (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2019
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1906.00285
- OpenAlex
- W2947305394
- Cited by
- 28
- References
- 50
- Citations by year
- 2026: 1 · 2025: 1 · 2024: 7 · 2023: 4 · 2022: 6 · 2021: 6 · 2020: 3
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Keywords
- algorithmic fairness, indirect discrimination, proxy variables, data combination, non-identifiability, sampling uncertainty, mortgage lending, U.S. Census
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
