The DOI matches its Crossref record; 29 of the 30 references it cites were found in a registry or an index.
The problem of infra-marginality in outcome tests for discrimination
Open accessAbstract
Outcome tests are a popular method for detecting bias in lending, hiring, and policing decisions. These tests operate by comparing the success rate of decisions across groups. For example, if loans made to minority applicants are observed to be repaid more often than loans made to whites, it suggests that only exceptionally qualified minorities are granted loans, indicating discrimination. Outcome tests, however, are known to suffer from the problem of infra-marginality: even absent discrimination, the repayment rates for minority and white loan recipients might differ if the two groups have different risk distributions. Thus, at least in theory, outcome tests can fail to accurately detect discrimination. We develop a new statistical test of discrimination—the threshold test—that mitigates the problem of infra-marginality by jointly estimating decision thresholds and risk distributions. Applying our test to a dataset of 4.5 million police stops in North Carolina, we find that the problem of infra-marginality is more than a theoretical possibility, and can cause the outcome test to yield misleading results in practice.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“thresholds cannot be identified by the observed data alone 7 , and so inferences are dependent on the specific functional form of the underlying Bayesian model, including the prior distributions.”
From A large-scale analysis of racial disparities in police stops across the United States (2020) · cites this paper as [7]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“the threshold test has been proposed as a more robust means for detecting discrimination 7,22 .”
From A large-scale analysis of racial disparities in police stops across the United States (2020) · cites this paper as [7]
In the citing paper
“However, since the protected attribute is frequently represented implicitly by some combination of proxy variables, the classifier still learns the discrimination reflected in training data.”
From Interventional Fairness: Causal Database Repair for Algorithmic Fairness (2019) · cites this paper as [46]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Equation of State Calculations by Fast Computing Machines
Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H., & Teller, E. (1953) · The Journal of Chemical Physics · doi:10.1063/1.1699114 · Cited by 37,192
Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (2003) · doi:10.1201/9780429258480 · Cited by 27,276
Inference from iterative simulation using multiple sequences
Gelman, A., & Rubin, D. B. (1992) · Statistical Science · doi:10.1214/ss/1177011136 · Cited by 16,627
Stan : A Probabilistic Programming Language
Carpenter, B., Gelman, A., Hoffman, M. D., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M., Guo, J., Li, P., & Riddell, A. (2017) · Journal of Statistical Software · doi:10.18637/jss.v076.i01 · Cited by 7,754 · cited 2×
“∗We thank Cheryl Phillips and Vignesh Ramachandran of the Stanford Computational Journalism Lab for helping to compile the North Carolina traffic stop data, and the John S. and James L. Knight Foundation for partial support of this research.”
MethodDuane, S., Kennedy, A. D., Pendleton, B. J., & Roweth, D. (1987) · Physics Letters B · doi:10.1016/0370-2693(87)91197-x · Cited by 4,081
The Statistical Theory of Racism and Sexism.
Phelps, E. S. (1972) · American Economic Review · Cited by 3,412
The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte CarloPreprint
Hoffman, M. D., & Gelman, A. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1111.4246 · Cited by 2,815
POSTERIOR PREDICTIVE ASSESSMENT OF MODEL FITNESS VIA REALIZED DISCREPANCIES
Gelman, A., Meng, X., & Stern, H. S. (1996) · Cited by 1,982
Nobel Lecture: The Economic Way of Looking at Behavior
Becker, G. S. (1993) · Journal of Political Economy · doi:10.1086/261880 · Cited by 1,619
Algorithmic Decision Making and the Cost of Fairness
Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., & Huq, A. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098095 · Cited by 1,149
Self-citationGelman, A., Fagan, J., & Kiss, A. (2007) · Journal of the American Statistical Association · doi:10.1198/016214506000001040 · Cited by 873
Pulled Over: How Police Stops Define Race and Citizenship
Epp, C. R., Maynard-Moody, S., & Haider-Markel, D. P. (2014) · doi:10.7208/chicago/9780226114040.001.0001 · Cited by 687
Racial Bias in Motor Vehicle Searches: Theory and Evidence
Knowles, J., Persico, N., & Todd, P. (2001) · Journal of Political Economy · doi:10.1086/318603 · Cited by 608
Jordan, M. I. (2004) · Statistical Science · doi:10.1214/088342304000000026 · Cited by 440
A New Look at Racial Profiling: Evidence from the Boston Police Department
Antonovics, K., & Knight, B. (2009) · The Review of Economics and Statistics · doi:10.1162/rest.91.1.163 · Cited by 296
An Alternative Test of Racial Prejudice in Motor Vehicle Searches: Theory and Evidence
Anwar, S., & Fang, H. (2006) · American Economic Review · doi:10.1257/000282806776157579 · Cited by 293
Precinct or prejudice? Understanding racial disparities in New York City’s stop-and-frisk policy
Goel, S., Rao, J. M., & Shroff, R. (2016) · The Annals of Applied Statistics · doi:10.1214/15-aoas897 · Cited by 285
Self-citationTesting for Racial Profiling in Traffic Stops From Behind a Veil of Darkness
Grogger, J., & Ridgeway, G. (2006) · Journal of the American Statistical Association · doi:10.1198/016214506000000168 · Cited by 260
An Improved Acceptance Procedure for the Hybrid Monte Carlo Algorithm
Neal, R. M. (1994) · Journal of Computational Physics · doi:10.1006/jcph.1994.1054 · Cited by 179
Assessing the Effect of Race Bias in Post-traffic Stop Outcomes Using Propensity Scores
Ridgeway, G. (2006) · Journal of Quantitative Criminology · doi:10.1007/s10940-005-9000-9 · Cited by 176
Outcome Tests of Racial Disparities in Police Practices
Ayres, I. (2002) · Justice Research and Policy · doi:10.3818/jrp.4.1.2002.131 · Cited by 156
Ridgeway, G. K., & MacDonald, J. M. (2009) · Journal of the American Statistical Association · doi:10.1198/jasa.2009.0034 · Cited by 89
ENGEL, R. S., & Cherkauskas, J. C. (2004) · Police Quarterly · doi:10.1177/1098611103257686 · Cited by 81
Combatting Police Discrimination in the Age of Big Data
Goel, S., Perelman, M., Shroff, R., & Sklansky, D. A. (2017) · New Criminal Law Review · doi:10.1525/nclr.2017.20.2.181 · Cited by 53
Self-citationAlpert, G. P., Smith, M. R., & Dunham, R. G. (2004) · Justice Research and Policy · doi:10.3818/jrp.6.1.2004.43 · Cited by 50
Searching for Equilibrium: The Tenuous Nature of the Outcome Test
Engel, R. S., & Tillyer, R. (2008) · Justice Quarterly · doi:10.1080/07418820701717243 · Cited by 30
The facts of lending discrimination cannot be argued away by examining default rates
Galster, G. (1993) · Housing Policy Debate · doi:10.1080/10511482.1993.9521127 · Cited by 29
Fast Threshold Tests for Detecting DiscriminationPreprint
Pierson, E., Corbett‐Davies, S., & Goel, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.08536 · Cited by 20
Self-citationThe Good and Bad News About Consent Searches in the Supreme Court
Maclin, T. (2008) · eYLS (Yale Law School) · Cited by 7
Research works printed but not linked: 2
- Arrow, K. (1973). The theory of discrimination. In Discrimination in Labor Markets Princeton Univ. Press, Princeton.
- Becker, G. S. (1957). The Economics of Discrimination. Univ. Chicago Press, Chicago, IL.
Cited by
Sorted by citations
Algorithmic Decision Making and the Cost of Fairness
Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., & Huq, A. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098095 · Cited by 1,149
Self-citationFairness definitions explained
Verma, S., & Rubin, J. (2018) · Proceedings of the International Workshop on Software Fairness · doi:10.1145/3194770.3194776 · Cited by 1,124
Algorithmic Fairness: Choices, Assumptions, and Definitions
Mitchell, S., Potash, E., Barocas, S., D'Amour, A., & Lum, K. (2021) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-042720-125902 · Cited by 687
A large-scale analysis of racial disparities in police stops across the United States
Pierson, E., Simoiu, C., Overgoor, J., Corbett-Davies, S., Jenson, D., Shoemaker, A., Ramachandran, V., Barghouty, P., Phillips, C., Shroff, R., et al. (2020) · Nature Human Behaviour · doi:10.1038/s41562-020-0858-1 · Cited by 552
Self-citationThe Measure and Mismeasure of Fairness
Corbett-Davies, S., Gaebler, J. D., Nilforoshan, H., Shroff, R., & Goel, S. (2018) · arXiv · doi:10.48550/arxiv.1808.00023 · Cited by 444
Self-citation
Metadata
- Authors
- Simoiu, C., Corbett-Davies, S., & Goel, S.
- Affiliations
- Stanford University (US)
- ORCID
- Goel, Sharad
- Venue
- The Annals of Applied Statistics
- Publisher
- Institute of Mathematical Statistics
- Published
- 2017-09-01
- Volume
- 11
- Issue
- 3
- Pages
- 1193-1216
- ISSN
- 1932-6157 (ISSN-L)
- 1941-7330
- OpenAlex
- W2950220130
- Cited by
- 154
- References
- 30
- Citations by year
- 2026: 6 · 2025: 10 · 2024: 16 · 2023: 21 · 2022: 17 · 2021: 22 · 2020: 28 · 2019: 17 · 2018: 15 · 2017: 2
- FWCI
- 38.87
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- The Annals of Applied Statistics (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Project Euclid (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Stanford Bio-X
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Sociology and Political Science › Names, Identity, and Discrimination Research
- Social Sciences › Law › Discrimination and Equality Law
- Keywords
- discrimination detection, threshold test, decision threshold, risk allocation, algorithmic fairness, police stops, racial disparities, loan repayment rates, hiring discrimination, credit discrimination
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
