The DOI matches its Crossref record; 115 of the 116 references it cites were found in a registry or an index; 13 of the 33 sources its claims cite were checked for support.
Administrative Records Mask Racially Biased Policing
Open accessAbstract
Researchers often lack the necessary data to credibly estimate racial discrimination in policing. In particular, police administrative records lack information on civilians police observe but do not investigate. In this article, we show that if police racially discriminate when choosing whom to investigate, analyses using administrative records to estimate racial discrimination in police behavior are statistically biased, and many quantities of interest are unidentified—even among investigated individuals—absent strong and untestable assumptions. Using principal stratification in a causal mediation framework, we derive the exact form of the statistical bias that results from traditional estimation. We develop a bias-correction procedure and nonparametric sharp bounds for race effects, replicate published findings, and show the traditional estimator can severely underestimate levels of racially biased policing or mask discrimination entirely. We conclude by outlining a general and feasible design for future studies that is robust to this inferential snare.
Overview
CoChat Summary
## TL;DR
The paper argues that analyses using police administrative records can systematically misstate racial bias because the data only include people police chose to stop, making the observed sample itself a product of potentially discriminatory selection. Using principal stratification and a causal mediation framework, the authors derive the bias, propose bounds and a bias-correction strategy, and show in a reanalysis of NYPD data that conventional estimates can dramatically understate racially discriminatory use of force.
## Key findings
- If police discriminate when deciding whom to stop, then estimates based only on stopped individuals are statistically biased; the paper says the bias is always negative, so naïve estimates understate racially discriminatory police violence.
- In their reanalysis of Fryer (2019) using NYPD Stop, Question and Frisk data from 2003–13, the paper says the naïve approach suggested only about 10,000 black and Hispanic civilians were handcuffed because of racial bias, while the bias-corrected estimate is approximately 56,000.
- For uses of force against black and Hispanic civilians in those data, the paper says the naïve estimate implied 10% were discriminatory, while the bias-corrected estimate is 39%.
- In one example, the naïve approach implied 2.4 instances per 1,000 encounters of pushing Hispanic suspects to a wall due to suspect race, while the revised estimate was at least 26 per 1,000 encounters.
## Methods
- The paper uses a principal stratification / causal mediation framework to define quantities of interest, derive the exact form of bias from conventional estimation, and distinguish estimates among stopped encounters from the overall causal effect of race.
- It proposes nonparametric sharp bounds for the average treatment effect among stops and the average treatment effect on the treated, and a bias-correction procedure based on an estimated share of racially discriminatory stops.
- It reanalyzes and extends Fryer (2019) using the NYPD Stop, Question and Frisk database (2003–13), which the paper says contains roughly 5 million pedestrian stop records, and compares naïve versus bias-corrected estimates across force thresholds.
## Limitations
- The paper explicitly says its bias-correction and bounding methods still rely on weaker assumptions that can be difficult to verify, and that some key quantities remain only partially identified without additional data.
- It states that point identification of the ATE requires additional data on unreported encounters; police administrative records alone are “virtually uninformative” about the unobserved proportion of discriminatory stops.
- Evident from the text: the reanalysis depends on assumptions about the proportion of discriminatory stops, including a conservative estimate from prior work, so corrected estimates still depend on an externally supplied sensitivity parameter.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures






Verification
Claims the paper makes
11 Valid · 5 Not valid · 15 Not checked · 114 Own results and interpretations
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“Replicating and extending the study of police behavior in New York in Fryer (2019), we show that the consequences of ignoring the selective process that generates police data are severe, leading analysts to dramatically underestimate or conceal entirely the differential police violence faced by civilians of color.”
What the sources say
- Could not verifycites Fryer 2019 sourceExperimental — accuracy not yet measured
The cited work is Fryer and Toros’s *Viktor Simov*, not Roland Fryer’s 2019 study of police behavior. Two paper-scoped searches returned no snippets, and we do not hold its full text. Semantic Scholar snippets from other policing papers discuss selection bias and identify Knox, Lowe, and Mummolo’s work as relevant, but none provides the high-confidence matching text required to establish a specific misattribution under this rubric. (LLM, conf=0.72)
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on 2 assumptions not shown here.
- Could not verify
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Knox and Mummolo(2020)showthat this conclusion rests on the erroneous assumption that police encounter minority and white civilians in equal number.”
What the sources say
- Partly supportedcites Knox and Mummolo 2020 sourceExperimental — accuracy not yet measured
In “Making inferences about racial disparities in police violence” (page unavailable), the authors say Johnson et al. did not account for how many minority or White civilians police encounter, so its data cannot support the claimed comparison of shooting rates. The paper does not say Johnson et al. assumed equal numbers of encounters or establish that such an assumption is false. (LLM, conf=0.86)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Using the SQF data and controlling for precinct, suspected crime, and prior local arrest rates by race, Gelman, Fagan, and Kiss (2007) produce estimates that -by our calculations -imply 32% of black-civilian stops made by the NYPD could not be explained even by differential criminality between racial groups of suspects, as proxied by prior arrest rates.”
What the sources say
- Partly supportedcites Gelman, Fagan, and Kiss (2007) sourceExperimental — accuracy not yet measured
Section 5.1 reports estimates by suspected crime and precinct showing that, relative to prior arrests, Black civilians were stopped more often than white civilians for the most common stop categories. Section 6 concludes that prior arrest rates and precinct do not explain the disparities. The cited paper does not give or verify the citing authors’ calculation that 32% of all Black-civilian stops were unexplained. (LLM, conf=0.81)
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Using a fl exible logistic regression to adjust for a vast array of indicators visible to of fi cers prestop, the study shows that white hit rates exceeded those of ' similarly situated ' black civilians.”
What the sources say
- Supportedcites Goel, Rao, and Shroff (2016) source
The Results (p. 15) describe using logistic regression to estimate hit rates for hypothetical similarly situated whites while preserving other aspects of stops of black civilians. They report a 3.8% hit rate for similarly situated whites versus 2.5% for black civilians. (LLM, conf=0.95)
boostedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“To state this more formally, note that the na¨ ı ve estimator recovers the weighted combination of violence rates in observed principal strata:”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
60 more claims
Create a free account to see all 145 claims and to verify the 4 references still open.
- It may state two separate claims as one. 33
- Probably not a substantial claim. 54
- The quote only partly states this claim. 2
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Self-check results appear here once an independent check has confirmed them.
Cited works that were retracted
Works this paper cites that were later retracted, from the retraction notices publishers deposit. Major when the notice came out before this paper was published.
A note, not an issue: for example, a cited work that was retracted only after this paper was published, which its authors could not have known.
In the paper
“10.1073/pnas.1903856116”
Cited work 10.1073/pnas.1903856116 · retracted 2020-07-10 · notice 10.1073/pnas.2014148117 · this paper published 2020-05-21
Citations
Cites
1 retracted
RETRACTED: Officer characteristics and racial disparities in fatal officer-involved shootings
Johnson, D. J., Tress, T., Burkel, N., Taylor, C., & Cesario, J. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1903856116 · Cited by 83
RetractedSample Selection Bias as a Specification Error
Heckman, J. J. (1979) · Econometrica · doi:10.2307/1912352 · Cited by 28,779
Estimating causal effects of treatments in randomized and nonrandomized studies
Rubin, D. B. (1974) · Journal of Educational Psychology · doi:10.1037/h0037350 · Cited by 9,436 · cited 1×
“We ground our analysis in the potential outcomes framework (Rubin 1974) often used in the study of causal mediation (Imai et al. 2011; Pearl 2001).”
Street-Level Bureaucracy: Dilemmas of the Individual in Public Services
Lipsky, M. (1981) · Michigan Law Review · doi:10.2307/1288305 · Cited by 8,981
Marginal Structural Models and Causal Inference in Epidemiology
Robins, J. M., Hernán, M. Á., & Brumback, B. (2000) · Epidemiology · doi:10.1097/00001648-200009000-00011 · Cited by 5,581
Statistics and Causal Inference
Holland, P. W. (1986) · Journal of the American Statistical Association · doi:10.1080/01621459.1986.10478354 · Cited by 5,293
Mostly Harmless Econometrics: An Empiricist's Companion
Angrist, J. D., & Pischke, J.-S. (2008) · doi:10.2307/j.ctvcm4j72 · Cited by 5,051
The new Jim Crow: mass incarceration in the age of colorblindness
(2010) · Choice Reviews Online · doi:10.5860/choice.48-1766 · Cited by 4,407
Identification of Causal Effects Using Instrumental Variables
Angrist, J. D., Imbens, G. W., & Rubin, D. B. (1996) · Journal of the American Statistical Association · doi:10.1080/01621459.1996.10476902 · Cited by 4,230
Orne, M. T. (1962) · American Psychologist · doi:10.1037/h0043424 · Cited by 4,041
Research works printed but not linked: 1
- Kocieniewski, 2002, Study Suggests Racial Gap in Speeding in New Jersey
Cited by
How Structural Racism Works — Racist Policies as a Root Cause of U.S. Racial Health Inequities
Bailey, Z. D., Feldman, J. M., & Bassett, M. T. (2021) · New England Journal of Medicine · doi:10.1056/nejmms2025396 · Cited by 1,598
Risk of being killed by police use of force in the United States by age, race–ethnicity, and sex
Edwards, F., Lee, H., & Esposito, M. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1821204116 · Cited by 916
What Is Your Estimand? Defining the Target Quantity Connects Statistical Evidence to Theory
Lundberg, I., Johnson, R., & Stewart, B. M. (2021) · American Sociological Review · doi:10.1177/00031224211004187 · Cited by 364
The Effects of Police Violence on Inner-City Students*
Ang, D. (2020) · The Quarterly Journal of Economics · doi:10.1093/qje/qjaa027 · Cited by 226
Changing the narrative: structural barriers and racial and ethnic inequities in COVID-19 vaccination
Njoku, A., Joseph, M., & Felix, R. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18189904 · Cited by 164
Metadata
- Authors
- KNOX, D., LOWE, W., & MUMMOLO, J.
- Affiliations
- Princeton University (US)
- Hertie School (DE)
- Corresponding author
- KNOX, DEAN
- LOWE, WILL
- MUMMOLO, JONATHAN
- Venue
- American Political Science Review
- Publisher
- Cambridge University Press (CUP)
- Published
- 2020-05-21
- Volume
- 114
- Issue
- 3
- Pages
- 619-637
- ISSN
- 0003-0554 (ISSN-L)
- 1537-5943
- OpenAlex
- W3122864926
- Cited by
- 314
- References
- 115
- Citations by year
- 2026: 39 · 2025: 39 · 2024: 47 · 2023: 68 · 2022: 48 · 2021: 34 · 2020: 33
- FWCI
- 161.39
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- American Political Science Review (publishedVersion)
- SSRN Electronic Journal (acceptedVersion)
- PhilPapers (PhilPapers Foundation) (submittedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- OPUS 4 (Zuse Institute Berlin) (submittedVersion)
- Topics
- Social Sciences › Political Science and International Relations › Policing Practices and Perceptions
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Social Sciences › Sociology and Political Science › Criminal Justice and Corrections Analysis
- Keywords
- racial discrimination in policing, administrative records, statistical bias, principal stratification, causal mediation analysis, bias correction, sharp bounds, unobserved heterogeneity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-29, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-29 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Administrative Records Mask Racially Biased Policing. https://app.cochat.ai/papers/10.1017/s0003055420000039
- The paper
- KNOX, D., LOWE, W., & MUMMOLO, J. (2020). Administrative Records Mask Racially Biased Policing. American Political Science Review. https://doi.org/10.1017/s0003055420000039
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.