Identifying Police Officers at Risk of Adverse Events
The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
Adverse events between police and the public, such as deadly shootings or instances of racial profiling, can cause serious or deadly harm, damage police legitimacy, and result in costly litigation. Evidence suggests these events can be prevented by targeting interventions based on an Early Intervention System (EIS) that flags police officers who are at a high risk for involvement in such adverse events. Today's EIS are not data-driven and typically rely on simple thresholds based entirely on expert intuition. In this paper, we describe our work with the Charlotte-Mecklenburg Police Department (CMPD) to develop a machine learning model to predict which officers are at risk for an adverse event. Our approach significantly outperforms CMPD's existing EIS, increasing true positives by ~12% and decreasing false positives by ~32%. Our work also sheds light on features related to officer characteristics, situational factors, and neighborhood factors that are predictive of adverse events. This work provides a starting point for police departments to take a comprehensive, data-driven approach to improve policing and reduce harm to both officers and members of the public.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The gaze of risk assessments can be turned from defendants to the actors and institutions that comprise the criminal justice system [21,34], enabling a more structural view of the system's operations.”
From The false promise of risk assessments: epistemic reform and the limits of fairness (2020) · cites this paper as [21]
In the citing paper
“Importantly, we included metrics at a certain percentage of the population (often referred to as metrics at k) because they are more commonly used in ML for public policy in order to specifically addresses real-world resource constraints [2,12,66].”
From It’s Just Not That Simple: An Empirical Study of the Accuracy-Explainability Trade-off in Machine Learning for Public Policy (2022) · cites this paper as [12]
In the citing paper
“These metrics were chosen because they reflect the performance measures typically used in ML for public policy [9,12,20,56].”
From It’s Just Not That Simple: An Empirical Study of the Accuracy-Explainability Trade-off in Machine Learning for Public Policy (2022) · cites this paper as [12]
In the citing paper
“We chose these classifiers because they are representative of the types of models commonly used in ML for public policy systems [9,12,20,56].”
From It’s Just Not That Simple: An Empirical Study of the Accuracy-Explainability Trade-off in Machine Learning for Public Policy (2022) · cites this paper as [12]
Citations
Cites
Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-day Readmission
Caruana, R., Lou, Y., Gehrke, J., Koch, P., Sturm, M., & Elhadad, N. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2788613 · Cited by 1,685
The United States Department of Justice
Fairlie, J. (1905) · Michigan Law Review · doi:10.2307/1273036 · Cited by 542
Neighborhood Context and Police Use of Force
Terrill, W., & Reisig, M. D. (2003) · Journal of Research in Crime and Delinquency · doi:10.1177/0022427803253800 · Cited by 518
Identifying situational predictors of police shootings using multivariate analysis
White, M. D. (2002) · Policing An International Journal · doi:10.1108/13639510210450659 · Cited by 81
Use of force in minority communities is related to police education, age, experience, and ethnicity
Chapman, C. (2011) · Police Practice and Research · doi:10.1080/15614263.2011.596711 · Cited by 77
Early Warning Systems: Responding to the Problem Police Officer
Walker, S., Alpert, G. P., & Kenney, D. J. (2001) · Cited by 72
(2024) · CQ Press eBooks · doi:10.4135/9781071920541.n114 · Cited by 65
White, M. D., & Kane, R. J. (2013) · Criminal Justice and Behavior · doi:10.1177/0093854813486269 · Cited by 59
Exploring the Impact of Police Officer Education Level on Allegations of Police Misconduct
Manis, J., Archbold, C. A., & Hassell, K. D. (2008) · International Journal of Police Science & Management · doi:10.1350/ijps.2008.10.4.102 · Cited by 55
Shjarback, J. A. (2015) · Policing · doi:10.1093/police/pav006 · Cited by 37
Research works printed but not linked: 6
- Davis R. C., 2003, Washington
- Fernandez M., 2015, New York Times
- Goldstein H., 1977, Ballinger Pub. Co
- Hyndman R. J., 2014, OTexts
- Jones N., 2014, Proactive policing and adolescent development among young, poor black men. New directions for child and adolescent development
- Shultz A., 2015, Early warning systems: What's new? what's working?
Cited by
The Ethics of Artificial Intelligence: Principles, Challenges, and Opportunities
Floridi, L. (2023) · doi:10.1093/oso/9780198883098.001.0001 · Cited by 507
How to Design AI for Social Good: Seven Essential Factors
Floridi, L., Cowls, J., King, T. C., & Taddeo, M. (2020) · Science and Engineering Ethics · doi:10.1007/s11948-020-00213-5 · Cited by 430
“…to game-theoretic models to prevent poaching (Fang et al. 2016); from online reinforcement learning to target HIV-education at homeless youths (Yadav et al. 2016a, b) to probabilistic models to prevent harmful policing (Carton et al. 2016) and support student retention (Lakkaraju et al. 2015).”
BackgroundMethodAlgorithmic bias: Senses, sources, solutions
Fazelpour, S., & Danks, D. (2021) · Philosophy Compass · doi:10.1111/phc3.12760 · Cited by 272
Data Science as Political Action: Grounding Data Science in a Politics of Justice
Green, B. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3658431 · Cited by 156
How to Design AI for Social Good: Seven Essential Factors
Floridi, L., Cowls, J., King, T. C., & Taddeo, M. (2021) · Philosophical Studies Series · doi:10.1007/978-3-030-81907-1_9 · Cited by 161
Considerations for Evaluation and Generalization in Interpretable Machine Learning
Doshi-Velez, F., & Kim, B. (2018) · The Springer Series on Challenges in Machine Learning · doi:10.1007/978-3-319-98131-4_1 · Cited by 176
Artificial Intelligence for Social Good
Hager, G. D., Drobnis, A., Fang, F., Ghani, R., Greenwald, A., Lyons, T., Parkes, D. C., Schultz, J., Saria, S., Smith, S. F., et al. (2019) · arXiv · doi:10.48550/arxiv.1901.05406 · Cited by 108
Self-citationBell, A., Solano-Kamaiko, I., Nov, O., & Stoyanovich, J. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533090 · Cited by 103
“These metrics were chosen because they reflect the performance measures typically used in ML for public policy [9, 12, 20, 56].”
MethodStapleton, L., Lee, M. H., Qing, D., Wright, M., Chouldechova, A., Holstein, K., Wu, Z. S., & Zhu, H. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533177 · Cited by 76
Designing AI for Social Good: Seven Essential Factors
Cowls, J., King, T., Taddeo, M., & Floridi, L. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3388669 · Cited by 83
The false promise of risk assessments: epistemic reform and the limits of fairness
Green, B. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372869 · Cited by 80
Explainable machine learning for public policy: Use cases, gaps, and research directions
Amarasinghe, K., Rodolfa, K. T., Lamba, H., & Ghani, R. (2023) · Data & Policy · doi:10.1017/dap.2023.2 · Cited by 73
“, counseling, training, adjustments to duties), many police departments make use of Early Intervention Systems (EIS), including several ML-based systems (see Carton et al. [2016] for an example).”
BackgroundInfluentialSelf-citationAlgorithms and values in justice and security
Hayes, P., van de Poel, I., & Steen, M. (2020) · AI & Society · doi:10.1007/s00146-019-00932-9 · Cited by 71
Hagos, D. H., & Rawat, D. B. (2022) · Sensors · doi:10.3390/s22249916 · Cited by 68
How Many Complaints Against Police Officers Can Be Abated by Incapacitating A Few 'Bad Apples?'
Chalfin, A. J., & Kaplan, J. (2020) · Criminology & Public Policy · doi:10.1111/1745-9133.12542 · Cited by 51
Public Trust, Institutional Legitimacy, and the Use of Algorithms in Criminal Justice
Purves, D., & Davis, J. (2022) · Public Affairs Quarterly · doi:10.5406/21520542.36.2.03 · Cited by 49
FATE in AI: Towards Algorithmic Inclusivity and Accessibility
Inuwa-Dutse, I. (2023) · ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO) · doi:10.1145/3617694.3623233 · Cited by 52
Kuppler, M., Kern, C. A., Bach, R. L., & Kreuter, F. (2022) · Frontiers in Sociology · doi:10.3389/fsoc.2022.883999 · Cited by 41
Early intervention systems for police: a state-of-the-art review
Gullion, C. L., & King, W. R. (2020) · Policing An International Journal · doi:10.1108/pijpsm-02-2020-0027 · Cited by 26
Tool for Surveillance or Spotlight on Inequality? Big Data and the Law
Johnson, R., & Rostain, T. (2020) · Annual Review of Law and Social Science · doi:10.1146/annurev-lawsocsci-061020-050543 · Cited by 29
Deploying Machine Learning Models for Public Policy: A Framework
Ackermann, K., Walsh, J., De Unánue, A., Naveed, H., Navarrete Rivera, A., Lee, S.-J., Bennett, J., Defoe, M., Cody, C., Haynes, L., et al. (2018) · Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3219819.3219911 · Cited by 22
Self-citationExplanatory Paradigms in Neural Networks: Towards relevant and contextual explanations
AlRegib, G., & Prabhushankar, M. (2022) · IEEE Signal Processing Magazine · doi:10.1109/msp.2022.3163871 · Cited by 23
Criminology in the age of data explosion: New directions
Ozkan, T. (2019) · The Social Science Journal · doi:10.1016/j.soscij.2018.10.010 · Cited by 25
Chandrasekaran, A., Currano, R., Batool, V., Chen, K., Murnane, E. L., Sirkin, D. M., & Mauriello, M. L. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3686957 · Cited by 15
Private security and public police
Grunwald, B., Rappaport, J., & Berg, M. (2024) · Journal of Empirical Legal Studies · doi:10.1111/jels.12393 · Cited by 11
A Unified Framework of Ethical Principles for AI
Floridi, L. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780198883098.003.0004 · Cited by 11
Gullion, C. L., & Ingram, J. R. (2024) · Justice Quarterly · doi:10.1080/07418825.2024.2333414 · Cited by 7
Gélvez, J. D. (2025) · Humanities and Social Sciences Communications · doi:10.1057/s41599-025-04967-w · Cited by 7
Design, Control, Predict: Cultural Politics In The Actually Existing Smart City
Shapiro, A. M. (2018) · ScholarlyCommons (University of Pennsylvania) · Cited by 11
Present: AI as a New Form of Agency, Not Intelligence
Floridi, L. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780198883098.003.0002 · Cited by 6
Yavaşgel, E., & Turdubaeva, E. (2021) · Istanbul University Press eBooks · doi:10.26650/b/ss07.2021.002.01 · Cited by 9
Repurposing recidivism models for forecasting police officer use of force
Khorshidi, S., Carter, J. G., & Mohler, G. (2020) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata50022.2020.9378173 · Cited by 12
Katz, C. M., Cheon, H., Freemon, K., & Wallace, D. (2026) · Police Quarterly · doi:10.1177/10986111251353487 · Cited by 4
Algorithmic Rural Road Planning in India: Constrained Capacities and Choices in Public Sector
Nisar, H., Gupta, D. K., Kumar, P., Murapaka, S. R., Rajesh, A. V., & Upadhyaya, A. (2022) · ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO) · doi:10.1145/3551624.3555299 · Cited by 4
Problem officers and problem behavior: on the object of early intervention systems
Worden, R. E., Harris, C. J., McLean, S. J., & Worden, K. M. (2025) · Policing A Journal of Policy and Practice · doi:10.1093/police/paaf002 · Cited by 3
Grandes datos, Google y desempleo
Campos-Vázquez, R. M., & Araiza, S. E. L. (2020) · Estudios Económicos de El Colegio de México · doi:10.24201/ee.v35i1.399 · Cited by 4
From Principles to Practices: The Risks of Being Unethical
Floridi, L. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780198883098.003.0005 · Cited by 2
Rentsch, S. (2023) · Sozialwissenschaften und Berufspraxis · doi:10.1007/978-3-658-38268-1_2 · Cited by 2
Gullion, C. L. (2024) · Policing A Journal of Policy and Practice · doi:10.1093/police/paae133 · Cited by 2
Fairness in Machine Learning for National Statistical Organizations
Schenk, P. O., Kern, C. A., & Kreuter, F. (2025) · Society, environment and statistics · doi:10.1007/978-3-032-10004-7_6 · Cited by 2
Расовое профилирование: история и современное состояние исследований
Григорьева, К. С. (2019) · Демографическое обозрение · doi:10.17323/demreview.v6i4.10429
Future: The Foreseeable Development of AI
Floridi, L. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780198883098.003.0003 · Cited by 2
AI and the UN Sustainable Development Goals
Floridi, L. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780198883098.003.0012 · Cited by 2
Simson, J., Fabris, A., Fröhner, C., Kreuter, F., & Kern, C. A. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.22363 · Cited by 2
The Promises and Pitfalls of Modern Early Intervention Systems for Police Misconduct
Cubitt, T. (2026) · Springer eBooks · doi:10.1007/978-3-032-14070-8_5 · Cited by 1
Soft Ethics and the Governance of AI
Floridi, L. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780198883098.003.0006 · Cited by 1
Floridi, L. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780198883098.002.0003 · Cited by 1
The Design and Evaluation of Neural Attention Mechanisms for Explaining Text Classifiers
Carton, S. (2019) · Deep Blue (University of Michigan)
Self-citationAudiovisual Big Data Analytics and Harm Prevention
Fan, M. D. (2019) · Cambridge University Press eBooks · doi:10.1017/9781108290364.005
Floridi, L. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780198883098.002.0009
Floridi, L. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780198883098.003.0001
Floridi, L. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780198883098.002.0004
Mapping the Ethics of Algorithms
Floridi, L. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780198883098.003.0007
How to Deliver a Good AI Society: Some Recommendations
Floridi, L. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780198883098.003.0010
Floridi, L. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780198883098.002.0007
Bad Practices: The Misuse of AI for Social Evil
Floridi, L. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780198883098.003.0008
List of Most Common Abbreviations and Acronyms
Floridi, L. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780198883098.002.0011
The Gambit: AI Impact on Climate Change
Floridi, L. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780198883098.003.0011
Floridi, L. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780198883098.002.0010
Good Practices: The Proper Use of AI for Social Good
Floridi, L. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780198883098.003.0009
Floridi, L. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780198883098.002.0005
Conclusion: The Green and the Blue
Floridi, L. (2023) · Oxford University Press eBooks · doi:10.1093/oso/9780198883098.003.0013
Wang, S. (2023) · Open Access Victoria University of Wellington | Te Herenga Waka (Figshare) · doi:10.26686/wgtn.24208311 · Cited by 2
(2024) · International Scientific Journal Monte · doi:10.33807/monte.2024v9n1
THE MERITS AND ETHICAL CONUNDRUMS OF PREDICTIVE POLICING
Rustemi, E., & Tahiri, M. (2024) · International Scientific Journal Monte · doi:10.33807/monte.20243099
International Journal of Digitalization and Applied Management
Bettioui, R. E., Gil, F. B., & Wieland, O. (2024) · IJDAM - International Journal of Digitalization and Applied Management. · doi:10.23882/ijdam
The relationship between officer misconduct and convictionless arrests
Ba, B., Rim, N., & Rivera, R. G. (2026) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2603437123 · Cited by 1
Metadata
- Authors
- Carton, S., Helsby, J., Joseph, K., Mahmud, A., Park, Y., Walsh, J., Cody, C., Patterson, C. E., Haynes, L., & Ghani, R.
- Affiliations
- University of Michigan (US)
- University of Chicago (US)
- Carnegie Mellon University (US)
- Princeton University (US)
- University of Arizona (US)
- Charlotte-Mecklenburg Police Department, Charlotte, NC, USA
- Venue
- Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
- Publisher
- ACM
- Published
- 2016-08-13
- Pages
- 67-76
- OpenAlex
- W2516651778
- Cited by
- 64
- References
- 15
- Citations by year
- 2026: 2 · 2025: 4 · 2024: 7 · 2023: 25 · 2022: 7 · 2021: 4 · 2020: 8 · 2019: 4 · 2018: 3
- FWCI
- 22.91
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) (publishedVersion)
- Funders
- Office of Science
- Topics
- Social Sciences › Political Science and International Relations › Policing Practices and Perceptions
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Law › Artificial Intelligence in Law
- Keywords
- early intervention systems, risk prediction, police shootings, racial profiling, police legitimacy, machine learning prediction models, situational factors, neighborhood factors
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.