CoChat

Evaluating the Effectiveness of a Police Early Intervention System: From the Predictive Validity of Officer Identification to the Impact of Intervention

The DOI matches its Crossref record; 35 of the 37 references it cites were found in a registry or an index.

Abstract

As a non-punitive approach to addressing and minimizing officer misconduct, early intervention systems (EISs) have become a best practice in policing. These systems focus on identifying officers at risk of future problematic behavior and providing effective early interventions. Despite the widespread use of EISs, limited research has evaluated these systems, and the evaluations that do exist often concentrate solely on either the selection of officers for intervention or the effects of interventions, which restricts our understanding of the overall effectiveness of EISs. Instead of concentrating on evaluating only a part of an EIS, we explore both the implementation and effectiveness of the Phoenix Police Department (PPD) EIS at two different stages. Utilizing data from over 2000 officers employed by the PPD between 2016 and 2020, we use receiver operating characteristic (ROC) curves to analyze the accuracy of officer selection for EIS involvement and a population-averaged interrupted time series model to determine whether EIS implementation is associated with a reduction in problematic officer behavior. Our findings indicated that the indicators and thresholds used by PPD to identify at-risk officers resulted in low predictive validity (none greater than 4%). Furthermore, establishing the EIS was not significantly associated with a decrease in problematic officer behavior. We conclude with a discussion of research and policy implications and future directions.

Overview

Integrity signals

Registry

Crossref · 10.1177/10986111251353487

Cites

35 of 35 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Citations

Cites

Show all 35 references

Research works printed but not linked: 11

  1. Benchmark Analytics. (2024). Identifying officers at risk of misconduct: How Benchmark analytics supports law enforcement agencies. Presented at the London Data Science Expo. https://www.benchmarkanalytics.com/wp-content/uploads/2024/04/2024-03-19-London-Data-Science-Presentation-Final.pdf
  2. Bobb M. J., Barge M., Mazar Y., Naguib C., Shugrue T. (2009). Los Angeles county sheriff’s department: 27th semiannual report. Police Assessment Resource Center. https://static1.squarespace.com/static/5498b74ce4b01fe317ef2575/t/54fc7497e4b
  3. Boston Globe. (1992). Wave of abuse claims laid to a few officers.
  4. Davis R. C., Henderson N. J., Mandelstam J., Ortiz C. W., Miller J. (2005). Federal intervention in local policing: Pittsburgh’s experience with a consent decree. Office of Community Oriented Policing Services. https://www.hsdl.org/?view&did5463256
  5. Davis R. C., Ortiz C. W., Henderson N. J., Miller J., Massie M. K. (2002). Turning necessity into virtue: Pittsburgh’s experience with a federal consent decree. Vera Institute of Justice. https://www.vera.org/downloads/Publications/turning-necessities-intovirtue-pittsburghs-experience-with-a-federal-consent-decree/legacy_downloads/Pittsburgh_consent_decree.pdf
  6. Fyfe J. J., Kane R. (2006). Bad cops: A study of career-ending misconduct among New York city police officers. John Jay College of Criminal Justice.
  7. Gullion C. L. (2022). Early intervention systems: An exploration of interventions handled by supervisors to address at-risk officer behavior (Publication No. 29605463) [Doctoral dissertation, Sam Huston State University]. ProQuest Dissertations & Theses Global.
  8. Hickey K. (2009). Report on an inquiry into early intervention systems in the new south wales police force. New South Wales Parliament, Committee on the Office of the Ombudsman and the Police Integrity Commission.
  9. Katz C. M., Walker S. (2004). An evaluation of Phoenix police department’s personnel assessment system training program. University of Nebraska at Omaha.
  10. Phoenix Police Department. (n.d.). Early intervention referral template.
  11. U.S. Census. (2020). Population estimates, July 1, 2019. U.S. Department of Commerce.

Metadata

Authors
Katz, C. M., Cheon, H., Freemon, K., & Wallace, D.
Affiliations
  • Arizona State University (US)
  • The University of Texas at El Paso (US)
  • DePaul University (US)
Corresponding author
Katz, Charles M.
Venue
Police Quarterly
Published
2025-06-22
Volume
29
Issue
1
Pages
6-39
ISSN
  • 1098-6111 (ISSN-L)
  • 1552-745X
OpenAlex
W4411532461
Cited by
4
References
37
Citations by year
2026: 3 · 2025: 1
FWCI
14.93
Type
journal article
Language
en
Open access
closed
Licence
journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
Locations
Police Quarterly (publishedVersion)
Funders
  • The City of Phoenix (153669-0)
  • The Watts Family
Topics
  • Social Sciences › Political Science and International Relations › Policing Practices and Perceptions
  • Social Sciences › Sociology and Political Science › Crime Patterns and Interventions
  • Social Sciences › Education › Education Discipline and Inequality
Keywords
officer misconduct, predictive validity, intervention effectiveness, police accountability

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Evaluating the Effectiveness of a Police Early Intervention System: From the Predictive Validity of Officer Identification to the Impact of Intervention. https://app.cochat.ai/papers/10.1177/10986111251353487
The paper
Katz, C. M., Cheon, H., Freemon, K., & Wallace, D. (2026). Evaluating the Effectiveness of a Police Early Intervention System: From the Predictive Validity of Officer Identification to the Impact of Intervention. Police Quarterly. https://doi.org/10.1177/10986111251353487

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.