‘Led by Intelligence': A Scoping Review on the Experimental Evaluation of Intelligence-Led Policing

Probably not a substantial claim: may not pass S3 (load-bearing).

In place-based predictive policing randomized controlled trials (RCTs), focusing on larger mission areas rather than micro-scaled intervention areas has been proposed as a more appropriate approach.

“In place-based predictive policing randomized controlled trials (RCTs), focusing on larger mission areas rather than micro-scaled intervention areas has been proposed as a more appropriate approach (Ratcliffe et al., 2021;Taylor & Ratcliffe, 2019).”

• cites Ratcliffe et al., 2021 sourcePartly supported

  The Background section (‘From mission grids to treatment areas as the unit of analysis’; page unavailable) says that analysing the mission grids alone did not reflect where officers actually patrolled, so the authors included adjacent cells in a larger treatment area. The Discussion also cautions that very small areas create prediction and statistical-power problems. The paper supports using larger areas for analysis, but does not propose replacing microscale mission grids with larger mission areas for the intervention. (LLM, conf=0.86)

  Experimental — accuracy not yet measured
• cites Taylor & Ratcliffe, 2019Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim.

SUTVA posits that the assignment of units to treatment or control groups should not influence the outcomes of other units.

“SUTVA posits that the assignment of units to treatment or control groups should not influence the outcomes of other units (Braga et al., 2012(Braga et al., , 2018)).”

• cites Braga et al., 2012 sourceCould not verify

  The cited paper’s sections on outcome measures (3.1.4) and crime effects (4.3) discuss spillover and diffusion into non-treated areas, but they do not define SUTVA or state what it posits. Searches for SUTVA, interference, and treatment assignment found no passage addressing the attributed definition. (LLM, conf=0.76)

  Experimental — accuracy not yet measured
• cites Braga et al., 2018Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim.

The study adopts Arksey and O'Malley's (2005) scoping review methodology, as redeveloped by Levac et al. (2010).

“This study adopts Arksey and O'Malley's (2005) scoping review methodology, as redeveloped by Levac et al. (2010).”

• cites Arksey & O'Malley, 2005 sourceSupported
• cites Levac et al., 2010 sourceSupported

May combine two claims.

Smaller geographical units may affect the crime base rates within those units, which can impact the size and strength of treatment effects.

“However, smaller geographical units may also affect the crime base rates within those units, which can impact the size and strength of treatment effects (Phillips et al., 2016;Ratcliffe et al., 2021).”

• cites Phillips et al., 2016 sourceNot checked — no full text of the source
• cites Ratcliffe et al., 2021 sourceSupported

Probably not a substantial claim: may not pass S3 (load-bearing).

Larger sample sizes are generally thought to increase statistical power to detect significant differences between treatment and control groups.

“One factor is sample size, with larger sample sizes generally thought to increase statistical power to detect significant differences between treatment and control groups (Ratcliffe et al., 2021;Taylor & Ratcliffe, 2019).”

• cites Ratcliffe et al., 2021 sourcePartly supported

  The Results say that small grid areas hampered statistical power, and the Discussion links low crime counts at fine spatial and temporal scales to reduced power. This supports the concern that sparse data can make treatment effects harder to detect, but the paper does not state the broader claim that increasing sample size generally increases power to detect treatment–control differences. (LLM, conf=0.78)

  Experimental — accuracy not yet measured
• cites Taylor & Ratcliffe, 2019Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

Crime hot spot analysis and mapping primarily involves retrospective analysis, description, and visualization of areas with high crime concentrations based on spatial intelligence in order to allocate police resources and implement specific tactics.

“This approach primarily involves the retrospective analysis, description, and visualization of areas with high crime concentrations based on spatial intelligence in order to allocate police resources and implement specific tactics (Ariel et al., 2016;Ariel & Partridge, 2017;Collazos et al., 2020;Corsaro et al., 2021).”

• cites Ariel et al., 2016 sourceNot checked — no full text of the source
• cites Ariel & Partridge, 2017 sourcePartly supported

  Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

  Experimental — accuracy not yet measured
• cites Collazos et al., 2020 sourceNot checked — no full text of the source

May combine two claims.

Widening the scale of treatment areas may make patrol officers' activities less monotonous and reduce the likelihood of boundary violations, but it could also lead to net-widening effects and pose challenges to police legitimacy.

“However, while widening the scale of treatment areas may make patrol officers' activities less monotonous and reduce the likelihood of boundary violations, it could also lead to net-widening effects and pose challenges to police legitimacy (Ratcliffe et al., 2021;Taylor & Ratcliffe, 2020).”

• cites Ratcliffe et al., 2021 sourcePartly supported

  The Background (page unavailable) says officers could not realistically patrol only the small mission grids and had to drive outside them; the Discussion (page unavailable) says the authors expanded the analytical areas to reflect policing on the ground. This supports the boundary-related rationale for a larger area, but the paper does not say that widening patrol areas would reduce monotony or boundary violations, or that it would cause net-widening effects or challenges to police legitimacy. (LLM, conf=0.78)

  Experimental — accuracy not yet measured
• cites Taylor & Ratcliffe, 2020 sourceNot checked — no full text of the source

May combine two claims.

Assessing data quality is crucial to mitigate biases in the data, as biased data can lead to feedback loops and unreliable crime predictions.

“It is crucial to assess data quality in order to mitigate biases in the data, as biased data can lead to feedback loops and unreliable crime predictions (Hunt et al., 2014;Mohler et al., 2016;Ratcliffe et al., 2021).”

• cites Hunt et al., 2014Not checked — no full text of the source
• cites Mohler et al., 2016Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Ratcliffe et al., 2021 sourcePartly supported

  The Discussion (page unavailable) says inaccuracies in property-crime reporting prevented the authors from using a preferred analytical approach, showing that data quality affected this study. It does not say that assessing data quality mitigates bias, or that biased data creates feedback loops and unreliable crime predictions. (LLM, conf=0.76)

  Experimental — accuracy not yet measured

May combine two claims.

Qualitative analyses of crime hot spots based on specific criteria or journey-after-crime analyses, as well as quantitative methods such as the Local Moran's I statistic or point-pattern analyses, were commonly employed in the included studies.

“Qualitative analyses of crime hot spots based on specific criteria or journey-after-crime analyses, as well as quantitative methods such as the Local Moran's I statistic or point-pattern analyses, were commonly employed (spatial and temporal analysis of crime or STAC) (Carter et al., 2021;Gerell, 2016;Lum et al., 2011;Mazeika, 2014).”

• cites Carter et al., 2021 sourcePartly supported

  Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

  Experimental — accuracy not yet measured
• cites Gerell, 2016 sourceNot checked — no full text of the source
• cites Lum et al., 2011 sourceSupported

May combine two claims.

These deliberative structures provided a platform for discussing tactical decisions, addressing requirements in the police intelligence infrastructure, and ensuring accountability among stakeholders.

“These structures provided a platform for discussing tactical decisions, addressing requirements in the police intelligence infrastructure, and ensuring accountability among stakeholders (e.g., Ariel et al., 2020;Caplan et al., 2021;Ratcliffe et al., 2021;Santos & Santos, 2016).”

• cites Ariel et al., 2020 sourceNot checked — no full text of the source
• cites Caplan et al., 2021 sourceNot checked — no full text of the source
• cites Ratcliffe et al., 2021 sourcePartly supported

  The Background (page unavailable) describes a command briefing for district leaders and a need for a decision-making framework that supports operational flexibility. It does not say these structures served as a forum for addressing police-intelligence infrastructure requirements or ensuring accountability among stakeholders. (LLM, conf=0.78)

  Experimental — accuracy not yet measured

May combine two claims.

Most included studies focused on establishing crime reduction effects, whereas only one explicitly focused on measuring and reducing social harm in so-called social harm spots.

“Appendix 1 clearly shows that most of the studies focused on establishing crime reduction effects, whereas only one explicitly focused on measuring and reducing social harm in so-called social harm spots (Carter et al., 2021).”

• cites Carter et al., 2021 sourceSupported

Probably not a substantial claim: may not pass S3 (load-bearing).

In the Mohler et al. study, potential savings to society from applying predictive policing versus hot spots policing were calculated using the methodology developed by McCollister et al. (2010).

“In this study, the potential savings to society from applying predictive policing versus hot spots policing were calculated using the methodology developed by McCollister et al. (2010).”

• cites Mohler et al., 2014Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites McCollister et al., 2010 sourceSupported

May combine two claims.

It was estimated that using predictive hot spot maps for 31 minutes per day for each hot spot would save the LAPD $17,258,801 per year compared to traditional hot spot mapping, which would save half as much.

“Based on these calculations, it was estimated that using predictive hot spot maps for 31 minutes per day for each hot spot would save the LAPD $17,258,801 per year vis-à-vis applying traditional hot spot mapping techniques by crime analysts, which would only save half as much (Mohler et al., 2015).”

• cites Mohler et al., 2015 sourcePartly supported

  The Discussion projects $17,258,801 in annual savings for LAPD using ETAS with 31 minutes of patrol per hotspot per day, and $8,223,519 under analyst-selected conditions—less than half as much. But the $17,258,801 figure is calculated against no patrol, not against traditional analyst-selected hot spots, as the citing sentence implies. (LLM, conf=0.90)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The logic behind TSE has lately been expanded to various (large) data sources in general, which is why the total error framework (TEF) was established.

“However, the logic behind TSE has lately been expanded to various (large) data sources in general, which is why the total error framework (TEF) was established (Amaya et al., 2020;Snaphaan et al., 2022).”

• cites Amaya et al., 2020Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Snaphaan et al., 2022 sourceSupported

May combine two claims.

The predominant operational tactics in the included studies focused on situational measures and law enforcement, although some combined traditional law enforcement approaches with problem-solving or community-oriented tactics.

“The predominant tactics focused on situational measures and law enforcement, although some combined traditional law enforcement approaches with problem-solving or community-oriented tactics.”

May combine two claims.

These messages gave strong warnings about the increased risks associated with offending or re-offending and were considered to be forms of focused and vicarious deterrence.

“These messages gave strong warnings about the increased risks associated with offending or re-offending and were considered to be forms of focused and vicarious deterrence (Ariel et al., 2019;Santos & Santos, 2016;Saunders et al., 2016).”

• cites Ariel et al., 2019 sourceNot checked — no full text of the source
• cites Santos & Santos, 2016 sourceNot checked — no full text of the source
• cites Saunders et al., 2016 sourceNot checked — no full text of the source

May combine two claims.

Directed patrol emerged as the most commonly used tactic in the included studies, primarily targeting crime hot spots based on spatial and/or temporal intelligence about these areas, or intelligence on prolific offenders residing/loitering in certain areas.

“Directed patrol emerged as the most commonly used tactic, primarily targeting crime hot spots based on spatial and/or temporal intelligence about these areas, or intelligence on prolific offenders residing/loitering in certain areas.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The aim of these interventions was to communicate messages to deter offenders, their connections, co-offending networks, or offender groups.

“The aim was to communicate messages to deter offenders, their connections, co-offending networks, or offender groups.”

May combine two claims.

Some studies within the domain of ILP have combined different types of crime intelligence or used solely offender-related intelligence.

“However, some studies have combined different types of crime intelligence or used solely offender-related intelligence.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Most included studies were published in 2016, followed by 2014, 2015, 2020 and 2021.

“As Figure 3 shows, most of the included studies were published in 2016, followed by 2014, 2015, 2020 and 2021.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The scoping review selected 355 studies after title and abstract screening.

“As a result, 355 studies were selected.”

May combine two claims.

Ariel et al. (2019) utilized social network analysis to identify prolific offenders and their cooffending networks, but without the intention of predicting future offending or victimization.

“Ariel et al. (2019) utilized social network analysis to identify prolific offenders and their cooffending networks, but without the intention of predicting future offending or victimization.”

• cites Ariel et al., 2019 sourceNot checked — no full text of the source

May combine two claims.

Offender-focused approaches in the included studies aimed to identify prolific offenders, collect and analyze intelligence on them, and direct police resources and tactics accordingly.

“These approaches aimed to identify prolific offenders, collect, and analyze intelligence on them, and direct police resources and tactics accordingly.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Five other crime types, such as vehicle crashes, also featured in the included studies.

“Several other (5) crime types, such as vehicle crashes, also featured.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

One included study was carried out in Uruguay and one in Colombia.

“Additionally, 1 study was carried in Uruguay and 1 in Colombia.”

May combine two claims.

Some included studies conducted general operational intelligence analyses to collect and analyze information on specific cases, or utilized investigative tools such as wiretaps, covert surveillance activities, or human intelligence sources.

“Furthermore, some studies conducted general operational intelligence analyses to collect and analyze information on specific cases, or utilized investigative tools such as wiretaps, covert surveillance activities, or human intelligence sources (e.g., Morton et al., 2019;Ratcliffe et al., 2017).”

• cites Morton et al., 2019 sourceNot checked — no full text of the source
• cites Ratcliffe et al., 2017 sourceNot checked — no full text of the source

May combine two claims.

Some interventions included initiatives to enhance positive police-community interactions or provide data-driven policing materials to citizens.

“These included initiatives to enhance positive policecommunity interactions or provide data-driven policing materials to citizens (e.g., Carter et al., 2014).”

• cites Carter et al., 2014 sourceNot checked — no full text of the source

May combine two claims.

The majority of included studies were conducted in the United States (31 studies), and only 5 were conducted in Europe: 1 in Italy, 1 in Sweden and 3 in England.

“Furthermore, the majority were conducted in the United States (31 studies), and only 5 were conducted in Europe: 1 in Italy, 1 in Sweden and 3 in England.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In 7 cases, the nature of the crime was not clearly specified or was reported as an aggregated crime type.

“In some cases (7 times), the nature of the crime was not clearly specified or was reported as an aggregated crime type.”

May combine two claims.

Most included studies focused on several types of crime at the same time; violent crime was the most commonly featured crime type (24 times), followed by property crime (11 times), drug crime (5 times) and disorder (3 times).

“Moreover, most focused on several types of crime at the same time, although we present the counts here separately: violent crime was the most commonly featured crime type (24 times), followed by property crime (11 times), drug crime (5 times) and disorder (3 times).”

May combine two claims.

Of the 38 included studies, 14 employed a quasi-experimental design and 24 conducted a randomized controlled trial.

“As reported in Table 1, the proportion of quasi-experimental versus experimental studies is more or less balanced, although there are more full experimental studies: 14 of the 38 studies employed a quasi-experimental design, whereas 24 conducted a randomized controlled trial.”

May combine two claims.

A wide range of spatial and spatial-temporal units of analysis were used throughout the included studies, with hot spots and street segments being the most common, followed by street blocks, grid cells, micro-time hot spots and districts.

“In addition, a wide range of spatial and spatial-temporal units of analysis were used throughout the studies, with hot spots 4 and street segments being the most common, followed by street blocks, grid cells, micro-time hot spots and districts.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Two included studies used individuals as the unit of analysis.

“Two studies used individuals as the unit of analysis.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Traditional approaches such as log-in systems, call-ins, and observations were also employed.

“More traditional approaches such as log-in systems, call-ins, and observations were also employed (Ratcliffe et al., 2011).”

• cites Ratcliffe et al., 2011 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S4 (based).

The secondary effects of policing interventions that are not crime-related are typically underreported in (quasi-)experimental studies evaluating policing programs.

“The secondary effects of policing interventions that are not crime-related are typically underreported in (quasi-)experimental studies evaluating policing programs.”

May combine two claims.

The costs in the Shreveport CSA were mainly related to distribution of predictive hot spot maps, collection of extra intelligence, running the predictive model monthly, making tactical/strategic decisions, and executing plans.

“The costs were mainly related to the distribution of predictive hot spot maps (intelligence products) to commanders, the collection of extra intelligence that was added to the predictive hot spot maps, the costs associated with running the predictive model on a monthly basis, but also the costs of making tactical and strategic decisions based on the intelligence derived from the predictive hot”

• cites Hunt et al., 2014Not checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

Technological applications such as AVL systems and GPS trackers were utilized to track police presence.

“For example, technological applications such as AVL systems and GPS trackers were utilized to track police presence (Telep et al., 2014;Weisburd et al., 2015;Ariel et al., 2016;Ariel & Partridge, 2016).”

• cites Telep et al., 2014Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Weisburd et al., 2015 sourceNot checked — no full text of the source
• cites Ariel et al., 2016 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

Researchers often argued that observed reductions in crime or harm measures were a result of the successful deployment of police resources to targeted areas or individuals identified through crime intelligence analysis.

“Nonetheless, researchers often argued that observed reductions in crime or harm measures were a result of the successful deployment of police resources to targeted areas or individuals identified through crime intelligence analysis.”

Probably not a substantial claim: may not pass S4 (based).

To the best of the authors' knowledge, this is the first study that endeavors to incorporate multiple (quasi-)experimental evaluation studies of various tactical policing approaches within a conceptual framework of ILP.

“To the best of our knowledge, this is the first study that endeavors to incorporate multiple (quasi-)experimental evaluation studies of various tactical policing approaches within a conceptual framework of ILP.”

May combine two claims.

Although most of the included studies did not specifically focus on the evaluation of secondary effects, some did.

“Although most of the studies did not specifically focus on the evaluation of secondary effects, some did.”

Probably not a substantial claim: may not pass S3 (load-bearing).

It was calculated that per 10.4 minutes of officer proactive activity in social harm spots, the costs of social harm were reduced by $38.6, totaling $118,232 during the experimental period.

“It was calculated that per 10.4 minutes of officer proactive activity in social harm (hot) spots, the costs of social harm were reduced by $38.6, a total of $118,232 during the experimental period (Carter et al., 2014).”

• cites Carter et al., 2014 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

Saunders et al. (2016) argued that there needs to be practical reasoning and guidance from police command staff to frontline officers on how to effectively employ intelligence in practice.

“In this context, Saunders et al. (2016), who conducted interviews with police officers on the use of tactical or strategic intelligence products, argued that there needs to be practical reasoning and guidance from police command staff to frontline officers on how to effectively employ intelligence in practice.”

• cites Saunders et al., 2016 sourceNot checked — no full text of the source

Classified as probably not a substantial claim.

They calculated societal costs per crime type according to costs related to the victim, the police and court system, and the offender.

“More specifically, they calculated the societal costs per crime type according to the costs related to the victim, the police and court system, and the offender.”

• cites Mohler et al., 2014Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim.

The review summarized the main insights and findings from 38 (quasi-)experimental studies that relate to the framework of intelligence-led policing.

“In this study, we summarized the main insights and findings from 38 (quasi-)experimental studies that relate to the framework of intelligence-led policing.”

May combine two claims.

Two studies had different emphases: one evaluated whether an ILP program resulted in increased human intelligence collection and detection, while the other primarily aimed to enhance police productivity through predictive policing to detect criminal activity.

“Two studies had different emphases, with one evaluating whether an ILP program resulted in increased human intelligence collection and detection (Morton et al., 2019), while the other primarily aimed to enhance police productivity through predictive policing to detect criminal activity (Mastrubuoni, 2020).”

• cites Morton et al., 2019 sourceNot checked — no full text of the source
• cites Mastrubuoni, 2020Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim.

These cost estimates were used to compute weekly savings per police division resulting from a weekly crime reduction of 4.3 crimes.

“These cost estimates were then used to compute the weekly savings per police division that would result from a weekly crime reduction of 4.3 crimes.”

• cites Mohler et al., 2014Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

The findings of most included studies did not support crime displacement, and only a few studies reported a diffusion of crime control benefits.

“As can be seen from Appendix 1, the findings of most studies did not support crime displacement, and only a few studies reported a diffusion of crime control benefits.”

May combine two claims.

While user experiences were a focus of several studies, a few also considered the potential ethical risks associated with proactive policing interventions based on criminal intelligence.

“While user experiences were a focus of several of the studies, a few also considered the potential ethical risks associated with proactive policing interventions based on criminal intelligence.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

It can be challenging to consider the diverse needs, objectives, capabilities, and limitations of the stakeholders involved in the experiment.

“However, it can be challenging to consider the diverse needs, objectives, capabilities, and limitations of the stakeholders involved in the experiment.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Evidence of cost-effectiveness has also been found in studies employing traditional hot spots policing approaches.

“Nonetheless, evidence of cost-effectiveness has also been found in studies employing traditional hot spots policing approaches.”

May combine two claims.

The vast majority of the included studies focused on evaluating local-level interventions, mostly tackling violent crime, followed by property crime, drug crime and disorder.

“The vast majority of the studies focused on evaluating local-level interventions, mostly tackling violent crime, followed by property crime, drug crime and disorder.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Various methods were employed to enhance monitoring and compliance in the included studies.

“Various methods were employed to enhance monitoring and compliance.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Some studies have advocated including other stakeholders such as citizens, criminals, or victims who are directly or indirectly affected by the intervention, to enhance awareness and communication about specific policing methods.

“In addition, some studies have advocated including other stakeholders, such as citizens, criminals, or victims who are directly or indirectly affected by the intervention, to enhance awareness and communication about specific policing methods (Groff et al., 2015;Ratcliffe et al., 2011).”

• cites Groff et al., 2015 sourceNot checked — no full text of the source
• cites Ratcliffe et al., 2011 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The initial search was conducted in November 2021, and additional searches were performed in July 2022 and February 2023.

“The initial search was conducted in November 2021, and additional searches were performed in July 2022 and February 2023.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The scoping review process entailed two distinct screening stages in which the relevance of the studies was evaluated by applying specific eligibility requirements.

“The scoping review process entailed two distinct screening stages in which the relevance of the studies was evaluated by applying specific eligibility requirements.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Hot spot maps or lists are integral to problem profiles, offering insights into spatial crime trends and assisting in prioritizing resource allocation.

“These hot spot maps or lists are integral to problem profiles, offering insights into spatial crime trends and assisting in prioritizing resource allocation.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Studies on ILP typically categorize intelligence products into four general categories: strategic assessments, tactical assessments, subject profiles, and problem profiles.

“Studies on ILP typically categorize intelligence products into four general categories: strategic assessments, tactical assessments, subject profiles, and problem profiles (Ratcliffe, 2016).”

• cites Ratcliffe, 2016 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

A specific search strategy was developed, employing coordinated search procedures across the databases using specific keywords related to the ILP framework.

“A specific search strategy was developed, employing coordinated search procedures across these databases using specific keywords related to the ILP framework (see Appendix 3 for the specific keywords).”

Probably not a substantial claim: may not pass S3 (load-bearing).

The reference lists of multiple systematic literature reviews of policing strategies were examined.

“The reference lists of multiple systematic literature reviews of policing strategies were also examined.”

May combine two claims.

Increasing sample sizes can result in increased variability between treatment and control groups, elevating the standard deviation within the sample, which reduces the probability of detecting substantial and statistically significant effects.

“First, increasing the sample sizes can result in increased variability between the treatment and control groups, which subsequently elevates the standard deviation within the sample.This increased variability reduces the probability of detecting substantial and statistically significant effects.”

Classified as probably not a substantial claim. May combine two claims.

Avoiding spillover effects is crucial for addressing SUTVA because spillover effects can obscure the true impact of the intervention and compromise causal inference.

“Avoiding spillover effects, which can obscure the true impact of the intervention and compromise causal inference, is crucial for addressing SUTVA.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Strategic assessments typically provide an overview of intelligence requirements and prevailing concerns impacting police operations.

“These typically provide an overview of intelligence requirements and prevailing concerns impacting police operations.”

Probably not a substantial claim: may not pass S3 (load-bearing).

A manual search was carried out to identify relevant publications that may not have been captured through the database search process.

“A manual search was also carried out to identify relevant publications that may not have been captured through the database search process.”

May combine two claims.

Intelligence-led policing (ILP) was first implemented in the early 1990s primarily in the UK as a proactive approach to policing intended to enhance the effectiveness and efficiency of police organizations.

“Intelligence-led policing (ILP), first implemented in the early 1990s primarily in the UK, is a proactive approach to policing intended to enhance the effectiveness and efficiency of police organizations.”

• cites Burcher, 2020 sourceNot checked — no full text of the source
• cites NCIS, 2000Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Ratcliffe, 2003Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim.

Three academic databases-Scopus, Web of Science, and ProQuest-were consulted to systematically identify relevant publications.

“To systematically identify relevant publications, three academic databases-Scopus, Web of Science, and ProQuest-were consulted.”

Classified as probably not a substantial claim.

ILP encompasses a policing paradigm rather than a specific program or intervention.

“However, there are inherent challenges in evaluating ILP's effectiveness, as it encompasses a policing paradigm rather than a specific program or intervention, with the result that there is a dearth of evidence on its overall effectiveness (Ratcliffe, 2021;Saunders et al., 2016).”

• cites Ratcliffe, 2021Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Saunders et al., 2016 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Duplicate studies were removed through an automated process using EndNote, as well as manually by deleting duplicates in the Excel file.

“Duplicate studies were removed through an automated process using EndNote, as well as manually by deleting duplicates in the Excel file.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Researchers have suggested matching treatment units to a larger number of comparison units and conducting panel regression analyses to address statistical powerlessness in quasi-experimental designs.

“To address the problem of statistical powerlessness in quasi-experimental designs, researchers have suggested matching treatment units to a larger number of comparison units, and conducting panel regression analyses (Phillips et al., 2016).”

• cites Phillips et al., 2016 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

Intelligence bulletins align closely with tactical assessments, as they define problems and subjects and recommend tactical options.

“They also align closely with tactical assessments, as they define problems and subjects and recommend tactical options.”

May combine two claims.

Strategic assessments are difficult to incorporate into evaluation studies because they do not consider the outcomes of the intervention, and therefore they have limited application in operational policing actions.

“However, they are difficult to incorporate into evaluation studies because they do not consider the outcomes of the intervention, and therefore they have limited application in operational policing actions (Ratcliffe, 2016).”

• cites Ratcliffe, 2016 sourceNot checked — no full text of the source

May combine two claims.

Numerous quasi-experimental and experimental studies have been conducted to evaluate proactive policing interventions/programs, some of which align with the central tenets of the ILP framework.

“Nonetheless, numerous quasi-experimental and experimental studies have been conducted in the past to evaluate proactive policing interventions/programs (for an overview, see for example the National Academies of Sciences' report on proactive policing by Weisburd & Majmundar, 2018), some of which align with the central tenets of the ILP framework.”

• cites Weisburd & Majmundar, 2018Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

Prospective hot spot methods differ from retrospective hot spot methods as they create a dynamic aggregation of crime risk zones centered around each criminal incident, providing a more short-term and forward-looking perspective.

“These methods differ from retrospective hot spot methods as they create a dynamic aggregation of crime risk zones centered around each criminal incident, providing a more short-term and forward-looking perspective (Caplan et al., 2021;Kennedy et al., 2022).”

• cites Caplan et al., 2021 sourceNot checked — no full text of the source
• cites Kennedy et al., 2022Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

A more in-depth thematic analysis of the content of the articles was conducted.

“A more in-depth thematic analysis of the content of the articles was also conducted.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Intelligence bulletins combine elements of both subject and problem profiles, providing spatial intelligence on crime hot spots and comprehensive information on suspects or victims.

“Intelligence bulletins combine elements of both subject and problem profiles, providing spatial intelligence on crime hot spots and comprehensive information on suspects or victims.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The results of the scoping review were presented to and discussed by the Working Group on Policing at the annual conference of the European Society of Criminology in 2022.

“The results of the scoping review were presented to and discussed by the Working Group on Policing at the annual conference of the European Society of Criminology in 2022.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Offender subject lists contribute to subject profiles by providing detailed information on individuals at whom resources can be directed.

“Offender subject lists contribute to subject profiles by providing detailed information on individuals at whom resources can be directed.”

May combine two claims.

These studies demonstrate positive effects of ILP-focused interventions on the internal processes of law enforcement agencies, improving their intelligence capabilities and relationships with specific communities.

“Although these studies did not focus primarily on crime or harm reduction, they are relevant within the framework of ILP, as they demonstrate the positive effects of ILP-focused interventions on the internal processes of law enforcement agencies, improving their intelligence capabilities and relationships with specific communities.”

Probably not a substantial claim: may not pass S3 (load-bearing).

These factors can influence officers' perceptions and attitudes towards the experiment and specific policing approaches.

“These factors can influence officers' perceptions and attitudes towards the experiment and specific policing approaches.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

The nature of the 'policing craft' encompasses the skills, knowledge, heuristics, attitudes, and acquired expertise of police officers in their daily duties.

“One important aspect to consider is the nature of the 'policing craft,' which encompasses the skills, knowledge, heuristics, attitudes, and acquired expertise of police officers in their daily duties (Caplan et al., 2021;Ratcliffe et al., 2011Ratcliffe et al., , 2021;;Wood et al., 2012).”

• cites Caplan et al., 2021 sourceNot checked — no full text of the source
• cites Ratcliffe et al., 2011 sourceNot checked — no full text of the source
• cites Wood et al., 2012Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

Deterrence mainly includes communicating deterrent messages of 'zero-tolerance' to deter potential future offenders.

“This mainly includes communicating deterrent messages of 'zero-tolerance' to deter potential future offenders (Ariel et al., 2019;Saunders et al., 2016).”

• cites Ariel et al., 2019 sourceNot checked — no full text of the source
• cites Saunders et al., 2016 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

Establishing statistically significant reduction effects is inherently challenging and influenced by various methodological factors.

“As discussed earlier, establishing statistically significant reduction effects is inherently challenging and influenced by various methodological factors.”

May combine two claims.

Predictive policing methods may be more cost-effective than traditional hot spots policing tactics, though the robustness and validity of these projections should be critically scrutinized.

“These finding suggest that predictive policing methods may be more cost-effective than traditional hot spots policing tactics, albeit the robustness and validity of these projections should be critically scrutinized at all times.”

May combine two claims.

The review excluded non-experimental evaluation studies, which may be regarded as a limitation because some additional findings may have been missed.

“First, we excluded non-experimental evaluation studies, which may be regarded a limitation because we may have missed some additional findings.”

Probably not a substantial claim: may not pass S3 (load-bearing).

One approach to address regression to the mean is to include the pre-intervention crime level as a covariate in the crime impact model.

“One approach is to use a tailored analysis that includes the pre-intervention crime level as a covariate in the crime impact model (Ratcliffe et al., 2011).”

• cites Ratcliffe et al., 2011 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

From a theoretical and methodological standpoint, patrolling smaller grid cells with high crime rates may increase the risk of apprehension; however, from a practical perspective, officers may experience constraints from being confined to such small patrol areas and may struggle with determining whom to target or how to operate in these high-crime locations.

“For example, from a theoretical and methodological standpoint, patrolling smaller grid cells (e.g., 200 by 200 meters) with high crime rates may increase the risk of apprehension.However, from a practical perspective, officers may experience constraints from being confined to such small patrol areas and may struggle with determining whom to target or how to operate in these high-crime locations.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

The novelty or level of institutionalization of the evaluated intervention within the police department can impact officers' willingness to actively participate.

“The novelty or level of institutionalization of the evaluated intervention within the police department, can also impact officers' willingness to actively participate.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Varying the frequency and intensity of police presence can be more effective in addressing criminals' potential adaptation to the policing approaches imposed.

“In some studies, it is argued that varying the frequency and intensity of police presence can be more effective in addressing criminals' potential adaptation to the policing approaches imposed (Ariel et al., 2016;Telep et al., 2014).”

• cites Ariel et al., 2016 sourceNot checked — no full text of the source
• cites Telep et al., 2014Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

Stopping and arresting the 'right people in the right areas' could help to create a perception of procedural justice in law enforcement activities.

“The argument was made that stopping and arresting the 'right people in the right areas' could help to create a perception of procedural justice in law enforcement activities.”

May combine two claims.

Hot spots policing has received considerable academic attention and evolved into an essential tactical policing approach that can be incorporated within the framework of ILP.

“It is clear that hot spots policing has received considerable academic attention as it evolved into an essential tactical policing approach that can be incorporated within the framework of ILP.”

May combine two claims.

Proactive policing interventions based on criminal intelligence can pose ethical risks, but when implemented judiciously and with consideration for procedural justice, they may not exacerbate disparities or negatively impact community perceptions.

“These findings suggest that while proactive policing interventions based on criminal intelligence can pose ethical risks, when they are implemented judiciously and with consideration for procedural justice, they may not exacerbate disparities or negatively impact community perceptions.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Deterrence has been recognized as a significant mechanism through which offender-focused policing interventions can yield crime reduction effects.

“Furthermore, deterrence has been recognized as a significant mechanism through which offender-focused policing interventions can yield crime reduction effects.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

To address regression to the mean, (quasi-)experimental studies should account for differences in extreme scores between treatment and control groups, a task that is typically easier to control for in RCTs.

“To address this challenge, it has been suggested that (quasi-)experimental studies should account for differences in extreme scores between the treatment and control groups, a task that is typically easier to control for in RCTs.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

(Quasi-)experimental studies necessitate institutional collaboration, forming coalitions between police administration and academic staff with potentially conflicting interests and aims.

“Moreover, (quasi-)experimental studies necessitate institutional collaboration, forming coalitions between police administration and academic staff with potentially conflicting interests and aims.”

Probably not a substantial claim: may not pass S4 (based).

These secondary effects contribute to the credibility of and support for these interventions.

“These effects contribute to the credibility of and support for these interventions.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

This raises practical questions about the adaptability of intelligence within the context of police administration.

“This raises practical questions about the adaptability of intelligence within the context of police administration.”

May combine two claims.

The inclusion of human oversight in decision-making processes aligns with citizens' preferences, emphasizing the importance of balancing technological advancements with human involvement to ensure accountability and transparency in law enforcement practices.

“The inclusion of human oversight in decision-making processes also aligns with citizens' preferences, emphasizing the importance of balancing technological advancements with human involvement to ensure accountability and transparency in law enforcement practices.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Citizens' increased willingness to provide information when officers patrolled in predicted hot spots could potentially contribute to improving police-community relationships and enhancing the intelligence capabilities of police departments.

“This could potentially contribute to improving police-community relationships and enhancing the intelligence capabilities of police departments (Hunt et al., 2014).”

• cites Hunt et al., 2014Not checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

A third concern is the involvement of only one researcher in the selection and screening process, despite extensive discussion by researchers involved.

“A third concern is the involvement of only one researcher in the selection and screening process, despite the fact that the various stages of this scoping review were extensively discussed by the researchers involved.”

Classified as probably not a substantial claim. May combine two claims.

The review only aimed to include original studies; thus, it excluded studies in which follow-up analyses were conducted based on original (quasi-)experimental data.

“We only aimed to include original studies; thus, we excluded studies in which follow-up analyses were conducted based on original (quasi-)experimental data.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The review attempted to overcome the limitation of using three databases by manually searching literature reviews on distinct policing tactics and strategies.

“although we attempted to overcome this limitation by manually searching literature reviews on distinct policing tactics and strategies.”

Probably not a substantial claim: may not pass S3 (load-bearing).

To distinguish between weak and strong quasi-experimental designs, the review adopted an inclusion criterion from the Global Policing Database Protocol.

“To make a distinction between weak and strong quasi-experimental designs, we adopted this inclusion criterion from the inclusion criteria in the Global Policing Database Protocol (seeHigginson et al., 2015).”

• cites Higginson et al., 2015Not checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

The total survey error framework (TSE) is a methodological quality assessment standard that has been used to identify possible errors in survey data.

“The total survey error framework (TSE), for example, is a methodological quality assessment standard that has been used to identify possible errors in survey data (Groves et al., 2004).”

• cites Groves et al., 2004Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

TEF provides a framework for analyzing and potentially mitigating various error components within data measurement, representation, and analysis processes.

“TEF provides a framework for analyzing and potentially mitigating various error components within data measurement, representation, and analysis processes (for an overview of TEF see Amaya et al., 2020).”

• cites Amaya et al., 2020Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

The review only used three academic databases.

“Second, we only used three academic databases”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The three databases have consistently served as reliable sources for previous scoping reviews conducted within the same domain, which underlines their credibility, and they were expected to provide a substantial volume of high-quality, peer-reviewed articles.

“These databases have consistently served as reliable sources for previous scoping reviews conducted within the same domain, which underlines their credibility (e.g., Snaphaan & Hardyns 2021), and it was expected that they would provide a substantial volume of high-quality, peer-reviewed articles.”

• cites Snaphaan & Hardyns 2021Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.