When is a bologna sandwich better than sex? A defense of small-n case study evaluations

“Eck, J.E. (2006). When is a bologna sandwich better than sex? A defense of small-n case study evaluations.”

“Support for the theory gives one greater reason for us to believe other interventions will work with other units, other observations, and, in other settings, to produce similar results ( Eck 2002, 2006; Townsley and Birks, in this volume).”

“Although it is true that criminology researchers like Eck (2006), Farrington (2006), and Harrell (2006) have recently stressed the importance of basing interventions on well-defined theories, and of conducting studies that are theory based and that document the causal chain(s) that produces the…”

“…rather than policy interventions, we are persuaded that the proper use of significance tests in evaluation research calls for a more judicious consideration of a levels and the relative risks of the two different inferential errors (Leahey, 2005; Schneider & Darcy, 1984; also see, Eck, 2006).”

“In total, the after-action reports captured daily counts of seven unique officer actions: arrest, field interrogations, quality of life summonses, citizen contacts, business checks, bus checks, and taxi inspections. These activities are different in nature, varying in their degree of severity and emphasis on tradition…”

“When randomized control trials are not possible, as is often the case in criminal justice research, quasi-experimental designs are considered the optimal evaluation method (Eck, 2006, 2017; Ratcliffe, 2019).”

“While a non-experimental study can never be as rigorous as a randomized experiment, evaluating the effects of a police interventions as they are carried out in regular practice has value for generalizing the effect (Eck 2006).”

“Eck (2006) suggested that stronger program theory would be an important aid to interpreting the many Bsmall n^ studies he would like to see in criminology evaluations.”

“It is also worth reiterating that we are drawing conclusions based on a case study, though case studies are still preferred over pure speculation (Eck, 2006).”

“Given that each case produced a large amount of casework, this is, as is the norm in crime prevention evaluations, a modest sample (Eck, 2006).”

“…into a “small” RCT, but it was also a practical reality and achieving statistical significance in any subsequent analyses was of little importance to HCSO, so this limitation was accepted by the research team (see Eck, 2006 for a discussion of design tradeoff applied studies often must undertake).”

“…been suggested that systematic reviews—given the inclusion criterion of an experimental or quasi-experimental design—can sacrifice generalizability (eck 2006; Knutsson and Tompson 2017, ch. 4), stripping away valuable knowledge and making it more difficult for practitioners to apply the findings…”

“Additionally, Eck (2006) highlights the intrinsic value of small-n case studies, pointing out that dismissing data from these studies limits our understanding of smaller communities, which may have unique dynamics compared to larger urban areas.”

“According to Eck (2006), there are two objectives in evaluating intervention programmes.”

“Scholars have been making this argument over the last two decades (Eck, 2006, 2017; Greene, 2018; Huey & Mitchell, 2016; Huey et al., 2021; Johnson et al., 2015; Piza & Welsh, 2022; Salvemini et al., 2015; Sherman, 2015; Sparrow, 2011; Tompson et al., 2021; Weisburd et al., 2017).”

“While we were able to conduct focus groups with a small sample, small samples have value (Eck, 2006).”

“Both Lauritsen (2006) and Eck (2006) recommended case studies or multiple nonrandomized evaluations with small samples.”

“This includes issues associated with evaluation data collection processes and the quality of the data itself (for example, the absence of baseline data, lack of appropriate and natural comparison group, small and nonrepresentative samples and missing data; Bamberger, Rugh, Church & Fort, 2004; Eck, 2006).”