Design Sensitivity in Criminal Justice Experiments
Abstract
Interest in randomized experiments with criminal justice subjects has grown, in recognition that experiments are much better suited for identifying and isolating program effects than are quasi-experimental or nonexperimental research designs. Relatively little attention, however, has been paid to methodological issues. Using the statistical concept of power-the likelihood that a test will lead to the rejection of a hypothesis of no effect, a survey examines the design sensitivity of experiments on sanctions. Contrary to conventional wisdom advocating large sample designs, little relationship is found in practice between sample size and statistical power. Difficulty in maintaining the integrity of treatments and the homogeneity of samples or treatments employed offsets the design advantages of larger investigations.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Studies with very small sample sizes may exhibit higher statistical power compared to those with very large sample sizes.
“In this regard, it has been found that studies with very small sample sizes may exhibit higher statistical power compared to those with very large sample sizes (Weisburd et al., 1993).”
From ‘Led by Intelligence': A Scoping Review on the Experimental Evaluation of Intelligence-Led Policing (2024) · cites this paper as Weisburd et al., 1993
Weisburd's paradox highlights a discrepancy between the expectation that increasing sample size will enhance statistical power and the reality that it often results in decreased statistical power.
“However, from an empirical point of view, this premise must be balanced against Weisburd's paradox (Groff et al., 2015;Weisburd et al., 1993), which highlights the discrepancy between the expectation that increasing the sample size will enhance statistical power, and the reality that it often results in decreased statistical power.”
From ‘Led by Intelligence': A Scoping Review on the Experimental Evaluation of Intelligence-Led Policing (2024) · cites this paper as Weisburd et al., 1993
Cites
Do studies of statistical power have an effect on the power of studies?
Sedlmeier, P., & Gigerenzer, G. (1989) · Psychological Bulletin · doi:10.1037/0033-2909.105.2.309 · Cited by 748 · see citing works
Relationship between psychotherapy with institutionalized boys and subsequent community adjustment
Persons, R. W. (1967) · Journal of Consulting Psychology · doi:10.1037/h0024443 · Cited by 26 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Effectiveness of school-based programs to reduce bullying: a systematic and meta-analytic review
Ttofi, M. M., & Farrington, D. P. (2011) · Journal of Experimental Criminology · doi:10.1007/s11292-010-9109-1 · Cited by 1,624 · see citing works
PROBLEM‐ORIENTED POLICING IN VIOLENT CRIME PLACES: A RANDOMIZED CONTROLLED EXPERIMENT*
BRAGA, A. A., WEISBURD, D. L., WARING, E. J., MAZEROLLE, L. G., SPELMAN, W., & GAJEWSKI, F. (1999) · Criminology · doi:10.1111/j.1745-9125.1999.tb00496.x · Cited by 583 · see citing works
Self-citationOrganizational stressors and police performance
Shane, J. M. (2010) · Journal of Criminal Justice · doi:10.1016/j.jcrimjus.2010.05.008 · Cited by 389 · see citing works
The relevance of systematic reviews to educational policy and practice
Davies, P. (2000) · Oxford Review of Education · doi:10.1080/713688543 · Cited by 286 · see citing works
Rigorous Large-Scale Educational RCTs Are Often Uninformative: Should We Be Concerned?
Lortie-Forgues, H., & Inglis, M. (2019) · Educational Researcher · doi:10.3102/0013189x19832850 · Cited by 234 · see citing works
Publication bias as a threat to the validity of meta-analytic results
Rothstein, H. R. (2008) · Journal of Experimental Criminology · doi:10.1007/s11292-007-9046-9 · Cited by 216 · see citing works
Sherman, L. W. (2007) · Journal of Experimental Criminology · doi:10.1007/s11292-007-9044-y · Cited by 183 · see citing works
The Effects of Problem-Oriented Policing on Crime and Disorder <sup/>
Weisburd, D., Telep, C. W., Hinkle, J. C., & Eck, J. E. (2008) · Campbell Systematic Reviews: Better Evidence for a Better World · doi:10.4073/csr.2008.14 · Cited by 100 · see citing works
Self-citationRisk‐Focused Policing at Places: An Experimental Evaluation
Weisburd, D., Morris, N. A., & Ready, J. (2008) · Justice Quarterly · doi:10.1080/07418820801954647 · Cited by 47 · see citing works
Self-citationAriel, B., Sutherland, A., & Sherman, L. W. (2019) · Journal of Experimental Criminology · doi:10.1007/s11292-018-9344-4 · Cited by 38 · see citing works
‘Led by Intelligence': A Scoping Review on the Experimental Evaluation of Intelligence-Led Policing
Khalfa, R., & Hardyns, W. (2024) · Evaluation Review · doi:10.1177/0193841x231204588 · Cited by 12 · see citing works
- Authors
- Weisburd, D., Petrosino, A., & Mason, G.
- Venue
- Crime and Justice
- Publisher
- University of Chicago Press
- Published
- 1993-01
- Volume
- 17
- Pages
- 337-379
- ISSN
- 0192-3234 (ISSN-L)
- 2153-0416
- OpenAlex
- W2012992066
- Cited by
- 124
- References
- 2
- Citations by year
- 2026: 3 · 2025: 5 · 2023: 5 · 2022: 3 · 2021: 7 · 2020: 5 · 2019: 6 · 2018: 7 · 2017: 5 · 2016: 2 · 2015: 2 · 2014: 6 · 2013: 7 · 2012: 4
- FWCI
- 1.52
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Crime and Justice (publishedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Social Sciences › Sociology and Political Science › Criminal Justice and Corrections Analysis
- Social Sciences › Decision Sciences › Management Science and Operations Research › Optimal Experimental Design Methods
- Keywords
- design sensitivity analysis, randomized experiments, statistical power, sample size, sanctions, implementation fidelity, sample homogeneity, program evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review