The DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Introducing EMMIE: an evidence rating scale to encourage mixed-method crime prevention synthesis reviews
Open accessAbstract
This paper describes the need for, and the development of, a coding system to distil the quality and coverage of systematic reviews of the evidence relating to crime prevention interventions. The starting point for the coding system concerns the evidence needs of policymakers and practitioners. The proposed coding scheme (EMMIE) builds on previous scales that have been developed to assess the probity, coverage and utility of evidence both in health and criminal justice. It also draws on the principles of realist synthesis and review. The proposed EMMIE scale identifies five dimensions to which systematic reviews intended to inform crime prevention should speak. These are the Effect of intervention, the identification of the causal Mechanism(s) through which interventions are intended to work, the factors that Moderate their impact, the articulation of practical Implementation issues, and the Economic costs of intervention. Systematic reviews of crime prevention, and the primary studies on which they are based, typically address the question of effect size, but are often silent on the other dimensions of EMMIE. This lacuna of knowledge is unhelpful to practitioners who want to know more than what might work to reduce crime. The EMMIE framework is intended to encourage the collection of primary data regarding these issues and the synthesis of such knowledge in future systematic reviews.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., Vandenbroucke, J. P., & Initiative, F. T. S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040296 · Cited by 23,020
von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., Vandenbroucke, J. P., & Initiative, F. T. S. (2007) · Annals of Internal Medicine · doi:10.7326/0003-4819-147-8-200710160-00010 · Cited by 17,796
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
the CONSORT Group, Schulz, K. F., Altman, D. G., & Moher, D. (2010) · BMC Medicine · doi:10.1186/1741-7015-8-18 · Cited by 13,371
A basic introduction to fixed-effect and random-effects models for meta-analysis
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2010) · Research Synthesis Methods · doi:10.1002/jrsm.12 · Cited by 6,793
Systematic Reviews in the Social Sciences: A Practical Guide
Petticrew, M., & Roberts, H. (2006) · doi:10.1002/9780470754887 · Cited by 3,586
What is ‘quality of evidence’ and why is it important to clinicians?
Guyatt, G. H., Oxman, A. D., Kunz, R., Vist, G. E., Falck-Ytter, Y., & Schünemann, H. J. (2008) · BMJ · doi:10.1136/bmj.39490.551019.be · Cited by 3,250
Robust variance estimation in meta‐regression with dependent effect size estimates
Hedges, L. V., Tipton, E., & Johnson, M. C. (2010) · Research Synthesis Methods · doi:10.1002/jrsm.5 · Cited by 2,086
Lipsey, M. W., & Wilson, D. B. (1993) · American Psychologist · doi:10.1037/0003-066x.48.12.1181 · Cited by 2,033 · cited 2×
“(Lipsey and Wilson 1993: 1201) Others have made similar suggestions (e.g., Cartwright and Hardie 2012; Rosenbaum 1988).”
BackgroundCONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
the CONSORT Group, Schulz, K. F., Altman, D. G., & Moher, D. (2010) · Trials · doi:10.1186/1745-6215-11-32 · Cited by 1,871
Pawson, R. (2006) · doi:10.4135/9781849209120 · Cited by 1,791
Research works printed but not linked: 30
- Adetugbo, K., & Williams, H. (2000). How well are randomized controlled trials reported in the dermatology literature? Archives of Dermatology, 136(3), 381–385. · DOI
- Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to meta-analysis. West Sussex: Wiley. · DOI · Paper page
- Bowers, K., Tompson, L., & Johnson, S. D. (2014). Implementing Information Science in Policing: Mapping the Evidence Base. Policing, advanced online access. · DOI
- Braga, A. A., & Weisburd, D. (2012). The effects of focused deterrence strategies on crime a systematic review and meta-analysis of the empirical evidence. Journal of Research in Crime and Delinquency, 49, 323–358. · DOI · Paper page
- Bryant, F. B., & Wortman, P. M. (1984). Methodological issues in the meta‐analysis of quasi‐experiments. New Directions for Program Evaluation, 24, 5–24. · DOI
- Campbell, D. T., & Stanley, J. C. (1963). Experimental and Quasi-Experimental Designs for Research. Boston: Houghton Mifflin.
- Cartwright, N., & Hardie, J. (2012). Evidence-Based Policy: A Practical Way of Doing it Better. Oxford: Oxford University Press. · DOI · Paper page
- Farrell, G., Bowers, K., & Johnson, S. D. (2004). Cost-benefit analysis for crime science: making cost-benefit analysis useful through a portfolio of outcomes. In M. Smith & N. Tilley (Eds.), Launching Crime Science. London: Willan.
- Gill, C. E. (2011). Missing links: how descriptive validity impacts the policy relevance of randomized controlled trials in criminology. Journal of Experimental Criminology, 7(3), 201–224. · DOI
- Hedges, L. V., & Vevea, J. L. (1996). Estimating effect size under publication bias: small sample properties and robustness of a random effects selection model. Journal of Educational and Behavioral Statistics, 21(4), 299–332. · doi:10.3102/10769986021004299 · Paper page
- Higgins J. P. T., & Green, S. (Eds.). (2011). Cochrane handbook for systematic reviews of interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration. Available from www.cochrane-handbook.org .
- Higgins, J., Altman, D. G., Gøtzsche, P. C., Jüni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K., Weeks, L., & Sterne, J. A. (2011). The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. British Medical Journal, 343, d5928. · DOI · Paper page
- Huffcutt, A. I., & Arthur, W. (1995). Development of a new outlier statistic for meta-analytic data. Journal of Applied Psychology, 80(2), 327–334. · doi:10.1037/0021-9010.80.2.327 · Paper page
- Johnson, S. D., Guerette, R. T., & Bowers, K. (2014) Crime displacement: what we know, what we don’t know, and what it means for crime reduction. Journal of Experimental Criminology, 10(4), 549–571. · DOI · Paper page
- Knutsson, J., & Tilley, N. (2009). Introduction. In J. Knutsson & N. Tilley (Eds.), Evaluating crime reduction initiatives (pp. 1–6). New Jersey: Prentice Hall.
- Lipsey, M., & Wilson, D. (2001). Practical Meta-Analysis. London: Sage.
- McDougal, C., Cohen, M. A., Perry, A., & Swaray, R. (2008). Benefit-Cost Analyses of Sentencing: A Systematic Review. Norway: Campbell Collaboration. · DOI
- Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G. (2009). Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Annals of Internal Medicine, 151(4), 264–269. · DOI · Paper page
- Ogrinc, G., Mooney, S. E., Estrada, C., Foster, T., Goldmann, D., Hall, L. W., Huizinga, M. M., Liu, S. K., Mills, P., Neily, J., Nelson, W., Pronovost, P. J., Provost, L., Rubenstein, L. V., Speroff, T., Splaine, M., Thomson, R., Tomolo, A. M., & Watts, B. (2008). The SQUIRE (standards for QUality improvement reporting excellence) guidelines for quality improvement reporting: explanation and elaboration. Quality and Safety in Health Care, 17(Suppl 1), i13–i32. · DOI · Paper page
- Pawson, R., & Tilley, N. (1997). Realistic Evaluation. London: Sage.
- Perry, A. E., Weisburd, D., & Hewitt, C. (2010). Are criminologists describing randomized controlled trials in ways that allow us to assess them? Findings from a sample of crime and justice trials. Journal of Experimental Criminology, 6(3), 245–262. · DOI
- Rosenbaum, D. (1988). Community crime prevention: a review and synthesis of the literature. Justice Quarterly, 5(3), 323–395. · DOI · Paper page
- Shea, B. J., Grimshaw, J. M., Wells, G. A., Boers, M., Andersson, N., Hamel, C., Porter, A. C., Tugwell, P., Moher, D., & Bouter, L. M. (2007). Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Medical Research Methodology, 7(10), 1–7.
- Sherman, L., Gottfredson, D., MacKenzie, D., Eck, J., Reuter, P., & Bushway, S. (1997). Preventing Crime: What Works, What Doesn’t, What’s Promising. Washington DC: US Department of Justice Office of Justice Programs.
- Sidebottom, A., & Tilley, N. (2012). Further improving reporting in crime and justice: an addendum to Perry, Weisburd and Hewitt (2010). Journal of Experimental Criminology, 8(1), 49–69. · DOI
- Stock, W. A., Okun, M. A., Haring, M. J., Miller, W., Kinney, C., & Ceurvorst, R. W. (1982). Rigor in data synthesis: a case study of reliability in meta-analysis. Educational Researcher, 11(6), 10–14. 20. · DOI
- Tilley, N. (1996). Demonstration, exemplification, duplication and replication in evaluation research. Evaluation: The International Journal of Theory, Research and Practice, 2(1), 35–50. · DOI
- Weisburd, D. (2010). Justifying the Use of Non-experimental methods and disqualifying the use of randomized controlled trials: challenging folklore in evaluation research in crime and justice. Journal of Experimental Criminology, 6(2), 209–227. · DOI
- Weisburd, D., Hinkle, J., Braga, A., and Wooditch, A. (2015). Understanding the Mechanisms Underlying Broken Windows Policing: The Need for Evaluation Evidence, Journal of Research in Crime and Delinquency (in press). · DOI
- Wilson, J. Q., & Kelling, G. L. (1982). Broken windows: the police and neighborhood safety. Atlantic Monthly, 211, 29–38.
Cited by
CCTV surveillance for crime prevention: A 40‐year systematic review with meta‐analysis
Piza, E. L., Welsh, B. C., Farrington, D. P., & Thomas, A. L. (2019) · Criminology & Public Policy · doi:10.1111/1745-9133.12419 · Cited by 305
The 40-year debate: a meta-review on what works for juvenile offenders
Pappas, L. N., & Dent, A. L. (2023) · Journal of Experimental Criminology · doi:10.1007/s11292-021-09472-z · Cited by 72
Rivas, C., Vigurs, C., Cameron, J., & Yeo, L. (2019) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013135.pub2 · Cited by 65
Police stops to reduce crime: A systematic review and meta-analysis
Petersen, K., Weisburd, D., Fay, S., Eggins, E., & Mazerolle, L. (2023) · Campbell Systematic Reviews: Better Evidence for a Better World · doi:10.1002/cl2.1302 · Cited by 63
Santos, R. B., & Santos, R. G. (2016) · Journal of Experimental Criminology · doi:10.1007/s11292-016-9268-9 · Cited by 22
Metadata
- Authors
- Johnson, S. D., Tilley, N., & Bowers, K. J.
- Venue
- Journal of Experimental Criminology
- Published
- 2015-07-01
- Volume
- 11
- Issue
- 3
- Pages
- 459-473
- ISSN
- 1572-8315 (ISSN-L)
- 1573-3750
- OpenAlex
- W376915030
- Cited by
- 166
- References
- 14
- FWCI
- 10.76
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Introducing EMMIE: an evidence rating scale to encourage mixed-method crime prevention synthesis reviews. https://app.cochat.ai/papers/10.1007/s11292-015-9238-7
- The paper
- Johnson, S. D., Tilley, N., & Bowers, K. J. (2015). Introducing EMMIE: an evidence rating scale to encourage mixed-method crime prevention synthesis reviews. Journal of Experimental Criminology. https://doi.org/10.1007/s11292-015-9238-7
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.