Evidence comes by replication, but needs differentiation: the reproducibility issue in science and its relevance for criminology
Open accessThe DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.
Abstract
Recent publications in Nature , Science , and other journals raised concerns about the reproducibility of empirical findings in psychology and other scientific disciplines. This article summarizes some of these arguments and results that led to discussions about a “replication crisis” in research. In criminology, there is not yet a similar discussion, although the need for more replications has been emphasized in the past. The present article addresses this topic with special consideration of program evaluations in early developmental crime prevention and offender treatment. In both fields, there has been substantial progress in research and practice. Most systematic reviews showed mean positive effects; however, nearly all of them demonstrated very heterogeneous findings that could not be attributed to the content of programs. This does not allow simple recommendations of “what works” for policy-making and practice. In addition, there is a serious lack of long-term follow-ups and independent evaluations. The article shows remarkable similarity of the findings and problems in both fields of intervention. Problems of reproducibility prove to be highly relevant for criminology, although there is no need for using the term “crisis”. The article proposes various strategies that can enhance the reproducibility of findings, i.e., more systematic investigation of those differentiated conditions under which interventions are most effective. An integrative model of relevant characteristics is briefly presented. It refers to factors of the programs, contexts, participants, and evaluation methods. Confirmatory meta-analyses can play an important role on the path toward more differentiated and replicated knowledge
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515
1,500 scientists lift the lid on reproducibility
Baker, M. (2016) · Nature · doi:10.1038/533452a · Cited by 4,209
Studies in the Logic of Explanation
Hempel, C. G., & Oppenheim, P. (1948) · Philosophy of Science · doi:10.1086/286983 · Cited by 3,174 · cited 1×
“In philosophy of science, this is known as Bexhaustion^, that is, further conditions are added to the deductivenomological model of explanation (Hempel and Oppenheim 1948).”
BackgroundThe Psychology of Criminal Conduct
Bonta, J., & Andrews, D. A. (2016) · doi:10.4324/9781315677187 · Cited by 2,107
DOES CORRECTIONAL TREATMENT WORK? A CLINICALLY RELEVANT AND PSYCHOLOGICALLY INFORMED META‐ANALYSIS *
ANDREWS, D. A., ZINGER, I., HOGE, R. D., BONTA, J., GENDREAU, P., & CULLEN, F. T. (1990) · Criminology · doi:10.1111/j.1745-9125.1990.tb01330.x · Cited by 2,084
Effectiveness of school-based programs to reduce bullying: a systematic and meta-analytic review
Ttofi, M. M., & Farrington, D. P. (2011) · Journal of Experimental Criminology · doi:10.1007/s11292-010-9109-1 · Cited by 1,624
Contradicted and Initially Stronger Effects in Highly Cited Clinical Research
Ioannidis, J. P. A. (2005) · JAMA · doi:10.1001/jama.294.2.218 · Cited by 1,465
Standards of Evidence: Criteria for Efficacy, Effectiveness and Dissemination
Flay, B. R., Biglan, A., Boruch, R. F., Castro, F. G., Gottfredson, D., Kellam, S., Mościcki, E. K., Schinke, S., Valentine, J. C., & Ji, P. (2005) · Prevention Science · doi:10.1007/s11121-005-5553-y · Cited by 1,338 · cited 1×
“Flay et al. (2005), Valentine et al. (2011), Gottfredson et al. (2015), and others addressed standards of evidence that should reduce replication problems in prevention science.”
BackgroundCore Implementation Components
Fixsen, D. L., Blase, K. A., Naoom, S. F., & Wallace, F. (2009) · Research on Social Work Practice · doi:10.1177/1049731509335549 · Cited by 1,103 · cited 1×
“If one takes also principles of implementation science (Fixsen et al. 2009) into account, there could be more than 30 factors that are relevant for outcome.”
BackgroundLipsey, M. W. (2009) · Victims & Offenders · doi:10.1080/15564880802612573 · Cited by 962 · cited 1×
“There is also a need for more data on the impact of combinations of programs or of specific program elements or modules (e.g., Hawkins et al. 2008; Lipsey 2009).”
Background
Cited by
Hinkle, J. C., Weisburd, D., Telep, C. W., & Petersen, K. (2020) · Campbell Systematic Reviews: Better Evidence for a Better World · doi:10.1002/cl2.1089 · Cited by 194
Depression, anxiety and delinquency: Results from the Pittsburgh Youth Study
Jolliffe, D., Farrington, D. P., Brunton-Smith, I., Loeber, R., Ahonen, L., & Palacios, A. P. (2019) · Journal of Criminal Justice · doi:10.1016/j.jcrimjus.2018.08.004 · Cited by 38
“Like all research, this study should be subject to replication to confirm the findings (e.g., Losel, 2018).”
ResultFerguson, C. J., & Smith, S. (2024) · Aggression and Violent Behavior · doi:10.1016/j.avb.2023.101905 · Cited by 14
Hein, S., Weeland, J., Square, A., Haeffel, G. J., Chapman, J., Macomber, D., Lee, M., Foley Geib, C., & Grigorenko, E. L. (2020) · International Journal of Law and Psychiatry · doi:10.1016/j.ijlp.2020.101626 · Cited by 12
Metadata
- Authors
- Lösel, F.
- Venue
- Journal of Experimental Criminology
- Published
- 2017-08-18
- Volume
- 14
- Issue
- 3
- Pages
- 257-278
- ISSN
- 1572-8315 (ISSN-L)
- 1573-3750
- OpenAlex
- W2745855877
- Cited by
- 76
- References
- 30
- FWCI
- 7.98
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Evidence comes by replication, but needs differentiation: the reproducibility issue in science and its relevance for criminology. https://app.cochat.ai/papers/10.1007/s11292-017-9297-z
- The paper
- Lösel, F. (2018). Evidence comes by replication, but needs differentiation: the reproducibility issue in science and its relevance for criminology. Journal of Experimental Criminology. https://doi.org/10.1007/s11292-017-9297-z
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.