Promoting Learning from Null or Negative Results in Prevention Science Trials
Open accessThe DOI matches its Crossref record; 64 of the 64 references it cites were found in a registry or an index.
Abstract
There can be a tendency for investigators to disregard or explain away null or negative results in prevention science trials. Examples include not publicizing findings, conducting spurious subgroup analyses, or attributing the outcome post hoc to real or perceived weaknesses in trial design or intervention implementation. This is unhelpful for several reasons, not least that it skews the evidence base, contributes to research "waste", undermines respect for science, and stifles creativity in intervention development. In this paper, we identify possible policy and practice responses when interventions have null (ineffective) or negative (harmful) results, and argue that these are influenced by: the intervention itself (e.g., stage of gestation, perceived importance); trial design, conduct, and results (e.g., pattern of null/negative effects, internal and external validity); context (e.g., wider evidence base, state of policy); and individual perspectives and interests (e.g., stake in the intervention). We advance several strategies to promote more informative null or negative effect trials and enable learning from such results, focusing on changes to culture, process, intervention design, trial design, and environment.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Prior work suggests that the success of prevention programs may depend on school culture, available resources, student engagement, and implementer characteristics [42,43].”
From Effectiveness of the Botvin High School LifeSkills Training Intervention on Adolescent Substance Use: A Cluster Randomized Trial (2026) · cites this paper as [42]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009) · Implementation Science · doi:10.1186/1748-5908-4-50 · Cited by 14,496 · cited 1×
“An implementation research framework (eg, Damschroder et al. 2009) and guidance on how to take account of context in intervention research (Craig et al. 2018) could usefully inform this exercise, shaping both intervention design and implementation strategy.”
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
the CONSORT Group, Schulz, K. F., Altman, D. G., & Moher, D. (2010) · BMC Medicine · doi:10.1186/1741-7015-8-18 · Cited by 13,371
Developing and evaluating complex interventions: the new Medical Research Council guidance
Craig, P., Dieppe, P., Macintyre, S., Michie, S., Nazareth, I., & Petticrew, M. (2008) · BMJ · doi:10.1136/bmj.a1655 · Cited by 10,995
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
Schulz, K. F., Altman, D. G., Moher, D., & for the CONSORT Group (2010) · BMJ · doi:10.1136/bmj.c332 · Cited by 9,329
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068 · cited 2×
“3 We readily acknowledge that the problems identified, such as failure to publish and conducting spurious sub-group analyses, are not unique to prevention science (Rosenthal 1979; Kasenda et al. 2014).”
Process evaluation of complex interventions: Medical Research Council guidance
Moore, G. F., Audrey, S., Barker, M., Bond, L., Bonell, C., Hardeman, W., Moore, L., O'Cathain, A., Tinati, T., Wight, D., et al. (2015) · BMJ · doi:10.1136/bmj.h1258 · Cited by 6,204 · cited 1×
“This is changing (see below, also Moore et al. 2015), but arguably having a sense of why something was ineffective or harmful makes it easier to accept the result and learn from the findings.”
Clinical Psychology: Science and Practice
(2021) · Clinical Psychology Science and Practice · doi:10.1111/(issn)1468-2850 · Cited by 3,672
Increasing value and reducing waste in research design, conduct, and analysis
Ioannidis, J. P. A., Greenland, S., Hlatky, M. A., Khoury, M. J., Macleod, M. R., Moher, D., Schulz, K. F., & Tibshirani, R. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62227-8 · Cited by 1,597
The need for a complex systems model of evidence for public health
Rutter, H., Savona, N., Glonti, K., Bibby, J., Cummins, S., Finegood, D. T., Greaves, F., Harper, L., Hawe, P., Moore, L., et al. (2017) · The Lancet · doi:10.1016/s0140-6736(17)31267-9 · Cited by 1,260 · cited 2×
“…a need for researchers, funders and developers to reflect on the fact that while simple behavioral interventions are easier to implement and to evaluate through trials, they are less likely to work in tackling complex social and health problems with complex causes (Ghate 2016; Rutter et al. 2017).”
Publication bias in clinical trials due to statistical significance or direction of trial results
Hopewell, S., Loudon, K., Clarke, M. J., Oxman, A. D., & Dickersin, K. (2009) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000006.pub3 · Cited by 898 · cited 1×
“The evidence for some of these is compelling; for example, several extensive reviews covering a range of disciplines demonstrate publication bias (Hopewell et al. 2009; Duyx et al. 2017; Chow and Eckholm 2018).”
Research works printed but not linked: 4
- Davies R Planning evaluability assessments: A synthesis of the literature with recommendations 2013 London Department for International Development
- FNP National Unit, & Dartington Service Design Lab FNP ADAPT: Using evidence, pragmatism and collaboration to change the Family Nurse Partnership programme in England 2020 London FNP National Unit
- Lipsey MW Howell JC Kelly MR Chapman G Carver D Improving the effectiveness of juvenile programs: A new perspective on evidence-based practice 2010 Washington, DC Georgetown University, Center for Juvenile Justice Reform
- Syed M Black box thinking: The surprising truth about success (and why some people never learn from their mistakes) 2015 London John Murray
Cited by
Beets, M. W., von Klinggraeff, L., Weaver, R. G., Armstrong, B., & Burkart, S. A. (2021) · Pilot and Feasibility Studies · doi:10.1186/s40814-021-00909-w · Cited by 130
Greene, D., Demeter, C., & Dolničar, S. (2023) · Journal of Travel Research · doi:10.1177/00472875231183701 · Cited by 97
Transparent, Open, and Reproducible Prevention Science
Grant, S., Wendt, K. E., Leadbeater, B. J. R., Supplee, L. H., Mayo‐Wilson, E., Gardner, F. E. M., & Bradshaw, C. P. (2022) · Prevention Science · doi:10.1007/s11121-022-01336-w · Cited by 46
Catlow, J., Bhardwaj-Gosling, R., Sharp, L., Rutter, M. D., & Sniehotta, F. F. (2021) · BMJ Quality & Safety · doi:10.1136/bmjqs-2021-013588 · Cited by 37
Allen, K., Meléndez‐Torres, G. J., Ford, T. J., Bonell, C. P., Finning, K., Fredlund, M., Gainsbury, A., & Berry, V. (2022) · PLoS ONE · doi:10.1371/journal.pone.0270894 · Cited by 33
Self-citation
Metadata
- Authors
- Axford, N., Berry, V., Lloyd, J., Hobbs, T., & Wyatt, K.
- Affiliations
- University of Plymouth (GB)
- University of Exeter (GB)
- Dartington Service Design Lab (GB)
- Corresponding author
- Berry, Vashti
- Venue
- Prevention Science
- Published
- 2020-08-04
- Volume
- 23
- Issue
- 5
- Pages
- 751-763
- ISSN
- 1389-4986 (ISSN-L)
- 1573-6695
- PMC
- PMC7398716
- OpenAlex
- W3041367671
- Cited by
- 34
- References
- 64
- Citations by year
- 2026: 10 · 2025: 6 · 2024: 1 · 2023: 6 · 2022: 7 · 2021: 4
- FWCI
- 4.44
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Prevention Science (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Open Research Exeter (University of Exeter) (publishedVersion, cc-by)
- PEARL (University of Plymouth) (submittedVersion, public-domain)
- Figshare (submittedVersion, cc-by)
- Funders
- University of Exeter
- Topics
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- negative results, research waste, subgroup analysis, trial design, intervention effectiveness, evidence base, research culture
- MeSH terms
- Negative Results, Humans, Research Design
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Promoting Learning from Null or Negative Results in Prevention Science Trials. https://app.cochat.ai/papers/10.1007/s11121-020-01140-4
- The paper
- Axford, N., Berry, V., Lloyd, J., Hobbs, T., & Wyatt, K. (2022). Promoting Learning from Null or Negative Results in Prevention Science Trials. Prevention Science. https://doi.org/10.1007/s11121-020-01140-4
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.