Reporting Quality of Social and Psychological Intervention Trials: A Systematic Review of Reporting Guidelines and Trial Publications
Open accessThe DOI matches its Crossref record; 62 of the 62 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Previous reviews show that reporting guidelines have improved the quality of trial reports in medicine, yet existing guidelines may not be fully suited for social and psychological intervention trials. OBJECTIVE/DESIGN: We conducted a two-part study that reviewed (1) reporting guidelines for and (2) the reporting quality of social and psychological intervention trials. DATA SOURCES: (1) To identify reporting guidelines, we systematically searched multiple electronic databases and reporting guideline registries. (2) To identify trials, we hand-searched 40 journals with the 10 highest impact factors in clinical psychology, criminology, education, and social work. ELIGIBILITY: (1) Reporting guidelines consisted of articles introducing a checklist of reporting standards relevant to social and psychological intervention trials. (2) Trials reported randomised experiments of complex interventions with psychological, social, or health outcomes. RESULTS: (1) We identified 19 reporting guidelines that yielded 147 reporting standards relevant to social and psychological interventions. Social and behavioural science guidelines included 89 standards not found in CONSORT guidelines. However, CONSORT guidelines used more recommended techniques for development and dissemination compared to other guidelines. (2) Our review of trials (n = 239) revealed that many standards were poorly reported, such as identification as a randomised trial in titles (20% reported the information) and abstracts (55%); information about blinding (15%), sequence generation (23%), and allocation concealment (17%); and details about actual delivery of experimental (43%) and control interventions (34%), participant uptake (25%), and service environment (28%). Only 11 of 40 journals referenced reporting guidelines in "Instructions to Authors." CONCLUSION: Existing reporting guidelines have important limitations in content, development, and/or dissemination. Important details are routinely missing from trial publications; most leading journals in social and behavioural sciences do not ask authors to follow reporting standards. Findings demonstrate a need to develop a CONSORT extension with updated standards for social and psychological intervention trials.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Unfortunately, reporting of interventions tested and the methods used to evaluate them is often suboptimal, even in our best journals (Grant et al. 2013), and this often makes it difficult to judge the quality of the evidence from potentially important prevention trials.”
From Standards of Evidence for Efficacy, Effectiveness, and Scale-up Research in Prevention Science: Next Generation (2015) · cites this paper as Grant et al. 2013
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Assessing the quality of reports of randomized clinical trials: Is blinding necessary?
Jadad, A. R., Moore, R. A., Carroll, D., Jenkinson, C., Reynolds, D. J. M., Gavaghan, D. J., & McQuay, H. J. (1996) · Controlled Clinical Trials · doi:10.1016/0197-2456(95)00134-4 · Cited by 17,582 · cited 1×
“Previous Guidance Through the literature search (see Figure 1), we identified 19 unique, eligible reporting guidelines and reporting quality assessment tools (see Table 1) developed between 1980 and 2010 (median 2004).[6,9,26,27,28,29,30,31,32,33,34,35,36,37,38,39, 40,41,42] Six were developed by the CONSORT Group for…”
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
the CONSORT Group, Schulz, K. F., Altman, D. G., & Moher, D. (2010) · BMC Medicine · doi:10.1186/1741-7015-8-18 · Cited by 13,371
Zhang, Q., Wang, C., Zheng, M., Li, Y., Li, J., Zhang, L., Shang, X., & Yan, C. (2015) · Cerebrovascular Diseases · doi:10.1159/000369778 · Cited by 11,533
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
Schulz, K. F., Altman, D. G., Moher, D., & for the CONSORT Group (2010) · BMJ · doi:10.1136/bmj.c332 · Cited by 9,329
Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gotzsche, P. C., Devereaux, P. J., Elbourne, D., Egger, M., & Altman, D. G. (2010) · BMJ · doi:10.1136/bmj.c869 · Cited by 8,883
(2005) · PubMed · Cited by 7,011
Durlak, J. A., & DuPre, E. P. (2008) · American Journal of Community Psychology · doi:10.1007/s10464-008-9165-0 · Cited by 5,304 · cited 1×
“[51] Details of trials not related to internal and external validity are also important, such as discussing other relevant research when interpreting trial findings,[52] referencing other reports about the trial that may have a different focus (e.”
BackgroundDong, Y., Hao, L., Shi, Z., Wang, G., Zhang, Z., & Han, C. (2013) · Urologia Internationalis · doi:10.1159/000351405 · Cited by 4,540
(2005) · Cited by 3,748
Consort 2010 statement: extension to cluster randomised trials
Campbell, M. K., Piaggio, G., Elbourne, D. R., Altman, D. G., & for the CONSORT Group (2012) · BMJ · doi:10.1136/bmj.e5661 · Cited by 3,510
Research works printed but not linked: 1
- L Moore, 2002, Research design for the rigorous evaluation of complex educational interventions: Lessons from health services research, Building Research Capacity, 1, 4
Cited by
Process evaluation of complex interventions: Medical Research Council guidance
Moore, G. F., Audrey, S., Barker, M., Bond, L., Bonell, C., Hardeman, W., Moore, L., O'Cathain, A., Tinati, T., Wight, D., et al. (2015) · BMJ · doi:10.1136/bmj.h1258 · Cited by 6,204
Mayo-Wilson, E., Dias, S., Mavranezouli, I., Kew, K., Clark, D. M., Ades, A. E., & Pilling, S. (2014) · The Lancet Psychiatry · doi:10.1016/s2215-0366(14)70329-3 · Cited by 680
Self-citationGottfredson, D. C., Cook, T. D., Gardner, F. E. M., Gorman-Smith, D., Howe, G. W., Sandler, I. N., & Zafft, K. M. (2015) · Prevention Science · doi:10.1007/s11121-015-0555-x · Cited by 627
“Unfortunately, reporting of interventions tested and the methods used to evaluate them is often suboptimal, even in our best journals (Grant et al. 2013), and this often makes it difficult to judge the quality of the evidence from potentially”
BackgroundMethodInfluentialLloyd-Evans, B., Mayo-Wilson, E., Harrison, B., Istead, H., Brown, E., Pilling, S., Johnson, S., & Kendall, T. (2014) · BMC Psychiatry · doi:10.1186/1471-244x-14-39 · Cited by 558
Self-citationA systematic review and meta-analysis of psychological interventions to improve mental wellbeing
van Agteren, J., Iasiello, M., Lo, L., Bartholomaeus, J., Kopsaftis, Z., Carey, M., & Kyrios, M. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01093-w · Cited by 510
Increasing value and reducing waste in biomedical research: who's listening?
Moher, D., Glasziou, P., Chalmers, I., Nasser, M., Bossuyt, P. M. M., Korevaar, D. A., Graham, I. D., Ravaud, P., & Boutron, I. (2016) · The Lancet · doi:10.1016/s0140-6736(15)00307-4 · Cited by 462
Hollands, G. J., Shemilt, I., Marteau, T. M., Jebb, S. A., Lewis, H. B., Wei, Y., Higgins, J. P. T., & Ogilvie, D. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd011045.pub2 · Cited by 327
O. B. O. T. C.-S. G., Grant, S., Mayo-Wilson, E., Montgomery, P., Macdonald, G., Michie, S., Hopewell, S., & Moher, D. (2018) · Trials · doi:10.1186/s13063-018-2735-z · Cited by 348
“In summary, we first conducted systematic reviews of existing guidance and quality of trial reporting [18].”
MethodSelf-citationShortt, M., Tilak, S., Kuznetcova, I., Martens, B., & Akinkuolie, B. (2023) · Computer Assisted Language Learning · doi:10.1080/09588221.2021.1933540 · Cited by 334
Szaszi, B., Palinkas, A., Palfi, B., Szollosi, A., & Aczel, B. (2018) · Journal of Behavioral Decision Making · doi:10.1002/bdm.2035 · Cited by 322
Beyond treatment – Psychosocial and behavioural issues in cancer survivorship research and practice
Aaronson, N. K., Mattioli, V., Minton, O., Weis, J., Johansen, C., Dalton, S. O., Verdonck-de Leeuw, I. M., Stein, K. D., Alfano, C. M., Mehnert, A., et al. (2014) · European Journal of Cancer Supplements · doi:10.1016/j.ejcsup.2014.03.005 · Cited by 234
“The observed discrepancies between these reviews may reflect varying quality of reporting in the various RCT’s, which makes it challenging to compare results across studies [35].”
Dignath, C., Rimm-Kaufman, S., van Ewijk, R., & Kunter, M. (2022) · Educational Psychology Review · doi:10.1007/s10648-022-09695-0 · Cited by 219
“While a randomized assignment of teachers to training conditions might not always be feasible, at least a control group design with pretest and posttest could facilitate verification of the effects of natural development (Gersten et al., 2005; Grant et al., 2013).”
BackgroundJonsson, U., Alaie, I., Parling, T., & Arnberg, F. K. (2014) · Contemporary Clinical Trials · doi:10.1016/j.cct.2014.02.005 · Cited by 175
“Unfortunately, the usefulness of many trials within this research field is limited by an unsatisfactory quality of reporting [1, 2].”
BackgroundBallard, M., & Montgomery, P. (2017) · Research Synthesis Methods · doi:10.1002/jrsm.1229 · Cited by 136
“…important methodological issues not mentioned in the literature (e.g. aggregation bias) (Pieper et al., 2014b), (2) refine the conditions and cutoffs, and (3) ensure its relevance to and uptake by potential authors of overviews and those who commission them (Grant et al., 2013; Moher et al., 2010).”
BackgroundMethodSelf-citationMayo-Wilson, E., Grant, S., Burton, J., Parsons, A., Underhill, K., & Montgomery, P. (2014) · PLoS ONE · doi:10.1371/journal.pone.0089257 · Cited by 163
“exacerbate difficulties in systematic reviews of complex interventions [203,204], yet under-reporting of intervention content and delivery is common [205–207].”
BackgroundSelf-citationProtocol for CONSORT-SPI: an extension for social and psychological interventions
Montgomery, P., Grant, S., Hopewell, S., Macdonald, G., Moher, D., Michie, S., & Mayo-Wilson, E. (2013) · Implementation Science · doi:10.1186/1748-5908-8-99 · Cited by 114
Self-citationMayo-Wilson, E., & Montgomery, P. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd005330.pub4 · Cited by 112
Self-citationAguilera, A., Figueroa, C. A., Hernandez-Ramos, R., Sarkar, U., Cemballi, A. G., Pathak, L. E., Miramontes, J., Yom‐Tov, E., Chakraborty, B., Yan, X., et al. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-034723 · Cited by 110
Cybulski, L., Mayo-Wilson, E., & Grant, S. (2016) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000115 · Cited by 90
Self-citationDeveloping a reporting guideline for social and psychological intervention trials
Mayo-Wilson, E., Grant, S., Hopewell, S., Macdonald, G., Moher, D., & Montgomery, P. (2013) · Trials · doi:10.1186/1745-6215-14-242 · Cited by 78
Self-citationKamada, M., Kitayuguchi, J., Abe, T., Taguri, M., Inoue, S., Ishikawa, Y., Bauman, A. E., Lee, I.-M. M., Miyachi, M., & Kawachi, I. I. (2017) · International Journal of Epidemiology · doi:10.1093/ije/dyx248 · Cited by 80
“RCTs of social interventions are increasingly popular in public health and other disciplines such as education and economics; however, these broad-reach trials typically face methodological challenges due to their complex nature and real-world research environments that are difficult to control.(11) Accumulation of em…”
BackgroundLeske, S., Harris, M. G., Charlson, F. J., Ferrari, A. J., Baxter, A. J., Logan, J. M., Toombs, M., & Whiteford, H. (2016) · Australian & New Zealand Journal of Psychiatry · doi:10.1177/0004867416662150 · Cited by 65
“This situation could potentially be improved by the implementation of intervention reporting standards or guidelines (Grant et al., 2013; Mayo-Wilson et al., 2013).”
BackgroundMeta-Analysis of Complex Interventions
Tanner-Smith, E. E., & Grant, S. (2018) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-040617-014112 · Cited by 62
Self-citationSystematic review and meta-analyses of studies analysing instructions to authors from 1987 to 2017.
Malički, M., Jerončić, A., Aalbersberg, I. J., Bouter, L., & ter Riet, G. (2021) · Nature Communications · doi:10.1038/s41467-021-26027-y · Cited by 53
Dignath, C., van Ewijk, R. J. G., Perels, F., & Fabriz, S. (2023) · Educational Psychology Review · doi:10.1007/s10648-023-09718-4 · Cited by 102
“Although random assignment of students to intervention conditions is not always possible, at least a quasi-experimental control group design with pre- and posttest allows testing the effects of natural development (Grant et al., 2013).”
BackgroundCommon Methodological Problems in Randomized Controlled Trials of Preventive Interventions
Steeger, C. M., Buckley, P. R., Pampel, F. C., Gust, C. J., & Hill, K. G. (2021) · Prevention Science · doi:10.1007/s11121-021-01263-2 · Cited by 61
Intervention Synthesis: A Missing Link between a Systematic Review and Practical Treatment(s)
Glasziou, P. P., Chalmers, I., Green, S., & Michie, S. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001690 · Cited by 55
“The most problematic is inadequate description of interventions in the reports of primary studies [23].”
BackgroundJonsson, U., Choque Olsson, N., & Bölte, S. (2016) · Autism · doi:10.1177/1362361315583817 · Cited by 51
“Items of relevance for external validity were selected from a review of reporting guidelines by Grant et al. (2013) and completed by additional items of clinical significance derived by face validity.”
BackgroundMethodStrengthening causal inference from randomised controlled trials of complex interventions
Leroy, J. L., Frongillo, E. A., Kase, B. E., Alonso, S., Chen, M., Dohoo, I. R., Huybregts, L., Kadiyala, S., & Saville, N. M. (2022) · BMJ Global Health · doi:10.1136/bmjgh-2022-008597 · Cited by 61
Mayo‐Wilson, E., Grant, S., Supplee, L. H., Kianersi, S., Amin, A., DeHaven, A., & Mellor, D. T. (2021) · Research Integrity and Peer Review · doi:10.1186/s41073-021-00112-8 · Cited by 57
“Based on previous meta-research concerning transparency and openness in social and behavioral intervention science [22, 23], we hypothesize that most journal policies will be rated Level 0 or Level 1 for most of the ten TOP Factor standards.”
MethodSelf-citationBartlett, L., Lovell, P., Otahal, P., & Sanderson, K. (2017) · Mindfulness · doi:10.1007/s12671-016-0643-4 · Cited by 42
Alexander, E. S., O’Connor, M., Rees, C. S., & Halkett, G. K. B. (2019) · Patient Education and Counseling · doi:10.1016/j.pec.2019.05.001 · Cited by 43
Kohrs, F. E., Auer, S., Bannach-Brown, A., Fiedler, S., Haven, T., Heise, V., Holman, C., Azevedo, F., Bernard, R., Bleier, A., et al. (2023) · eLife · doi:10.7554/elife.89736 · Cited by 34
“Despite these developments, reproducible research and open science practices remain uncommon in many fields ( Blanco et al., 2019 ; Grant et al., 2013 ; Hardwicke et al., 2022 ; Hardwicke et al., 2020 ; Page and Moher, 2017 ).”
BackgroundMovsisyan, A., Dennis, J. A., Rehfuess, E. A., Grant, S., & Montgomery, P. (2018) · Research Synthesis Methods · doi:10.1002/jrsm.1290 · Cited by 49
Self-citationOpen science practices in clinical psychology journals: An audit study.
Nutu, D., Gentili, C., Naudet, F., & Cristea, I. A. (2019) · Journal of Abnormal Psychology · doi:10.1037/abn0000414 · Cited by 34
Transparent, Open, and Reproducible Prevention Science
Grant, S., Wendt, K. E., Leadbeater, B. J. R., Supplee, L. H., Mayo‐Wilson, E., Gardner, F. E. M., & Bradshaw, C. P. (2022) · Prevention Science · doi:10.1007/s11121-022-01336-w · Cited by 46
Self-citationGrant, S., Pedersen, E. R., Osilla, K. C., Kulesza, M., & D’Amico, E. J. (2015) · Addiction · doi:10.1111/add.13136 · Cited by 34
Self-citationFuller, T., Peters, J., Pearson, M., & Anderson, R. (2014) · American Journal of Public Health · doi:10.2105/ajph.2014.302195 · Cited by 30
Fidelity in After-School Program Intervention Research: A Systematic Review
Maynard, B. R., Peters, K. E., Vaughn, M. G., & Sarteschi, C. M. (2013) · Research on Social Work Practice · doi:10.1177/1049731513491150 · Cited by 30
Howard, M. O., & Garland, E. L. (2015) · Journal of the Society for Social Work and Research · doi:10.1086/681099 · Cited by 30
“A new guideline for trials of social and psychological interventions—the CONSORT-SPI—is under development (Grant et al., 2013).”
BackgroundToward the Development of Reporting Standards for Evaluations
Montrosse‐Moorhead, B., & Griffith, J. C. (2017) · American Journal of Evaluation · doi:10.1177/1098214017699275 · Cited by 30
Hensel, W. (2020) · European Journal for Philosophy of Science · doi:10.1007/s13194-020-00317-6
“I discuss four empirical studies: (a) by Grant et al. (2013), who investigate quality of reporting in psychological and social intervention trials in clinical psychology, criminology, education and social work, (b) by Robb et al. (2018), who examine compliance with field specific reporting…”
BackgroundInfluentialPerspectives on the application of CONSORT guidelines to randomised controlled trials in nutrition
Rigutto‐Farebrother, J., Ahles, S., Cade, J. E., Murphy, K. J., Plat, J., Schwingshackl, L., Roche, H. M., Shyam, S., Lachat, C. K., Minihane, A. M., et al. (2023) · European Journal of Nutrition · doi:10.1007/s00394-023-03137-5 · Cited by 37
“This presents some specific challenges regarding the potential for a multiplicity of tests and inflation of false positive or negative findings, and ‘p-value hacking’ [52].”
BackgroundStandards for design and measurement would make clinical research reproducible and usable
Dickersin, K., & Mayo‐Wilson, E. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1708273114 · Cited by 27
“Nevertheless, journal articles often omit important information about study methods (17, 24, 25), a major contributor to research waste (26).”
BackgroundSelf-citationKoopman, M. Y., Pieterse, M. E., Bohlmeijer, E. T., & Drossaert, C. H. C. (2017) · International Journal of Mental Health Promotion · doi:10.1080/14623730.2017.1328367 · Cited by 38
Impact and perceived value of journal reporting guidelines among Radiology authors and reviewers
Dewey, M., Levine, D., Bossuyt, P. M., & Kressel, H. Y. (2019) · European Radiology · doi:10.1007/s00330-018-5980-3 · Cited by 30
Cargo, M., Stankov, I., Thomas, J., Saini, M., Rogers, P., Mayo-Wilson, E., & Hannes, K. (2015) · BMC Medical Research Methodology · doi:10.1186/s12874-015-0037-7 · Cited by 30
“Despite the importance of understanding ‘implementation in context’, intervention descriptions and process evaluation measures are poorly reported in medicine, social and psychological interventions [15–17], social work [18] and for a broad range of occupational health, public health and health promotion interventions…”
BackgroundSelf-citationReporting quality and risk of bias in randomised trials in health professions education
Horsley, T., Galipeau, J., Petkovic, J., Zeiter, J., Hamstra, S. J., & Cook, D. A. (2016) · Medical Education · doi:10.1111/medu.13130 · Cited by 24
Network Meta-Analysis for Complex Social Interventions: Problems and Potential
Grant, E. S., & Calderbank-Batista, T. (2013) · Journal of the Society for Social Work and Research · doi:10.5243/jsswr.2013.25 · Cited by 24
Increase value and reduce waste in research on psychological therapies
Cristea, I. A., & Naudet, F. (2019) · Behaviour Research and Therapy · doi:10.1016/j.brat.2019.103479 · Cited by 22
Use of publication procedures to improve research integrity by addiction journals
Gorman, D. M. (2019) · Addiction · doi:10.1111/add.14604 · Cited by 24
Saarijärvi, M., Wallin, L., & Bratt, E. (2020) · European Journal of Cardiovascular Nursing · doi:10.1177/1474515120906561 · Cited by 20
Developing a reporting guideline for social and psychological intervention trials
Mayo‐Wilson, E., Montgomery, P., Hopewell, S., Macdonald, G. M., Moher, D., & Grant, S. (2013) · The British Journal of Psychiatry · doi:10.1192/bjp.bp.112.123745 · Cited by 24
Self-citationNorth, C. A., Shortt, M., Bowman, M. A., & Akinkuolie, B. (2021) · TechTrends · doi:10.1007/s11528-021-00636-2 · Cited by 29
Mayo‐Wilson, E., Grant, S., & Supplee, L. H. (2021) · Prevention Science · doi:10.1007/s11121-021-01284-x · Cited by 27
“Journal articles also often omit information needed to assess external validity or applicability to different populations and settings (Grant et al., 2013; Westbrook et al., 2017).”
BackgroundSelf-citationLevack, W., Engkasan, J. P., Heinemann, A. W., & Négrini, S. (2019) · American Journal of Physical Medicine & Rehabilitation · doi:10.1097/phm.0000000000001368 · Cited by 19
Trial Registration: Understanding and Preventing Reporting Bias in Social Work Research
Harrison, B. A., & Mayo-Wilson, E. (2014) · Research on Social Work Practice · doi:10.1177/1049731513512374 · Cited by 18
Self-citationArnberg, F. K., Alaie, I., Parling, T., & Jonsson, U. (2013) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2013.08.019 · Cited by 20
“The recently initiated work with reporting guidelines specifically for RCTs of social and psychological interventions is an important step [17,18].”
BackgroundResultScientific Reporting: Raising the Standards
McLeroy, K. R., Garney, W. R., Mayo‐Wilson, E., & Grant, S. (2016) · Health Education & Behavior · doi:10.1177/1090198116668522 · Cited by 19
“For instance, authors of this article are involved with developing an extension of the CONSORT Statement for the reporting of trials of social and psychological interventions, known as CONSORTSPI (Grant et al., 2013; Mayo-Wilson et al., 2013; Montgomery, Grant, et al., 2013).”
Self-citationChurches, R., Dommett, E. J., Devonshire, I. M., Hall, R., Higgins, S., & Korin, A. (2020) · Mind Brain and Education · doi:10.1111/mbe.12243 · Cited by 28
Grant, S., & on behalf of the CONSORT‐SPI Group (2018) · Addiction · doi:10.1111/add.14411 · Cited by 21
Self-citationGardner, F. E. M., Mayo‐Wilson, E., Montgomery, P., Hopewell, S., Macdonald, G. M., Moher, D., & Grant, S. (2013) · Journal of Child Psychology and Psychiatry · doi:10.1111/jcpp.12106 · Cited by 16
“This guideline will incorporate lessons from previous extensions (Grant et al., 2013) and other reporting guidelines to aid the critical appraisal, replication, and uptake of this research.”
Self-citationHome Visits for Prevention of Impairment and Death in Older Adults: A Systematic Review
Grant, S., Parsons, A., Burton, J., Montgomery, P., Underhill, K., & Wilson, E. (2014) · Campbell Systematic Reviews · doi:10.4073/csr.2014.3 · Cited by 17
Self-citationConsolidated Standards of Reporting Trials (CONSORT): Considerations for Neuropsychological Research
Miller, J. B., Schoenberg, M. R., & Bilder, R. M. (2014) · The Clinical Neuropsychologist · doi:10.1080/13854046.2014.907445 · Cited by 14
Valdez, D., Vorland, C. J., Brown, A. W., Mayo-Wilson, E., Otten, J. M., Ball, R. J., Grant, S., Levy, R., Svetina, D., & Allison, D. B. (2020) · F1000Research · doi:10.12688/f1000research.26594.1 · Cited by 16
Self-citationWiehn, J., Nonte, J., & Prugger, C. (2022) · BMJ Open · doi:10.1136/bmjopen-2022-061873 · Cited by 16
Open Science Standards at Journals that Inform Evidence-Based Policy
Grant, S., Mayo‐Wilson, E., Kianersi, S., Naaman, K., & Henschel, B. (2023) · Prevention Science · doi:10.1007/s11121-023-01543-z · Cited by 22
“Unfortunately, many of these open research practices are not common in the intervention research reviewed by clear-inghouses for evidence-based policy (Buckley et al., 2022; Grant et al., 2013).”
BackgroundMethodInfluentialSelf-citationHuynh, N., Tariq, S., Charron, C. B., Hayes, T., Bhanushali, O., Kaur, T., Jama, S., Ambade, P., Bignell, T., Hegarty, T., et al. (2022) · Journal of Epidemiology & Community Health · doi:10.1136/jech-2021-216783 · Cited by 14
Promising behavior change techniques for climate-friendly behavior change – a systematic review
Masciangelo, L., Lumbi, S. L., Rinderhagen, M., Hornberg, C., Liebig‐Gonglach, M., & McCall, T. (2024) · Frontiers in Public Health · doi:10.3389/fpubh.2024.1396958 · Cited by 15
A Systematic Review to Inform the Development of a Reporting Guideline for Concept Mapping Research
Pantha, S., Jones, M., Gartoulla, P., & Gray, R. J. (2023) · Methods and Protocols · doi:10.3390/mps6050101 · Cited by 11
Kaplan, D. M., Tidwell, C. A., Chung, J. M., Alisic, E., Demiray, B., Bruni, M. R., Evora, S., Gajewski‐Nemes, J. A., Macbeth, A., Mangelsdorf, S. N., et al. (2023) · Behavior Research Methods · doi:10.3758/s13428-023-02293-0 · Cited by 12
Hildenbrand, A. K. (2017) · Journal of Pediatric Psychology · doi:10.1093/jpepsy/jsx136 · Cited by 8
Sénéquier, A., Draper‐Rodi, J., Álvarez, G., BRAITHWAITE, F., Brown, J., Corcoran, D., Forrest, L., Godsi, E., MacMillan, A., Ménard, M., et al. (2025) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2025.111788 · Cited by 10
Blithikioti, C., Tomei, G., Visconti, F., Pizzocri, L., Cadorin, C., Gómez‐Gómez, I., & Cristea, I. A. (2025) · BMJ Mental Health · doi:10.1136/bmjment-2025-301578 · Cited by 7
“Trials of psychological interventions are often poorly reported, 7 8 and interventions in particular are cursorily and incompletely described 9 as ‘packages’, simply referencing the manual.”
Schemata, CONSORT, and the Salk Polio Vaccine Trial
Kowalski, C. J., & Mrdjenovich, A. J. (2017) · The Journal of Medicine and Philosophy A Forum for Bioethics and Philosophy of Medicine · doi:10.1093/jmp/jhx032 · Cited by 6
New guidelines are needed to improve the reporting of trials in addiction sciences
Grant, S., Mayo‐Wilson, E., Hopewell, S., Macdonald, G. M., Moher, D., & Montgomery, P. (2013) · Addiction · doi:10.1111/add.12249 · Cited by 7
Self-citationThe good, the bad and systematic reviews
Bölte, S. (2014) · Autism · doi:10.1177/1362361314561393 · Cited by 6
“The lack of consideration of external validity is the most frequent criticism by clinicians (Rothwell, 2006), and the need for increased attention on the selection of participants and their characteristics, differences between the protocol and routine practice, choice of outcome measures, follow-up timing, providers’…”
BackgroundKohrs, F. E., Auer, S., Bannach‐Brown, A., Fiedler, S., Haven, T. L., Heise, V., Holman, C., Azevedo, F., Bernard, R., Bleier, A., et al. (2023) · doi:10.31219/osf.io/kcvra · Cited by 6
Havsteen-Franklin, D., Oley, M., Sellors, S. J., & Eagles, D. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.588661 · Cited by 8
Magura, S., Lee-Easton, M. J., Abu-Obaid, R. N., Landsverk, J. A., DeCamp, W., Rolls-Reutz, J., Moore, K. A., Firpo‐Triplett, R., Buckley, P. R., Stout, E. R., et al. (2022) · Evaluation and Program Planning · doi:10.1016/j.evalprogplan.2022.102214 · Cited by 5
Ngo, H. L., Sokolovic, N., Hu, J., & Jenkins, J. M. (2025) · BMC Medical Education · doi:10.1186/s12909-025-07917-x · Cited by 3
“This could be addressed by standardizing interventions, methods and reporting [75, 76].”
Chan, E. W., Wong, J., Chan, C. S., & Cheung, F. (2022) · Springer eBooks · doi:10.1007/978-3-031-04968-2_18 · Cited by 4
Dierckx, C., Zaman, B., & Hannes, K. (2022) · Arts and Humanities in Higher Education · doi:10.1177/14740222221132952 · Cited by 3
“, 2007), and reporting guidelines for social and psychological intervention trials (Grant et al., 2013).”
BackgroundInvestigating Issues with Reproducibility in Journal Articles Published in Psychological SciencePreprint
Homewood, C. (2023) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/6d5nq · Cited by 3
Improving the Reporting of Health and Psychological Research
Fuller, T. E., Pearson, M., & Peters, J. (2016) · Australian Psychologist · doi:10.1111/ap.12135 · Cited by 3
Report Quality of Randomized Controlled Trials in Rehabilitation
Fujimoto, S., Komukai, K., Mitsutake, S., Sugita, S., & Kobayashi, M. (2018) · Rigakuryoho Kagaku · doi:10.1589/rika.33.507 · Cited by 3
Open science practices in clinical psychology journals: an audit studyPreprint
Cristea, I. A., Naudet, F., Nutu, D., & Gentili, C. (2018) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/t2k56 · Cited by 5
Ravichandran, C. T., Babb, S. M., Öngür, D., Harris, P. Q., & Cohen, B. M. (2022) · BMJ Open · doi:10.1136/bmjopen-2022-061882 · Cited by 1
Modi, N., Timmer, Z., Taylor, A., Bose, S., Spencer, A. J., & Smeds, M. R. (2024) · Annals of Vascular Surgery · doi:10.1016/j.avsg.2024.06.013 · Cited by 1
Stansfield, C., & O’Mara-Eves, A. (2025) · London Review of Education · doi:10.14324/lre.23.1.11 · Cited by 1
Spécificités et recherches méthodologiques pour l’évaluation en soins de support en oncologie
Paillard, M., Pulido, M., Metges, J., Almont, T., Heutte, N., & Bonnetain, F. (2017) · ONCOLOGIE · doi:10.1007/s10269-017-2686-4 · Cited by 1
“Des critères CONSORT (Consolidated Standards of Reporting Clinical Trials) sur les thérapeutiques non médicamenteuses en général [29], mais également dans quelques cas précis notamment pour les essais d’intervention en psychologie [30,31] et les essais cliniques en acupuncture [32], ont été publiés.”
BackgroundOliveira, L. M. (2023) · International Journal of Dental Hygiene · doi:10.1111/idh.12670 · Cited by 1
The Solution to Science's Replication CrisisPreprint
Knuteson, B. (2016) · SSRN Electronic Journal · doi:10.2139/ssrn.2835131 · Cited by 2
“[97] Grant SP, Mayo-Wilson E, Melendez-Torres G, Montgomery P.”
Introduction to reproducible psychological researchPreprint
Kunisato, Y. (2019) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/x8js5 · Cited by 1
Bhui, K. S. (2014) · The British Journal of Psychiatry · doi:10.1192/bjp.205.1.82 · Cited by 1
Kaplan, D. M., Tidwell, C. A., & Mehl, M. R. (2023) · doi:10.31234/osf.io/rnfps · Cited by 1
Setting-up a clinical trial: Some methodological recommendations
Colombo, D., Cipresso, P., Pedroli, E., & Riva, G. (2017) · Anuario de Psicología/The UB Journal of Psychology · doi:10.1016/j.anpsic.2017.12.001 · Cited by 1
Research Integrity in Public Health
Gorman, D. M. (2018) · Oxford University Press eBooks · doi:10.1093/obo/9780199756797-0189
The Solution to Science's Replication CrisisPreprint
Knuteson, B. (2016) · RePEc: Research Papers in Economics · doi:10.48550/arxiv.1609.03223 · Cited by 1
“[97] Grant SP, Mayo-Wilson E, Melendez-Torres G, Montgomery P.”
Mayo-Wilson, E., Grant, S., Supplee, L. H., Kianersi, S., Amin, A., DeHaven, A., & Mellor, D. (2021) · Figshare · doi:10.6084/m9.figshare.14722173
Self-citation
Metadata
- Authors
- Grant, S. P., Mayo-Wilson, E., Melendez-Torres, G. J., & Montgomery, P.
- Affiliations
- University of Oxford (GB)
- Corresponding author
- Grant, Sean P.
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2013-05-29
- Volume
- 8
- Issue
- 5
- Pages
- e65442
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC3666983
- OpenAlex
- W2162696461
- Cited by
- 111
- References
- 62
- Citations by year
- 2026: 2 · 2025: 5 · 2024: 2 · 2023: 10 · 2022: 9 · 2021: 16 · 2020: 5 · 2019: 7 · 2018: 9 · 2017: 9 · 2016: 8 · 2015: 8 · 2014: 11 · 2013: 10
- FWCI
- 2.50
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- UCL Discovery (University College London) (submittedVersion)
- Figshare (submittedVersion, cc-by)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion)
- Funders
- Economic and Social Research Council (ES/K00087X/1)
- Clarendon Fund
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Keywords
- reporting guidelines, reporting quality, complex interventions, CONSORT extension, randomised trials, blinding, allocation concealment
- MeSH terms
- Mental Disorders, Humans, Research Design, Social Adjustment, Randomized Controlled Trials as Topic, Guidelines as Topic, Information Dissemination, Biomedical Research, Research Report
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.


