The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Protocol for the development of a tool (INSPECT-SR) to identify problematic randomised controlled trials in systematic reviews of health interventions
Open accessPublished version of a preprint
Abstract
Introduction: Randomised controlled trials (RCTs) inform healthcare decisions. It is now apparent that some published RCTs contain false data and some appear to have been entirely fabricated. Systematic reviews are performed to identify and synthesise all RCTs that have been conducted on a given topic. While it is usual to assess methodological features of the RCTs in the process of undertaking a systematic review, it is not usual to consider whether the RCTs contain false data. Studies containing false data therefore go unnoticed and contribute to systematic review conclusions. The INveStigating ProblEmatic Clinical Trials in Systematic Reviews (INSPECT-SR) project will develop a tool to assess the trustworthiness of RCTs in systematic reviews of healthcare-related interventions. Methods and analysis: The INSPECT-SR tool will be developed using expert consensus in combination with empirical evidence, over five stages: (1) a survey of experts to assemble a comprehensive list of checks for detecting problematic RCTs, (2) an evaluation of the feasibility and impact of applying the checks to systematic reviews, (3) a Delphi survey to determine which of the checks are supported by expert consensus, culminating in, (4) a consensus meeting to select checks to be included in a draft tool and to determine its format and (5) prospective testing of the draft tool in the production of new health systematic reviews, to allow refinement based on user feedback. We anticipate that the INSPECT-SR tool will help researchers to identify problematic studies and will help patients by protecting them from the influence of false data on their healthcare. Ethics and dissemination: The University of Manchester ethics decision tool was used, and this returned the result that ethical approval was not required for this project (30 September 2022), which incorporates secondary research and surveys of professionals about subjects relating to their expertise. Informed consent will be obtained from all survey participants. All results will be published as open-access articles. The final tool will be made freely available.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“The development process for the tool has previously been described (17) and included five stages”
From INSPECT-SR: a tool for assessing trustworthiness of randomised controlled trials (2025) · cites this paper as [17]
The cited source says what the claim attributes to it.
In the citing paper
“A protocol describing the INSPECT-SR project methods has previously been published”
From Assessing the feasibility and impact of clinical trial trustworthiness checks via an application to Cochrane Reviews: Stage 2 of the INSPECT-SR project (2025) · cites this paper as [9]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“The Statcheck and GRIM-Test are especially good at spotting statistical errors.”
From Artificial Intelligence in Detecting Statistical Errors: Implications for Authors, Reviewers, and Editors (2025) · cites this paper as [31]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“It has been proposed as a reliable tool by validity studies in which its sensitivity exceeded 83%, specificity was over 96%, and overall accuracy reached above 92%.31”
From Artificial Intelligence in Detecting Statistical Errors: Implications for Authors, Reviewers, and Editors (2025) · cites this paper as [31]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“The GRIM-Test is a mathematical tool that ensures reported means are possible given a certain sample size.”
From Artificial Intelligence in Detecting Statistical Errors: Implications for Authors, Reviewers, and Editors (2025) · cites this paper as [31]
Experimental — accuracy not yet measured
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
1 retracted
Retracted: Meta-analysis of Randomized Trials of Ivermectin to Treat SARS-CoV-2 Infection
Hill, A. M., Abdulamir, A., Ahmed, S., Asghar, A., Babalola, O. E., Basri, R., Chaccour, C., Chachar, A. Z. K., Chowdhury, A. T. M., Elgazzar, A., et al. (2021) · Open Forum Infectious Diseases · doi:10.1093/ofid/ofab358 · Cited by 98
RetractedRoB 2: a revised tool for assessing risk of bias in randomised trials
Sterne, J. A. C., Savović, J., Page, M. J., Elbers, R. G., Blencowe, N. S., Boutron, I., Cates, C. J., Cheng, H.-Y., Corbett, M. S., Eldridge, S. M., et al. (2019) · BMJ · doi:10.1136/bmj.l4898 · Cited by 34,787 · cited 1×
“Criteria relating to internal or external validity of results produced by RCTs, such as those included in Risk of Bias, 12 Risk of Bias V.2 13 or GRADE (Grading of Recommendations Assessment, Development and Evaluation) 14 frameworks, are not within the scope of the INSPECT-SR project.”
BackgroundSelf-citationThe Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817
Dermyshi, E., Wang, Y., Yan, C., Hong, W., Qiu, G., Gong, X., & Zhang, T. (2017) · Neonatology · doi:10.1159/000454668 · Cited by 24,451
What is ‘quality of evidence’ and why is it important to clinicians?
Guyatt, G. H., Oxman, A. D., Kunz, R., Vist, G. E., Falck-Ytter, Y., & Schünemann, H. J. (2008) · BMJ · doi:10.1136/bmj.39490.551019.be · Cited by 3,250
Psychological therapies for the management of chronic pain (excluding headache) in adults
Williams, A. C. D. C., Eccleston, C., & Morley, S. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007407.pub3 · Cited by 1,634
Psychological therapies for the management of chronic pain (excluding headache) in adults
Williams, A. C. D. C., Fisher, E., Hearn, L., & Eccleston, C. (2020) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd007407.pub4 · Cited by 605
Cockayne, S., Adamson, J., Lanham-New, S., Shearer, M. J., Gilbody, S., & Torgerson, D. J. (2006) · Archives of Internal Medicine · doi:10.1001/archinte.166.12.1256 · Cited by 361
Self-citationBryant, A., Lawrie, T. A., Dowswell, T., Fordham, E. J., Mitchell, S., Hill, S. R., & Tham, T. C. (2021) · American Journal of Therapeutics · doi:10.1097/mjt.0000000000001402 · Cited by 264
Mol, B. W., Lai, S., Rahim, A., Bordewijk, E. M., Wang, R., van Eekelen, R., Gurrin, L. C., Thornton, J. G., van Wely, M., & Li, W. (2023) · Research Integrity and Peer Review · doi:10.1186/s41073-023-00130-8 · Cited by 154 · cited 1×
“While the need to prevent problematic studies from contributing to systematic reviews is recognised, with several recent laudable efforts to tackle the issue, 18–20 there is currently limited agreement on how this should be done.”
BackgroundSelf-citation
Research works printed but not linked: 1
- Cochrane . Cochrane database of systematic reviews: editorial policies Cochrane library. 2021 Available: https://www.cochranelibrary.com/cdsr/editorial-policies
Cited by
Xu, C., Fan, S., Tian, Y., Liu, F., Furuya-Kanamori, L., Clark, J., Zhang, C., Li, S., Lin, L., Chu, H., et al. (2025) · BMJ · doi:10.1136/bmj-2024-082068 · Cited by 39
Wilkinson, J., Heal, C., Antoniou, G. A., Flemyng, E., Ahnström, L., Alteri, A., Avenell, A., Barker, T. H., Borg, D. N., Brown, N. J. L., et al. (2025) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2025.111824 · Cited by 28
Self-citationINSPECT-SR: a tool for assessing trustworthiness of randomised controlled trials
Wilkinson, J., Heal, C., Flemyng, E., Antoniou, G. A., Aburrow, T., Alfirevic, Z., Avenell, A., Barbour, V., Berghella, V., Bishop, D. V. M., et al. (2025) · medRxiv · doi:10.1101/2025.09.03.25334905 · Cited by 15
Self-citationReeves‐McLaren, N., & Christensen, S. M.-L. (2025) · Journal of Materials Chemistry A · doi:10.1039/d5ta05512a · Cited by 14
Zeraatkar, D., Ling, M., Kirsh, S., Jassal, T., Pitre, T., Chakraborty, S. P., Turner, T., Turkstra, L. S., McIntyre, R. S., Izcovich, A., et al. (2025) · BMJ Open · doi:10.1136/bmjopen-2024-086407 · Cited by 18
Metadata
- Authors
- Wilkinson, J., Heal, C., Antoniou, G. A., Flemyng, E., Alfirevic, Z., Avenell, A., Barbour, G., Brown, N. J. L., Carlisle, J., Clarke, M., Dicker, P., Dumville, J. C., Grey, A., Grohmann, S., Gurrin, L., Hayden, J. A., Heathers, J., Hunter, K. E., Lasserson, T., Lam, E., Lensen, S., Li, T., Li, W., Loder, E., Lundh, A., Meyerowitz-Katz, G., Mol, B. W., O'Connell, N. E., Parker, L., Redman, B. K., Seidler, L., Sheldrick, K. A., Sydenham, E., Torgerson, D., van Wely, M., Wang, R., Bero, L., & Kirkham, J. J.
- Affiliations
- Manchester Academic Health Science Centre (GB)
- Cochrane Canada (CA)
- Department of Women's and Children's Health, University of Liverpool, Liverpool, UK
- University of Aberdeen (GB)
- Medical Journal of Australia, Sydney, New South Wales, Australia
- Linnaeus University (SE)
- Torbay Hospital (GB)
- Queen's University Belfast (GB)
- University of Auckland (NZ)
- The University of Melbourne (AU)
- Dalhousie University (CA)
- SafeBeat Rx Inc, St Louis, Missouri, USA
- The University of Sydney (AU)
- Independent Lay Member, Cheshire, UK
- Royal Women's Hospital (AU)
- University of Colorado Anschutz (US)
- Monash University (AU)
- Brigham and Women's Hospital (US)
- University of Southern Denmark (DK)
- School of Health and Society, University of Wollongong, Wollongong, New South Wales, Australia
- Brunel University of London (GB)
- New York University (US)
- Evidence Integration, NHMRC Clinical Trials Centre, University of Sydney, Sydney, New South Wales, Australia
- UNSW Sydney (AU)
- Cochrane (GB)
- University of York (GB)
- Cochrane Gynaecology and Fertility Satellite and Cochrane Sexually Transmitted Infection Group, Cochrane, Duivendrecht, The Netherlands
- Corresponding author
- Wilkinson, Jack
- ORCID
- Wilkinson, Jack
- Heal, Calvin
- Antoniou, George A
- Flemyng, Ella
- Avenell, Alison
- Brown, Nicholas J L
- Carlisle, John
- Clarke, Mike
- Dicker, Patrick
- Dumville, Jo C
- Grey, Andrew
- Grohmann, Steph
- Gurrin, Lyle
- Hayden, Jill Alison
- Heathers, James
- Hunter, Kylie Elizabeth
- Lasserson, Toby
- Lensen, Sarah
- Li, Tianjing
- Li, Wentao
- Loder, Elizabeth
- Lundh, Andreas
- Mol, Ben W
- O'Connell, Neil E
- Parker, Lisa
- Redman, Barbara K
- Sheldrick, Kyle A
- Sydenham, Emma
- Torgerson, David
- Wang, Rui
- Bero, Lisa
- Kirkham, Jamie J
- Venue
- BMJ Open
- Publisher
- BMJ
- Published
- 2024-03
- Volume
- 14
- Issue
- 3
- Pages
- e084164
- ISSN
- 2044-6055 (ISSN-L)
- PMC
- PMC10936473
- OpenAlex
- W4392712333
- Cited by
- 45
- References
- 20
- Citations by year
- 2026: 11 · 2025: 27 · 2024: 7
- FWCI
- 5.66
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- BMJ Open (publishedVersion, cc-by)
- medRxiv (acceptedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Research Explorer (The University of Manchester) (publishedVersion, cc-by)
- Research Portal (Queen's University Belfast) (publishedVersion, cc-by)
- DiVA (Linnaeus University) (submittedVersion, other-oa)
- Aberdeen University Research Archive (Aberdeen University) (submittedVersion, other-oa)
- PubMed Central (submittedVersion, cc-by)
- Research Online (University of Wollongong) (submittedVersion)
- Brunel University Research Archive (BURA) (Brunel University London) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Minerva Access (University of Melbourne) (submittedVersion, cc-by)
- HighWire Press Open Archive (submittedVersion)
- Pure Amsterdam UMC (submittedVersion, other-oa)
- University of Southern Denmark Research Portal (University of Southern Denmark) (submittedVersion, cc-by)
- Funders
- NIHR (NIHR203568)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Education › Academic integrity and plagiarism
- Health Sciences › Nursing › Issues, ethics and legal aspects › Pharmaceutical industry and healthcare
- Keywords
- randomized controlled trials, research integrity, data fabrication, research misconduct, systematic reviews, trustworthiness assessment, expert consensus, Delphi method, health interventions
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Protocol for the development of a tool (INSPECT-SR) to identify problematic randomised controlled trials in systematic reviews of health interventions. https://app.cochat.ai/papers/10.1136/bmjopen-2024-084164
- The paper
- Wilkinson, J., Heal, C., Antoniou, G. A., Flemyng, E., Alfirevic, Z., Avenell, A., Barbour, G., Brown, N. J. L., Carlisle, J., Clarke, M., Dicker, P., Dumville, J. C., Grey, A., Grohmann, S., Gurrin, L., Hayden, J. A., Heathers, J., Hunter, K. E., Lasserson, T., . . . Kirkham, J. J. (2024). Protocol for the development of a tool (INSPECT-SR) to identify problematic randomised controlled trials in systematic reviews of health interventions. BMJ Open. https://doi.org/10.1136/bmjopen-2024-084164
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
