Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index; 18 of the 30 sources its claims cite were checked for support.

INSPECT-SR: a tool for assessing trustworthiness of randomised controlled trials

Open access
Wilkinson, Jack, Heal, Calvin, Flemyng, Ella, Antoniou, Georgios A., Aburrow, Tony, Alfirevic, Zarko, Avenell, Alison, Barbour, Virginia, Berghella, Vincenzo, Bishop, Dorothy V. M., Bordewijk, Esmée M, Brown, Nicholas J. L., Christopher, Jana, Clarke, Mike, Dahly, Darren, Dennis, Jane, Dicker, Patrick, Dumville, Jo, Frankish, Helen, Grey, Andrew, Grohmann, Steph, Gurrin, Lyle C., Hayden, Jill A., Heathers, James A.J., Hunter, Kylie E, Hussey, Ian, Jung, Lukas, Lam, Emily, Lasserson, Toby J., Lensen, Sarah, Li, Tianjing, Li, Wentao, Liu, Jianping, Loder, Elizabeth, Lundh, Andreas, Meyerowitz-Katz, Gideon, Mol, Ben W., Naudet, Florian, Noel-Storr, Anna, O’Connell, Neil E., Parker, Lisa, Redberg, Rita F., Redman, Barbara K., Richardson, Rachel, Seidler, Anna Lene, Sheldrick, Kyle, Sydenham, Emma, Wely, Madelon van, Vorland, Colby J., Wang, Rui, Weibel, Stephanie, Wjst, Matthias, Bero, Lisa, Kirkham, Jamie J. · 2025 · medRxiv

Abstract

The integrity of evidence synthesis is threatened by problematic randomised controlled trials (RCTs). These are RCTs where there are serious concerns about the trustworthiness of the data or findings. This could be due to research misconduct, including fraud, or due to honest critical errors. If these RCTs are not detected, they may be inadvertently included in systematic reviews and guidelines, potentially distorting their results. To address this problem, the INSPECT-SR (INveStigating ProblEmatic Clinical Trials in Systematic Reviews) tool has been developed to assess the trustworthiness of RCTs. This will allow problematic RCTs to be identified and excluded from systematic reviews. This paper describes the development of INSPECT-SR. The tool and an associated guidance document are presented.

Overview

CoChat Summary

TL;DR

INSPECT-SR is a systematically developed tool for identifying potentially problematic randomised controlled trials (RCTs) so they can be excluded or handled cautiously in systematic reviews. The paper reports development through Delphi consensus, consensus meetings, and user testing, resulting in a 21-check tool across four domains.

Key findings

  • The tool was developed from an initial list of 76 trustworthiness checks, refined to 69 checks, then narrowed to 29 checks that met Delphi consensus and 21 checks selected in consensus meetings.
  • The Delphi survey ran from November 2023 to April 2024; 323 and 333 individuals were invited to Rounds 1 and 2, and 148 and 134 completed them, respectively.
  • User testing invited 148 individuals; 75 consented, 40 provided survey feedback, and the median number of trials assessed was 3 (IQR 1.5 to 4).
  • In user testing, the median time to assess one RCT was 45 minutes (IQR 27 to 74); 28 (70%) said they would choose to use the tool in future work, and 37 (93%) said their judgements were reasonable.

Methods

  • Development followed five stages: check generation from literature and expert input, application to 50 Cochrane Reviews, Delphi consensus, online consensus meetings, and user testing with feedback.
  • The Delphi survey asked participants to rate each check from 1 to 9 on usefulness and feasibility; a check was supported if it scored 7 or more for usefulness by at least 80% of participants at Round 2.
  • Consensus meetings were held in duplicate across four domains; the final tool was built from checks selected through anonymous voting and resolution of disagreements, then refined through user testing.

Limitations

  • The paper states that INSPECT-SR does not assess risk of bias, generalisability, or conflicts of interest, and it is not designed as a diagnostic test for fraud.
  • The authors note a learning curve and that some checks may be difficult to implement; they say user feedback highlighted challenges with implementing several checks.
  • Evident from the text: the reported user testing involved only 40 survey respondents and an 8-person workshop, so practical usability evidence is limited and based on a relatively small, self-selected group of testers.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1101/2025.09.03.25334905

Cites

30 of 30 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1: Summary of the development of INSPECT-SR
Figure 1: Summary of the development of INSPECT-SRCC BY · PubMed Central
Figure 2: Structure of the INSPECT-SR tool
Figure 2: Structure of the INSPECT-SR toolCC BY · PubMed Central

Verification

Claims the paper makes

9 Valid · 3 Not valid · 5 Not checked · 21 Own results and interpretations

7 more claims

Create a free account to see all 38 claims and every source verdict.

See all claims — free account
  • It may state two separate claims as one. 13
  • Probably not a substantial claim. 17

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 29 references

Research works printed but not linked: 4

  1. Cochrane. Cochrane Policy for managing potentially problematic studies. Cochrane Database of Systematic Reviews: editorial policies Cochrane Library [Available from: https://www.cochranelibrary.com/cdsr/editorial-policies.
  2. 2020, Probiotic treatment for women with gestational diabetes to improve maternal and infant health and well-being, Cochrane Database Syst Rev, 6, CD012970
  3. Group OEI . Trustworthiness criteria for meta-analyses of randomized controlled studies: OBGYN journal guidelines. BJOG. 2024.
  4. 2022, The power of weak signals: how systematic reviews direct researchers away from potentially biased primary studies, Cochrane Database Syst Rev, 11, ED000160

Cited by

See all 15

Metadata

Authors
Wilkinson, J., Heal, C., Flemyng, E., Antoniou, G. A., Aburrow, T., Alfirevic, Z., Avenell, A., Barbour, V., Berghella, V., Bishop, D. V. M., Bordewijk, E. M., Brown, N. J. L., Christopher, J., Clarke, M., Dahly, D., Dennis, J., Dicker, P., Dumville, J., Frankish, H., Grey, A., Grohmann, S., Gurrin, L. C., Hayden, J. A., Heathers, J. A. J., Hunter, K. E., Hussey, I., Jung, L., Lam, E., Lasserson, T. J., Lensen, S., Li, T., Li, W., Liu, J., Loder, E., Lundh, A., Meyerowitz-Katz, G., Mol, B. W., Naudet, F., Noel-Storr, A., O’Connell, N. E., Parker, L., Redberg, R. F., Redman, B. K., Richardson, R., Seidler, A. L., Sheldrick, K., Sydenham, E., Wely, M. V., Vorland, C. J., Wang, R., Weibel, S., Wjst, M., Bero, L., & Kirkham, J. J.
Affiliations
  • Manchester Academic Health Science Centre (GB)
  • Cochrane Canada (CA)
  • University of Aberdeen (GB)
  • Queensland University of Technology (AU)
  • Thomas Jefferson University (US)
  • University of Oxford (GB)
  • Amsterdam University Medical Centers (NL)
  • Linnaeus University (SE)
  • Queen's University Belfast (GB)
  • University College Cork (IE)
  • University of Bristol (GB)
  • Royal College of Surgeons in Ireland (IE)
  • University of Manchester (GB)
  • University of Auckland (NZ)
  • The University of Melbourne (AU)
  • Dalhousie University (CA)
  • Center for Scientific Integrity (US)
  • The University of Sydney (AU)
  • University of Bern (CH)
  • Royal Women's Hospital (AU)
  • University of Colorado Anschutz (US)
  • UNSW Sydney (AU)
  • Beijing University of Chinese Medicine (CN)
  • Harvard University (US)
  • University of Southern Denmark (DK)
  • Monash University (AU)
  • Inserm (FR)
  • Brunel University of London (GB)
  • University of California, San Francisco (US)
  • New York University (US)
  • Universitätsmedizin Rostock (DE)
  • Indiana University Bloomington (US)
  • Universitätsklinikum Würzburg (DE)
  • TUM Klinikum (DE)
Corresponding author
Wilkinson, Jack
Venue
medRxiv
Publisher
openRxiv
Published
2025-09-05
OpenAlex
W4414008064
Cited by
15
References
29
Citations by year
2026: 13 · 2025: 2
Type
posted content
Language
en
Open access
Locations
  • medRxiv (acceptedVersion, cc-by)
  • PubMed (publishedVersion)
  • PubMed Central (publishedVersion, cc-by)
  • Research Explorer (The University of Manchester) (submittedVersion, cc-by)
  • Open Access CRIS of the University of Bern
Funders
  • Neuroscience Research Australia
  • National Institute for Health and Care Research (NIHR203568)
  • Queensland University of Technology
  • National Eye Institute (UG1EY020522)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Social Sciences › Education › Academic integrity and plagiarism
  • Health Sciences › Nursing › Issues, ethics and legal aspects › Pharmaceutical industry and healthcare
Keywords
randomized controlled trials, research integrity, research misconduct, data integrity, systematic reviews, evidence synthesis

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: INSPECT-SR: a tool for assessing trustworthiness of randomised controlled trials. https://app.cochat.ai/papers/10.1101/2025.09.03.25334905
The paper
Wilkinson, J., Heal, C., Flemyng, E., Antoniou, G. A., Aburrow, T., Alfirevic, Z., Avenell, A., Barbour, V., Berghella, V., Bishop, D. V. M., Bordewijk, E. M., Brown, N. J. L., Christopher, J., Clarke, M., Dahly, D., Dennis, J., Dicker, P., Dumville, J., Frankish, H., . . . Kirkham, J. J. (2025). INSPECT-SR: a tool for assessing trustworthiness of randomised controlled trials. medRxiv. https://doi.org/10.1101/2025.09.03.25334905

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.