The DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index; 18 of the 30 sources its claims cite were checked for support.
INSPECT-SR: a tool for assessing trustworthiness of randomised controlled trials
Open accessAbstract
The integrity of evidence synthesis is threatened by problematic randomised controlled trials (RCTs). These are RCTs where there are serious concerns about the trustworthiness of the data or findings. This could be due to research misconduct, including fraud, or due to honest critical errors. If these RCTs are not detected, they may be inadvertently included in systematic reviews and guidelines, potentially distorting their results. To address this problem, the INSPECT-SR (INveStigating ProblEmatic Clinical Trials in Systematic Reviews) tool has been developed to assess the trustworthiness of RCTs. This will allow problematic RCTs to be identified and excluded from systematic reviews. This paper describes the development of INSPECT-SR. The tool and an associated guidance document are presented.
Overview
CoChat Summary
TL;DR
INSPECT-SR is a systematically developed tool for identifying potentially problematic randomised controlled trials (RCTs) so they can be excluded or handled cautiously in systematic reviews. The paper reports development through Delphi consensus, consensus meetings, and user testing, resulting in a 21-check tool across four domains.
Key findings
- The tool was developed from an initial list of 76 trustworthiness checks, refined to 69 checks, then narrowed to 29 checks that met Delphi consensus and 21 checks selected in consensus meetings.
- The Delphi survey ran from November 2023 to April 2024; 323 and 333 individuals were invited to Rounds 1 and 2, and 148 and 134 completed them, respectively.
- User testing invited 148 individuals; 75 consented, 40 provided survey feedback, and the median number of trials assessed was 3 (IQR 1.5 to 4).
- In user testing, the median time to assess one RCT was 45 minutes (IQR 27 to 74); 28 (70%) said they would choose to use the tool in future work, and 37 (93%) said their judgements were reasonable.
Methods
- Development followed five stages: check generation from literature and expert input, application to 50 Cochrane Reviews, Delphi consensus, online consensus meetings, and user testing with feedback.
- The Delphi survey asked participants to rate each check from 1 to 9 on usefulness and feasibility; a check was supported if it scored 7 or more for usefulness by at least 80% of participants at Round 2.
- Consensus meetings were held in duplicate across four domains; the final tool was built from checks selected through anonymous voting and resolution of disagreements, then refined through user testing.
Limitations
- The paper states that INSPECT-SR does not assess risk of bias, generalisability, or conflicts of interest, and it is not designed as a diagnostic test for fraud.
- The authors note a learning curve and that some checks may be difficult to implement; they say user feedback highlighted challenges with implementing several checks.
- Evident from the text: the reported user testing involved only 40 survey respondents and an 8-person workshop, so practical usability evidence is limited and based on a relatively small, self-selected group of testers.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
9 Valid · 3 Not valid · 5 Not checked · 21 Own results and interpretations
Some cited sources support this claim and some don’t — read each source’s verdict below.
In the paper
“A comprehensive list containing 76 trustworthiness checks was produced using existing evidence (a scoping review (19) and qualitative study (20)) and a new survey of experts (18).”
§ INSPECT-SR: a tool for assessing trustworthiness of randomised controlled trials > Development of the tool > Candidate item list generation
What the sources say
- Supportedcites [19] source
The citation is attached specifically to “a scoping review” as a source of existing evidence, not to the 76-check total or the new expert survey. The Methods (section 2.3) say the review extracted methods concerning research misconduct, and the Results (section 3.2) report 27 methods. Our copy provides no page numbers for these passages. (LLM, conf=0.91)
- Not checked — no full text of the sourcecites [20] source
- Disputedcites [18] 2024, A survey of experts to identify methods to detect problematic studies: stage 1 of the INveStigating ProblEmatic Clinical Trials in Systematic Reviews project, J Clin Epidemiol sourceExperimental — accuracy not yet measured
The Results section says 76 checks came from papers referenced by the scoping review; the list entering the expert survey contained 102. Section 3.5 reports that the updated list following the survey contained 116 checks, not 76. (LLM, conf=0.94)
Checked by a reasoning model — may be wrong.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“Examples of potentially problematic trials threatening the integrity of systematic reviews can be found across a diverse range of clinical areas (3, 10, 28).”
§ INSPECT-SR: a tool for assessing trustworthiness of randomised controlled trials > Discussion
What the sources say
- Not checked — no full text of the sourcecites [3] 2016, Does tranexamic acid prevent postpartum haemorrhage? A systematic review of randomised controlled trials, BJOG source
- Supportedcites [10] 2023, Investigating the veracity of a sample of divergent published trial data in spinal pain, Pain source
The Abstract identifies ten spinal-pain trials with trustworthiness concerns and says they should probably not inform systematic reviews. This paper supplies an example in one clinical area; the sentence’s broader range is attributed to the citations collectively. (LLM, conf=0.90)
- Not checked — no full text of the sourcecites [28] 2018, Caution to Readers About Systematic Review on Vitamin K and Prevention of Fractures That Included Problematic Trials, JAMA Intern Med source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“37 (93%) said that they thought the judgements they reached using the tool were reasonable.”
§ INSPECT-SR: a tool for assessing trustworthiness of randomised controlled trials > Development of the tool > User testing of the draft tool
plainn = 40- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“INSPECT-SR implements a set of trustworthiness checks, selected on the basis of empirical evidence, expert consensus methods, and theoretical considerations in the form of a practical tool that will help systematic reviewers and users of primary research to assess whether an RCT is trustworthy.”
§ INSPECT-SR: a tool for assessing trustworthiness of randomised controlled trials > Introduction In systematic reviews of randomised controlled trials (RCTs) the established steps are to identify all eligible trials, to appraise them using risk of bias tools, and to synthesise the results to reach a conclusion about the effectiveness and harms of an intervention. This paradigm is threatened by the presence of problematic studies in the literature. As defined by Cochrane, these are studies “where there are serious questions about the trustworthiness of the data or findings” (1). A study could be problematic due to research misconduct (including plagiarism, falsification, and fabrication of data or methods) or due to critical errors in study conduct that would not be flagged by risk of bias tools. If these studies are not identified as untrustworthy, they may inadvertently be included in evidence synthesis, potentially leading to treatment recommendations that are misleading or even actively harmful (2–4). Because many of these studies are not easily identifiable, it is difficult to assess the prevalence of problematic RCTs, but the available evidence is not encouraging. Carlisle estimated that 26% of RCTs submitted to the journal Anaesthesia were problematic (5). A recent review identified 847 systematic reviews that included one or more retracted RCTs in meta-analysis, and found that excluding these studies changed the size of effect in 16% of meta-analyses, and the direction of effect in just over 8% (6). It may take years for a problematic trial to be recognised and retracted, if it is retracted at all (7, 8), so these figures are likely to underestimate the problem. These delays also mean that systematic reviewers cannot rely on retraction status alone to identify problematic trials. Risk of bias tools are not designed to identify problems of this nature, nor do they appear to be successful in doing so (9, 10). Bespoke tools for trustworthiness assessment are therefore needed. A variety of trustworthiness tools have recently been proposed (11–15), although they differ in terms of their development, content and structure, and the appropriateness of some of the trustworthiness checks that have been included in some of these tools is unclear (16). Given the number of problematic studies that are now being identified, and the growing number of proposed methods for detecting these studies, it was timely to assess and reach international consensus on a set of checks and principles for trustworthiness assessment.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The median (IQR) time to assess an RCT using the tool was 45 (27 to 74) minutes.”
§ INSPECT-SR: a tool for assessing trustworthiness of randomised controlled trials > Development of the tool > User testing of the draft tool
plainn = 40- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
7 more claims
Create a free account to see all 38 claims and every source verdict.
- It may state two separate claims as one. 13
- Probably not a substantial claim. 17
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The INSPECT-SR tool, developed with Cochrane backing and endorsed in October 2025, provides 21 trustworthiness checks for assessing RCT reliability during systematic review”
From Artificial intelligence to enhance research integrity in evidence-based medicine: toward scalable trustworthiness assessment (2026) · cites this paper as Wilkinson et al., 2025
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Unlike conventional risk-of-bias tools, INSPECT-SR targets signals of potentially problematic trials (including critical error or possible fabrication) that can slip into meta-analyses and guidelines.”
From Artificial intelligence to enhance research integrity in evidence-based medicine: toward scalable trustworthiness assessment (2026) · cites this paper as Wilkinson et al., 2025
Citations
Cites
Page, M. J., Moher, D., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · BMJ · doi:10.1136/bmj.n160 · Cited by 13,262
Self-citationFalse individual patient data and zombie randomised controlled trials submitted to Anaesthesia
Carlisle, J. B. (2021) · Anaesthesia · doi:10.1111/anae.15263 · Cited by 173
Mol, B. W., Lai, S., Rahim, A., Bordewijk, E. M., Wang, R., van Eekelen, R., Gurrin, L. C., Thornton, J. G., van Wely, M., & Li, W. (2023) · Research Integrity and Peer Review · doi:10.1186/s41073-023-00130-8 · Cited by 154
Self-citationCheck for publication integrity before misconduct
Grey, A., Bolland, M. J., Avenell, A., Klein, A. A., & Gunsalus, C. K. (2020) · Nature · doi:10.1038/d41586-019-03959-6 · Cited by 114
Self-citationData extraction methods for systematic review (semi)automation: a living systematic review
Schmidt, L., Olorisade, B. K., McGuinness, L. A., Thomas, J., & Higgins, J. P. T. (2021) · F1000Research · doi:10.12688/f1000research.51117.1 · Cited by 106
Chambers, L., Michener, C., & Falcone, T. (2019) · BJOG: An International Journal of Obstetrics & Gynaecology · doi:10.1111/1471-0528.15689 · Cited by 99
Weibel, S., Popp, M., Reis, S., Skoetz, N., Garner, P., & Sydenham, E. (2023) · Research Synthesis Methods · doi:10.1002/jrsm.1599 · Cited by 93
Self-citationThe lesson of ivermectin: meta-analyses based on summary data alone are inherently unreliable
Lawrence, J. M., Meyerowitz-Katz, G., Heathers, J. A. J., Brown, N. J. L., & Sheldrick, K. A. (2021) · Nature Medicine · doi:10.1038/s41591-021-01535-y · Cited by 88
Self-citationNaggie, S., Boulware, D. R., Lindsell, C. J., Stewart, T. G., Slandzicki, A. J., Lim, S. C., Cohen, J., Kavtaradze, D., Amon, A. P., Gabriel, A., et al. (2023) · JAMA · doi:10.1001/jama.2023.1650 · Cited by 79
Tóth, B., Berek, L., Gulàcsi, L., Péntek, M., & Zrubka, Z. (2024) · Systematic Reviews · doi:10.1186/s13643-024-02592-3 · Cited by 70
Research works printed but not linked: 4
- Cochrane. Cochrane Policy for managing potentially problematic studies. Cochrane Database of Systematic Reviews: editorial policies Cochrane Library [Available from: https://www.cochranelibrary.com/cdsr/editorial-policies.
- 2020, Probiotic treatment for women with gestational diabetes to improve maternal and infant health and well-being, Cochrane Database Syst Rev, 6, CD012970
- Group OEI . Trustworthiness criteria for meta-analyses of randomized controlled studies: OBGYN journal guidelines. BJOG. 2024.
- 2022, The power of weak signals: how systematic reviews direct researchers away from potentially biased primary studies, Cochrane Database Syst Rev, 11, ED000160
Cited by
Hou, L., Feng, Q., Mazi, M., Lensen, S. F., Maheshwari, A., & Mol, B. W. (2026) · Human Fertility · doi:10.1080/14647273.2026.2696818 · Cited by 1
Therapeutic hypothermia for newborns with hypoxic-ischaemic encephalopathy
Fink, A., Berg, M. T., Soll, R. F., Fiander, M., Hunter, K. E., Aagerup, J., Sotiropoulos, J. X., Cracknell, J., Bruschettini, M., Seidler, A. L., et al. (2026) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd016345 · Cited by 4
Self-citationNeale, E. P. (2025) · Nutrition & Dietetics · doi:10.1111/1747-0080.70065 · Cited by 1
“42 In order to maintain the rigour of evidence syntheses, and ensure their value in guiding nutrition and dietetics policy and practice, review authors must be aware of the hallmarks of these problematic studies and subsequently exclude them from their reviews.”
Shokraee, K., Bongaerts, B. W. C., Taneri, P. E., Liquitay, C. M. E., Olfati, S., Zorger, A.-M., Skoetz, N., & Franco, J. V. A. (2026) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd016387
Bruton, A. M., Staab, C. A., Gray, O., Chua, W. R. (., Alsakhita, N., Goldenberg, J. Z., & Roberts, J. L. (2026) · BMJ Open · doi:10.1136/bmjopen-2026-123353
“35”
Metadata
- Authors
- Wilkinson, J., Heal, C., Flemyng, E., Antoniou, G. A., Aburrow, T., Alfirevic, Z., Avenell, A., Barbour, V., Berghella, V., Bishop, D. V. M., Bordewijk, E. M., Brown, N. J. L., Christopher, J., Clarke, M., Dahly, D., Dennis, J., Dicker, P., Dumville, J., Frankish, H., Grey, A., Grohmann, S., Gurrin, L. C., Hayden, J. A., Heathers, J. A. J., Hunter, K. E., Hussey, I., Jung, L., Lam, E., Lasserson, T. J., Lensen, S., Li, T., Li, W., Liu, J., Loder, E., Lundh, A., Meyerowitz-Katz, G., Mol, B. W., Naudet, F., Noel-Storr, A., O’Connell, N. E., Parker, L., Redberg, R. F., Redman, B. K., Richardson, R., Seidler, A. L., Sheldrick, K., Sydenham, E., Wely, M. V., Vorland, C. J., Wang, R., Weibel, S., Wjst, M., Bero, L., & Kirkham, J. J.
- Affiliations
- Manchester Academic Health Science Centre (GB)
- Cochrane Canada (CA)
- University of Aberdeen (GB)
- Queensland University of Technology (AU)
- Thomas Jefferson University (US)
- University of Oxford (GB)
- Amsterdam University Medical Centers (NL)
- Linnaeus University (SE)
- Queen's University Belfast (GB)
- University College Cork (IE)
- University of Bristol (GB)
- Royal College of Surgeons in Ireland (IE)
- University of Manchester (GB)
- University of Auckland (NZ)
- The University of Melbourne (AU)
- Dalhousie University (CA)
- Center for Scientific Integrity (US)
- The University of Sydney (AU)
- University of Bern (CH)
- Royal Women's Hospital (AU)
- University of Colorado Anschutz (US)
- UNSW Sydney (AU)
- Beijing University of Chinese Medicine (CN)
- Harvard University (US)
- University of Southern Denmark (DK)
- Monash University (AU)
- Inserm (FR)
- Brunel University of London (GB)
- University of California, San Francisco (US)
- New York University (US)
- Universitätsmedizin Rostock (DE)
- Indiana University Bloomington (US)
- Universitätsklinikum Würzburg (DE)
- TUM Klinikum (DE)
- Corresponding author
- Wilkinson, Jack
- ORCID
- Wilkinson, Jack
- Heal, Calvin
- Flemyng, Ella
- Avenell, Alison
- Barbour, Virginia
- Berghella, Vincenzo
- Bishop, Dorothy V. M.
- Bordewijk, Esmée M
- Brown, Nicholas J. L.
- Christopher, Jana
- Clarke, Mike
- Dahly, Darren
- Dennis, Jane
- Dicker, Patrick
- Dumville, Jo
- Grohmann, Steph
- Hunter, Kylie E
- Hussey, Ian
- Jung, Lukas
- Lasserson, Toby J.
- Lensen, Sarah
- Li, Tianjing
- Li, Wentao
- Liu, Jianping
- Mol, Ben W.
- Naudet, Florian
- O’Connell, Neil E.
- Redman, Barbara K.
- Richardson, Rachel
- Sydenham, Emma
- Wely, Madelon van
- Vorland, Colby J.
- Wang, Rui
- Weibel, Stephanie
- Bero, Lisa
- Kirkham, Jamie J.
- Venue
- medRxiv
- Publisher
- openRxiv
- Published
- 2025-09-05
- PMC
- PMC12424918
- OpenAlex
- W4414008064
- Cited by
- 15
- References
- 29
- Citations by year
- 2026: 13 · 2025: 2
- Type
- posted content
- Language
- en
- Open access
- green
- doi.org
- Licence
- Locations
- medRxiv (acceptedVersion, cc-by)
- PubMed (publishedVersion)
- PubMed Central (publishedVersion, cc-by)
- Research Explorer (The University of Manchester) (submittedVersion, cc-by)
- Open Access CRIS of the University of Bern
- Funders
- Neuroscience Research Australia
- National Institute for Health and Care Research (NIHR203568)
- Queensland University of Technology
- National Eye Institute (UG1EY020522)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Education › Academic integrity and plagiarism
- Health Sciences › Nursing › Issues, ethics and legal aspects › Pharmaceutical industry and healthcare
- Keywords
- randomized controlled trials, research integrity, research misconduct, data integrity, systematic reviews, evidence synthesis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: INSPECT-SR: a tool for assessing trustworthiness of randomised controlled trials. https://app.cochat.ai/papers/10.1101/2025.09.03.25334905
- The paper
- Wilkinson, J., Heal, C., Flemyng, E., Antoniou, G. A., Aburrow, T., Alfirevic, Z., Avenell, A., Barbour, V., Berghella, V., Bishop, D. V. M., Bordewijk, E. M., Brown, N. J. L., Christopher, J., Clarke, M., Dahly, D., Dennis, J., Dicker, P., Dumville, J., Frankish, H., . . . Kirkham, J. J. (2025). INSPECT-SR: a tool for assessing trustworthiness of randomised controlled trials. medRxiv. https://doi.org/10.1101/2025.09.03.25334905
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.

