The DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index; 7 of the 13 sources its claims cite were checked for support.
Checklist to assess Trustworthiness in RAndomised Controlled Trials (TRACT checklist): concept proposal and pilot
Open accessPublished version of a preprint
Abstract
OBJECTIVES: To propose a checklist that can be used to assess trustworthiness of randomized controlled trials (RCTs). DESIGN: A screening tool was developed using the four-stage approach proposed by Moher et al. This included defining the scope, reviewing the evidence base, suggesting a list of items from piloting, and holding a consensus meeting. The initial checklist was set-up by a core group who had been involved in the assessment of problematic RCTs for several years. We piloted this in a consensus panel of several stakeholders, including health professionals, reviewers, journal editors, policymakers, researchers, and evidence-synthesis specialists. Each member was asked to score three articles with the checklist and the results were then discussed in consensus meetings. OUTCOME: The Trustworthiness in RAndomised Clinical Trials (TRACT) checklist includes 19 items organised into seven domains that are applicable to every RCT: 1) Governance, 2) Author Group, 3) Plausibility of Intervention Usage, 4) Timeframe, 5) Drop-out Rates, 6) Baseline Characteristics, and 7) Outcomes. Each item can be answered as either no concerns, some concerns/no information, or major concerns. If a study is assessed and found to have a majority of items rated at a major concern level, then editors, reviewers or evidence synthesizers should consider a more thorough investigation, including assessment of original individual participant data. CONCLUSIONS: The TRACT checklist is the first checklist developed specifically to detect trustworthiness issues in RCTs. It might help editors, publishers and researchers to screen for such issues in submitted or published RCTs in a transparent and replicable manner.
Overview
CoChat Summary
TL;DR
The authors propose TRACT, a 19-item, seven-domain screening checklist to assess trustworthiness of randomized controlled trials and triage possible research misconduct. In a pilot consensus process, users showed 65% agreement on item ratings, and the checklist is intended to prompt further investigation when most items are rated as major concerns.
Key findings
- The TRACT checklist includes 19 items organized into seven domains: Governance, Author Group, Plausibility of Intervention Usage, Timeframe, Drop-out Rates, Baseline Characteristics, and Outcomes.
- Each item is rated as no concerns, some concerns/no information, or major concerns; if a study has a majority of items rated at major concern level, editors, reviewers, or evidence synthesizers should consider a more thorough investigation, including assessment of original individual participant data.
- In pilot comparisons, there was 65% overall agreement on item ratings between users, and all item numbers except one had a 50% or more chance of similar results between users; the exception was “Effect size that is much larger than in other RCTs regarding the same topic” in the Outcomes domain.
- The paper states the TRACT checklist is the first checklist developed specifically to detect trustworthiness issues in RCTs, and the authors report having assessed the integrity of over 300 RCTs in women’s health over recent years.
Methods
- Development was adapted from Moher et al.’s four-stage approach: define the scope, review the evidence base, suggest a list of items from piloting, and hold a consensus process.
- A steering group of evidence-synthesis experts, clinical researchers, reviewers, editors, biostatisticians, and evidence-synthesis specialists defined the scope; a scoping review and four published systematic analyses informed the first checklist version.
- In the consensus process, stakeholders used the draft checklist to screen up to three articles and completed a questionnaire rating each item’s usefulness and ease on a five-point Likert scale; items with a median rating of 3 or more in both usefulness and ease were eligible for inclusion, and responses were compared where assessors rated the same article.
Limitations
- The paper states the checklist is not yet validated, that a formalized Delphi method would allow greater verification of utility and useability, that some concern patterns are crude and may miss other misconduct patterns, that fabricators could use the checklist to fabricate a paper fitting all items, that it may misclassify papers, and that using it can be time consuming.
- The paper also states no formal cut-off at which the checklist scores positive has been provided, and that this will be addressed in subsequent detection accuracy studies.
- Evident from the text: the pilot agreement was 65%, and one item had less than a 50% chance of similar results between users, suggesting inter-user reliability may be limited.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
6 Valid · 3 Not valid · 4 Not checked · 46 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“A similar drop-out rate for different treatments arms is often seen as a reassurance against bias; however, Bell et al. determined that equal number of drop-outs rates do not guarantee unbiased results in RCTs [21].”
§ Checklist to assess Trustworthiness in RAndomised Controlled Trials (TRACT checklist): concept proposal and pilot > Rationale for included items > Drop-out rates
What the sources say
- Supportedcites [21] Bell ML, Kenward MG, Fairclough DL, et al. Differential dropout and bias in randomised controlled trials: when it matters and when it may not. BMJ. 2013;346:e8668–768. https://doi.org/10.1136/bmj.e8668. source
The Introduction explicitly identifies as a myth the idea that similar dropout rates between trial arms remove concern about bias. The Discussion states that equal dropout rates can still yield biased estimates of treatment effects. (LLM, conf=0.95)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“The availability of participant data and the willingness to share these data may be a good indicator of quality, methodological soundness and integrity of RCTs [23, 24].”
§ Checklist to assess Trustworthiness in RAndomised Controlled Trials (TRACT checklist): concept proposal and pilot > Discussion
What the sources say
- Not checked — no full text of the sourcecites [23] Tsujimoto Y, Fujii T, Onishi A, et al. No consistent evidence of data availability bias existed in recent individual participant data meta-analyses: a meta-epidemiological study. J Clin Epidemiol. 2020;118:107-14.e5. https://doi.org/10.1016/j.jclinepi.2019.10.004. publishedOnlineFirst:2019/10/28. source
- Supportedcites [24] Bordewijk EM, Wang R, van Wely M, et al. To share or not to share data: how valid are trials evaluating first-line ovulation induction for polycystic ovary syndrome? Hum Reprod Update. 2020;26:929–41. Accepted for publication. source
The Discussion (p. 9) says that willingness to share participant data may indicate trial quality, methodological soundness and integrity. It cautions that sharing should not be used as the sole indicator, a limitation consistent with the citing sentence’s qualified wording. (LLM, conf=0.91)
hedgedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We received positive feedback from users – mostly junior academics and students from several countries – that in their assessments with this first version of the checklist it was able to successfully identify trustworthiness issues in RCTs that were later confirmed in more formal investigations led by journals or publishers.”
§ Checklist to assess Trustworthiness in RAndomised Controlled Trials (TRACT checklist): concept proposal and pilot > Methods > Piloting of the preliminary checklist
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Over the last few years, we have assessed the integrity of over 300 RCTs in women’s health for a variety of reasons.”
§ Checklist to assess Trustworthiness in RAndomised Controlled Trials (TRACT checklist): concept proposal and pilot > Methods > Piloting of the preliminary checklist
plainn = 300- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Additionally, all item numbers except one had a 50% or more chance of similar results between users.”
§ Checklist to assess Trustworthiness in RAndomised Controlled Trials (TRACT checklist): concept proposal and pilot > Methods > Consensus process
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
14 more claims
Create a free account to see all 59 claims and every source verdict.
- Probably not a substantial claim. 35
- It may state two separate claims as one. 17
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Integrity assessment was integrated into guideline evidence synthesis processes and followed the Research Integrity in Guideline Development framework, with studies assessed against criteria from the Research Integrity Assessment tool and the Trustworthiness in RAndomised Controlled Trials checklist. 10-12”
From Recommendations from the 2023 international evidence-based guideline for the assessment and management of polycystic ovary syndrome (2023) · cites this paper as [12]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Integrity assessment was integrated into guideline evidence synthesis processes and followed the Research Integrity in Guideline Development (RIGID) framework, with studies assessed against criteria from the Research Integrity Assessment (RIA) tool and the Trustworthiness in RAndomised Controlled Trials (TRACT) checklist (10-12).”
From Recommendations From the 2023 International Evidence-based Guideline for the Assessment and Management of Polycystic Ovary Syndrome (2023) · cites this paper as [12]
The cited source says what the claim attributes to it.
In the citing paper
“A variety of trustworthiness tools have recently been proposed (11–15), although they differ in terms of their development, content and structure, and the appropriateness of some of the trustworthiness checks that have been included in some of these tools is unclear (16).”
From INSPECT-SR: a tool for assessing trustworthiness of randomised controlled trials (2025) · cites this paper as [11]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“However, none of these have involved a comprehensive evaluation and subsequent selection of potential trustworthiness checks.”
From Assessing the feasibility and impact of clinical trial trustworthiness checks via an application to Cochrane Reviews: Stage 2 of the INSPECT-SR project (2025) · cites this paper as [10]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Checklist to assess Trustworthiness in RAndomised Controlled Trials (TRACT checklist): concept proposal and pilot > Abstract > Outcome
“Each item can be answered as either no concerns, some concerns/no information, or major concerns.”
§ Checklist to assess Trustworthiness in RAndomised Controlled Trials (TRACT checklist): concept proposal and pilot > Using the checklist
“The TRACT checklist (Additional file 1: Table S1) is designed to efficiently highlight the areas and as well as levels of concern within each domain via the item ratings – either ‘No Concerns’, ‘Some Concerns’/’Not Applicable’, or ‘Major Concerns’.”
Citations
Cites
CONSORT 2010 statement: Updated guidelines for reporting parallel group randomised trials
Schulz, K. F., Altman, D. G., & Moher, D. (2010) · Journal of Pharmacology and Pharmacotherapeutics · doi:10.4103/0976-500x.72352 · Cited by 2,165
Fanelli, D. (2009) · PLoS ONE · doi:10.1371/journal.pone.0005738 · Cited by 1,911 · cited 1×
“The prevalence of researchers who have been involved in scientific misconduct is estimated to be around 2% [2] and, for randomized clinical trials (RCTs), this percentage is likely to be higher.”
BackgroundCONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
the CONSORT Group, Schulz, K. F., Altman, D. G., & Moher, D. (2010) · Trials · doi:10.1186/1745-6215-11-32 · Cited by 1,871
Hariton, E., & Locascio, J. J. (2018) · BJOG: An International Journal of Obstetrics & Gynaecology · doi:10.1111/1471-0528.15199 · Cited by 1,604
Guidance for Developers of Health Research Reporting Guidelines
Moher, D., Schulz, K. F., Simera, I., & Altman, D. G. (2010) · PLoS Medicine · doi:10.1371/journal.pmed.1000217 · Cited by 1,437 · cited 1×
“Methods Development of our screening tool was adapted from the four-stage approach proposed by Moher and colleagues: define the scope, review the evidence base, suggest a list of items from piloting, and hold a consensus process [18].”
MethodSystematic reviews and meta-analysis: Understanding the best evidence in primary healthcare
Ganeshkumar, P., & Gopalakrishnan, S. (2013) · Journal of Family Medicine and Primary Care · doi:10.4103/2249-4863.109934 · Cited by 743
Differential dropout and bias in randomised controlled trials: when it matters and when it may not
Bell, M. L., Kenward, M. G., Fairclough, D. L., & Horton, N. J. (2013) · BMJ · doi:10.1136/bmj.e8668 · Cited by 460 · cited 1×
“determined that equal number of drop-outs rates do not guarantee unbiased results in RCTs [21].”
ResultThe analysis of 168 randomised controlled trials to test data integrity
Carlisle, J. B. (2012) · Anaesthesia · doi:10.1111/j.1365-2044.2012.07128.x · Cited by 255
False individual patient data and zombie randomised controlled trials submitted to Anaesthesia
Carlisle, J. B. (2021) · Anaesthesia · doi:10.1111/anae.15263 · Cited by 173
The Evolution of Trial Registries and Their Use to Assess the Clinical Trial Enterprise
Dickersin, K. (2012) · JAMA · doi:10.1001/jama.2012.4230 · Cited by 127
Cited by
Teede, H. J., Tay, C. T., Laven, J. J. E., Dokras, A., Moran, L. J., Piltonen, T. T., Costello, M. F., Boivin, J., Redman, L. M., Boyle, J. A., et al. (2023) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/clinem/dgad463 · Cited by 946
Teede, H. J., Tay, C. T., Laven, J. J. E., Dokras, A., Moran, L. J., Piltonen, T. T., Costello, M. F., Boivin, J., Redman, L. M., Boyle, J. A., et al. (2023) · European Journal of Endocrinology · doi:10.1093/ejendo/lvad096 · Cited by 525
“Category includes evidence-based (sufficient evidence in PCOS) or consensus (insufficient evidence in PCOS, also evidence in general or relevant populations was considered) recommendations and accompanying practice points (implementation considerations) (Table 1).”
Self-citationTeede, H. J., Tay, C. T., Laven, J., Dokras, A., Moran, L. J., Piltonen, T. T., Costello, M. F., Boivin, J., M. Redman, L., A. Boyle, J., et al. (2023) · Fertility and Sterility · doi:10.1016/j.fertnstert.2023.07.025 · Cited by 417
Self-citationTeede, H. J., Tay, C. T., Laven, J., Dokras, A., Moran, L. J., Piltonen, T. T., Costello, M. F., Boivin, J., Redman, L. M., Boyle, J. A., et al. (2023) · Human Reproduction · doi:10.1093/humrep/dead156 · Cited by 244
“Integrity assessment was integrated into guideline evidence synthesis processes and followed the Research Integrity in Guideline Development (RIGID) framework, with studies assessed against criteria from the Research Integrity Assessment (RIA) tool and the Trustworthiness in RAndomised Controlled Trials (TRACT) checkl…”
MethodSelf-citationInterventions for the management of long covid (post-covid condition): living systematic review
Zeraatkar, D., Ling, M., Kirsh, S., Jassal, T., Shahab, M., Movahed, H., Talukdar, J. R., Walch, A., Chakraborty, S. P., Turner, T., et al. (2024) · BMJ · doi:10.1136/bmj-2024-081318 · Cited by 149
“63 This checklist includes 19 items in seven domains: governance, author group, plausibility of intervention, timeframe, dropouts, baseline characteristics, and outcomes.”
Metadata
- Authors
- Mol, B. W., Lai, S., Rahim, A., Bordewijk, E. M., Wang, R., van Eekelen, R., Gurrin, L. C., Thornton, J. G., van Wely, M., & Li, W.
- Affiliations
- University of Aberdeen (GB)
- Monash University (AU)
- Amsterdam UMC Location University of Amsterdam (NL)
- The University of Melbourne (AU)
- University of Nottingham (GB)
- Corresponding author
- Gurrin, Lyle C.
- ORCID
- Venue
- Research Integrity and Peer Review
- Published
- 2023-06-20
- Volume
- 8
- Issue
- 1
- Pages
- 6-6
- ISSN
- 2058-8615 (ISSN-L)
- PMC
- PMC10280869
- OpenAlex
- W4381250920
- Cited by
- 154
- References
- 24
- Citations by year
- 2026: 48 · 2025: 49 · 2024: 43 · 2023: 14
- FWCI
- 15.81
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Research Integrity and Peer Review (publishedVersion, cc-by)
- Research Square (submittedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Pure Amsterdam UMC (publishedVersion, other-oa)
- Aberdeen University Research Archive (Aberdeen University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Nursing › Issues, ethics and legal aspects › Pharmaceutical industry and healthcare
- Social Sciences › Education › Academic integrity and plagiarism
- Keywords
- trustworthiness assessment, randomized controlled trials, screening tool, research integrity, questionable research practices, peer review, individual patient data, dropout rates
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-30, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-30 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Checklist to assess Trustworthiness in RAndomised Controlled Trials (TRACT checklist): concept proposal and pilot. https://app.cochat.ai/papers/10.1186/s41073-023-00130-8
- The paper
- Mol, B. W., Lai, S., Rahim, A., Bordewijk, E. M., Wang, R., van Eekelen, R., Gurrin, L. C., Thornton, J. G., van Wely, M., & Li, W. (2023). Checklist to assess Trustworthiness in RAndomised Controlled Trials (TRACT checklist): concept proposal and pilot. Research Integrity and Peer Review. https://doi.org/10.1186/s41073-023-00130-8
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.