The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index; 2 of the 14 sources its claims cite were checked for support.
Systematic review and statistical analysis of the integrity of 33 randomized controlled trials
Open accessAbstract
Background: Statistical techniques can investigate data integrity in randomized controlled trials (RCTs). We systematically reviewed and analyzed all human RCTs undertaken by a group of researchers, about which concerns have been raised. Methods: We compared observed distributions of p values for between-groups differences in baseline variables, for standardized sample means for continuous baseline variables, and for differences in treatment group participant numbers with the expected distributions. We assessed productivity, recruitment rates, outcome data, textual consistency, and ethical oversight. Results: The researchers were remarkably productive, publishing 33 RCTs over 15 years involving large numbers of older patients with substantial comorbidity, recruited over very short periods. Treatment groups were improbably similar. The distribution of p values for differences in baseline characteristics differed markedly from the expected uniform distribution (p = 5.2 × 10−82). The distribution of standardized sample means for baseline continuous variables and the differences between participant numbers in randomized groups also differed markedly from the expected distributions (p = 4.3 × 10−4, p = 1.5 × 10−5, respectively). Outcomes were remarkably positive, with very low mortality and study withdrawals despite substantial comorbidity. There were very large reductions in hip fracture incidence, regardless of intervention (relative risk 0.22, 95% confidence interval 0.15–0.31, p < 0.0001, range of relative risk 0.10–0.33), that greatly exceed those reported in meta-analyses of other trials. There were multiple examples of inconsistencies between and within trials, errors in reported data, misleading text, duplicated data and text, and uncertainties about ethical oversight. Conclusions: A systematic approach using statistical techniques to assess randomization outcomes can evaluate data integrity, in this case suggesting these RCT results may be unreliable.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
2 of 14 cited sources checked · 1 supported · 0 flagged · accuracy not yet measured
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In both, .85% of people approached were enrolled (A9, A23).”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > RESULTS Productivity and recruitment rates.
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Y. Sato was the first author for 26 RCTs.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > RESULTS Productivity and recruitment rates.
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Considering these practical requirements, it was surprising that there was no statement regarding funding for 30 of the 33 human RCTs.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > RESULTS Productivity and recruitment rates.
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This distribution is highly unlikely to have arisen by chance (p 5 5.2 3 10 282 ).”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > RESULTS Productivity and recruitment rates.
Load-bearingStrength:boosted· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Furthermore, none of the RCTs contained a statement on trial registration, although many trials were published before this became customary.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > RESULTS Productivity and recruitment rates.
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In November 2012, we searched MEDLINE and Embase for all RCTs with Y. Sato or Yoshihiro Sato listed as an author, published in English or Japanese.”
- Also stated in: § METHODS · plain
“After excluding unrelated articles and articles by other individuals with the same name, we identified 33 RCTs in humans (table e-1A).”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > METHODS
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes- Also stated in: § METHODS · plain
None of the sources cited for this claim have been checked yet.
In the paper
“In one, ,10% of potential participants were ineligible despite very restrictive inclusion criteria (A9).”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > RESULTS Productivity and recruitment rates.
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In the 25 RCTs with Y. Sato as the first author, the distribution of p values differed significantly from the expected distribution (p 5 3.8 3 10 2100 ) (figure 1C).”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > RESULTS Productivity and recruitment rates.
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesThe paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In one study (A13), the researchers claimed to have recruited 628 trial participants from a single site.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > 8% (A16, A31).
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesThe paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In a later correction, they stated that there were 2 additional unnamed hospitals.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > 8% (A16, A31).
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passes
- It may state two separate claims as one. 37
- Probably not a substantial claim. 18
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“There are insufficient data for hip fracture outcomes from trials of alphacalcidiol, 26 B 12 /folate, or vitamin K by other investigators, but vitamin K is reported to have no effect on bone density 29 or vertebral fracture risk.30”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > Duplicate data and text.
What the sources say
- Not checked — no full text of the sourcecites [26] 26. Avenell A, Gillespie WJ, Gillespie LD, O'Connell D. Vitamin D and vitamin D analogues for preventing fractures associated with involutional and post-menopausal osteopo- rosis. Cochrane Database Syst Rev 2009:CD000227. source
- Supportedcites [29] 29. Booth SL, Dallal G, Shea MK, Gundberg C, Peterson JW, Dawson-Hughes B. Effect of vitamin K supplementation on bone loss in elderly men and women. J Clin Endocrinol Metab 2008;93:1217-1223. source
Citation 29 is attached to the bone-density claim, not the separate vertebral-fracture claim. The Results report no difference in bone mineral density changes between groups. This finding concerns phylloquinone supplementation in older adults also receiving calcium and vitamin D; the study did not assess fracture risk. (LLM, conf=0.91)
- Partly supportedcites [30] 30. Stevenson M, Lloyd-Jones M, Papaioannou D. Vitamin K to prevent fractures in older women: systematic review and economic evaluation. Health Technol Assess 2009;13:iii- xi, 1-134. DOI 10.1212/WNL.0000000000003387 2016;87;2391-2402 Published Online before print November 9, 2016 Neurology sourceExperimental — accuracy not yet measured
In the Menatetrenone (vitamin K2) section (page unavailable), the review reports no statistically significant reduction in vertebral fracture risk when four trials are pooled. That does not establish that vitamin K has no effect: the review notes that the trials are heterogeneous, and its efficacy-model section says site-specific vertebral data for vitamin K1 are unavailable. The bone-density statement carries a separate citation. (LLM, conf=0.88)
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Of note, these 33 RCTs were published over a 15-year period and included 5,894 participants.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > RESULTS Productivity and recruitment rates.
Strength:plain· not yet calibratedn = 5,894Tests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The researchers were remarkably productive, publishing 33 RCTs over 15 years involving large numbers of older patients with substantial comorbidity, recruited over very short periods.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > Abstract
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In 32 human RCTs, baseline data for all participants in all randomized groups were presented for 513 variables; data were only presented for study completers in one RCT (A1).”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > RESULTS Productivity and recruitment rates.
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Of note, 52% of p values were .0.8, while only 6%, 14%, and 27% of p values were ,0.2, ,0.4, and ,0.6, respectively.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > RESULTS Productivity and recruitment rates.
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Compounding the workload of recruiting for these 3 trials, participants were also reviewed every 4 weeks, and during this period, there was also ongoing intensive follow-up in 3 trials involving 774 participants (A13, A14, A20) and recruitment and intensive follow-up for 2 other trials involving 292 participants (A11, A23).”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > RESULTS Productivity and recruitment rates.
Strength:plain· not yet calibratedn = 774Tests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In one study (A29), it is stated that randomization was stratified by site, yet only one site was mentioned in the methods section.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > 8% (A16, A31).
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The distribution of p values from 918 baseline variables from 13 RCTs in the control dataset was consistent with the expected independent uniform distribution (p 5 0.07, figure e-1), and the SD of the standardized sample means from 726 baseline variables was 1.02, also consistent with the expected distribution (p 5 0.78).”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > RESULTS Productivity and recruitment rates.
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“For example, in a trial in Alzheimer disease (A29), the control group received risedronate and calcium and had a hip fracture rate of 86/1,000 patient-years.Yet, the active treatment arm of another trial in patients with Alzheimer disease (A18) who received risedronate, calcium, and vitamin D had a hip fracture rate of 15/1,000 patient-years.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > RESULTS Productivity and recruitment rates.
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Duplicated text is also apparent when logical errors arise.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > Duplicate data and text.
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The researchers stated that the reason for this misinformation was that “these hospitals were reluctant to have their names in the article.””
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > 8% (A16, A31).
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However in A17, 10 participants withdrew because of death or intercurrent illness, 87 experienced at least one fall, and 8 had a hip fracture.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > 8% (A16, A31).
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Dr. Sato is the first author on 79% of these trials and holds primary oversight of the majority of these publications.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > Abstract
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, comparison of baseline p values and standardized sample means from the RCTs of interest and the control RCTs using bootstrap resampling showed marked differences in distributions (baseline p values: p , 0.001 in all 1,000 comparisons; standardized sample means: median p value 6 3 10 24 , 95% confidence interval 2 3 10 27 to 0.021).”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > RESULTS Productivity and recruitment rates.
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Likewise, in A23, 7 participants withdrew because of death or intercurrent illness, 12 had a hip fracture, and there was an average of 1.5 falls per participant during the trial.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > 8% (A16, A31).
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“(A) The results from 20 trials presumed to use simple randomization, and (B) the results from 10 trials that used block randomization.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > Duplicate data and text.
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“There are examples of strikingly similar text in the reports, including duplicate outcome data.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > Duplicate data and text.
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“By using all the published information regarding the ethics committee approval and the location of the study, it appears that ethical approval was granted by the ethics committee of Futase Social Insurance Hospital in 11 RCTs, Mitate Hospital for 10 RCTs, Keiyu Orthopaedic Hospital for 5 RCTs, multiple hospital committees in 2 RCTs, and in 5 RCTs, the committee could not be identified.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > Duplicate data and text.
Load-bearingStrength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Despite studying frail elderly individuals with substantial comorbidity, table 2 shows that participant withdrawals from the trials were few in number and very well matched between treatment groups.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > RESULTS Productivity and recruitment rates.
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Table 3 shows that the reported mean and SD for fall rates were identical in 3 studies (A4, A6, A9) for both the treatment and control groups, and another study (A19) had identical SD to the earlier studies but the mean differed by l unit for both the treatment and control groups.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > Duplicate data and text.
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
In the paper
“Furthermore, figure 3 also shows that the magnitude of these reductions is substantially greater than those observed in metaanalyses of other trials for these agents, [24][25][26] and the confidence intervals around the pooled result for the researchers' trials do not overlap the confidence intervals around the pooled results for trials conducted by other investigators.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > RESULTS Productivity and recruitment rates.
What the sources say
- Not matchedcites [24][25][26]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- Not matched
A claim is checked only against the cited paper’s full text, never its abstract, and none is available for its sources.
In the paper
“Recently, the Journal of the American Medical Association has issued an expression of concern about one trial (A13), and the Journal of Bone and Mineral Research an expression of concern followed by retraction of another trial (A16), both precipitated by our concerns.23,27,28”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > Duplicate data and text.
What the sources say
- Not checked — no full text of the sourcecites [23] 23. Sato Y, Iwamoto J, Kanoko T, Satoh K. Retraction: "ame- lioration of osteoporosis and hypovitaminosis D by sun- light exposure in hospitalized elderly women with Alzheimer's disease: a randomized controlled trial." J Bone Miner Res 2015;30:2328. source
- Not checked — no full text of the sourcecites [27] 27. Bauchner H, Fontanarosa PB. Expression of Concern: Sato et al. Effect of folate and mecobalamin on hip fractures in patients with stroke: a randomized controlled trial. JAMA. 2005;293(9):1082-1088. JAMA 2015;313: 1914. source
- Not checked — no full text of the sourcecites [28] 28. Sato Y, Iwamoto J, Kanoko T, Satoh K. Amelioration of osteoporosis and hypovitaminosis D by sunlight exposure in hospitalized elderly women with Alzheimer's disease: a randomized controlled trial. J Bone Miner Res 2015; 30:1745. source
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In another study (A15), participants were eligible for inclusion if they had sustained a stroke at least 3 months before the study began, but the mean duration of illness at baseline in both randomized groups was 90 days, or slightly less than 3 months, which appears implausible.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > 8% (A16, A31).
Strength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Some examples of duplicated text are extensive.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > Duplicate data and text.
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In a later correction, they stated that there were 3 additional unnamed hospitals and that the originally stated site actually recruited only 53 of 628 participants, the smallest number by some degree.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > 8% (A16, A31).
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The observed and expected distribution of participant numbers between randomized groups in 30 two-arm randomized controlled trials by the researchers.Trial A11, which was pseudorandomized, is excluded from these analyses.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > Duplicate data and text.
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Likewise, when vitamin D was used in the placebo arm of trials, bone density decreased substantially by 0.9% to 3.2% (A18, A22, A23, A30), whereas when sunlight exposure was studied as active treatment, bone mineral density increased by 2.7% to 3.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > RESULTS Productivity and recruitment rates.
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For all 33 trials, randomization in blocks was stated specifically for 10 RCTs, using computergenerated random numbers for 13 RCTs, no details were reported for 9 RCTs, and one trial was pseudorandomized using alternate allocation.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > RESULTS Productivity and recruitment rates.
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Hip fracture was the most frequently reported fracture outcome, reported for 23 trials.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > METHODS
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For example, in one Neurology paper (A9), the authors state that 109 participants were randomized to one group and 108 to the other.However, the participant flow diagram shows 129 participants in each group with 109 and 108, respectively, completing the trial.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > 8% (A16, A31).
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Similarly, in 2 studies (A15, A18), the researchers claimed to have recruited 280 male patients with stroke and 500 patients with Alzheimer disease, respectively, from a single site.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > 8% (A16, A31).
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Similarly, trials in Parkinson disease in which the control group received placebo plus vitamin D reported very high rates of hip fracture (49, 37, and 55/1,000 patient-years) (A22, A23, A30) but when sunlight exposure was used as the active treatment arm to improve vitamin D status, the hip fracture rate was only 9/1,000 patient-years (A31).”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > RESULTS Productivity and recruitment rates.
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Table 3 shows an example of 2 studies with identical exclusion criteria, but one criterion does not make sense for one trial because it only applies to women, but only men were eligible to participate in the trial.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > Duplicate data and text.
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The expected number of trials with equal participants is 22/30 and with a difference of 2 participants is 5/30, whereas the observed number of trials was 27 and 0, respectively (p 5 0.04).”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > RESULTS Productivity and recruitment rates.
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“A limitation of this work, and the previous work by Carlisle, is that the statistical theory underpinning the methods used has not been fully developed.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > Duplicate data and text.
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, any differences would be small and could not explain the marked departure from the expected distributions that we found.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > Duplicate data and text.
Load-bearingStrength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes? Based: unclearIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“There will be differences in p values and standardized sample means calculated from published summary data compared to values calculated from raw data because of rounding of variables.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > Duplicate data and text.
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“For block randomization, the number of participants in each trial arm is equal in each block.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > METHODS
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A second issue is that some of the baseline variables may not be independent.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > Duplicate data and text.
Strength:hedged· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Similarly, pooling all continuous variables produced a distribution similar to expected for the 366 RCTs from other authors, whereas the distribution was markedly different to expected for the variables from the 168 RCTs of interest.2”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > Duplicate data and text.
What the sources say
- Not checked — no full text of the sourcecites [2] 2. Carlisle JB. The analysis of 168 randomised controlled trials to test data integrity. Anaesthesia 2012;67:521-537. source
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may state two separate claims as one.
In the paper
“Previously, Carlisle analyzed the distribution of standardized sample means from 168 RCTs published by an individual author, the integrity of whose work had been questioned, along with data from 366 RCTs from other authors. 2”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > Duplicate data and text.
What the sources say
- Not checked — no full text of the sourcecites [2] 2. Carlisle JB. The analysis of 168 randomised controlled trials to test data integrity. Anaesthesia 2012;67:521-537. source
Strength:plain· not yet calibratedn = 366Tests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, clustering may not occur in the same direction or magnitude in other trials.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > Duplicate data and text.
Load-bearingStrength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“A randomized group in an RCT can be considered as a random sample from the entire trial population.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > METHODS
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“There were multiple examples of inconsistencies between and within trials, errors in reported data, misleading text, duplicated data and text, and uncertainties about ethical oversight.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > Abstract
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Baseline variables within an RCT are not always independent and may be correlated.However, this interdependence is unlikely to translate into substantial between-group similarities where simple randomization is used.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > METHODS
Load-bearingStrength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For example, if by chance, one randomized group is older than the other, this group may have similar differences in variables correlated with age, such as bone mineral density.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > Duplicate data and text.
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This could lead to clustering of p values and standardized means within an individual RCT.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > Duplicate data and text.
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In fact, only 519 participants (8.8%) did not complete the trials, which is surprising because participants had very high rates of hip fracture (table 2) and substantial background comorbidity, both of which are associated with high mortality.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > RESULTS Productivity and recruitment rates.
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Remarkably positive outcomes.The outcomes reported by the researchers are generally extremely positive and inconsistent with contemporaneous literature.”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > RESULTS Productivity and recruitment rates.
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
In the paper
“He reported that the observed distribution of standardized sample means for weight, height, and age from the 366 control RCTs were consistent with expected (range of standardized SD 0.93-1.06),whereas the distributions of sample means from the 168 RCTs of interest were not (range SD 0.55-0.62).”
§ Systematic review and statistical analysis of the integrity of 33 randomized controlled trials > Duplicate data and text.
What the sources say
- Not checked — no full text of the sourcecites [2] 2. Carlisle JB. The analysis of 168 randomised controlled trials to test data integrity. Anaesthesia 2012;67:521-537. source
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may state two separate claims as one.
The authors state this as an assumption.
Randomization was stratified by site for trials A13 and A15.
“We assumed randomization was stratified by site for these 2 trials.”
The paper relies on this without support.
The researchers' RCTs collectively have substantially influenced relevant systematic reviews.
“This group of authors has published a very large number of RCTs (appendix e-1, table e-1A, references A1-A33 at Neurology.org) that collectively have substantially influenced relevant systematic reviews.”
The authors state this as an assumption.
The trials were effectively funded by the hospitals involved, with substantial trial input from other unnamed hospital staff.
“Given the large size and intensive nature of these trials, and the absence of statements regarding funding, we presume the trials were effectively funded by the hospitals involved, with substantial trial input from other unnamed hospital staff.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The technique has recently identified systematic problems with data in 33 randomised trials by Yoshihiro Sato, who is not an anaesthetist [5].”
From Data fabrication and other reasons for non‐random sampling in 5087 randomised, controlled trials in anaesthetic and general medical journals (2017) · cites this paper as [5]
Experimental — accuracy not yet measuredIn the citing paper
“Fraudulent researchers have so far been found in ad hoc ways, including concerns being raised by whistleblowers and researchers noticing strange patterns whilst reading papers in their field or conducting systematic reviews. 8,9”
From Automated detection of over- and under-dispersion in baseline tables in randomised controlled trials. (2022) · cites this paper as [9]
In the citing paper
“A software called Statcheck can extract reported statistical values from the text to reproduce the P-value for the reported statistical result.19”
From Assessing Research Misconduct in Randomized Controlled Trials (2021) · cites this paper as [19]
In the citing paper
“Finally, and most importantly, it cannot be emphasized enough that scientific misconduct occurs in many disciplines and many countries in the world, including Japan, the UK and the USA, as well as many other countries [23–28].”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Self-check results appear here once an independent check has confirmed them.
Cited works that were retracted
Works this paper cites that were later retracted, from the retraction notices publishers deposit. Major when the notice came out before this paper was published.
Cites a retracted work: 10.1212/wnl.65.9.1513
MajorThe cited work was retracted on 2016-07-12 (notice 10.1212/wnl.0000000000002788), before this paper was published (2016-12-06). Its authors could have known.
Cited work 10.1212/wnl.65.9.1513 · retracted 2016-07-12 · notice 10.1212/wnl.0000000000002788 · this paper published 2016-12-06
“10.1212/WNL.65.9.1513”
Cites a retracted work: 10.1002/jbmr.2725
InfoThe cited work was retracted; the date of the notice is not on record.
Cited work 10.1002/jbmr.2725 · retraction date not known · this paper published 2016-12-06
“10.1002/jbmr.2725”
Citations
Cites
2 retracted
Risedronate therapy for prevention of hip fracture after stroke in elderly women
Poole, K. E. S., Warburton, E. A., & Reeve, J. (2005) · Neurology · doi:10.1212/wnl.65.9.1513 · Cited by 13
RetractedSato, Y., Iwamoto, J., Kanoko, T., & Satoh, K. (2015) · Journal of Bone and Mineral Research · doi:10.1002/jbmr.2725 · Cited by 6
RetractedAvenell, A., Gillespie, W. J., Gillespie, L. D., & O'Connell, D. (2009) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000227.pub3 · Cited by 667
Self-citationWells, G. A., Cranney, A., Peterson, J., Boucher, M., Shea, B., Welch, V. A., Coyle, D., & Tugwell, P. (2008) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd003376.pub3 · Cited by 518
Effect of Calcium Supplementation on Bone Loss in Postmenopausal Women
Reid, I. R., Ames, R. W., Evans, M. C., Gamble, G. D., & Sharpe, S. J. (1993) · New England Journal of Medicine · doi:10.1056/nejm199302183280702 · Cited by 512
Self-citationRandomisation and baseline comparisons in clinical trials
Altman, D. G., & Doré, C. J. (1990) · The Lancet · doi:10.1016/0140-6736(90)90014-v · Cited by 508
Statistics notes: How to randomise
Altman, D. G., & Bland, J. M. (1999) · BMJ · doi:10.1136/bmj.319.7211.703 · Cited by 506
Grey, A. B., Bolland, M. J., Gamble, G. D., Wattie, D. J., Horne, A. M., Davidson, J. S., & Reid, I. R. (2007) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2006-2646 · Cited by 397
Self-citationStevenson, M., Jones, M. L., Nigris, E. D., Brewer, N., Davis, S., & Oakley, J. E. (2005) · Health Technology Assessment · doi:10.3310/hta9220 · Cited by 291
Randomized Controlled Trial of Calcium in Healthy Older Women
Reid, I. R., Mason, B., Horne, A., Ames, R., Reid, H. E., Bava, U., Bolland, M. J., & Gamble, G. D. (2006) · The American Journal of Medicine · doi:10.1016/j.amjmed.2006.02.038 · Cited by 281
Self-citation
Flagged first, then by citations
Research works printed but not linked: 1
- Clarke CE. Efficacy of methyprednisolone pulse therapy on neuroleptic malignant syndrome in Parkinson's disease. J Neurol Neurosurg Psychiatry 2004;75:510–511; author reply 511.
Cited by
Sorted by citations
Interventions for preventing falls in older people in care facilities and hospitals
Cameron, I. D., Dyer, S. M., Panagoda, C. E., Murray, G., Hill, K., Cumming, R. G., & Kerse, N. M. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd005465.pub4 · Cited by 623
Carlisle, J. B. (2017) · Anaesthesia · doi:10.1111/anae.13938 · Cited by 229
“I used three methods to generate p values for individual variables: independent t-test; ANOVA; and Monte Carlo simulations [5], adjusted for the precision to which mean (SD) were reported.”
MethodFusaro, M., Cianciolo, G., Brandi, M. L., Ferrari, S., Nickolas, T. L., Tripepi, G. L., Plebani, M., Zaninotto, M., Iervasi, G., Manna, G. L., et al. (2020) · Nutrients · doi:10.3390/nu12123625 · Cited by 129
“[77] published an update of their previous meta-analysis [78], mainly in light of the results of a review and statistical analysis, made by Bolland and colleagues [79] relative to 33 RCTs which raised serious doubts regarding their integrity and validity but also into consideration that more RCTs have been published i…”
ResultCOVID-19 research: pandemic versus “paperdemic”, integrity, values and risks of the “speed science”
Dinis‐Oliveira, R. J. (2020) · Forensic Sciences Research · doi:10.1080/20961790.2020.1767754 · Cited by 137
Sato, Y., Manabe, S., Kuno, H., & Oizumi, K. (1999) · Journal of Neurology, Neurosurgery & Psychiatry · doi:10.1136/jnnp.66.1.64 · Cited by 131
Retracted
Metadata
- Authors
- Bolland, M. J., Avenell, A., Gamble, G. D., & Grey, A.
- Affiliations
- University of Auckland (NZ)
- Venue
- Neurology
- Published
- 2016-12-06
- Volume
- 87
- Issue
- 23
- Pages
- 2391-2402
- ISSN
- 0028-3878 (ISSN-L)
- 1526-632X
- OpenAlex
- W2549563032
- Semantic Scholar
- 5d35e18505037eccc3b60cf04e14da0063ec2ac2
- Cited by
- 106
- References
- 35
- Citations by year
- 2026: 5 · 2025: 8 · 2024: 13 · 2023: 10 · 2022: 8 · 2021: 8 · 2020: 14 · 2019: 17 · 2018: 9 · 2017: 12 · 2016: 2
- FWCI
- 3.71
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Neurology (publishedVersion)
- PubMed (publishedVersion)
- Funders
- University of Otago
- University of Aberdeen
- Topics
- Social Sciences › Education › Academic integrity and plagiarism
- Physical Sciences › Mathematics › Statistics and Probability › Benford’s Law and Fraud Detection
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Keywords
- data integrity, randomized controlled trials, p-value distribution, baseline characteristics, research misconduct, hip fracture incidence, older patients with comorbidities, ethical oversight
- MeSH terms
- Data Accuracy, Data Interpretation, Statistical, Humans, Meta-Analysis as Topic, Randomized Controlled Trials as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
