Assessing Research Misconduct in Randomized Controlled Trials

May combine two claims.

In an examination of all RCTs by two authors from the same university, 21 of 35 evaluated trials had more than 10 identical or extremely similar values between trials, revealing a pattern of self-plagiarism of data.

“Using pairwise comparisons of baseline characteristics and outcome tables, for 21 of the 35 evaluated trials, more than 10 identical or extremely similar values between trials were observed, uncovering a pattern of self-plagiarism of data.”

• cites [14] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

In the authors' experience, misconduct cases are likely to meet several checklist criteria.

“In our experience, cases of misconduct are likely to meet several of the criteria in the checklist.”

May combine two claims.

The median time from publication to retraction was 11.2 years, with an interquartile range of 4.3 to 13.3 years.

“The median duration between publication to retraction was 11.2 years (interquartile range 4.3-13.3).”

May combine two claims.

There is no consistent protocol or motivation for monitoring feedback on articles after publication.

“There is no consistent protocol nor motivation to monitor feedback on articles once they are published”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Placebo preparation is technically challenging and usually requires involvement of the intervention manufacturers.

“Placebo preparation is technically challenging and usually requires the involvement of manufacturers of interventions.”

Classified as probably not a substantial claim.

The authors could not determine the most appropriate threshold for triggering an investigation.

“We have not been able to determine the most appropriate threshold to trigger an investigation.”

May combine two claims.

Not all research numbers occur naturally, and some naturally occurring numbers may have unequal digit distributions, such as rounded laboratory readings.

“However, not all numbers in research occur naturally and some naturally occurring numbers may have unequal distributions of digits; for example, laboratory readings might be rounded.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The REAPPRAISED checklist supports systematic evaluation through 11 categories and can be used during manuscript review and postpublication evaluation.

“This checklist facilitates systematic evaluation through 11 categories and, as the authors suggested, can be used during both manuscript review and postpublication evaluation.”

• cites [15]Not checked yet

  Not checked in the free run.

May combine two claims.

A validation study found that applying Benford's law to RCTs is unlikely to miss fraud cases (100% sensitivity) but may misclassify many controls as cases (poor specificity).

“A validation study using cases of fraud and nonverified controls found that applying Benford's law to RCTs is unlikely to miss cases (100% sensitivity) but may misclassify many controls as cases (poor specificity).23”

• cites [23] sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

The COPE guidelines use a rigid branching logic and depend heavily on responses from authors and their institutions.

“The COPE guidelines follow a rigid branching logic and rely heavily on responses from authors and their institutions.”

• AssumptionRests on an assumption: In most research-misconduct investigations, authors and their institutions do not respond adequately or in a timely manner.

May combine two claims.

GRIMMER can test variances, standard deviations, and standard errors.

“This method can test variances, SDs, and standard errors.”

• cites [20] sourceNot checked yet

  Not checked in the free run.

• cites [21] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Benford's law describes digit distributions in naturally occurring numbers: first digits follow a special distribution with 1, 2, and 3 accounting for more than 60%, while last digits approach uniform distribution.

“Benford's law describes the distribution of the digits in naturally occurring numbers.23 The first digits tend to follow a special distribution where digits 1, 2, and 3 account for more than 60% of the total probability distribution.The last digits, however, approach uniform distribution.”

• cites [23] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

Few people have enough knowledge to detect data-integrity problems in RCTs, and no overview of available detection methods exists.

“However, few people are equipped with adequate knowledge to detect data integrity issues in RCTs and there is no overview of available methods that disclose these problems.”

Probably not a substantial claim: may not pass S2 (checkable).

Judgement using common sense, the literature, and local data can be very helpful in assessing data integrity.

“More importantly, judgement according to common sense, the literature, and local data can be extremely helpful.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Some checklist criteria carry more weight than others; any issue involving author group, intervention, or timeframe could be enough to question a study's veracity.

“However, some criteria have more weight than others; for example, any issue regarding author group, intervention, and timeframe could offer enough caution to question the veracity of the study.”

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

Baseline characteristics and outcomes can provide insight into data integrity and should be assessed methodologically, including for extreme baseline balance and consistency between outcomes.

“Information in baseline characteristics and outcomes can offer insight into integrity.They should be assessed from a methodologic point of view such as extreme balance for baseline characteristics and consistency between outcomes.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Grey et al. formalized assessment of individual articles, not limited to RCTs, as the REAPPRAISED checklist.

“Grey et al formalized the assessment of individual articles, not restricted to RCTs, as the “REAPPRAISED” checklist.15”

• cites [15]Not checked yet

  Not checked in the free run.

May combine two claims.

Although most biomedical journals endorse the COPE guidelines, they are not implemented consistently.

“Another concern is that the COPE guidelines, although endorsed by most biomedical journals, are not implemented consistently.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

These registration and ethics problems are usually found in trials determined to be fabricated.

“These issues are usually found in trials that were determined to be fabricated.”

May combine two claims.

SPRITE complements GRIMMER, has no sample-size limitation, can handle per-cell sample sizes in the thousands, and considers the range of possible raw values.

“SPRITE complements GRIMMER tests.It does not have a sample size limitation and can be used with per-cell sample sizes in the thousands.SPRITE also takes into consideration the range of possible values of the raw data.”

• cites [22] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

A survey found that more than 14% of scientists had observed fabrication or falsification at least once among colleagues, while only 2% admitted such behavior themselves.

“A survey found that more than 14% of scientists observed fabrication or falsification at least once among their colleagues but only 2% admitted that they had such behaviors.10”

• cites [10] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based).

Absent or vague descriptions of research ethics may indicate that a trial lacks ethics approval.

“Similarly, absent or vague descriptions of research ethics may indicate that the trial lacks ethics approval.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

A pilot study used the REAPPRAISED checklist as a screening tool for one journal's editorial work, but no formal validation has been reported.

“A pilot study was conducted in which this checklist was used as a screening tool for editorial works of one journal but no formal validation has been reported.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Most investigations of compromised data integrity begin with suspicious clues about implausible study design, conduct, or reporting.

“Most investigations regarding cases of compromised data integrity are triggered by suspicious clues of implausible study design, conduct, or report.”

Probably not a substantial claim: may not pass S3 (load-bearing).

SPRITE reconstructs potential discrete data sets using the mean, standard deviation, sample size, and lower and upper bounds of item values.

“SPRITE (Sample Parameter Reconstruction via Iterative Techniques) is a technique for reconstructing potential discrete data sets using basic summary information about a sample, namely the mean, the SD, the sample size, and the lower and upper bounds of the range of item values.22”

• cites [22] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Several methods can assess data integrity using summary statistics without individual participant data, but abnormalities may reflect honest error, poor reporting, or misconduct.

“Several methods can be applied to assess data integrity using summary statistics in publications without individual participant data available (Table 2); however, abnormalities found with summary statistics could be explained by honest error, poor reporting, or misconduct.”

Probably not a substantial claim: may not pass S4 (based).

Absent or retrospective trial registration suggests poor transparency of the trial plan.

“Absent or retrospective registration suggests poor transparency of the trial plan.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The GRIMMER test assesses whether reported measures of variability are possible given the sample size and granularity.

“The GRIMMER (Granularity-Related Inconsistency of Means Mapped to Error Repeats) test allows for testing whether reported measures of variability are possible by assessing whether a reported statistic is consistent with the sample size and granularity.20,21”

• cites [20] sourceNot checked yet

  Not checked in the free run.

• cites [21] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Fabricated trials frequently report zero participants lost to follow-up or loss to follow-up that yields perfectly balanced analysis numbers.

“Besides implausible time, we also noticed that fabricated trials frequently reported zero participants lost to follow-up or reported loss to follow-up leading to perfectly balanced numbers for analysis (eg, three groups of 50; two groups of 100).”

Probably not a substantial claim: may not pass S4 (based).

In obstetrics trials, gestational age and birthweight summary values should be reasonably correlated because fraudulent researchers may ignore necessary correlations between these related parameters.

“For example, when assessing obstetrics trials, one could assess whether summary values of gestational age and birthweight are reasonably correlated because researchers who commit fraud may ignore necessary correlations between these two related parameters.”

Probably not a substantial claim: may not pass S3 (load-bearing).

A limitation of GRIMMER software is that it only works for sample sizes of 99 or less.

“A limitation of the software is that it only works for a sample size of 99 or less.”

• cites [20] sourceNot checked yet

  Not checked in the free run.

• cites [21] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Authors who commit fraud may create variables that are too similar for unrelated variables, which can be assessed with coefficients of variation.

“Authors who commit fraud may come up with variables that are too similar for unrelated variables.This can be assessed with coefficients of variation.38”

• cites [38] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

There is ongoing debate about the importance of underlying assumptions, including independent variables, exclusive simple randomization, no rounding of summary statistics, and no publication bias.

“However, there is an ongoing debate about the importance of the underlying assumptions including independent variables, exclusive use of simple randomization, no rounding of summary statistics, and no publication bias.29,30”

• cites [29] sourceNot checked yet

  Not checked in the free run.

• cites [30] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

When manipulating data, people tend to avoid values far from the mean because such values are more noticeable.

“When manipulating data, people tend to avoid values that are far from the mean as they are more noticeable.”

Probably not a substantial claim: may not pass S4 (based).

An increasing number of methods can be used to assess data integrity in randomized controlled trials.

“A growing number of methods can be applied to assess data integrity in RCTs.”

May combine two claims.

No dates should occur before first recruitment, and no events should happen after death or the end of the study.

“For example, no dates should occur before the first recruitment, and, similarly, no events should happen after death or the end of the study.36”

• cites [36] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Trial registration and the CONSORT statement are endorsed by most journals as prepublication measures to safeguard RCT trustworthiness.

“Measures to safeguard the trustworthiness of RCTs before publication, mainly trial registration and the CONSORT (Consolidated Standards of Reporting Trials) statement, are endorsed by most journals.”

Classified as probably not a substantial claim. May combine two claims.

CONSORT aims to promote transparent reporting of trial methodology, which is different from detecting data fabrication.

“The latter, however, aims to promote transparent reporting of trial methodology, which is different from the detection of data fabrication.”

May combine two claims.

Fabricated data tend to have no or few missing values, whereas 100% follow-up is extremely rare in real trials.

“Also, fabricated data tend to be perfect in terms of no or few missing values, whereas, in real trials, 100% follow-up is extremely rare.35”

• cites [35] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

It is difficult to fabricate several correlated variables that remain consistent with real data.

“It is difficult to fabricate several variables that are correlated and be consistent with real data.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Compulsory prospective registration for clinical trials was launched 17 years ago.

“The requirement of compulsory prospective registration for any clinical trials was launched 17 years ago.9”

• cites [9] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Further simulation studies suggest that only rounding could significantly alter the expected distribution and that continuous baseline characteristics conform to a uniform distribution.

“Reassuringly, further simulation studies suggest that only rounding could significantly alter the expected distribution, 31 and that continuous baseline characteristics conform to uniform distribution.32”

• cites [31] sourceNot checked yet

  Not checked in the free run.

• cites [32] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Real participant recruitment or randomization rates are unlikely to be perfectly constant over time because trials usually have a climbing period.

“Also, the rate of real participant recruitment or randomization is unlikely to be perfectly constant over time as recruitments in trials usually have a “climbing” period.”

Classified as probably not a substantial claim.

Central statistical monitoring compares data from one center with data from other centers using statistical methods.

“Central statistical monitoring refers to comparing data from one center with data from other centers using statistical methods.39”

• cites [39] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Inconsistent retraction practices and failure to state retraction reasons account for the paradoxical finding that some retracted articles are cited more frequently after retraction.

“This explains a paradoxical phenomenon that some retracted articles were cited far more often after their retractions.45”

Probably not a substantial claim: may not pass S4 (based).

Manipulated observations might have different characteristics from true observations.

“As a result, manipulated observations might present different characteristics from true observations.”

• AssumptionThe authors state this as an assumption: In central statistical monitoring, it is assumed that anyone who manipulated data did not have access to trial data from other staff or centers.

May combine two claims.

Because journals do not share submission information with one another, authors who commit research misconduct can easily leave a bad record behind and submit the compromised paper to a different journal.

“Journals do not share information of submissions with their counterparts, which makes it easy for authors who commit research misconduct to walk away from a bad record and submit the compromised paper to another journal.”

• AssumptionRests on an assumption: Journals do not share information about submissions with other journals.

May combine two claims.

Journals' retraction practices are inconsistent, and many retractions do not state the reason for retraction.

“At present, journals' retraction practices are inconsistent and many retractions fail to state the reason for retraction.44”

• cites [44]Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Publications containing falsified data harm the reputation of journals and publishers, misguide future research, and put patients at risk.

“Publications containing falsified data not only harm the reputation of journals and publishers but also misguide future research and put patients at risk.”

Probably not a substantial claim: may not pass S3 (load-bearing).

It might be possible to check whether too many participant visits occurred during weekends or public holidays.

“It might be possible to check whether there are too many participant visits that took place during weekends or public holidays.37”

• cites [37] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based).

The biggest hurdle for individual-participant-data assessments is the unavailability of data for submitted or published trials.

“However, the biggest hurdle for these assessments is the unavailability of data for submitted or published trials.”

May combine two claims.

Some variables should be correlated based on knowledge or common sense; for example, blood loss and hemoglobin drop should be related in a postpartum hemorrhage trial, and the correlation strength should be consistent across treatment groups.

“Some variables should be correlated based on knowledge or common sense; for example, the volume of blood loss and drop in hemoglobin should be related in a trial studying postpartum hemorrhage and, the strength of correlation should be consistent across treatment groups.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Further work estimated expected distributions using rounded summary statistics from a large collection of authentic RCTs.

“Further works were also performed to estimate the expected distributions using rounded summary statistics of a large collection of authentic RCTs.33”

• cites [33] sourceNot checked yet

  Not checked in the free run.