A Case-Control Comparison of Retracted and Non-Retracted Clinical Trials: Can Retraction Be Predicted?
Open accessThe DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
Abstract
Does scientific misconduct severe enough to result in retraction disclose itself with warning signs? We test a hypothesis that variables in the results section of randomized clinical trials (RCTs) are associated with retraction, even without access to raw data. We evaluated all English-language RCTs retracted from the PubMed database prior to 2011. Two controls were selected for each case, matching publication journal, volume, issue, and page as closely as possible. Number of authors, subjects enrolled, patients at risk, and patients treated were tallied in cases and controls. Among case RCTs, 17.5% had ≤2 authors, while 6.3% of control RCTs had ≤2 authors. Logistic regression shows that having few authors is associated with retraction (p < 0.03), although the number of subjects enrolled, patients at risk, or treated patients is not. However, none of the variables singly, nor all of the variables combined, can reliably predict retraction, perhaps because retraction is such a rare event. Exploratory analysis suggests that retraction rate varies by medical field (p < 0.001). Although retraction cannot be predicted on the basis of the variables evaluated, concern is warranted when there are few authors, enrolled subjects, patients at risk, or treated patients. Ironically, these features urge caution in evaluating any RCT, since they identify studies that are statistically weaker.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures


Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“The retracted trials (89.0%) were not usually multicentre studies, typically received academic funding or failed to report funding sources (89.5%), and were more likely to be unregistered (81.3%); these findings are in accordance with previous studies.14 98 99”
From Investigating the impact of trial retractions on the healthcare evidence ecosystem (VITALITY Study I): retrospective cohort study (2025) · cites this paper as [99]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
1 retracted
Bezwoda, W. R., Seymour, L., & Dansey, R. D. (1995) · Journal of Clinical Oncology · doi:10.1200/jco.1995.13.10.2483 · Cited by 400 · cited 2×
“Werner Bezwoda claimed to show that high-dose chemotherapy with stem cell rescue was a useful treatment for breast cancer [17].”
BackgroundRetractedWhy Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 1×
“We concur that the best overall predictor of the replication of published work is the sample size of subjects enrolled [26], not only because a large sample size is more likely to lead to a replicable finding [27], but also because a large sample size may be harder to fabricate.”
BackgroundContradicted and Initially Stronger Effects in Highly Cited Clinical Research
Ioannidis, J. P. A. (2005) · JAMA · doi:10.1001/jama.294.2.218 · Cited by 1,465 · cited 1×
“We concur that the best overall predictor of the replication of published work is the sample size of subjects enrolled [26], not only because a large sample size is more likely to lead to a replicable finding [27], but also because a large sample size may be harder to fabricate.”
BackgroundStadtmauer, E. A., O'Neill, A., Goldstein, L. J., Crilley, P. A., Mangan, K. F., Ingle, J. N., Brodsky, I., Martino, S., Lazarus, H. M., Erban, J. K., et al. (2000) · New England Journal of Medicine · doi:10.1056/nejm200004133421501 · Cited by 460
Retractions in the scientific literature: is the incidence of research fraud increasing?
Steen, R. G. (2011) · Journal of Medical Ethics · doi:10.1136/jme.2010.040923 · Cited by 405 · cited 1×
“What should balance these factors is an ethical consideration for the wellbeing of the patient, since fraudulent clinical research may put patients at risk [2–5].”
BackgroundSelf-citationBergh, J., Wiklund, T., Erikstein, B., Lidbrink, E., Lindman, H., Malmström, P., Kellokumpu-Lehtinen, P., Bengtsson, N.-O., Söderlund, G., Anker, G., et al. (2000) · The Lancet · doi:10.1016/s0140-6736(00)02841-5 · Cited by 339 · cited 1×
“Subsequent clinical trials showed that high-dose treatment with stem cell rescue did little [20] or nothing to extend survival of breast cancer patients [21–24], though such treatment was associated with more frequent adverse events [22], at least in the short term [25].”
BackgroundHigh-Dose Chemotherapy with Hematopoietic Stem-Cell Rescue for High-Risk Breast Cancer
Rodenhuis, S., Bontenbal, M., Beex, L. V. A. M., Wagstaff, J., Richel, D. J., Nooij, M. A., Voest, E. E., Hupperets, P., van Tinteren, H., Peterse, H. L., et al. (2003) · New England Journal of Medicine · doi:10.1056/nejmoa022794 · Cited by 258 · cited 1×
“Subsequent clinical trials showed that high-dose treatment with stem cell rescue did little [20] or nothing to extend survival of breast cancer patients [21–24], though such treatment was associated with more frequent adverse events [22], at least in the short term [25].”
BackgroundRetractions in the scientific literature: do authors deliberately commit research fraud?
Steen, R. G. (2011) · Journal of Medical Ethics · doi:10.1136/jme.2010.038125 · Cited by 217 · cited 2×
“Papers that met these criteria were further evaluated, using an established analytic approach [3].”
BackgroundMethodSelf-citationTallman, M. S., Gray, R., Robert, N. J., LeMaistre, C. F., Osborne, C. K., Vaughan, W. P., Gradishar, W. J., Pisansky, T. M., Fetting, J., Paietta, E., et al. (2003) · New England Journal of Medicine · doi:10.1056/nejmoa030684 · Cited by 191 · cited 1×
“Subsequent clinical trials showed that high-dose treatment with stem cell rescue did little [20] or nothing to extend survival of breast cancer patients [21–24], though such treatment was associated with more frequent adverse events [22], at least in the short term [25].”
BackgroundHigh-dose chemotherapy for high-risk primary breast cancer: an on-site review of the Bezwoda study
Weiss, R. B., Rifkin, R. M., Stewart, F. M., Theriault, R. L., Williams, L. A., Herman, A. A., & Beveridge, R. A. (2000) · The Lancet · doi:10.1016/s0140-6736(00)90024-2 · Cited by 161 · cited 3×
“Some years ago, a very controversial issue in medical oncology was the use of high-dose chemotherapy to treat metastatic breast cancer, with autologous stem cell transplantation after ablation of the patients’ immune system [16].”
BackgroundRetractions in the medical literature: how many patients are put at risk by flawed research?
Steen, R. G. (2011) · Journal of Medical Ethics · doi:10.1136/jme.2011.043133 · Cited by 156 · cited 1×
“Such data were extracted, as follows [5]:”
Self-citationAre these data real? Statistical methods for the detection of data fabrication in clinical trials
Al-Marzouki, S., Evans, S., Marshall, T., & Roberts, I. (2005) · BMJ · doi:10.1136/bmj.331.7511.267 · Cited by 131 · cited 2×
“Specifically, differences between treated and control groups at baseline following “randomization” were large enough to suggest that patients had actually not been allocated randomly [9].”
BackgroundMethodInfluentialSuspected research fraud: difficulties of getting at the truth
White, C. (2005) · BMJ · doi:10.1136/bmj.331.7511.281 · Cited by 125
Trikalinos, N. A., Evangelou, E., & Ioannidis, J. P. A. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.11.019 · Cited by 100 · cited 3×
“The Trikalinos study concluded that the number of authors did not differ between fraudulent and non-fraudulent papers [15], yet we found that there was a difference for RCTs, though that difference was too small to be useful as a predictor.”
BackgroundResultInfluentialInvestigating the previous studies of a fraudulent author
Smith, R. (2005) · BMJ · doi:10.1136/bmj.331.7511.288 · Cited by 71 · cited 2×
“The Reuben and Boldt cases [13,14] illustrate a common pattern; in-depth investigation inspired by a single retracted paper reveals that a first author has engaged in questionable practices, including other fraudulent papers, sometimes dating back many years [11].”
BackgroundWeiss, R. B., Gill, G. G., & Hudis, C. A. (2001) · Journal of Clinical Oncology · doi:10.1200/jco.2001.19.11.2771 · Cited by 62 · cited 2×
“Yet, it gradually emerged that the key paper [17] was marred by fraud and that at least nine additional papers were also problematic [18].”
BackgroundPerioperative Analgesia: What Do We Still Know?
White, P. F., Kehlet, H., & Liu, S. (2009) · Anesthesia & Analgesia · doi:10.1213/ane.0b013e3181a16835 · Cited by 50 · cited 1×
“The Reuben and Boldt cases [13,14] illustrate a common pattern; in-depth investigation inspired by a single retracted paper reveals that a first author has engaged in questionable practices, including other fraudulent papers, sometimes dating back many years [11].”
BackgroundHarrison, W. T. A., Simpson, J., & Weil, M. (2009) · Acta Crystallographica Section E Structure Reports Online · doi:10.1107/s1600536809051757 · Cited by 45
Hanrahan, E. O., Broglio, K. R., Frye, D., Buzdar, A. U., Theriault, R. L., Valero, V., Booser, D. J., Singletary, S. E., Strom, E. A., Gajewski, J., et al. (2006) · Cancer · doi:10.1002/cncr.21906 · Cited by 33 · cited 1×
“Subsequent clinical trials showed that high-dose treatment with stem cell rescue did little [20] or nothing to extend survival of breast cancer patients [21–24], though such treatment was associated with more frequent adverse events [22], at least in the short term [25].”
BackgroundPeppercorn, J. M., Herndon, J. E., Kornblith, A. B., Peters, W. P., Ahles, T. A., Vredenburgh, J. J., Schwartz, G. N., Shpall, E. J., Hurd, D. D., Holland, J. C., et al. (2005) · Cancer · doi:10.1002/cncr.21363 · Cited by 32
Statistical Techniques to Detect Fraud and Other Data Irregularities in Clinical Questionnaire Data
Taylor, R. N., McEntegart, D. J., & Stillman, E. C. (2002) · Drug Information Journal · doi:10.1177/009286150203600115 · Cited by 30 · cited 1×
“Statistical methods have been devised to detect fraud in clinical questionnaire data [8] and in clinical trial data [6], and statistical comparison of a retracted randomized clinical trial (RCT) to a non-retracted RCT found that statistical features of data in the retracted RCT were so strongly suggestive of data fabr…”
MethodShafer, S. L. (2011) · Anesthesia & Analgesia · doi:10.1213/ane.0b013e31821a8542 · Cited by 30 · cited 1×
“The Reuben and Boldt cases [13,14] illustrate a common pattern; in-depth investigation inspired by a single retracted paper reveals that a first author has engaged in questionable practices, including other fraudulent papers, sometimes dating back many years [11].”
BackgroundDeceit and fraud in medical research
Jaffer, U., & Cameron, A. E. P. (2006) · International Journal of Surgery · doi:10.1016/j.ijsu.2006.02.004 · Cited by 29 · cited 1×
“Factors that potentially promote research misconduct in clinical research are legion; financial gain, personal fame, scientific hubris and the competitive nature of research funding may all contribute [1].”
BackgroundAntisepsis in the Time of Antibiotics: Following in the Footsteps of John Snow and Joseph Lister
Hollmann, M. W., & Roy, R. C. (2010) · Anesthesia & Analgesia · doi:10.1213/ane.0b013e3181fa354e · Cited by 12
Hanrahan, E. O., Frye, D., Buzdar, A. U., Theriault, R. L., Booser, D. J., Valero, V., Singletary, S. E., Gajewski, J., Champlin, R. E., & Hortobágyi, G. N. (2005) · Journal of Clinical Oncology · doi:10.1200/jco.2005.23.16_suppl.663 · Cited by 3
Buyse, M. (1999) · doi:10.1002/(sici)1097-0258(19991230)18:24<3435::aid-sim365>3.0.co;2-o · cited 5×
“Multi-center clinical trials, in particular, offer an opportunity to check the plausibility of submitted clinical data, by comparing data submitted by different centers [6].”
BackgroundMethodInfluential
Research works printed but not linked: 1
- Steen, R.G. Retractions in the medical literature: Who is responsible for scientific integrity? AMWA J. 2011, 26, 2-7.
Cited by
The visibility of scientific misconduct: A review of the literature on retracted journal articles
Hesselmann, F., Graf, V., Schmidt, M., & Reinhart, M. (2017) · Current Sociology · doi:10.1177/0011392116663807 · Cited by 153
“(Continued) Publication Data sampling strategies Sample size, retraction rates Reasons for retraction Snodgrass and Pfeifer, 1992 PubMed 1975–1991 Publication type: ‘Retraction of Publication’ 97 retraction notices Steen and Hamer, 2014 PubMed ?”
BackgroundMethodSelf-correction in biomedical publications and the scientific impact
Gasparyan, A. Y., Ayvazyan, L., Akazhanov, N. A., & Kitas, G. D. (2014) · Croatian Medical Journal · doi:10.3325/cmj.2014.55.61 · Cited by 73
Xu, C., Fan, S., Tian, Y., Liu, F., Furuya-Kanamori, L., Clark, J., Zhang, C., Li, S., Lin, L., Chu, H., et al. (2025) · BMJ · doi:10.1136/bmj-2024-082068 · Cited by 39
Schneider, J., Woods, N. D., Proescholdt, R., Fu, Y., & The RISRS Team (2021) · MetArXiv (OSF Preprints) · doi:10.31222/osf.io/ms579 · Cited by 18
Research on retractions: A systematic review and research agenda
Mertkan, Ş., Mills, D., Takır, A., & Sarıkaya, E. E. (2025) · Accountability in Research · doi:10.1080/08989621.2025.2542203 · Cited by 11
Metadata
- Authors
- Steen, R., & Hamer, R.
- Affiliations
- Medical Communications Consultants LLC, 103 Van Doren Place, Chapel Hill, NC 27517, USA
- University of North Carolina at Chapel Hill (US)
- Corresponding author
- Steen, R.
- ORCID
- Steen, R.
- Venue
- Publications
- Publisher
- MDPI AG
- Published
- 2014-01-27
- Volume
- 2
- Issue
- 1
- Pages
- 27-37
- ISSN
- 2304-6775 (ISSN-L)
- OpenAlex
- W2044381503
- Cited by
- 7
- References
- 27
- Citations by year
- 2025: 3 · 2024: 1 · 2021: 1 · 2016: 1 · 2014: 1
- FWCI
- 1.86
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Publications (publishedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Social Sciences › Education › Academic integrity and plagiarism
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Keywords
- retraction, research misconduct, randomized controlled trials, number of authors, study size, warning signs, research integrity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.