Trial Publication after Registration in ClinicalTrials.Gov: A Cross-Sectional Analysis
Open accessThe DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Abstract
BACKGROUND: ClinicalTrials.gov is a publicly accessible, Internet-based registry of clinical trials managed by the US National Library of Medicine that has the potential to address selective trial publication. Our objectives were to examine completeness of registration within ClinicalTrials.gov and to determine the extent and correlates of selective publication. METHODS AND FINDINGS: We examined reporting of registration information among a cross-section of trials that had been registered at ClinicalTrials.gov after December 31, 1999 and updated as having been completed by June 8, 2007, excluding phase I trials. We then determined publication status among a random 10% subsample by searching MEDLINE using a systematic protocol, after excluding trials completed after December 31, 2005 to allow at least 2 y for publication following completion. Among the full sample of completed trials (n = 7,515), nearly 100% reported all data elements mandated by ClinicalTrials.gov, such as intervention and sponsorship. Optional data element reporting varied, with 53% reporting trial end date, 66% reporting primary outcome, and 87% reporting trial start date. Among the 10% subsample, less than half (311 of 677, 46%) of trials were published, among which 96 (31%) provided a citation within ClinicalTrials.gov of a publication describing trial results. Trials primarily sponsored by industry (40%, 144 of 357) were less likely to be published when compared with nonindustry/nongovernment sponsored trials (56%, 110 of 198; p<0.001), but there was no significant difference when compared with government sponsored trials (47%, 57 of 122; p = 0.22). Among trials that reported an end date, 75 of 123 (61%) completed prior to 2004, 50 of 96 (52%) completed during 2004, and 62 of 149 (42%) completed during 2005 were published (p = 0.006). CONCLUSIONS: Reporting of optional data elements varied and publication rates among completed trials registered within ClinicalTrials.gov were low. Without greater attention to reporting of all data elements, the potential for ClinicalTrials.gov to address selective publication of clinical trials will be limited. Please see later in the article for the Editors' Summary.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Clinical trial registries are an important source of information for tracking clinical trials from their inception through to their reporting, and have been used to measure publication bias and outcome reporting bias.”
From Systematic review protocol assessing the processes for linking clinical trial registries and their published results (2016) · cites this paper as [14]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Effect of Rosiglitazone on the Risk of Myocardial Infarction and Death from Cardiovascular Causes
Nissen, S. E., & Wolski, K. (2007) · New England Journal of Medicine · doi:10.1056/nejmoa072761 · Cited by 4,675 · cited 2×
“Similarly unpublished clinical trials of rosiglitazone [3] identified from a company-maintained website and of erythropoiesis-stimulating agents [4] and antidepressants [5] found among data submitted to the FDA revealed important efficacy and safety information to be missing from the medical literature.”
BackgroundSelf-citationSilverstein, F. E., Faich, G., Goldstein, J. L., Simon, L. S., Pincus, T., Whelton, A., Makuch, R., Eisen, G., Agrawal, N. M., Stenson, W. F., et al. (2000) · JAMA · doi:10.1001/jama.284.10.1247 · Cited by 3,154
Publication bias in clinical research
Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R. (1991) · The Lancet · doi:10.1016/0140-6736(91)90201-y · Cited by 3,109 · cited 1×
“Many studies have attempted to evaluate the extent of selective publication in the biomedical literature and found similarly low rates of publication [5,12–24], although none have used, to our knowledge, as large and as broadly representative a registry as ClinicalTrials.”
BackgroundSelective publication of antidepressant trials and its influence on apparent efficacy
Turner, E. H., Matthews, A. M., Linardatos, E., Tell, R. A., & Rosenthal, R. (2008) · New England Journal of Medicine · doi:10.1056/nejmsa065779 · Cited by 2,435 · cited 3×
“Similarly unpublished clinical trials of rosiglitazone [3] identified from a company-maintained website and of erythropoiesis-stimulating agents [4] and antidepressants [5] found among data submitted to the FDA revealed important efficacy and safety information to be missing from the medical literature.”
BackgroundResultPharmaceutical industry sponsorship and research outcome and quality: systematic review
Lexchin, J., Bero, L. A., Djulbegovic, B., & Clark, O. (2003) · BMJ · doi:10.1136/bmj.326.7400.1167 · Cited by 2,147 · cited 1×
“However, we also found significantly different publication rates among study types and primary sponsors, consistent with prior research [11].”
ResultChan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004) · JAMA · doi:10.1001/jama.291.20.2457 · Cited by 1,845 · cited 1×
“Given the documented presence of outcome reporting bias among trials studied in other settings [5,12,13], the potential impact of ClinicalTrials.”
ResultRisk of Cardiovascular Events Associated with Selective COX-2 Inhibitors
Mukherjee, D. (2001) · JAMA · doi:10.1001/jama.286.8.954 · Cited by 1,741 · cited 1×
“Other evidence concerning selective publication is anecdotal, such as the absence of 6 mo of trial data from a key publication describing the efficacy of celecoxib [25,26], the delay of publication for two early trials of rofecoxib until after the medication was withdrawn from the market [27–29], and the aforementione…”
BackgroundSystematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias
Dwan, K., Altman, D. G., Arnaiz, J. A., Bloom, J., Chan, A.-W., Cronin, E., Decullier, E., Easterbrook, P. J., Von Elm, E., Gamble, C., et al. (2008) · PLoS ONE · doi:10.1371/journal.pone.0003081 · Cited by 1,546 · cited 1×
“Many studies have attempted to evaluate the extent of selective publication in the biomedical literature and found similarly low rates of publication [5,12–24], although none have used, to our knowledge, as large and as broadly representative a registry as ClinicalTrials.”
BackgroundClinical Trial Registration: A Statement from the International Committee of Medical Journal Editors
De Angelis, C., Drazen, J. M., Frizelle, F. A., Haug, C., Hoey, J., Horton, R., Kotzin, S., Laine, C., Marusic, A., Overbeke, A. J. P. M., et al. (2004) · New England Journal of Medicine · doi:10.1056/nejme048225 · Cited by 1,241
Dickersin, K. (1992) · JAMA · doi:10.1001/jama.1992.03480030052036 · Cited by 854
Cited by
SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials
Chan, A.-W., Tetzlaff, J. M., Gotzsche, P. C., Altman, D. G., Mann, H., Berlin, J. A., Dickersin, K., Hrobjartsson, A., Schulz, K. F., Parulekar, W. R., et al. (2013) · BMJ · doi:10.1136/bmj.e7586 · Cited by 6,251
Increasing value and reducing waste: addressing inaccessible research
Chan, A.-W., Song, F., Vickers, A., Jefferson, T., Dickersin, K., Gøtzsche, P. C., Krumholz, H. M., Ghersi, D., & van der Worp, H. B. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62296-5 · Cited by 887
Self-citationThe ClinicalTrials.gov Results Database — Update and Key Issues
Zarin, D. A., Tse, T., Williams, R. J., Califf, R. M., & Ide, N. C. (2011) · New England Journal of Medicine · doi:10.1056/nejmsa1012065 · Cited by 838
ResearchMatch: A National Registry to Recruit Volunteers for Clinical Research
Harris, P. A., Scott, K. W., Lebo, L., Hassan, N., Lightner, C., & Pulley, J. (2012) · Academic Medicine · doi:10.1097/acm.0b013e31823ab7d2 · Cited by 387
Publication of NIH funded trials registered in ClinicalTrials.gov: cross sectional analysis
Ross, J. S., Tse, T., Zarin, D. A., Xu, H., Zhou, L., & Krumholz, H. M. (2012) · BMJ · doi:10.1136/bmj.d7292 · Cited by 371
Self-citationNon-publication of large randomized clinical trials: cross sectional analysis
Jones, C. W., Handler, L., Crowell, K. E., Keil, L. G., Weaver, M. A., & Platts-Mills, T. F. (2013) · BMJ · doi:10.1136/bmj.f6104 · Cited by 350
Outcome Reporting Among Drug Trials Registered in ClinicalTrials.gov
Bourgeois, F. T., Murthy, S., & Mandl, K. D. (2010) · Annals of Internal Medicine · doi:10.7326/0003-4819-153-3-201008030-00006 · Cited by 332
“Tracking the existence of clinical trials and a priori primary outcome measures provides the potential to monitor the selective publication of trials and trial results (5, 6).”
Publication bias: what is it? How do we measure it? How do we avoid it?
Song, F., Hooper, & Loke, Y. (2013) · Open Access Journal of Clinical Trials · doi:10.2147/oajct.s34419 · Cited by 275
“gov of a relevant published article.(59) In the sample of .”
Timing and Completeness of Trial Results Posted at ClinicalTrials.gov and Published in Journals
Riveros, C., Dechartres, A., Perrodeau, E., Haneef, R., Boutron, I., & Ravaud, P. (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001566 · Cited by 269
“gov and showed that fewer than half were published [20].”
BackgroundSchmucker, C., Schell, L. K., Portalupi, S., Oeller, P., Cabrera, L., Bassler, D., Schwarzer, G., Scherer, R. W., Antes, G., von Elm, E., et al. (2014) · PLoS ONE · doi:10.1371/journal.pone.0114023 · Cited by 269
Comparison of protocols and registry entries to published reports for randomised controlled trials
Dwan, K., Altman, D. G., Cresswell, L., Blundell, M., Gamble, C. L., & Williamson, P. R. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000031.pub2 · Cited by 243
How evidence‐based medicine is failing due to biased trials and selective publication
Every‐Palmer, S., & Howick, J. (2014) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.12147 · Cited by 223
“However, in reality only 51% of the completed trials in the FDA database were positive, resulting in a 32% overestimation of effect size [54].”
BackgroundComparison of registered and published outcomes in randomized controlled trials: a systematic review
Jones, C. W., Keil, L. G., Holland, W. C., Caughey, M. C., & Platts-Mills, T. F. (2015) · BMC Medicine · doi:10.1186/s12916-015-0520-3 · Cited by 237
Viergever, R. F., & Li, K. (2015) · BMJ Open · doi:10.1136/bmjopen-2015-008932 · Cited by 228
Chen, R., Desai, N. R., Ross, J. S., Zhang, W., Chau, K. H., Wayda, B., Murugiah, K., Lu, D. Y., Mittal, A., & Krumholz, H. M. (2016) · BMJ · doi:10.1136/bmj.i637 · Cited by 227
Self-citationDiscontinuation and non-publication of surgical randomised controlled trials: observational study
Chapman, S. J., Shelton, B., Mahmood, H., Fitzgerald, J. E., Harrison, E. M., & Bhangu, A. (2014) · BMJ · doi:10.1136/bmj.g6870 · Cited by 221
Frequency and reasons for outcome reporting bias in clinical trials: interviews with trialists
Smyth, R. M. D., Kirkham, J. J., Jacoby, A., Altman, D. G., Gamble, C., & Williamson, P. R. (2011) · BMJ · doi:10.1136/bmj.c7153 · Cited by 220
Discontinuation and Nonpublication of Randomized Clinical Trials Conducted in Children
Pica, N., & Bourgeois, F. (2016) · Pediatrics · doi:10.1542/peds.2016-0223 · Cited by 191
Naci, H., Smalley, K. R., & Kesselheim, A. S. (2017) · JAMA · doi:10.1001/jama.2017.9415 · Cited by 195
Transparency in Ecology and Evolution: Real Problems, Real Solutions
Parker, T. H., Forstmeier, W., Koricheva, J., Fidler, F., Hadfield, J. D., Chee, Y. E., Kelly, C. D., Gurevitch, J., & Nakagawa, S. (2016) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2016.07.002 · Cited by 201
Publication of Tumor Marker Research Results: The Necessity for Complete and Transparent Reporting
McShane, L. M., & Hayes, D. F. (2012) · Journal of Clinical Oncology · doi:10.1200/jco.2012.42.6858 · Cited by 185
“gov provided evidence that the trials database could help identify a substantial number of unpublished studies.(57) A public registry specifically for tumor marker studies does not currently exist.”
BackgroundBest Practice in Systematic Reviews: The Importance of Protocols and Registration
Editors, T. P. M. (2011) · PLoS Medicine · doi:10.1371/journal.pmed.1001009 · Cited by 160
“It also allows studies to be traced from inception through to completion and publication [2].”
BackgroundThe Quality of Registration of Clinical Trials
Viergever, R. F., & Ghersi, D. (2011) · PLoS ONE · doi:10.1371/journal.pone.0014701 · Cited by 150
“Although subsequent studies suggest that this has improved to less than two or three percent [33,34,46], more information is required about the intervention than the name.”
BackgroundDickersin, K., & Chalmers, I. (2011) · Journal of the Royal Society of Medicine · doi:10.1258/jrsm.2011.11k042 · Cited by 165
The Quality of Registration of Clinical Trials: Still a Problem
Viergever, R. F., Karam, G., Reis, A., & Ghersi, D. (2014) · PLoS ONE · doi:10.1371/journal.pone.0084727 · Cited by 141
A Plea for Neutral Comparison Studies in Computational Sciences
Boulesteix, A.-L., Lauer, S., & Eugster, M. J. A. (2013) · PLoS ONE · doi:10.1371/journal.pone.0061562 · Cited by 142
“ms with their own areas of expertise, etc. 4 Negative results Comparison studies can be a good vehicle for negative research findings. Publication biases and the necessity to ”accentuate the negative” [8] are w ell-documented in the context of medical and pharmaceutical research. In applied statistics and data analysis…”
BackgroundBell, S., & Smith, C. T. (2014) · Orphanet Journal of Rare Diseases · doi:10.1186/s13023-014-0170-0 · Cited by 145
All trials must be registered and the results published
Chalmers, I., Glasziou, P., & Godlee, F. (2013) · BMJ · doi:10.1136/bmj.f105 · Cited by 131
Zwierzyna, M., Davies, M., Hingorani, A. D., & Hunter, J. (2018) · BMJ · doi:10.1136/bmj.k2130 · Cited by 127
Li, T., Higgins, J. P., & Deeks, J. J. (2019) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9781119536604.ch5 · Cited by 127
Ushering in a New Era of Open Science Through Data Sharing: The Wall Must Come Down
Ross, J. S., & Krumholz, H. M. (2013) · JAMA · doi:10.1001/jama.2013.1299 · Cited by 136
“Nearly half of clinical research trials are never published.(3,4) Moreover, publications are often incomplete, selectively reporting favorable outcomes and infrequently reporting relevant safety findings.”
BackgroundSelf-citationBiomarker studies: a call for a comprehensive biomarker study registry
André, F., McShane, L. M., Michiels, S., Ransohoff, D. F., Altman, D. G., Reis-Filho, J. S., Hayes, D. F., & Pusztai, L. (2011) · Nature Reviews Clinical Oncology · doi:10.1038/nrclinonc.2011.4 · Cited by 118
“gov to increase access to clinical trials information”
Time to publication among completed clinical trials
Ross, J. S., Mocanu, M., Lampropulos, J. F., Tse, T., & Krumholz, H. M. (2013) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2013.136 · Cited by 114
“14 Efficacy only 359 (38) 21 (14-30) Safety only 22 (2) 28 (14-46) Pharmacokinetics 26 (3) 17 (12-22) Trial phasec Phase 1 33 (4) 20 (14-35)”
Self-citationMayo‐Wilson, E., Li, T., Fusco, N., Dickersin, K., & for the MUDS investigators (2018) · Research Synthesis Methods · doi:10.1002/jrsm.1277 · Cited by 114
“gov) are useful for identifying trials, comparing published results with planned outcomes and methods and obtaining results not reported elsewhere.(11,14,15) Registries may also include data about potential harms that cannot be obtained through other sources.”
ResultHuić, M., Marušić, M., & Marušić, A. (2011) · PLoS ONE · doi:10.1371/journal.pone.0025258 · Cited by 114
“In a cross-sectional study of trials in ClinicalTrials.gov registry, Ross et al [28] demonstrated that, while the registration of mandatory registry data was satisfactory, the reporting of optional data for ClinicalTrials.gov but mandatory data for ICMJE, such as primary outcome and trial end date, was unsatisfactory.”
BackgroundRathi, V. K., Krumholz, H. M., Masoudi, F. A., & Ross, J. S. (2015) · JAMA · doi:10.1001/jama.2015.8761 · Cited by 112
“Completed 50 (21-114) 241 (147-415) 202 (126-694) 100 (43-252)”
Self-citationLorkeers, S. J. J. O., Eding, J., Vesterinen, H. M., van der Spoel, T. I. G., Sena, E. S., Duckers, H. J., Doevendans, P. A., Macleod, M. R., & Chamuleau, S. A. J. (2014) · Circulation Research · doi:10.1161/circresaha.116.304872 · Cited by 116
“Publication bias is a serious problem in both clinical and preclinical studies, 40,43 and impedes transition from preclinical toward clinical studies, by skewing the expected effect size.”
The Project Data Sphere Initiative: Accelerating Cancer Research by Sharing Data
Green, A. K., Reeder-Hayes, K. E., Corty, R. W., Basch, E., Milowsky, M. I., Dusetzina, S. B., Bennett, A. V., & Wood, W. A. (2015) · The Oncologist · doi:10.1634/theoncologist.2014-0431 · Cited by 109
de Vries, Y. A., Roest, A. M., de Jonge, P., Cuijpers, P., Munafò, M. R., & Bastiaansen, J. A. (2018) · Psychological Medicine · doi:10.1017/s0033291718001873 · Cited by 98
“gov were published within two years of completion (Ross et al., 2009), and non-reporting of protocol-specified outcomes or the silent addition of new outcomes is also common (Jones et al.”
BackgroundNoncompletion and nonpublication of trials studying rare diseases: A cross-sectional analysis
Rees, C. A., Pica, N., Monuteaux, M. C., & Bourgeois, F. T. (2019) · PLoS Medicine · doi:10.1371/journal.pmed.1002966 · Cited by 111
The Importance of Clinical Trial Data Sharing: Toward More Open Science
Ross, J. S., Lehman, R., & Gross, C. P. (2012) · Circulation: Cardiovascular Quality and Outcomes · doi:10.1161/circoutcomes.112.965798 · Cited by 104
Self-citationReporting of Results in ClinicalTrials.gov and High-Impact Journals
Becker, J. E., Krumholz, H. M., Ben-Josef, G., & Ross, J. S. (2014) · JAMA · doi:10.1001/jama.2013.285634 · Cited by 100
Self-citationNon-publication and delayed publication of randomized trials on vaccines: survey
Manzoli, L., Flacco, M. E., D'Addario, M., Capasso, L., De Vito, C., Marzuillo, C., Villari, P., & Ioannidis, J. P. A. (2014) · BMJ · doi:10.1136/bmj.g3058 · Cited by 99
Trial Quality in Nephrology: How Are We Measuring Up?
Palmer, S. C., Sciancalepore, M., & Strippoli, G. F. (2011) · American Journal of Kidney Diseases · doi:10.1053/j.ajkd.2011.06.006 · Cited by 109
Miller, J. E., Korn, D., & Ross, J. S. (2015) · BMJ Open · doi:10.1136/bmjopen-2015-009758 · Cited by 92
“▪ This study takes a uniquely comprehensive approach, analysing five critical elements of transparency for trials of new drugs: (1) registration, (2) results reporting, (3) publication in a medical journal, (4) compliance with legal disclosure requirements and (5) adherence with the ethics standards enshrined in the C…”
BackgroundSelf-citationIs it time for a paradigm shift in drug research and development in endometriosis/adenomyosis?
Guo, S., & Groothuis, P. G. (2018) · Human Reproduction Update · doi:10.1093/humupd/dmy020 · Cited by 97
Linking ClinicalTrials.gov and PubMed to Track Results of Interventional Human Clinical Trials
Huser, V., & Cimino, J. J. (2013) · PLoS ONE · doi:10.1371/journal.pone.0068409 · Cited by 90
“04 years and that 81% of studies that eventually publish and link a result article do so within 3 years after completion date [3,4,6,18].”
BackgroundResultInfluentialPublic Availability of Results of Trials Assessing Cancer Drugs in the United States
Nguyen, T.-A.-H., Dechartres, A., Belgherbi, S., & Ravaud, P. (2013) · Journal of Clinical Oncology · doi:10.1200/jco.2012.46.9577 · Cited by 83
The Financing of Drug Trials by Pharmaceutical Companies and Its Consequences
Schott, G., Pachl, H., Limbach, U., Gundert‐Remy, U., Lieb, K., & Ludwig, W.-D. (2010) · Deutsches Ärzteblatt international · doi:10.3238/arztebl.2010.0295 · Cited by 111
Over-optimism in bioinformatics research
Boulesteix, A. (2009) · Bioinformatics · doi:10.1093/bioinformatics/btp648 · Cited by 85
Abdulatif, M., Mukhtar, A. M., & Obayah, G. M. (2015) · British Journal of Anaesthesia · doi:10.1093/bja/aev166 · Cited by 87
“Our assumption is in line with the recommendation of the World Health Organization Minimal Registration Data Set(68) and is supported by the finding that perfect 100% compliance was reported in obligatory data, while a variable adherence was observed in optional data in one of the most popular trial registration datab…”
ResultIoannidis, J. P. A. (2014) · BMJ · doi:10.1136/bmj.g7089 · Cited by 93
Promoting Transparency in Pharmaceutical Industry–Sponsored Research
Ross, J. S., Gross, C. P., & Krumholz, H. M. (2011) · American Journal of Public Health · doi:10.2105/ajph.2011.300187 · Cited by 85
Self-citationOutcome Discrepancies and Selective Reporting: Impacting the Leading Journals?
Fleming, P. S., Koletsi, D., Dwan, K., & Pandis, N. (2015) · PLOS ONE · doi:10.1371/journal.pone.0127495 · Cited by 80
“gov) [21], while optional fields are completed with significantly lower frequency.”
BackgroundTrial Registration Numbers Are Underreported in Biomedical Publications
van de Wetering, F. T., Scholten, R. J. P. M., Haring, T., Clarke, M., & Hooft, L. (2012) · PLoS ONE · doi:10.1371/journal.pone.0049599 · Cited by 79
The new BMJ policy on sharing data from drug and device trials
Godlee, F., & Groves, T. (2012) · BMJ · doi:10.1136/bmj.e7888 · Cited by 74
Translational Research in Cardiovascular Repair
Chamuleau, S. A. J., van der Naald, M., Climent, A. M., Kraaijeveld, A. O., Wever, K., Duncker, D. J., Fernández‐Avilés, F., & Bolli, R. (2018) · Circulation Research · doi:10.1161/circresaha.117.311565 · Cited by 78
Association between trial registration and treatment effect estimates: a meta-epidemiological study
Dechartres, A., Ravaud, P., Atal, I., Riveros, C., & Boutron, I. (2016) · BMC Medicine · doi:10.1186/s12916-016-0639-x · Cited by 71
“Registration aims to make information about the existence and methods of clinical trials publicly available to limit the effect of selective publication of trials and outcomes with positive results resulting in exaggerated treatment effect estimates [8–13].”
Use of Trial Register Information during the Peer Review Process
Mathieu, S., Chan, A., & Ravaud, P. (2013) · PLoS ONE · doi:10.1371/journal.pone.0059910 · Cited by 85
Saito, H., & Gill, C. J. (2014) · PLoS ONE · doi:10.1371/journal.pone.0101826 · Cited by 74
“While other investigators have challenged the quality of information entered on CTG [21], this too falls into the category of limitations for which there is no current solution since we are blind to information that may have been hidden.”
BackgroundWallach, J. D., Egilman, A. C., Dhruva, S. S., McCarthy, M. E., Miller, J. E., Woloshin, S., Schwartz, L. M., & Ross, J. S. (2018) · BMJ · doi:10.1136/bmj.k2031 · Cited by 75
Self-citationYou, B., Gan, H. K., Pond, G. R., & Chen, E. X. (2011) · Journal of Clinical Oncology · doi:10.1200/jco.2011.37.0890 · Cited by 77
Obtaining evidence by a single well-powered trial or several modestly powered trials
IntHout, J., Ioannidis, J. P. A., & Borm, G. F. (2012) · Statistical Methods in Medical Research · doi:10.1177/0962280212461098 · Cited by 79
“Ross et al.14 investigated the publication rates for a cross section of trials that had been registered at ClinicalTrials.gov after 31 December 1999.”
BackgroundRaftery, J., Young, A., Stanton, L., Milne, R., Cook, A., Turner, D., & Davidson, P. (2015) · Health Technology Assessment · doi:10.3310/hta19110 · Cited by 74
Chen, T., Li, C., Qin, R., Wang, Y., Yu, D., Dodd, J., Wang, D., & Cornelius, V. (2019) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.7242 · Cited by 68
Risks of phase I research with healthy participants: A systematic review
Johnson, R., Rid, A., Emanuel, E., & Wendler, D. S. (2015) · Clinical Trials · doi:10.1177/1740774515602868 · Cited by 67
“This window allows for a roughly 2-year lag between completion of the trial and publication of the trial results.(24) We found 3851 registered phase I trials, suggesting that our sample reflects fewer than 12% of phase I trials with healthy participants conducted during a 5-year period.”
BackgroundA Decade of Cell Therapy Clinical Trials (2000–2010)
Culme-Seymour, E. J., Davie, N. L., Brindley, D. A., Edwards‐Parton, S., & Mason, C. (2012) · Regenerative Medicine · doi:10.2217/rme.12.45 · Cited by 67
Rasmussen, N., Lee, K., & Bero, L. (2009) · Trials · doi:10.1186/1745-6215-10-116 · Cited by 62
Developing a guideline to standardize the citation of bioresources in journal articles (CoBRA)
Bravo, E., Calzolari, A., Castro, P. D., Mabile, L., Napolitani, F., Rossi, A. M., & Cambon‐Thomsen, A. (2015) · BMC Medicine · doi:10.1186/s12916-015-0266-y · Cited by 64
“In clinical research, for instance, nearly half of trials remain unpublished [2].”
BackgroundMiller, J. E., Wilenzick, M., Ritcey, N., Ross, J. S., & Mello, M. M. (2017) · BMJ Open · doi:10.1136/bmjopen-2017-017917 · Cited by 62
“For pharmaceutical companies with only one drug approved by the FDA in 2014, we averaged their scores on (1) the trials-in-patients analysis, excluding trials in healthy volunteers, and (2) a FDAAA compliance standard that counted a trial as compliant if it satisfied either interpretation of the reporting requirements.”
BackgroundMethodInfluentialSelf-citationFleminger, J., & Goldacre, B. M. (2018) · PLoS ONE · doi:10.1371/journal.pone.0193088 · Cited by 64
Out of sight but not out of mind: how to search for unpublished clinical trial evidence
Chan, A.-W. (2012) · BMJ · doi:10.1136/bmj.d8013 · Cited by 62
Jones, C. W., Keil, L. G., Weaver, M. A., & Platts-Mills, T. F. (2014) · Systematic Reviews · doi:10.1186/2046-4053-3-126 · Cited by 61
Wieschowski, S., Riedel, N., Wollmann, K., Kahrass, H., Müller-Ohlraun, S., Schuermann, C., Kelley, S., Kszuk, U., Siegerink, B., Dirnagl, U., et al. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.06.002 · Cited by 60
“For three decades studies investigated and criticized this challenge [9-11].”
BackgroundClinical Research Involving Children: Registration, Completeness, and Publication
Shamliyan, T., & Kane, R. L. (2012) · Pediatrics · doi:10.1542/peds.2010-2847 · Cited by 59
“Gender Both 212 (29) 516 728 Female 5 (26) 14 19 Male 1 (9) 10 11 Study type Interventional 201 (30) 470 671 Observational 17 (20) 70 87 RCT 158 (32) 336 494 Not RCT 56 (23) 188 244 Posted results Has results 16 (24) 50 66 No results available 202 (29) 490 692 Phase of clinical trials Phase 0 0 1 1 Phase I 16 (32) 34…”
Estimation of Potential Savings Through Therapeutic Substitution
Johansen, M. E., & Richardson, C. R. (2016) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2016.1704 · Cited by 63
Genetic factors associated with reasons for clinical trial stoppage
Razuvayevskaya, O., López, I., Dunham, I., & Ochoa, D. (2024) · Nature Genetics · doi:10.1038/s41588-024-01854-z · Cited by 64
“Since 2007, the FDA has required the submission of clinical trial results to ClinicalTrials.gov, a free-to-access global databank aimed at registering clinical research studies and their results 16,17 .”
P-hacking in clinical trials and how incentives shape the distribution of results across phases
Adda, J., Decker, C., & Ottaviani, M. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1919906117 · Cited by 67
“, information about scientific leadership (15), consistency of reported primary outcomes (16, 17), comparisons to results published in academic journals (14, 18), and the provision of Individual Participant Data (IPD) (19).”
Pinto, R. Z., Elkins, M. R., Moseley, A. M., Sherrington, C., Herbert, R. D., Maher, C. G., Ferreira, P. H., & Ferreira, M. L. (2013) · Physical Therapy · doi:10.2522/ptj.20120206 · Cited by 51
Feys, F., Bekkering, G. E., Singh, K., & Devroey, D. (2014) · Systematic Reviews · doi:10.1186/2046-4053-3-14 · Cited by 51
Carpenter, C. R., Kane, B. G., Carter, M., Lucas, R. H., Wilbur, L., & Graffeo, C. S. (2010) · Academic Emergency Medicine · doi:10.1111/j.1553-2712.2010.00889.x · Cited by 53
“Hospital with no EM residents 2 (2) Other 3 (3) Academic position Chair or vice chair 9 (9) Program director 29 (30)”
Herbal Medicines for Parkinson's Disease: A Systematic Review of Randomized Controlled Trials
Kim, T., Cho, K., Jung, W., & Lee, M. S. (2012) · PLoS ONE · doi:10.1371/journal.pone.0035695 · Cited by 50
“selective outcome reporting and publication bias [92].”
Thaler, K. J., Kien, C., Nußbaumer-Streit, B., Noord, M. V., Griebler, U., Klerings, I., & Gartlehner, G. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.01.008 · Cited by 55
Öztürk, S., Elçin, A. E., Koca, A., & Elçin, Y. M. (2020) · Stem Cell Reviews and Reports · doi:10.1007/s12015-020-10029-2 · Cited by 52
Bashir, R., Bourgeois, F. T., & Dunn, A. G. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0518-3 · Cited by 47
“, terminated and withdrawn trials [50, 51], trials funded by specific organisations or from certain countries [52, 53]), and by specialty and condition (e.”
BackgroundReveiz, L., Chan, A.-W., Krleža-Jerić, K., Granados, C. E., Pinart, M., Etxeandia, I., Rada, D., Martinez, M., Bonfill, X., & Cardona, A. F. (2010) · PLoS ONE · doi:10.1371/journal.pone.0012484 · Cited by 47
“Conversely, reporting of optional data elements varied (66% for primary outcome measures; 56% for secondary outcome measures) [19].”
BackgroundComparison of published orthopaedic trauma trials following registration in Clinicaltrials.gov
Gandhi, R., Jan, M., Smith, H. N., Mahomed, N. N., & Bhandari, M. (2011) · BMC Musculoskeletal Disorders · doi:10.1186/1471-2474-12-278 · Cited by 47
Publication and Reporting of Test Accuracy Studies Registered in ClinicalTrials.gov
Korevaar, D. A., Ochodo, E. A., Bossuyt, P. M. M., & Hooft, L. (2014) · Clinical Chemistry · doi:10.1373/clinchem.2013.218149 · Cited by 47
Walker, K. F., Stevenson, G., & Thornton, J. G. (2014) · JRSM Open · doi:10.1177/2042533313517688 · Cited by 45
“Ross et al.(10) scrutinised the completeness of trial registry information in ClinicalTrials.”
BackgroundBlumle, A., Meerpohl, J. J., Rucker, G., Antes, G., Schumacher, M., & von Elm, E. (2011) · BMJ · doi:10.1136/bmj.d1828 · Cited by 44
Jones, C. W., Safferman, M. R., Adams, A., & Platts‐Mills, T. F. (2017) · BMJ Open · doi:10.1136/bmjopen-2017-017719 · Cited by 45
Re‐inventing the randomized controlled trial in medical oncology: The registry‐based trial
Foroughi, S., Wong, H., Gately, L., Lee, M., Simons, K., Tie, J., Burgess, A. W., & Gibbs, P. (2018) · Asia-Pacific Journal of Clinical Oncology · doi:10.1111/ajco.12992 · Cited by 43
A Model for Dissemination and Independent Analysis of Industry Data
Krumholz, H. M., & Ross, J. S. (2011) · JAMA · doi:10.1001/jama.2011.1459 · Cited by 44
“Many clinical studies, including randomized clinical trials, are never published in the biomedical literature.(1,2) Among those that are published, key information is often not presented, such as data on specific outcomes and safety end points.”
BackgroundSelf-citationConditional equivalence testing: An alternative remedy for publication bias
Campbell, H., & Gustafson, P. (2018) · PLoS ONE · doi:10.1371/journal.pone.0195145 · Cited by 46
Krumholz, H. M. (2012) · Circulation: Cardiovascular Quality and Outcomes · doi:10.1161/circoutcomes.112.965848 · Cited by 42
Self-citationMassey, P. R., Wang, R., Prasad, V. K., Bates, S. E., & Fojo, T. (2016) · The Oncologist · doi:10.1634/theoncologist.2015-0516 · Cited by 42
Discontinuation and non-publication of clinical trials in cardiovascular medicine
Roddick, A., Chan, F., Stefaniak, J. D., & Zheng, S. L. (2017) · International Journal of Cardiology · doi:10.1016/j.ijcard.2017.06.020 · Cited by 42
Pranić, S. M., & Marušić, A. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.07.007 · Cited by 46
Obstacles to the reuse of study metadata in ClinicalTrials.gov
Miron, L., Gonçalves, R. S., & Musen, M. A. (2020) · Scientific Data · doi:10.1038/s41597-020-00780-z · Cited by 41
Lack of Transparency of Clinical Trials on Endometriosis
Guo, S., & Evers, J. L. H. (2013) · Obstetrics and Gynecology · doi:10.1097/aog.0b013e318291f299 · Cited by 41
Metadata
- Authors
- Ross, J. S., Mulvey, G. K., Hines, E. M., Nissen, S. E., & Krumholz, H. M.
- Affiliations
- Health Services Research & Development (US)
- Yale New Haven Hospital (US)
- Amherst College (US)
- Cleveland Clinic (US)
- Corresponding author
- Ross, Joseph S.
- Venue
- PLoS Medicine
- Publisher
- Public Library of Science (PLoS)
- Published
- 2009-09-08
- Volume
- 6
- Issue
- 9
- Pages
- e1000144
- ISSN
- 1549-1277 (ISSN-L)
- 1549-1676
- PMC
- PMC2728480
- OpenAlex
- W1970223153
- Cited by
- 401
- References
- 31
- Citations by year
- 2026: 17 · 2025: 10 · 2024: 15 · 2023: 10 · 2022: 16 · 2021: 20 · 2020: 18 · 2019: 14 · 2018: 23 · 2017: 27 · 2016: 38 · 2015: 39 · 2014: 36 · 2013: 35 · 2012: 32
- FWCI
- 24.36
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS Medicine (publishedVersion, public-domain)
- PubMed Central (publishedVersion, public-domain)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- U.S. Department of Veterans Affairs (K08 AG032886)
- Hartford Foundation for Public Giving
- Amgen
- Eli Lilly and Company
- Pfizer
- Sanofi
- Health Services Research and Development
- Cleveland Clinic
- Massachusetts Medical Society
- Daiichi-Sankyo
- Novo Nordisk
- National Institute on Aging (K08AG032886)
- U.S. National Library of Medicine
- Topics
- Health Sciences › Nursing › Issues, ethics and legal aspects › Pharmaceutical industry and healthcare
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- ClinicalTrials.gov, clinical trial registration, selective publication, publication bias, registration completeness, industry funding, publication rates, clinical trial registries, primary outcome reporting
- MeSH terms
- Clinical Trials as Topic, Cross-Sectional Studies, Drug Industry, Financing, Government, Humans, National Library of Medicine (U.S.), Periodicals as Topic, Publishing, Registries, United States, Bibliometrics, Databases, Factual, Treatment Outcome, Internet, Information Dissemination
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
