The DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Design characteristics, risk of bias, and reporting of randomised controlled trials supporting approvals of cancer drugs by European Medicines Agency, 2014-16: cross sectional analysis
Open accessAbstract
Objective: To examine the design characteristics, risk of bias, and reporting adequacy of pivotal randomised controlled trials of cancer drugs approved by the European Medicines Agency (EMA). Design: Cross sectional analysis. Setting: European regulatory documents, clinical trial registry records, protocols, journal publications, and supplementary appendices. Eligibility criteria: Pivotal randomised controlled trials of new cancer drugs approved by the EMA between 2014 and 2016. Main outcome measures: Study design characteristics (randomisation, comparators, and endpoints); risk of bias using the revised Cochrane tool (bias arising from the randomisation process, deviations from intended interventions, missing outcome data, measurement of the outcome, and selection of the reported result); and reporting adequacy (completeness and consistency of information in trial protocols, publications, supplementary appendices, clinical trial registry records, and regulatory documents). Results: Between 2014 and 2016, the EMA approved 32 new cancer drugs on the basis of 54 pivotal studies. Of these, 41 (76%) were randomised controlled trials and 13 (24%) were either non-randomised studies or single arm studies. 39/41 randomised controlled trials had available publications and were included in our study. Only 10 randomised controlled trials (26%) measured overall survival as either a primary or coprimary endpoint, with the remaining trials evaluating surrogate measures such as progression free survival and response rates. Overall, 19 randomised controlled trials (49%) were judged to be at high risk of bias for their primary outcome. Concerns about missing outcome data (n=10) and measurement of the outcome (n=7) were the most common domains leading to high risk of bias judgments. Fewer randomised controlled trials that evaluated overall survival as the primary endpoint were at high risk of bias than those that evaluated surrogate efficacy endpoints (2/10 (20%)v 16/29 (55%), respectively). When information available in regulatory documents and the scientific literature was considered separately, overall risk of bias judgments differed for eight randomised controlled trials (21%), which reflects reporting inadequacies in both sources of information. Regulators identified additional deficits beyond the domains captured in risk of bias assessments for 10 drugs (31%). These deficits included magnitude of clinical benefit, inappropriate comparators, and non-preferred study endpoints, which were not disclosed as limitations in scientific publications. Conclusions: Most pivotal studies forming the basis of EMA approval of new cancer drugs between 2014 and 2016 were randomised controlled trials. However, almost half of these were judged to be at high risk of bias based on their design, conduct, or analysis, some of which might be unavoidable because of the complexity of cancer trials. Regulatory documents and the scientific literature had gaps in their reporting. Journal publications did not acknowledge the key limitations of the available evidence identified in regulatory documents.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
RoB 2: a revised tool for assessing risk of bias in randomised trials
Sterne, J. A. C., Savović, J., Page, M. J., Elbers, R. G., Blencowe, N. S., Boutron, I., Cates, C. J., Cheng, H.-Y., Corbett, M. S., Eldridge, S. M., et al. (2019) · BMJ · doi:10.1136/bmj.l4898 · Cited by 34,787
Self-citationThe Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817
Self-citationSchulz (1995) · doi:10.1001/jama.1995.03520290060030 · Cited by 5,634
Combined BRAF and MEK inhibition in melanoma with BRAF V600 mutations
Flaherty, K. T., Infante, J. R., Daud, A., Gonzalez, R., Kefford, R. F., Sosman, J., Hamid, O., Schuchter, L., Cebon, J., Ibrahim, N., et al. (2012) · New England Journal of Medicine · doi:10.1056/nejmoa1210093 · Cited by 2,810
Wood, L., Egger, M., Gluud, L. L., Schulz, K. F., Jüni, P., Altman, D. G., Gluud, C., Martin, R. M., Wood, A. J. G., & Sterne, J. A. C. (2008) · BMJ · doi:10.1136/bmj.39465.451748.ad · Cited by 2,538
Self-citationSavović, J., Jones, H. E., Altman, D. G., Harris, R. J., Jüni, P., Pildal, J., Als-Nielsen, B., Balk, E. M., Gluud, C., Gluud, L. L., et al. (2012) · Annals of Internal Medicine · doi:10.7326/0003-4819-157-6-201209180-00537 · Cited by 1,049
Self-citationPage, M. J., Shamseer, L., Altman, D. G., Tetzlaff, J., Sampson, M., Tricco, A. C., Catalá-López, F., Li, L., Reid, E. K., Sarkis-Onofre, R., et al. (2016) · PLOS Medicine · doi:10.1371/journal.pmed.1002028 · Cited by 768
Turner, L., Shamseer, L., Altman, D. G., Schulz, K. F., & Moher, D. (2012) · Systematic Reviews · doi:10.1186/2046-4053-1-60 · Cited by 628
Biases in Randomized Trials: A Conversation Between Trialists and Epidemiologists
Mansournia, M. A., Higgins, J. P. T., Sterne, J. A. C., & Hernán, M. A. (2017) · Epidemiology · doi:10.1097/ede.0000000000000564 · Cited by 285
Self-citationPage, M. J., Higgins, J. P. T., Clayton, G., Sterne, J. A. C., Hróbjartsson, A., & Savović, J. (2016) · PLOS ONE · doi:10.1371/journal.pone.0159267 · Cited by 284
Self-citation
Cited by
Minozzi, S., Cinquini, M., Gianola, S., Gonzalez-Lorenzo, M., & Banzi, R. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.06.015 · Cited by 471
Minozzi, S., Dwan, K., Borrelli, F., & Filippini, G. (2022) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.09.021 · Cited by 150
Azoulay, L. (2022) · The Oncologist · doi:10.1093/oncolo/oyac114 · Cited by 79
Metadata
- Authors
- Naci, H., Davis, C., Savović, J., Higgins, J. P. T., Sterne, J. A. C., Gyawali, B., Romo-Sandoval, X., Handley, N., & Booth, C. M.
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 2019-09-18
- Volume
- 366
- Pages
- l5221
- ISSN
- 1756-1833 (ISSN-L)
- 0959-8138
- OpenAlex
- W2974839292
- Cited by
- 243
- References
- 19
- FWCI
- 16.19
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Design characteristics, risk of bias, and reporting of randomised controlled trials supporting approvals of cancer drugs by European Medicines Agency, 2014-16: cross sectional analysis. https://app.cochat.ai/papers/10.1136/bmj.l5221
- The paper
- Naci, H., Davis, C., Savović, J., Higgins, J. P. T., Sterne, J. A. C., Gyawali, B., Romo-Sandoval, X., Handley, N., & Booth, C. M. (2019). Design characteristics, risk of bias, and reporting of randomised controlled trials supporting approvals of cancer drugs by European Medicines Agency, 2014-16: cross sectional analysis. BMJ. https://doi.org/10.1136/bmj.l5221
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.