When are ‘positive’ clinical trials in oncology truly positive?
Abstract
The approval of a new drug for cancer treatment by the regulatory authorities, such as the United States Food and Drug Administration or European Medicines Agency, is usually based on the positive results of one or more randomized phase III clinical trials comparing the investigational treatment with the standard treatment. A clinical trial is presented as positive if the new drug tested on an experimental group shows a statistically significant difference with the control group (P < .05) in the primary endpoint, which is usually a time-to-event endpoint (overall survival or progression-free survival). Such apparently positive clinical trials disregard whether the final value of the difference in the primary endpoints between the experimental and control groups (δ) meets the criterion that was predefined in the protocol. Currently, the trend is to design large trials that may detect statistically significant, but often trivial, differences in survival endpoints. However, recent appeals have been made in the oncology literature for the design of smaller clinical trials to detect or exclude only larger, clinically important, values of δ. Here, we have evaluated 18 randomized phase III clinical trials that were used for the approval of molecular-targeted anticancer drugs by the United States Food and Drug Administration. Results showed that in some of the articles the magnitude of the reported values of δ were lower than the values predefined in the protocol. We suggest that trials should not be declared positive based only on a statistically significant P value, but should also require detection of a difference in survival outcome that equals or exceeds a clinically important value that is specified in the protocol.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
No standard instrument exists for grading how much clinical benefit cancer therapies provide.
“To date, there is no standard tool for grading the magnitude of clinical benefit of cancer therapies [25,26]”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Sorafenib in advanced hepatocellular carcinoma
Llovet, J. M., Ricci, S., Mazzaferro, V., Hilgard, P., Gane, E., Blanc, J.-F., de Oliveira, A. C., Santoro, A., Raoul, J.-L., Forner, A., et al. (2008) · New England Journal of Medicine · doi:10.1056/nejmoa0708857 · Cited by 13,097 · see citing works
Slamon, D. J., Leyland-Jones, B., Shak, S., Fuchs, H., Paton, V., Bajamonde, A., Fleming, T., Eiermann, W., Wolter, J., Pegram, M., et al. (2001) · New England Journal of Medicine · doi:10.1056/nejm200103153441101 · Cited by 11,500 · see citing works
Bevacizumab plus Irinotecan, Fluorouracil, and Leucovorin for Metastatic Colorectal Cancer
Hurwitz, H., Fehrenbacher, L., Novotny, W., Cartwright, T., Hainsworth, J., Heim, W., Berlin, J., Baron, A., Griffing, S., Holmgren, E., et al. (2004) · New England Journal of Medicine · doi:10.1056/nejmoa032691 · Cited by 10,968 · see citing works
Paclitaxel–Carboplatin Alone or with Bevacizumab for Non–Small-Cell Lung Cancer
Sandler, A., Gray, R., Perry, M. C., Brahmer, J., Schiller, J. H., Dowlati, A., Lilenbaum, R., & Johnson, D. H. (2006) · New England Journal of Medicine · doi:10.1056/nejmoa061884 · Cited by 5,761 · see citing works
Sunitinib versus Interferon Alfa in Metastatic Renal-Cell Carcinoma
Motzer, R. J., Hutson, T. E., Tomczak, P., Michaelson, M. D., Bukowski, R. M., Rixe, O., Oudard, S. M., Négrier, S., Szczylik, C. A., Kim, S. T., et al. (2007) · New England Journal of Medicine · doi:10.1056/nejmoa065044 · Cited by 5,752 · see citing works
Sorafenib in Advanced Hepatocellular Carcinoma
정, 재. (2008) · Cited by 5,606 · see citing works
Erlotinib in Previously Treated Non–Small-Cell Lung Cancer
Shepherd, F. A., Rodrigues Pereira, J., Ciuleanu, T., Tan, E. H., Hirsh, V., Thongprasert, S., Campos, D., Maoleekoonpiroj, S., Smylie, M., Martins, R., et al. (2005) · New England Journal of Medicine · doi:10.1056/nejmoa050753 · Cited by 5,419 · see citing works
Sorafenib in Advanced Clear-Cell Renal-Cell Carcinoma
Escudier, B. J., Eisen, T. G., Stadler, W. M., Szczylik, C. A., Oudard, S. M., Siebels, M., Négrier, S., Chevreau, C., Solska, E., Desai, A. A., et al. (2007) · New England Journal of Medicine · doi:10.1056/nejmoa060655 · Cited by 4,802 · see citing works
Moore, M. J., Goldstein, D., Hamm, J., Figer, A., Hecht, J. R., Gallinger, S., Au, H. J., Murawa, P., Walde, D., Wolff, R. A., et al. (2007) · Journal of Clinical Oncology · doi:10.1200/jco.2006.07.9525 · Cited by 3,848 · see citing works
Temsirolimus, Interferon Alfa, or Both for Advanced Renal-Cell Carcinoma
Hudes, G. R., Carducci, M. A., Tomczak, P., Dutcher, J. P., Figlin, R. A., Kapoor, A., Starosławska, E., Sosman, J. A., McDermott, D. F., Bodrogi, I., et al. (2007) · New England Journal of Medicine · doi:10.1056/nejmoa066838 · Cited by 3,645 · see citing works
Lapatinib plus Capecitabine for HER2-Positive Advanced Breast Cancer
Geyer, C. E., Forster, J. K., Lindquist, D. L., Chan, S. Y. T., Romieu, C. G., Pieńkowski, T., Jagiełło-Gruszfeld, A., Crown, J. P., Chan, A., Kaufman, B., et al. (2006) · New England Journal of Medicine · doi:10.1056/nejmoa064320 · Cited by 3,426 · see citing works
Paclitaxel plus Bevacizumab versus Paclitaxel Alone for Metastatic Breast Cancer
Miller, K. D., Wang, M., Gralow, J. R., Dickler, M. N., Cobleigh, M., Perez, E. A., Shenkier, T. N., Cella, D. F., & Davidson, N. E. (2007) · New England Journal of Medicine · doi:10.1056/nejmoa072113 · Cited by 3,077 · see citing works
Motzer, R. J., Escudier, B., Oudard, S., Hutson, T. E., Porta, C., Bracarda, S., Grünwald, V., Thompson, J. A., Figlin, R. A., Hollaender, N., et al. (2008) · The Lancet · doi:10.1016/s0140-6736(08)61039-9 · Cited by 3,019 · see citing works
Demetri, G. D., van Oosterom, A. T., Garrett, C. R., Blackstein, M. E., Shah, M. H., Verweij, J., McArthur, G. A., Judson, I. R., Heinrich, M. C., Morgan, J. A., et al. (2006) · The Lancet · doi:10.1016/s0140-6736(06)69446-4 · Cited by 2,646 · see citing works
Giantonio, B. J., Catalano, P. J., Meropol, N. J., O'Dwyer, P. J., Mitchell, E. P., Alberts, S. R., Schwartz, M. A., & Benson, A. B. (2007) · Journal of Clinical Oncology · doi:10.1200/jco.2006.09.6305 · Cited by 2,279 · see citing works
Escudier, B. J., Pluzanska, A., Koralewski, P., Ravaud, A., Sergio Bracarda, A., Szczylik, C. A., Chevreau, C., Filipek, M., Melichar, B., Bajetta, E., et al. (2007) · The Lancet · doi:10.1016/s0140-6736(07)61904-7 · Cited by 2,216 · see citing works
Van Cutsem, E., Peeters, M., Siena, S., Humblet, Y., Hendlisz, A., Neyns, B., Canon, J.-L., Van Laethem, J.-L., Maurel, J., Richardson, G., et al. (2007) · Journal of Clinical Oncology · doi:10.1200/jco.2006.08.1620 · Cited by 1,979 · see citing works
Cetuximab for the Treatment of Colorectal Cancer
Jonker, D. J., O'Callaghan, C. J., Karapetis, C. S., Zalcberg, J. R., Tu, D., Au, H.-J., Berry, S. R., Krahn, M., Price, T., Simes, R. J., et al. (2007) · New England Journal of Medicine · doi:10.1056/nejmoa071834 · Cited by 1,904 · see citing works
Lamon, D. E. J. S., Eyland, B. R. L., Hak, S. T. S., Uchs, H. A. F., Aton, V. I. P., Ajamonde, A. L. B., LeMing, T., Iermann, W. O. E., Olter, J. A. W., Egram, M. A. P., et al. (2009) · Cited by 1,199 · see citing works
Johnston, S. R. D., Pippen, J., Pivot, X. B., Lichinitser, M., Sadeghi, S., Dieras, V. C., Gómez, H., Romieu, G., Manikhas, A., Kennedy, M. J., et al. (2009) · Journal of Clinical Oncology · doi:10.1200/jco.2009.23.3734 · Cited by 1,016 · see citing works
Detecting an Overall Survival Benefit that Is Derived From Progression-Free Survival
Broglio, K. R., & Berry, D. A. (2009) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djp369 · Cited by 492 · see citing works
Lapatinib plus Capecitabine for HER2-Positive Advanced Breast Cancer
(2007) · New England Journal of Medicine · doi:10.1056/nejmx070016 · Cited by 350 · see citing works
How much is life worth: cetuximab, non-small cell lung cancer, and the $440 billion question
Fojo, T., & Grady, C. (2009) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djp177 · Cited by 349 · see citing works
What Does It Mean to Understand a Risk? Evaluating Risk Comprehension
Weinstein, N. D. (1999) · JNCI Monographs · doi:10.1093/oxfordjournals.jncimonographs.a024192 · Cited by 315 · see citing works
Endpoints for Assessing Drug Activity in Clinical Trials
Pazdur, R. M. (2008) · The Oncologist · doi:10.1634/theoncologist.13-s2-19 · Cited by 287 · see citing works
Sloan, J. A. (2005) · COPD Journal of Chronic Obstructive Pulmonary Disease · doi:10.1081/copd-200053374 · Cited by 169 · see citing works
Incremental Advance or Seismic Shift? The Need to Raise the Bar of Efficacy for Drug Approval
Sobrero, A. F., & Bruzzi, P. (2009) · Journal of Clinical Oncology · doi:10.1200/jco.2009.22.4162 · Cited by 99 · see citing works
Chao, C., Studts, J. L., Abell, T. D., et al. (2003) · Journal of Clinical Oncology · doi:10.1200/jco.2003.06.025 · Cited by 90 · see citing works
Sargent, D. J. (2010) · The Oncologist · doi:10.1634/theoncologist.2010-s1-19 · Cited by 36 · see citing works
The $64,000 question—What is a quality-adjusted life-year worth?
Rascati, K. L. (2006) · Clinical Therapeutics · doi:10.1016/j.clinthera.2006.07.002 · Cited by 29 · see citing works
Endpoints and surrogate endpoints in colorectal cancer: a review of recent developments
Piedbois, P., & Buyse, M. E. (2008) · Current Opinion in Oncology · doi:10.1097/cco.0b013e32830218fe · Cited by 20 · see citing works
Do We Have to Change the Way Targeted Drugs Are Developed?
Ocaña, A., Amir, E., Šeruga, B., et al. (2010) · Journal of Clinical Oncology · doi:10.1200/jco.2010.28.9918 · Cited by 12 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Increasing value and reducing waste in research design, conduct, and analysis
Ioannidis, J. P. A., Greenland, S., Hlatky, M. A., Khoury, M. J., Macleod, M. R., Moher, D., Schulz, K. F., & Tibshirani, R. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62227-8 · Cited by 1,597 · see citing works
Cherny, N. I., Sullivan, R., Dafni, U., Kerst, J. M., Sobrero, A., Zielinski, C., de Vries, E. G. E., & Piccart, M. J. (2015) · Annals of Oncology · doi:10.1093/annonc/mdv249 · Cited by 818 · see citing works
A peculiar prevalence of <i>p</i> values just below .05
Masicampo, E. J., & Lalande, D. R. (2012) · Quarterly Journal of Experimental Psychology · doi:10.1080/17470218.2012.711335 · Cited by 266 · see citing works
Vera-Badillo, F. E., Shapiro, R., Ocana, A., Amir, E., & Tannock, I. F. (2013) · Annals of Oncology · doi:10.1093/annonc/mds636 · Cited by 117 · see citing works
Self-citationBenefit and harms of new anti-cancer drugs
Vera-Badillo, F. E., Al-Mubarak, M., Templeton, A. J., & Amir, E. (2013) · Current Oncology Reports · doi:10.1007/s11912-013-0303-y · Cited by 35 · see citing works
Evidence appraisal: a scoping review, conceptual framework, and research agenda
Goldstein, A., Venker, E., & Weng, C. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocx050 · Cited by 30 · see citing works
- Authors
- Ocana, A., & Tannock, I. F.
- Affiliations
- University of Toronto (CA)
- ORCID
- Tannock, I. F.
- Venue
- JNCI Journal of the National Cancer Institute
- Publisher
- Oxford University Press (OUP)
- Published
- 2010-12-03
- Volume
- 103
- Issue
- 1
- Pages
- 16-20
- ISSN
- 0027-8874 (ISSN-L)
- 1460-2105
- OpenAlex
- W2115837083
- Cited by
- 110
- References
- 32
- Citations by year
- 2025: 1 · 2024: 2 · 2023: 1 · 2022: 1 · 2021: 5 · 2020: 3 · 2019: 15 · 2018: 9 · 2017: 9 · 2016: 7 · 2015: 10 · 2014: 14 · 2013: 10 · 2012: 13
- FWCI
- 9.00
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- JNCI Journal of the National Cancer Institute (publishedVersion)
- PubMed (publishedVersion)
- HighWire Press Open Archive (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- oncology drug approval, clinically important difference, statistical significance, time-to-event outcomes, overall survival, progression-free survival, trial design, FDA drug approval
- MeSH terms
- Antineoplastic Agents, Data Interpretation, Statistical, Europe, Humans, Neoplasms, Quality of Life, United States, United States Food and Drug Administration, Odds Ratio, Survival Analysis, Randomized Controlled Trials as Topic, Treatment Outcome, Drug Approval, Clinical Trials, Phase III as Topic, Sample Size
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
