Bias in reporting of end points of efficacy and toxicity in randomized, clinical trials for women with breast cancer
Open accessThe DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
Abstract
Background Phase III randomized, clinical trials (RCTs) assess clinically important differences in end points that reflect benefit to patients. Here, we evaluate the quality of reporting of the primary end point (PE) and of toxicity in RCTs for breast cancer.Methods PUBMED was searched from 1995 to 2011 to identify RCTs for breast cancer. Bias in the reporting of the PE and of toxicity was assessed using pre-designed algorithms. Associations of bias with the Journal Impact Factor (JIF), changes in the PE compared with information in ClinicalTrials.gov and funding source were evaluated.Results Of 164 included trials, 33% showed bias in reporting of the PE and 67% in the reporting of toxicity. The PE was more likely to be reported in the concluding statement of the abstract when significant differences favoring the experimental arm were shown; 59% of 92 trials with a negative PE used secondary end points to suggest benefit of experimental therapy. Only 32% of articles indicated the frequency of grade 3 and 4 toxicities in the abstract. A positive PE was associated with under-reporting of toxicity.Conclusion Bias in reporting of outcome is common for studies with negative PEs. Reporting of toxicity is poor, especially for studies with positive PEs.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Other methodological systematic reviews focusing on two-arm parallel-group RCTs with nonstatistically significant primary outcomes in specific medical fields found consistent results (48–54).”
From Misrepresentation and distortion of research in biomedical literature (2018) · cites this paper as [48]
In the citing paper
“Overestimating or overstating the benefits from new intervention can cause harm: it confounds public policy decision making [29], undermines the credibility of oncology research reporting [26,29,30], harms patients who choose to undertake treatments based on exaggerated expectations that may subject them to either risk of adverse effects, inconvenience or substantial personal costs [26,28] and, in”
Citations
Cites
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 1×
“Selection bias can affect not only the interpretation of the trial itself but also the interpretation of subsequent systematic reviews or overviews, producing inaccurate summaries of research [2, 9] and misrepresentation of toxicity [10].”
BackgroundSiegel, R., Ward, E., Brawley, O., & Jemal, A. (2011) · CA: A Cancer Journal for Clinicians · doi:10.3322/caac.20121 · Cited by 4,070
Chan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004) · JAMA · doi:10.1001/jama.291.20.2457 · Cited by 1,845 · cited 3×
“The pharmaceutical industry is increasingly influential in clinical trial sponsorship with data showing an increase in industry sponsorship of phase III RCTs from 24 to 72% over a 30 year period [3, 19, 20].”
BackgroundResultBetter Reporting of Harms in Randomized Trials: An Extension of the CONSORT Statement
Ioannidis, J. P. A., Evans, S. J. W., Gøtzsche, P. C., O'Neill, R. T., Altman, D. G., Schulz, K., Moher, D., & for the CONSORT Group* (2004) · Annals of Internal Medicine · doi:10.7326/0003-4819-141-10-200411160-00009 · Cited by 1,593 · cited 1×
“Guidelines such as Consolidated Standards of Reporting Trials can improve the quality of reporting of clinical trials [6].”
BackgroundSystematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias
Dwan, K., Altman, D. G., Arnaiz, J. A., Bloom, J., Chan, A.-W., Cronin, E., Decullier, E., Easterbrook, P. J., Von Elm, E., Gamble, C., et al. (2008) · PLoS ONE · doi:10.1371/journal.pone.0003081 · Cited by 1,546 · cited 3×
“Such trials provide the gold standard to evaluate the efficacy and toxicity of new drugs before approval by regulatory authorities [2, 3].”
BackgroundGianni, L., Eiermann, W., Semiglazov, V., Manikhas, A., Lluch, A., Tjulandin, S., Zambetti, M., Vazquez, F., Byakhow, M., Lichinitser, M., et al. (2010) · The Lancet · doi:10.1016/s0140-6736(09)61964-4 · Cited by 1,255
Kirkham, J. J., Dwan, K. M., Altman, D. G., Gamble, C., Dodd, S., Smyth, R., & Williamson, P. R. (2010) · BMJ · doi:10.1136/bmj.c365 · Cited by 1,133 · cited 1×
“Trial registration does not necessarily remove bias in reporting outcome, although it does make it easier to detect [18].”
BackgroundCortes, J., O'Shaughnessy, J., Loesch, D., Blum, J. L., Vahdat, L. T., Petrakova, K., Chollet, P., Manikas, A., Diéras, V., Delozier, T., et al. (2011) · The Lancet · doi:10.1016/s0140-6736(11)60070-6 · Cited by 1,094
Boutron, I. (2010) · JAMA · doi:10.1001/jama.2010.651 · Cited by 842 · cited 1×
“The pharmaceutical industry is increasingly influential in clinical trial sponsorship with data showing an increase in industry sponsorship of phase III RCTs from 24 to 72% over a 30 year period [3, 19, 20].”
BackgroundRising, K., Bacchetti, P., & Bero, L. (2008) · PLoS Medicine · doi:10.1371/journal.pmed.0050217 · Cited by 328
Progression-Free Survival: Meaningful or Simply Measurable?
Booth, C. M., & Eisenhauer, E. A. (2012) · Journal of Clinical Oncology · doi:10.1200/jco.2011.38.7571 · Cited by 324 · cited 1×
“PFS or DFS may be suitable end points if these measures are valid surrogates for OS [24], or possibly in trials where there is a high rate of cross-over to the experimental arm which confounds interpretation of OS.”
BackgroundFactors Associated With Failure to Publish Large Randomized Trials Presented at an Oncology Meeting
Krzyzanowska, M. K. (2003) · JAMA · doi:10.1001/jama.290.4.495 · Cited by 314 · cited 1×
“This may be influenced by publication bias—the association between positive results and acceptance of reports for publication [4, 8].”
BackgroundSelf-citationReporting of Safety Results in Published Reports of Randomized Controlled Trials
(2009) · Archives of Internal Medicine · doi:10.1001/archinternmed.2009.306 · Cited by 287 · cited 1×
“Spin, a type of bias, is defined as use of reporting strategies to highlight that the experimental treatment is beneficial, despite a statistically non-significant difference in the primary outcome, or to distract the reader from statistically nonsignificant results [11].”
Endpoints for Assessing Drug Activity in Clinical Trials
Pazdur, R. (2008) · The Oncologist · doi:10.1634/theoncologist.13-s2-19 · Cited by 286 · cited 1×
“OS is the gold standard for the assessment of benefit: it is unambiguous and is not subject to investigator interpretation [23].”
BackgroundAdverse Events in Randomized Trials: Neglected, Restricted, Distorted, and Silenced
Ioannidis, J. P. A. (2009) · Archives of Internal Medicine · doi:10.1001/archinternmed.2009.313 · Cited by 243 · cited 1×
“Reviews have shown that a substantial proportion of clinical trials have suboptimal reporting of harm—a large number of trials have shown deficiency in the report of toxicity, especially severe toxicity, graded as 3 and 4 [5].”
BackgroundFrequency and reasons for outcome reporting bias in clinical trials: interviews with trialists
Smyth, R. M. D., Kirkham, J. J., Jacoby, A., Altman, D. G., Gamble, C., & Williamson, P. R. (2011) · BMJ · doi:10.1136/bmj.c7153 · Cited by 220 · cited 1×
“The pharmaceutical industry is increasingly influential in clinical trial sponsorship with data showing an increase in industry sponsorship of phase III RCTs from 24 to 72% over a 30 year period [3, 19, 20].”
BackgroundIndexes and boundaries for “quantitative significance” in statistical decisions
Burnand, B., Kernan, W. N., & Feinstein, A. R. (1990) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(90)90093-5 · Cited by 209
Adjuvant Docetaxel for High-Risk, Node-Negative Breast Cancer
Martín, M., Seguí, M. A., Antón, A., Ruiz, A., Ramos, M., Adrover, E., Aranda, I., Rodríguez-Lescure, A., Große, R., Calvo, L., et al. (2010) · New England Journal of Medicine · doi:10.1056/nejmoa0910320 · Cited by 209
Outcome selection bias in meta-analysis
Williamson, P. R., Gamble, C., Altman, D. G., & Hutton, J. L. (2005) · Statistical Methods in Medical Research · doi:10.1191/0962280205sm415oa · Cited by 188
Limitations are not properly acknowledged in the scientific literature
Ioannidis, J. P. A. (2007) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.09.011 · Cited by 180 · cited 1×
“It is important to recognize the presence of bias and spin in reports of clinical trials, and to evaluate their importance when placing a RCT in context and ascribing a level of credibility [12].”
BackgroundBias Due to Changes in Specified Outcomes during the Systematic Review Process
Kirkham, J. J., Altman, D. G., & Williamson, P. R. (2010) · PLoS ONE · doi:10.1371/journal.pone.0009810 · Cited by 164 · cited 2×
“It is good practice to design RCTs with no more than three outcomes for which hypothesis testing is planned [4].”
BackgroundFamily physicians' use of medical abstracts to guide decision making: style or substance?
Barry, H. C., Ebell, M. H., Shaughnessy, A. F., Slawson, D. C., & Nietzke, F. (2001) · PubMed · Cited by 135
Amir, E., Šeruga, B., Kwong, R., Tannock, I. F., & Ocaña, A. (2011) · European Journal of Cancer · doi:10.1016/j.ejca.2011.10.028 · Cited by 133 · cited 1×
“For women with breast cancer, neither DFS nor PFS have been shown to be adequate surrogates for OS [25, 26] but 83.”
BackgroundSelf-citationWhen are ‘positive’ clinical trials in oncology truly positive?
Ocana, A., & Tannock, I. F. (2011) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djq463 · Cited by 110 · cited 1×
“For women with breast cancer, neither DFS nor PFS have been shown to be adequate surrogates for OS [25, 26] but 83.”
BackgroundSelf-citationBalancing benefits and harms in health care
Cuervo, L. G., & Clarke, M. (2003) · BMJ · doi:10.1136/bmj.327.7406.65 · Cited by 104
You, B., Gan, H. K., Pond, G., & Chen, E. X. (2012) · Journal of Clinical Oncology · doi:10.1200/jco.2011.37.0890 · Cited by 74 · cited 1×
“[27] evaluated reports of RCTs published between 2005 and 2009, and found that there was misinterpretation of the PE in 21.”
Method
Research works printed but not linked: 1
- Ocana, 2011, When are ‘positive’ clinical trials in oncology truly positive?, J Natl Cancer Inst, 103, 16, 10.1093/jnci/djq463 · DOI · Paper page
Cited by
Reducing waste from incomplete or unusable reports of biomedical research
Glasziou, P., Altman, D. G., Bossuyt, P., Boutron, I., Clarke, M., Julious, S., Michie, S., Moher, D., & Wager, E. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62228-x · Cited by 1,319
Cherny, N. I., Sullivan, R., Dafni, U., Kerst, J. M., Sobrero, A., Zielinski, C., de Vries, E. G. E., & Piccart, M. J. (2015) · Annals of Oncology · doi:10.1093/annonc/mdv249 · Cited by 818
“Overestimating or overstating the benefits from new intervention can cause harm: It confounds public policy decision making [29], undermines the credibility of oncology research reporting [26, 29, 30], harms patients who choose to undertake treatments based on exaggerated expectations that may subject them to either r…”
BackgroundBooth, C. M., & Tannock, I. F. (2014) · British Journal of Cancer · doi:10.1038/bjc.2013.725 · Cited by 469
“…therapy is providing benefit to patients when there is no difference in the primary outcome measure as compared with the control arm and/or increased toxicity associated with the new treatment (Ohorodnyk et al, 2009; Boutron et al, 2010; Altwairgi et al, 2012; Vera-Badillo et al, 2013).”
BackgroundSelf-citationBoutron, I., Altman, D. G., Hopewell, S., Vera-Badillo, F., Tannock, I., & Ravaud, P. (2014) · Journal of Clinical Oncology · doi:10.1200/jco.2014.56.7503 · Cited by 368
Self-citation‘Spin’ in published biomedical literature: A methodological systematic review
Chiu, K., Grundy, Q., & Bero, L. (2017) · PLOS Biology · doi:10.1371/journal.pbio.2002173 · Cited by 317
Metadata
- Authors
- Vera-Badillo, F. E., Shapiro, R., Ocana, A., Amir, E., & Tannock, I. F.
- Affiliations
- University of Toronto (CA)
- Corresponding author
- Tannock, I.F.
- Venue
- Annals of Oncology
- Publisher
- Elsevier BV
- Published
- 2013-05
- Volume
- 24
- Issue
- 5
- Pages
- 1238-1244
- ISSN
- 0923-7534 (ISSN-L)
- OpenAlex
- W2120241552
- Cited by
- 117
- References
- 27
- Citations by year
- 2026: 3 · 2025: 1 · 2023: 3 · 2022: 4 · 2021: 8 · 2020: 8 · 2019: 13 · 2018: 11 · 2017: 8 · 2016: 13 · 2015: 13 · 2014: 14 · 2013: 18
- FWCI
- 21.33
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/open-access/userlicense/1.0/ (vor, 2442 d)
- Locations
- Annals of Oncology (publishedVersion)
- PubMed (publishedVersion)
- HighWire Press Open Archive (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Health Sciences › Nursing › Issues, ethics and legal aspects › Pharmaceutical industry and healthcare
- Keywords
- breast cancer, reporting bias, primary endpoint, adverse event reporting, randomized controlled trials, women, abstract reporting, negative trial results, funding disclosure
- MeSH terms
- Algorithms, Antineoplastic Agents, Breast Neoplasms, Female, Humans, Research Design, Observer Variation, Bias, Randomized Controlled Trials as Topic, Journal Impact Factor
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.