Cohort study of trials submitted to ethics committee identified discrepant reporting of outcomes in publications
Abstract
Objectives To identify factors associated with discrepant outcome reporting in randomized drug trials.Study design and setting Cohort study of protocols submitted to a Swiss ethics committee 1988-1998: 227 protocols and amendments were compared with 333 matching articles published during 1990-2008. Discrepant reporting was defined as addition, omission, or reclassification of outcomes.Results Overall, 870 of 2,966 unique outcomes were reported discrepantly (29.3%). Among protocol-defined primary outcomes, 6.9% were not reported (19 of 274), whereas 10.4% of reported outcomes (30 of 288) were not defined in the protocol. Corresponding percentages for secondary outcomes were 19.0% (284 of 1,495) and 14.1% (334 of 2,375). Discrepant reporting was more likely if P values were <0.05 compared with P ≥ 0.05 [adjusted odds ratio (aOR): 1.38; 95% confidence interval (CI): 1.07, 1.78], more likely for efficacy compared with harm outcomes (aOR: 2.99; 95% CI: 2.08, 4.30) and more likely for composite than for single outcomes (aOR: 1.48; 95% CI: 1.00, 2.20). Cardiology (aOR: 2.34; 95% CI: 1.44, 3.79) and infectious diseases (aOR: 1.77; 95% CI: 1.01, 3.13) had more discrepancies compared with all specialties combined.Conclusion Discrepant reporting was associated with statistical significance of results, type of outcome, and specialty area. Trial protocols should be made freely available, and the publications should describe and justify any changes made to protocol-defined outcomes.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Problems arise when trial protocols do not designate any primary outcome.
“Problems also arise when trial protocols do not designate any primary outcome, as seen in half of protocols for a sample of trials published from 2002 to 2008,254 and in 18% and 25% of randomised trial protocols that received ethics approval in Switzerland and Denmark, respectively.249 255”
From SPIRIT 2025 explanation and elaboration: updated guideline for protocols of randomised trials (2025) · cites this paper as [255]
18% and 25% of randomised trial protocols that received ethics approval in Switzerland and Denmark, respectively, did not designate any primary outcome.
“and in 18% and 25% of randomised trial protocols that received ethics approval in Switzerland and Denmark, respectively.249 255”
From SPIRIT 2025 explanation and elaboration: updated guideline for protocols of randomised trials (2025) · cites this paper as [255]
Even when a report exists, the P value, size, or direction of findings can lead to some assessed outcomes being omitted or only partly reported (selective nonreporting or outcome reporting bias).
“Even when a study report is available, results for some outcomes that were assessed may be missing or incompletely reported because of the P value, or the magnitude or direction of the results (selective nonreporting bias or outcome reporting bias).”
From Investigating and dealing with publication bias and other reporting biases in meta-analyses of health research: a review (2021) · cites this paper as [24]
Studies comparing journal articles with protocols, trial register entries, or regulator documents have found frequent omission of prespecified outcomes.
“Several studies have compared journal article reports of studies with their corresponding protocols 23,24 , trials register entries 25,26 , or documents submitted to regulators 27 , and identified frequent nonreporting of results for outcomes that were prespecified.”
From Investigating and dealing with publication bias and other reporting biases in meta-analyses of health research: a review (2021) · cites this paper as [24]
The largest such study compared 227 protocols and amendments with 333 matching articles from 1990 to 2008; it found that 7% of primary outcomes and 19% of secondary outcomes defined in protocols were not reported.
“The largest such study compared 227 protocols This article is protected by copyright.All rights reserved.and amendments with 333 matching articles published between 1990 and 2008 24 and found that 7% of protocol-defined primary outcomes and 19% of secondary outcomes were not reported.”
From Investigating and dealing with publication bias and other reporting biases in meta-analyses of health research: a review (2021) · cites this paper as [24]
Cites
Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gotzsche, P. C., Devereaux, P. J., Elbourne, D., Egger, M., & Altman, D. G. (2010) · BMJ · doi:10.1136/bmj.c869 · Cited by 8,883 · see citing works
Self-citationImproving the Quality of Reporting of Randomized Controlled Trials: The CONSORT Statement
Begg, C. (1996) · JAMA · doi:10.1001/jama.1996.03540080059030 · Cited by 3,127 · see citing works
Publication bias in clinical research
Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R. (1991) · The Lancet · doi:10.1016/0140-6736(91)90201-y · Cited by 3,109 · cited 1× · see citing works
“This result must be interpreted in the context of our previous analysis of the determinants of publication of drug trials: we and others found that the probability of publication was reduced with commercial funding [11,14,15].”
ResultGluud, L. L., Villumsen, J., & Gluud, C. N. (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-135-11-200112040-00010 · Cited by 2,262 · see citing works
Chan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004) · JAMA · doi:10.1001/jama.291.20.2457 · Cited by 1,845 · see citing works
<i>meta-analysis</i> bias in location and selection of studies
Egger, M., & Smith, G. D. (1998) · BMJ · doi:10.1136/bmj.316.7124.61 · Cited by 1,374 · see citing works
Self-citationWhittington, C. J., Kendall, T., Fonagy, P., Cottrell, D. J., Cotgrove, A. J., & Boddington, E. (2004) · The Lancet · doi:10.1016/s0140-6736(04)16043-1 · Cited by 1,055 · see citing works
Dickersin, K., Yuan, M.-Q., & Meinert, C. (1992) · Journal of the American Medical Association (JAMA) · doi:10.1001/jama.1992.03480030052036 · Cited by 900 · see citing works
Publication bias in clinical trials due to statistical significance or direction of trial results
Hopewell, S., Loudon, K., Clarke, M. J., Oxman, A. D., & Dickersin, K. (2009) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000006.pub3 · Cited by 898 · see citing works
Publication bias: evidence of delayed publication in a cohort study of clinical research projects
Stern, J. M., & Simes, R. J. (1997) · BMJ · doi:10.1136/bmj.315.7109.640 · Cited by 818 · see citing works
Cha, M., Eastwaad, S., Hartan, R., Maher, D., Olkin, I., Pitkin, R., Rennie, D., Schulz, K. F., Simel, D. L., & Straup, D. F. (1996) · Cited by 736 · see citing works
Comparison of Registered and Published Primary Outcomes in Randomized Controlled Trials
Mathieu, S. (2009) · JAMA · doi:10.1001/jama.2009.1242 · Cited by 730 · see citing works
Chan, A.-W., & Altman, D. G. (2005) · BMJ · doi:10.1136/bmj.38356.424606.8f · Cited by 613 · cited 2× · see citing works
“Among 281 authors who denied the existence of unreported outcomes, there was evidence to the contrary in their publications in one-third of authors [19].”
BackgroundWithholding Research Results in Academic Life Science
Blumenthal, D. (1997) · JAMA · doi:10.1001/jama.1997.03540390054035 · Cited by 581 · see citing works
Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research
Chan, A.-W. (2004) · Canadian Medical Association Journal · doi:10.1503/cmaj.1041086 · Cited by 444 · see citing works
Córdoba, G., Schwartz, L., Woloshin, S., Bae, H. T., & Gøtzsche, P. C. (2010) · BMJ · doi:10.1136/bmj.c3920 · Cited by 404 · see citing works
The Impact of Model Selection on Inference in Linear Regression
Hurvich, C. M., & Tsai, C. (1990) · The American Statistician · doi:10.1080/00031305.1990.10475722 · Cited by 279 · see citing works
Comparison of protocols and registry entries to published reports for randomised controlled trials
Dwan, K., Altman, D. G., Cresswell, L., Blundell, M., Gamble, C. L., & Williamson, P. R. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000031.pub2 · Cited by 243 · see citing works
Bias in meta-analytic research
Felson, D. T. (1992) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(92)90072-u · Cited by 231 · see citing works
Development of The Cochrane Collaboration's Central Register of Controlled Clinical Trials
Dickersin, K., Manheimer, E., Wieland, S., Robinson, K. A., Lefebvre, C., & McDonald, S. (2002) · Evaluation & the Health Professions · doi:10.1177/0163278702025001004 · Cited by 198 · see citing works
Blümle, A., Huwiler, K., & Witschi, M. (2008) · Swiss Medical Weekly · doi:10.4414/smw.2008.12027 · Cited by 112 · see citing works
Self-citationHahn, S., Williamson, P. R., & Hutton, J. L. (2002) · Journal of Evaluation in Clinical Practice · doi:10.1046/j.1365-2753.2002.00314.x · Cited by 97 · see citing works
Comparison of protocols and registry entries to published reports for randomised controlled trials
Dwan, K. (2011) · Journal of Evidence-Based Medicine · doi:10.1111/j.1756-5391.2011.01149.x · Cited by 38 · see citing works
Reporting of noninferiority trials was incomplete in trial registries
Dekkers, O. M., Soonawala, D., Vandenbroucke, J. P., & Egger, M. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.12.008 · Cited by 15 · see citing works
Self-citation
Flagged first, then by citations
1 of the 25 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Page, M. J., Sterne, J. A. C., Higgins, J. P. T., & Egger, M. (2021) · Research Synthesis Methods · doi:10.1002/jrsm.1468 · Cited by 294 · see citing works
Self-citationSPIRIT 2025 explanation and elaboration: updated guideline for protocols of randomised trials
Hróbjartsson, A., Boutron, I., Hopewell, S., Moher, D., Schulz, K. F., Collins, G. S., Tunn, R., Aggarwal, R., Berkwits, M., Berlin, J. A., et al. (2025) · BMJ · doi:10.1136/bmj-2024-081660 · Cited by 171 · see citing works
Finding the power to reduce publication bias
Stanley, T. D., Doucouliagos, H., & Ioannidis, J. P. A. (2017) · Statistics in Medicine · doi:10.1002/sim.7228 · Cited by 131 · see citing works
Promoting public access to clinical trial protocols: challenges and recommendations
Chan, A.-W., & Hróbjartsson, A. (2018) · Trials · doi:10.1186/s13063-018-2510-1 · Cited by 84 · see citing works
Li, G., Abbade, L. P. F., Nwosu, I., Jin, Y., Leenus, A., Maaz, M., Wang, M., Bhatt, M., Zielinski, L., Sanger, N., et al. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-017-0465-7 · Cited by 67 · see citing works
Bohlius, J., Tonia, T., Nüesch, E., Jüni, P., Fey, M. F., Egger, M., & Bernhard, J. (2014) · British Journal of Cancer · doi:10.1038/bjc.2014.171 · Cited by 66 · see citing works
Self-citationButcher, N. J., Monsour, A., Mew, E. J., Szatmári, P., Pierro, A., Kelly, L. E., Farid‐Kapadia, M., Chee-A-Tow, A., Saeed, L. H., Monga, S., et al. (2019) · Trials · doi:10.1186/s13063-019-3248-0 · Cited by 59 · see citing works
Outcome reporting recommendations for clinical trial protocols and reports: a scoping review
Butcher, N. J., Mew, E. J., Monsour, A., Chan, A., Moher, D., & Offringa, M. (2020) · Trials · doi:10.1186/s13063-020-04440-w · Cited by 47 · see citing works
TARG Meta-Research Group & Collaborators (2023) · BMJ Open · doi:10.1136/bmjopen-2023-076264 · Cited by 33 · see citing works
Perlmutter, A. S., Tran, V.-T., Dechartres, A., & Ravaud, P. (2017) · Annals of Oncology · doi:10.1093/annonc/mdw682 · Cited by 31 · see citing works
Time to publication for results of clinical trials
Showell, M. G., Cole, S., Clarke, M. J., DeVito, N., Farquhar, C. M., & Jordan, V. (2024) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000011.pub3 · Cited by 33 · see citing works
A retrospective analysis of dissemination biases in the brief alcohol intervention literature.
Tanner-Smith, E. E., & Polanin, J. R. (2015) · Psychology of Addictive Behaviors · doi:10.1037/adb0000014 · Cited by 26 · see citing works
“Prospective registration of research studies may be one way to promote complete outcome reporting, although recent research suggests this may not be enough (Dwan et al., 2011; Mathieu et al., 2009; Rasmussen et al., 2009; Redmond et al., 2013).”
Backgroundvan Lent, M., Rongen, G. A. P. J. M., & Out, H. J. (2014) · BMC Medical Ethics · doi:10.1186/1472-6939-15-83 · Cited by 20 · see citing works
Dekkers, O. M., Cevallos, M., Bührer, J., Poncet, A., Rau, S. A., Perneger, T., & Egger, M. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.09.015 · Cited by 17 · see citing works
Self-citationThibault, R. T., Clark, R. W. A., Pedder, H., van den Akker, O. R., Westwood, S. J., Thompson, J., & Munafo, M. R. (2021) · medRxiv · doi:10.1101/2021.07.07.21259868 · Cited by 18 · see citing works
Stanley, T. D., & Massey, S. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.03.024 · Cited by 17 · see citing works
Stanley, T. D., Ioannidis, J. P. A., Maier, M., Doucouliagos, H., Otte, W. M., & Bartos̆, F. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.03.004 · Cited by 15 · see citing works
Alqaidoom, Z., Nguyen, P., Awadh, M., & Page, M. J. (2022) · Research Synthesis Methods · doi:10.1002/jrsm.1583 · Cited by 15 · see citing works
Peng, S., Liu, S., Lei, J., Ren, W., Xiao, L., Liu, X., Lü, M., & Zhou, K. (2022) · PLoS ONE · doi:10.1371/journal.pone.0266780 · Cited by 15 · see citing works
Peer review before trial conduct could increase research value and reduce waste
Siebert, M., Naudet, F., & Ioannidis, J. P. A. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.05.024 · Cited by 10 · see citing works
Investigating and Dealing with Publication Bias and Other Reporting Biases
Page, M. J., Sterne, J. A. C., Higgins, J. P. T., & Egger, M. (2022) · Wiley eBooks · doi:10.1002/9781119099369.ch5 · Cited by 9 · see citing works
Self-citationGoswami, N., Tsalik, E., Naggie, S., Miller, W., Horton, J., Pfeiffer, C. D., & Hicks, C. (2014) · Trials · doi:10.1186/1745-6215-15-31 · Cited by 7 · see citing works
Continuing review of ethics in clinical trials: a surveillance study in Iran.
Mohamadi, A., Asghari, F., & Rashidian, A. (2013) · Europe PMC (PubMed Central) · Cited by 6 · see citing works
Assessing the Risk of Bias in Randomized Trials
Page, M. J., Altman, D. G., & Egger, M. (2022) · doi:10.1002/9781119099369.ch4 · Cited by 3 · see citing works
Self-citationOn the Randomisation Gallery:Random Reflections on a Popular Method in Medical Research
Hardt, J. (2016) · Journal of Headache & Pain Management · doi:10.4172/2472-1913.100019 · Cited by 1 · see citing works
Outcome Reporting in Surgical Randomized Controlled Trials
Glen, P. (2016) · uO Research (University of Ottawa) · doi:10.20381/ruor-5642
JPEN Journal Club 56. Primary Outcomes
Koretz, R. L. (2020) · Journal of Parenteral and Enteral Nutrition · doi:10.1002/jpen.1947
The Nuts and Bolts of Critically Appraising Observational and Randomized Controlled Studies
Koretz, R. L. (2021) · Nutrition in Clinical Practice · doi:10.1002/ncp.10620
Butcher, N. J., Mew, E. J., Monsour, A., Chan, A.-W., Moher, D., & Offringa, M. (2020) · Figshare · doi:10.6084/m9.figshare.12629389
Butcher, N. J., Mew, E. J., Monsour, A., Chan, A.-W., Moher, D., & Offringa, M. (2020) · Figshare · doi:10.6084/m9.figshare.12629395
Butcher, N. J., Mew, E. J., Monsour, A., Chan, A.-W., Moher, D., & Offringa, M. (2020) · Figshare · doi:10.6084/m9.figshare.12629401
Butcher, N. J., Mew, E. J., Monsour, A., Chan, A.-W., Moher, D., & Offringa, M. (2020) · Figshare · doi:10.6084/m9.figshare.12629392
- Authors
- Redmond, S., von Elm, E., Blümle, A., Gengler, M., Gsponer, T., & Egger, M.
- Affiliations
- University of Bern (CH)
- University Medical Center Freiburg (DE)
- Corresponding author
- Egger, Matthias
- Venue
- Journal of Clinical Epidemiology
- Publisher
- Elsevier BV
- Published
- 2013-12
- Volume
- 66
- Issue
- 12
- Pages
- 1367-1375
- ISSN
- 0895-4356 (ISSN-L)
- OpenAlex
- W2089918264
- Cited by
- 32
- References
- 25
- Citations by year
- 2025: 1 · 2024: 1 · 2023: 3 · 2022: 5 · 2021: 2 · 2020: 7 · 2019: 1 · 2018: 2 · 2017: 1 · 2016: 4 · 2014: 4 · 2013: 1
- FWCI
- 3.30
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Clinical Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- Bern Open Repository and Information System (University of Bern) (submittedVersion)
- IRIS (submittedVersion)
- SERVAL (Université de Lausanne) (submittedVersion)
- Funders
- University of Bern
- Compute Canada
- Inselspital, Universitätsspital Bern
- Canadian Institutes of Health Research
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Life Sciences › Pharmacology, Toxicology and Pharmaceutics › Pharmacology › Pharmaceutical industry and healthcare
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Keywords
- primary and secondary outcomes, statistical significance, composite endpoints
- MeSH terms
- Drug Therapy, Humans, Switzerland, Cohort Studies, Randomized Controlled Trials as Topic, Treatment Outcome, Publication Bias, Ethics Committees, Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
