Bias due to selective inclusion and reporting of outcomes and analyses in systematic reviews of randomised trials of healthcare interventions
The DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Abstract
This is a protocol for a Cochrane Review (Methodology). The objectives are as follows: To summarise the characteristics and synthesise the results of empirical studies that have (i) investigated the prevalence of selective inclusion or reporting in systematic reviews of RCTs, (ii) investigated the factors associated with this (e.g. statistical significance or direction of effect), and (iii) quantified the bias.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Chan, A.-W., Hróbjartsson, A., Haahr, M. T., Gøtzsche, P. C., & Altman, D. G. (2004) · JAMA · doi:10.1001/jama.291.20.2457 · Cited by 1,845
Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias
Dwan, K., Altman, D. G., Arnaiz, J. A., Bloom, J., Chan, A.-W., Cronin, E., Decullier, E., Easterbrook, P. J., Von Elm, E., Gamble, C., et al. (2008) · PLoS ONE · doi:10.1371/journal.pone.0003081 · Cited by 1,546
Self-citationThe Existence of Publication Bias and Risk Factors for Its Occurrence
Dickersin, K. (1990) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1990.03440100097014 · Cited by 1,417
Kirkham, J. J., Dwan, K. M., Altman, D. G., Gamble, C., Dodd, S., Smyth, R., & Williamson, P. R. (2010) · BMJ · doi:10.1136/bmj.c365 · Cited by 1,133
Self-citationDissemination and publication of research findings: an updated review of related biases
Song, F., Parekh, Hooper, L., Loke, Y., Ryder, J., Sutton, A., Hing, C., Kwok, C., Pang, C., & Harvey, I. (2010) · Health Technology Assessment · doi:10.3310/hta14080 · Cited by 1,039
Publication bias—a problem in interpreting medical data
Begg, C. B., & Berlin, J. A. (1988) · Journal of the Royal Statistical Society. Series A (Statistics in Society) · doi:10.2307/2982993 · Cited by 814
Comparison of Registered and Published Primary Outcomes in Randomized Controlled Trials
Mathieu, S. (2009) · JAMA · doi:10.1001/jama.2009.1242 · Cited by 730
Chan, A.-W., & Altman, D. G. (2005) · BMJ · doi:10.1136/bmj.38356.424606.8f · Cited by 613
An international registry of systematic-review protocols
Booth, A., Clarke, M., Ghersi, D., Moher, D., Petticrew, M., & Stewart, L. (2011) · The Lancet · doi:10.1016/s0140-6736(10)60903-8 · Cited by 512
Standardising outcomes for clinical trials and systematic reviews
Clarke, M. (2007) · Trials · doi:10.1186/1745-6215-8-39 · Cited by 483
Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research
Chan, A.-W. (2004) · Canadian Medical Association Journal · doi:10.1503/cmaj.1041086 · Cited by 444
Publication and related biases.
Song, Eastwood, Gilbody, Duley, & Sutton (2000) · Health Technology Assessment · doi:10.3310/hta4100 · Cited by 427
Comparison of protocols and registry entries to published reports for randomised controlled trials
Dwan, K., Altman, D. G., Cresswell, L., Blundell, M., Gamble, C. L., & Williamson, P. R. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000031.pub2 · Cited by 243
Self-citationChan, A.-W., Hrobjartsson, A., Jorgensen, K. J., Gotzsche, P. C., & Altman, D. G. (2008) · BMJ · doi:10.1136/bmj.a2299 · Cited by 202
Outcome selection bias in meta-analysis
Williamson, P. R., Gamble, C., Altman, D. G., & Hutton, J. L. (2005) · Statistical Methods in Medical Research · doi:10.1191/0962280205sm415oa · Cited by 188
Bias Due to Changes in Specified Outcomes during the Systematic Review Process
Kirkham, J. J., Altman, D. G., & Williamson, P. R. (2010) · PLoS ONE · doi:10.1371/journal.pone.0009810 · Cited by 164
Self-citationIdentification and impact of outcome selection bias in meta‐analysis
Williamson, P. R., & Gamble, C. (2005) · Statistics in Medicine · doi:10.1002/sim.2025 · Cited by 132
Booth, A., Clarke, M., Ghersi, D., Moher, D., Petticrew, M., & Stewart, L. (2011) · PLoS ONE · doi:10.1371/journal.pone.0027319 · Cited by 96
Tendal, B., Higgins, J. P. T., Juni, P., Hrobjartsson, A., Trelle, S., Nuesch, E., Wandel, S., Jorgensen, A. W., Gesser, K., Ilsoe-Kristensen, S., et al. (2009) · BMJ · doi:10.1136/bmj.b3128 · Cited by 77
Selective outcome reporting: telling and detecting true lies. The state of the science
Macura, A., Abraha, I., Kirkham, J., Gensini, G. F., Moja, L., & Iorio, A. (2010) · Internal and Emergency Medicine · doi:10.1007/s11739-010-0371-z · Cited by 19
Self-citation
Research works printed but not linked: 20
- Bax, 2011, Beyond publication bias, Journal of Clinical Epidemiology, 64, 459, 10.1016/j.jclinepi.2010.09.003 · DOI
- Bender, 2008, Attention should be given to multiplicity issues in systematic reviews, Journal of Clinical Epidemiology, 61, 857, 10.1016/j.jclinepi.2008.03.004 · DOI · Paper page
- Bjordal, 2005, Can Cochrane reviews in controversial areas be biased? A sensitivity analysis based on the protocol of a systematic Cochrane review on low-level laser therapy in osteoarthritis, Photomedicine and Laser Surgery, 23, 453, 10.1089/pho.2005.23.453 · DOI
- Chalmers, 1990, Underreporting research is scientific misconduct, JAMA, 263, 1405, 10.1001/jama.1990.03440100121018 · DOI · Paper page
- Deeks JJ Higgins JPT Altman DG Chapter 9: Analysing data and undertaking meta-analyses. In: Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011 Available from www.cochrane-handbook.org
- Felson, 1992, Bias in meta analytic research, Journal of Clinical Epidemiology, 45, 885, 10.1016/0895-4356(92)90072-U · DOI · Paper page
- Higgins, 2002, Quantifying heterogeneity in a meta-analysis, Statistics in Medicine, 21, 1539, 10.1002/sim.1186 · DOI · Paper page
- Higgins JPT Altman DG Sterne JAC Chapter 8: Assessing risk of bias in included studies. In: Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011 Available from www.cochrane-handbook.org
- Hutton, 2000, Bias in meta-analysis due to outcome variable selection within studies, Applied Statistics, 49, 359
- Liberati, 2009, The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration, Journal of Clinical Epidemiology, 62, e1, 10.1016/j.jclinepi.2009.06.006 · DOI · Paper page
- Moher, 2007, Epidemiology and reporting characteristics of systematic reviews, PLoS Medicine, 4, e78, 10.1371/journal.pmed.0040078 · DOI · Paper page
- Moher, 2009, Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement, Journal of Clinical Epidemiology, 62, 1006, 10.1016/j.jclinepi.2009.06.005 · DOI · Paper page
- PLoS Medicine, 2011, Best practice in systematic reviews: the importance of protocols and registration, PLoS Medicine, 8, e1001009, 10.1371/journal.pmed.1001009 · DOI · Paper page
- The Nordic Cochrane Centre, The Cochrane Collaboration Review Manager (RevMan) 2011
- Schünemann HJ Oxman AD Vist GE Higgins JPT Deeks JJ Glasziou P Chapter 12: Interpreting results and drawing conclusions. In: Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011 Available from www.cochrane-handbook.org
- Sterne JAC Egger M Moher D Chapter 10: Addressing reporting biases. In: Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011 Available from www.cochrane-handbook.org
- Tannock, 1996, False-positive results in clinical trials: multiple significance tests and the problem of unreported comparisons, Journal of the National Cancer Institute, 88, 206, 10.1093/jnci/88.3-4.206 · DOI
- Tendal B Nuesch E Higgins JPT Juni P Gotzsche PC Multiplicity of data in trial reports and the reliability of meta-analyses: an empirical study BMJ 2011 d4829 · DOI · Paper page
- Tricco, 2009, Non-Cochrane vs. Cochrane reviews were twice as likely to have positive conclusion statements: cross-sectional study, Journal of Clinical Epidemiology, 62, 380, 10.1016/j.jclinepi.2008.08.008 · DOI
- Tricco, 2009, An international survey indicated that unpublished systematic reviews exist, Journal of Clinical Epidemiology, 62, 617, 10.1016/j.jclinepi.2008.09.014 · DOI
Cited by
Shamseer, L., Moher, D., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., Stewart, L. A., & the PRISMA-P Group (2015) · BMJ · doi:10.1136/bmj.g7647 · Cited by 13,120
CINeMA: an approach for assessing confidence in the results of a network meta-analysis
Nikolakopoulou, A., Higgins, J. P. T., Papakonstantinou, T., Chaimani, A., Del Giovane, C., Egger, M., & Salanti, G. (2020) · PLOS Medicine · doi:10.1371/journal.pmed.1003082 · Cited by 1,935
Page, M. J., Moher, D., Bossuyt, P., Boutron, I., Hoffmann, T., mulrow, C., Shamseer, L., Tetzlaff, J., Akl, E., Brennan, S. E., et al. (2020) · doi:10.31222/osf.io/gwdhk · Cited by 916
Self-citationPRISMA harms checklist: improving harms reporting in systematic reviews
Zorzela, L., Loke, Y. K., Ioannidis, J. P., Golder, S., Santaguida, P., Altman, D. G., Moher, D., Vohra, S., & PRISMA harms group (2016) · BMJ · doi:10.1136/bmj.i157 · Cited by 557
‘Spin’ in published biomedical literature: A methodological systematic review
Chiu, K., Grundy, Q., & Bero, L. (2017) · PLOS Biology · doi:10.1371/journal.pbio.2002173 · Cited by 317
Metadata
- Authors
- Page, M. J., McKenzie, J. E., Dwan, K., Kirkham, J., Green, S., & Forbes, A.
- Venue
- Cochrane Database of Systematic Reviews
- Publisher
- John Wiley & Sons, Ltd
- Published
- 2012-05-16
- ISSN
- 1361-6137 (ISSN-L)
- OpenAlex
- W4234931888
- Cited by
- 30
- References
- 20
- Type
- reference entry
- Language
- en
- Update notices
- new_version (2014-10-01)
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.