Why the Cochrane Risk of Bias Tool Should Include Funding Source as a Standard Item
Open accessThe DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Abstract
GP Kidd/Science Photo Library This editorial is based on a presentation given as part of a debate held at the Cochrane Methods Symposium in Québec City, Canada, October 2013. You can also read Jonathan Sterne's editorial presenting the case for not including funding source in the risk of bias tool. Rigorous systematic reviews aim to identify and reduce any bias that can adversely influence interpretation of study outcomes. The Cochrane Handbook defines bias as “a systematic error, or deviation from the truth, in results or inferences” and goes on to describe a number of biases, including allocation concealment and blinding, that have been shown empirically to systematically influence results.[1] As Cochrane strives to improve its methods of bias assessment, we must consider the empirical data showing that funding source is a risk of bias. A recent Cochrane Review provides evidence that there is bias associated with study funding sources.[2] The review examined the association of study funding source with clinical drug study outcomes, and the included studies defined ‘favorable’ results as those showing greater efficacy or less harm for the sponsor's product than for the comparator. Pharmaceutical industry‐sponsored studies were more likely to have favourable efficacy results (risk ratio 1.32, 95% confidence interval 1.21 to 1.44) and harm results (risk ratio 1.87, 95% confidence interval 1.54 to 2.27) than studies not sponsored by industry. The review found mixed results for effect size. This is completely expected since the effect sizes of different drugs can vary widely and depend on dose, the outcome being measured, and other factors. For example, the influence of reporting biases on effect sizes has varies considerably among drugs.[3] Industry‐sponsored studies have favourable conclusions more often than non‐industry‐sponsored studies, even when controlling for effect sizes.[4] Importantly, the consequence of industry funding is the same, independent of the effect size: industry‐funded trials show treatments to be more efficacious and less harmful than non‐industry funded trials. The current Cochrane risk of bias tool is insufficient to assess bias related to study funding sources. The Cochrane tool includes sequence generation, allocation sequence concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, and selective outcome reporting. The Cochrane Review that examined funding bias found no difference between industry‐funded and non‐industry‐funded drug studies in sequence generation, concealment of allocation, or loss to follow‐up.[2] Industry‐funded studies did have lower risk of bias related to blinding, but this is not surprising as many drug studies are conducted for regulatory approval and must adhere to certain study design standards. Furthermore, the lower risk of performance and detection bias in industry trials would be expected to lead to less, not more, favourable results. Since the observed funding bias cannot be fully explained by the study characteristics assessed with the Cochrane risk of bias tool, there must be other possible mechanisms by which this works. Bias related to funding source can results from systematic influences on how the study is actually conducted, the methodology of the study, whether the full results and analyses of the study are published, or a combination of these mechanisms.[5] Drug study results can be biased to maximise efficacy and minimise harm through such mechanisms as choice of inferior comparators (either by dose, drug or administration route),[6, 7] biased coding of outcomes,[6] bias in how data are analysed,[8] and selective outcome reporting and publication bias.[9] Risks of bias are not mutually exclusive; a study may have multiple risks of bias and we may not be able to identify all of them. Funding source bias is a known bias that should be assessed. Furthermore, bias may be related to funding source even when all studies are industry‐funded. For head‐to‐head comparisons of statins with other drugs, funding bias is associated with which statin manufacturer funds the study, and not just industry funding per se. The head‐to‐head comparison was more likely to favour the sponsor's product than the competitor drug, even when other risks of bias in the studies were taken into account.[10] So, when there is competition between drugs in a class, funding bias can influence the outcome to favour the commercial interests of the sponsor. If readers of Cochrane Reviews wanted to do their own assessment of bias related to funding source, they would find this difficult. Cochrane Reviews are not doing an adequate job of disclosing funding sources of included trials. Roseman et al have shown that the funding sources of the included trials were fully or partially disclosed in only 46 of 151 Cochrane Reviews and that these disclosures were scattered around seven different places in the reviews.[11] The MECIR (Method
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Industry sponsorship and research outcome
Lundh, A., Lexchin, J., Mintzes, B., Schroll, J. B., & Bero, L. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000033.pub3 · Cited by 1,697
Self-citationAls-Nielsen, B., Chen, W., Gluud, C., & Kjaergard, L. L. (2003) · JAMA · doi:10.1001/jama.290.7.921 · Cited by 746
Industry sponsorship and research outcome
Lundh, A., Sismondo, S., Lexchin, J., Busuioc, O. A., & Bero, L. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000033.pub2 · Cited by 605
Self-citationEffect of reporting bias on meta-analyses of drug trials: reanalysis of meta-analyses
Hart, B., Lundh, A., & Bero, L. (2012) · BMJ · doi:10.1136/bmj.d7202 · Cited by 255
Self-citationBero, L., Oostvogel, F., Bacchetti, P., & Lee, K. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040184 · Cited by 249
Self-citationDESIGN AND REPORTING MODIFICATIONS IN INDUSTRY-SPONSORED COMPARATIVE PSYCHOPHARMACOLOGY TRIALS
SAFER, D. J. (2002) · The Journal of Nervous and Mental Disease · doi:10.1097/00005053-200209000-00002 · Cited by 180
Cited by
Margulis, A. V., Pladevall, M., Riera-Guardia, N., Varas-Lorenzo, C., Hazell, L., Berkman, N., Viswanathan, M., & Perez-Gutthann, S. (2014) · Clinical Epidemiology · doi:10.2147/clep.s66677 · Cited by 441
Woodruff, T. J., & Sutton, P. (2014) · Environmental Health Perspectives · doi:10.1289/ehp.1307175 · Cited by 427
“This domain has been proposed—but not yet adopted—by Cochrane and GRADE as an important risk of bias (Bero 2013).”
MethodCatalá-López, F., Hutton, B., Núñez-Beltrán, A., Page, M. J., Ridao, M., Macías Saint-Gerons, D., Catalá, M. A., Tabarés-Seisdedos, R., & Moher, D. (2017) · PLOS ONE · doi:10.1371/journal.pone.0180355 · Cited by 396
Jørgensen, L., Paludan-Müller, A. S., Laursen, D. R. T., Savović, J., Boutron, I., Sterne, J. A. C., Higgins, J. P. T., & Hróbjartsson, A. (2016) · Systematic Reviews · doi:10.1186/s13643-016-0259-8 · Cited by 396
Methylphenidate for children and adolescents with attention deficit hyperactivity disorder (ADHD)
Storebø, O. J., Ramstad, E., Krogh, H. B., Nilausen, T. D., Skoog, M., Holmskov, M., Rosendal, S., Groth, C., Magnusson, F. L., Moreira-Maia, C. R., et al. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd009885.pub2 · Cited by 347
Metadata
- Authors
- Bero, L. A.
- Venue
- Cochrane Database of Systematic Reviews
- Publisher
- Wiley
- Published
- 2013-12-20
- Volume
- 2013
- Issue
- 12
- Pages
- ED000075-ED000075
- ISSN
- 1361-6137 (ISSN-L)
- 1465-1858
- OpenAlex
- W2428069039
- Cited by
- 96
- References
- 6
- FWCI
- 12.76
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Why the Cochrane Risk of Bias Tool Should Include Funding Source as a Standard Item. https://app.cochat.ai/papers/10.1002/14651858.ed000075
- The paper
- Bero, L. A. (2013). Why the Cochrane Risk of Bias Tool Should Include Funding Source as a Standard Item. Cochrane Database of Systematic Reviews. https://doi.org/10.1002/14651858.ed000075
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.