CoChat
Sign Up
Cited by 122Meta-analysis and systematic reviewsStatistical Methods in Clinical TrialsPharmaceutical industry and healthcare

The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.

Adjustment for reporting bias in network meta-analysis of antidepressant trials

Open access
Trinquart, Ludovic, Chatellier, Gilles, Ravaud, Philippe·2012·BMC Medical Research Methodology
DOI 10.1186/1471-2288-12-150PMC PMC3537713
Open this paper in CoChat — free accountOpen PDF

Abstract

BACKGROUND: Network meta-analysis (NMA), a generalization of conventional MA, allows for assessing the relative effectiveness of multiple interventions. Reporting bias is a major threat to the validity of MA and NMA. Numerous methods are available to assess the robustness of MA results to reporting bias. We aimed to extend such methods to NMA. METHODS: We introduced 2 adjustment models for Bayesian NMA. First, we extended a meta-regression model that allows the effect size to depend on its standard error. Second, we used a selection model that estimates the propensity of trial results being published and in which trials with lower propensity are weighted up in the NMA model. Both models rely on the assumption that biases are exchangeable across the network. We applied the models to 2 networks of placebo-controlled trials of 12 antidepressants, with 74 trials in the US Food and Drug Administration (FDA) database but only 51 with published results. NMA and adjustment models were used to estimate the effects of the 12 drugs relative to placebo, the 66 effect sizes for all possible pair-wise comparisons between drugs, probabilities of being the best drug and ranking of drugs. We compared the results from the 2 adjustment models applied to published data and NMAs of published data and NMAs of FDA data, considered as representing the totality of the data. RESULTS: Both adjustment models showed reduced estimated effects for the 12 drugs relative to the placebo as compared with NMA of published data. Pair-wise effect sizes between drugs, probabilities of being the best drug and ranking of drugs were modified. Estimated drug effects relative to the placebo from both adjustment models were corrected (i.e., similar to those from NMA of FDA data) for some drugs but not others, which resulted in differences in pair-wise effect sizes between drugs and ranking. CONCLUSIONS: In this case study, adjustment models showed that NMA of published data was not robust to reporting bias and provided estimates closer to that of NMA of FDA data, although not optimal. The validity of such methods depends on the number of trials in the network and the assumption that conventional MAs in the network share a common mean bias mechanism.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/1471-2288-12-150

Cites

57 of 57 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Contour-enhanced funnel plots forthe antidepressant trials withpublished results. Each funnel plot is the scatter plot of the treatment effect estimates from individual trials against the associated standard errors; the vertical solid line represents the pooled estimate. In the absence of reporting bias, we might expect a symmetrical funnel plot. We may find the funnel plot is not symmetrical, ie does not resemble an inverted funnel, which may be due to reporting bias, however there are other possible sources of asymmetry. The contour lines represent perceived milestones of statistical significance (long dash p = 0.1; dash p = 0.05; short dash p = 0.01). If studies seem to be missing in areas of non-significance then asymmetry may be due to reporting bias rather than other factors.
Figure 1 Contour-enhanced funnel plots forthe antidepressant trials withpublished results. Each funnel plot is the scatter plot of the treatment effect estimates from individual trials against the associated standard errors; the vertical solid line represents the pooled estimate. In the absence of reporting bias, we might expect a symmetrical funnel plot. We may find the funnel plot is not symmetrical, ie does not resemble an inverted funnel, which may be due to reporting bias, however there are other possible sources of asymmetry. The contour lines represent perceived milestones of statistical significance (long dash p = 0.1; dash p = 0.05; short dash p = 0.01). If studies seem to be missing in areas of non-significance then asymmetry may be due to reporting bias rather than other factors.CC BY · PubMed Central
Figure 2 Difference plots of estimatesof pair-wise comparisons ofthe 12 antidepressant agentsand placebo: regression modelof published data vs.standard network meta-analysis (NMA)model of published data(left panel); selection modelsof published vs. standardNMA model of publisheddata (right panel). The x-axes show the estimates from the standard NMA model applied to published data, the y-axes show the differences between the estimates from the adjustment (regression or selection) model of published data and the estimates from the standard NMA model of published data. Black dots are the 12 estimated drug effects relative to placebo; white dots are the 66 possible pair-wise comparisons between the 12 drugs.
Figure 2 Difference plots of estimatesof pair-wise comparisons ofthe 12 antidepressant agentsand placebo: regression modelof published data vs.standard network meta-analysis (NMA)model of published data(left panel); selection modelsof published vs. standardNMA model of publisheddata (right panel). The x-axes show the estimates from the standard NMA model applied to published data, the y-axes show the differences between the estimates from the adjustment (regression or selection) model of published data and the estimates from the standard NMA model of published data. Black dots are the 12 estimated drug effects relative to placebo; white dots are the 66 possible pair-wise comparisons between the 12 drugs.CC BY · PubMed Central
Figure 3 Probabilities that each antidepressantdrug is the bestaccording to standard NMAof FDA data, regressionmodel, selection model orstandard NMA model ofpublished data.
Figure 3 Probabilities that each antidepressantdrug is the bestaccording to standard NMAof FDA data, regressionmodel, selection model orstandard NMA model ofpublished data.CC BY · PubMed Central
Figure 4 Cumulative ranking probability plotsfor the 12 antidepressantagents from the standardNMA model applied toFDA data (bold solidline) and published data(bold dotted line) andfrom the 2 adjustmentmodels applied to publisheddata (regression model inplain dashed line andselection model in plaindouble-dashed line). On each plot, the x-axis shows possible ranks from r = 1 up to r = 13 and the y-axis shows the cumulative probabilities that the corresponding treatment is among the top r treatments. The closer the curve is to the upper left corner, the better the treatment. The surface under the cumulative ranking line is 1 when a treatment is the best and 0 when a treatment is the worst. FDA: standard NMA model applied to FDA data (bold plain line); Pub.: standard NMA model applied to published data (bold dash line); Reg.: regression model applied to published data (dash line); Sln.: selection model applied to published data (long-dash short-dash line).
Figure 4 Cumulative ranking probability plotsfor the 12 antidepressantagents from the standardNMA model applied toFDA data (bold solidline) and published data(bold dotted line) andfrom the 2 adjustmentmodels applied to publisheddata (regression model inplain dashed line andselection model in plaindouble-dashed line). On each plot, the x-axis shows possible ranks from r = 1 up to r = 13 and the y-axis shows the cumulative probabilities that the corresponding treatment is among the top r treatments. The closer the curve is to the upper left corner, the better the treatment. The surface under the cumulative ranking line is 1 when a treatment is the best and 0 when a treatment is the worst. FDA: standard NMA model applied to FDA data (bold plain line); Pub.: standard NMA model applied to published data (bold dash line); Reg.: regression model applied to published data (dash line); Sln.: selection model applied to published data (long-dash short-dash line).CC BY · PubMed Central
Figure 5 Difference plots of estimatesof pair-wise comparisons ofthe 12 antidepressant agentsand placebo: standard NMAmodel of published datavs. standard NMA modelof FDA data (upperpanel); regression model ofpublished data vs. standardNMA model of FDAdata (bottom left panel);selection model of publishedvs. standard NMA modelof FDA data (bottomright panel). The x-axes show the estimates from the standard NMA model applied to FDA data, the y-axes show the differences between the estimates from the adjustment (regression or selection) model of published data and the estimates from the standard NMA model of FDA data. Black dots are the 12 estimated drug effects relative to placebo; white dots are the 66 possible pair-wise comparisons between the 12 drugs.
Figure 5 Difference plots of estimatesof pair-wise comparisons ofthe 12 antidepressant agentsand placebo: standard NMAmodel of published datavs. standard NMA modelof FDA data (upperpanel); regression model ofpublished data vs. standardNMA model of FDAdata (bottom left panel);selection model of publishedvs. standard NMA modelof FDA data (bottomright panel). The x-axes show the estimates from the standard NMA model applied to FDA data, the y-axes show the differences between the estimates from the adjustment (regression or selection) model of published data and the estimates from the standard NMA model of FDA data. Black dots are the 12 estimated drug effects relative to placebo; white dots are the 66 possible pair-wise comparisons between the 12 drugs.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 55 references

Cited by

Sorted by citations

See all 122

Related

Metadata

Authors
Trinquart, L., Chatellier, G., & Ravaud, P.
Affiliations
  • Cochrane (GB)
  • Inserm (FR)
Corresponding author
Trinquart, Ludovic
ORCID
Venue
BMC Medical Research Methodology
Publisher
Springer Science and Business Media LLC
Published
2012-09-27
Volume
12
Issue
1
Pages
150-150
ISSN
1471-2288 (ISSN-L)
DOI
10.1186/1471-2288-12-150
PMC
PMC3537713
OpenAlex
W2103017208
Cited by
122
References
55
Citations by year
2026: 2 · 2025: 13 · 2024: 8 · 2023: 9 · 2022: 5 · 2021: 18 · 2020: 16 · 2019: 11 · 2018: 11 · 2017: 9 · 2016: 5 · 2015: 4 · 2014: 6 · 2013: 5
FWCI
1.32
Type
journal article
Language
en
Open access
Licence
Locations
  • BMC Medical Research Methodology (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • HAL (submittedVersion, other-oa)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
  • Health Sciences › Nursing › Issues, ethics and legal aspects › Pharmaceutical industry and healthcare
Keywords
reporting bias, network meta-analysis, antidepressants, Bayesian network meta-analysis, meta-regression, selection model, publication bias, drug ranking, bias correction
MeSH terms
Antidepressive Agents, Clinical Trials as Topic, Humans, Regression Analysis, Research Design, Meta-Analysis as Topic, Bias, Case-Control Studies, Publication Bias

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review