Using the contribution matrix to evaluate complex study limitations in a network meta-analysis: a case study of bipolar maintenance pharmacotherapy review
Open accessThe DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Limitations in the primary studies constitute one important factor to be considered in the grading of recommendations assessment, development, and evaluation (GRADE) system of rating quality of evidence. However, in the network meta-analysis (NMA), such evaluation poses a special challenge because each network estimate receives different amounts of contributions from various studies via direct as well as indirect routes and because some biases have directions whose repercussion in the network can be complicated. FINDINGS: In this report we use the NMA of maintenance pharmacotherapy of bipolar disorder (17 interventions, 33 studies) and demonstrate how to quantitatively evaluate the impact of study limitations using netweight, a STATA command for NMA. For each network estimate, the percentage of contributions from direct comparisons at high, moderate or low risk of bias were quantified, respectively. This method has proven flexible enough to accommodate complex biases with direction, such as the one due to the enrichment design seen in some trials of bipolar maintenance pharmacotherapy. CONCLUSIONS: Using netweight, therefore, we can evaluate in a transparent and quantitative manner how study limitations of individual studies in the NMA impact on the quality of evidence of each network estimate, even when such limitations have clear directions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures

![Fig. 2 Contributions from studies at high, moderate or low risk of bias to RR to prevent any mood episodes. ARP aripiprazole, CBZ carbamazepine, FLX fluoxetine, IMP imipramine, LIT lithium, LTG lamotrigine, OLZ olanzapine, OXC oxcarbazepine, PAL paliperidone, PLB placebo, QTP quetiapine, RisLAI risperidone long-acting injection, VPA valproate (Figure dapted from p. 98 of the Appendix in Miura et al. [4])](https://figures.cochat.ai/paper-figures/20a96efbabea40168c96b08c008c3f6b/using-the-contribution-matrix-to-evaluate-complex-study-limitations-in-a-network-figure-2.jpg)
![Fig. 3 Contributions from enriched vs non-enriched studies to RR to prevent depressive episodes. ARP aripiprazole, CBZ carbamazepine, FLX fluoxetine, IMP imipramine, LIT lithium, LTG lamotrigine, OLZ olanzapine, OXC oxcarbazepine, PAL paliperidone, PLB placebo, QTP quetiapine, RisLAI risperidone long-acting injection, VPA valproate (Figures adapted from p. 90 of the Appendix in Miura et al. [4])](https://figures.cochat.ai/paper-figures/44decd06055940cc86f34443ecf18721/using-the-contribution-matrix-to-evaluate-complex-study-limitations-in-a-network-figure-3.jpg)
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This limitation in the primary trials might affect the validity of the findings for some antidepressants, but the incorporation of both direct and indirect comparisons might have contributed to reduce the potential risk of bias.30”
From Comparative efficacy and acceptability of 21 antidepressant drugs for the acute treatment of adults with major depressive disorder: a systematic review and network meta-analysis (2018) · cites this paper as [30]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817
Dermyshi, E., Wang, Y., Yan, C., Hong, W., Qiu, G., Gong, X., & Zhang, T. (2017) · Neonatology · doi:10.1159/000454668 · Cited by 24,451
Wood, L., Egger, M., Gluud, L. L., Schulz, K. F., Jüni, P., Altman, D. G., Gluud, C., Martin, R. M., Wood, A. J. G., & Sterne, J. A. C. (2008) · BMJ · doi:10.1136/bmj.39465.451748.ad · Cited by 2,538 · cited 1×
“In practice, few network meta-analyses would have enough power to detect material differences between high and low risk of bias studies, so that the question about comparability of results between low and high risk of bias studies has to be answered by large-scale empirical studies [11, 12].”
BackgroundGraphical tools for network meta-analysis in STATA
Chaimani, A., Higgins, J. P. T., Mavridis, D., Spyridonos, P., & Salanti, G. (2013) · PLoS ONE · doi:10.1371/journal.pone.0076654 · Cited by 2,473 · cited 3×
“With netweight, a command for NMA in STATA [8], we can obtain the contribution matrix showing contributions from each direct comparison to the network estimates even in a large network like our example, and then we can calculate the composition of each level of risk of bias in network estimates quantitatively.”
BackgroundMethodSelf-citationThe BUGS project: evolution, critique and future directions
Lunn, D., Spiegelhalter, D., Thomas, A., & Best, N. (2009) · Statistics in Medicine · doi:10.1002/sim.3680 · Cited by 1,876 · cited 1×
“[6].”
Puhan, M. A., Schunemann, H. J., Murad, M. H., Li, T., Brignardello-Petersen, R., Singh, J. A., Kessels, A. G., Guyatt, G. H., & for the GRADE Working Group (2014) · BMJ · doi:10.1136/bmj.g5630 · Cited by 1,812 · cited 3×
“[2] rates the quality of evidence separately for direct and indirect estimates, and each rating is more impressionistic than quantitative.”
BackgroundMethodEvaluating the quality of evidence from a network meta-analysis
Salanti, G., Del Giovane, C., Chaimani, A., Caldwell, D. M., & Higgins, J. P. T. (2014) · PLoS ONE · doi:10.1371/journal.pone.0099682 · Cited by 1,402 · cited 3×
“[3] uses weights more extensively and makes more quantitative evaluations of all the involved evidence.”
BackgroundSelf-citationConceptual and technical challenges in network meta-analysis
Cipriani, A., Higgins, J. P. T., Geddes, J. R., & Salanti, G. (2013) · Annals of Internal Medicine · doi:10.7326/0003-4819-159-2-201307160-00008 · Cited by 1,072 · cited 1×
“Background The number of network meta-analyses (NMA) has been increasing rapidly in recent years [1], and concomitantly the methodology for NMA is also quickly developing and expanding.”
BackgroundSelf-citationA graphical tool for locating inconsistency in network meta-analyses
Krahn, U., Binder, H., & König, J. (2013) · BMC Medical Research Methodology · doi:10.1186/1471-2288-13-35 · Cited by 458 · cited 1×
“Using the contribution matrix to quantify the influence of RoB and of enrichment design in each network estimate We used a recently developed tool for NMA, called the contribution matrix, that quantifies how much each direct comparison in the network contributes to each network estimate in the NMA [8, 9].”
MethodMiura, T., Noma, H., Furukawa, T. A., Mitsuyasu, H., Tanaka, S., Stockton, S., Salanti, G., Motomura, K., Shimano-Katsuki, S., Leucht, S., et al. (2014) · The Lancet Psychiatry · doi:10.1016/s2215-0366(14)70314-1 · Cited by 364
Self-citation
Cited by
Cipriani, A., Furukawa, T. A., Salanti, G., Chaimani, A., Atkinson, L. Z., Ogawa, Y., Leucht, S., Ruhe, H. G., Turner, E. H., Higgins, J. P. T., et al. (2018) · The Lancet · doi:10.1016/s0140-6736(17)32802-7 · Cited by 3,541
Self-citationCipriani, A., Furukawa, T. A., Salanti, G., Chaimani, A., Atkinson, L. Z., Ogawa, Y., Leucht, S., Ruhe, H. G., Turner, E. H., Higgins, J. P. T., et al. (2018) · Focus · doi:10.1176/appi.focus.16407 · Cited by 328
Self-citationFurukawa, T. A., Salanti, G., Atkinson, L. Z., Leucht, S., Ruhe, H. G., Turner, E. H., Chaimani, A., Ogawa, Y., Takeshima, N., Hayasaka, Y., et al. (2016) · BMJ Open · doi:10.1136/bmjopen-2015-010919 · Cited by 214
“Analyses for statistical evaluation of the inconsistency and production of network graphs and result figures will be carried out in Stata using the mvmeta command(56) and a collection of routines described elsewhere.(30) All analyses of the primary outcomes will be duplicated using the netmeta package in R.”
MethodSelf-citationCombining randomized and non-randomized evidence in network meta-analysis
Efthimiou, O., Mavridis, D., Debray, T. P. A., Samara, M., Belger, M., Siontis, G. C. M., Leucht, S., Salanti, G., & 4, O. B. O. G. W. P. (2017) · Statistics in Medicine · doi:10.1002/sim.7223 · Cited by 183
Self-citationFurukawa, T. A., Shinohara, K., Sahker, E., Karyotaki, E., Miguel, C., Ciharova, M., Bockting, C. L. H., Breedvelt, J. J. F., Tajika, A., Imai, H., et al. (2021) · World Psychiatry · doi:10.1002/wps.20906 · Cited by 126
Self-citation
Metadata
- Authors
- Furukawa, T. A., Miura, T., Chaimani, A., Leucht, S., Cipriani, A., Noma, H., Mitsuyasu, H., Kanba, S., & Salanti, G.
- Affiliations
- Kyoto University (JP)
- Kyushu University (JP)
- University of Ioannina (GR)
- Technical University of Munich (DE)
- University of Oxford (GB)
- The Institute of Statistical Mathematics (JP)
- University of Bern (CH)
- Corresponding author
- Furukawa, Toshi A.
- ORCID
- Venue
- BMC Research Notes
- Published
- 2016-04-14
- Volume
- 9
- Issue
- 1
- Pages
- 218-218
- ISSN
- 1756-0500 (ISSN-L)
- PMC
- PMC4831112
- OpenAlex
- W2336859207
- Cited by
- 23
- References
- 13
- Citations by year
- 2026: 2 · 2025: 1 · 2024: 3 · 2023: 3 · 2022: 3 · 2021: 2 · 2020: 1 · 2019: 2 · 2018: 4 · 2017: 1 · 2016: 1
- FWCI
- 2.49
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0/ (tdm)
- creativecommons.org/licenses/by/4.0/ (vor)
- Locations
- BMC Research Notes (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Institutional Repositories DataBase (IRDB) (submittedVersion)
- Bern Open Repository and Information System (University of Bern) (submittedVersion, cc-by)
- Kyoto University Research Information Repository (Kyoto University) (submittedVersion)
- Open Access CRIS of the University of Bern
- Funders
- Japan Society for the Promotion of Science (26670314)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Keywords
- research limitations, risk of bias, quality of evidence, GRADE, network meta-analysis, bipolar disorder, maintenance pharmacotherapy, enrichment design
- MeSH terms
- Algorithms, Bipolar Disorder, Humans, Models, Theoretical, Review Literature as Topic, Bias, Maintenance Chemotherapy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Using the contribution matrix to evaluate complex study limitations in a network meta-analysis: a case study of bipolar maintenance pharmacotherapy review. https://app.cochat.ai/papers/10.1186/s13104-016-2019-1
- The paper
- Furukawa, T. A., Miura, T., Chaimani, A., Leucht, S., Cipriani, A., Noma, H., Mitsuyasu, H., Kanba, S., & Salanti, G. (2016). Using the contribution matrix to evaluate complex study limitations in a network meta-analysis: a case study of bipolar maintenance pharmacotherapy review. BMC Research Notes. https://doi.org/10.1186/s13104-016-2019-1
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.