Bias in emerging biomarkers for bipolar disorder
Open accessAbstract
Background: To date no comprehensive evaluation has appraised the likelihood of bias or the strength of the evidence of peripheral biomarkers for bipolar disorder (BD). Here we performed an umbrella review of meta-analyses of peripheral non-genetic biomarkers for BD. Method: The Pubmed/Medline, EMBASE and PsycInfo electronic databases were searched up to May 2015. Two independent authors conducted searches, examined references for eligibility, and extracted data. Meta-analyses in any language examining peripheral non-genetic biomarkers in participants with BD (across different mood states) compared to unaffected controls were included. Results: Six references, which examined 13 biomarkers across 20 meta-analyses (5474 BD cases and 4823 healthy controls) met inclusion criteria. Evidence for excess of significance bias (i.e. bias favoring publication of ‘positive’ nominally significant results) was observed in 11 meta-analyses. Heterogeneity was high for (I2 ⩾ 50%) 16 meta-analyses. Only two biomarkers met criteria for suggestive evidence namely the soluble IL-2 receptor and morning cortisol. The median power of included studies, using the effect size of the largest dataset as the plausible true effect size of each meta-analysis, was 15.3%. Conclusions: Our findings suggest that there is an excess of statistically significant results in the literature of peripheral biomarkers for BD. Selective publication of ‘positive’ results and selective reporting of outcomes are possible mechanisms.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The weaknesses of the ICD-11 and DSM-5 systems are debatable and unavoidable given the lack of any objective pathophysiological compass and the limited specificity of most currently available biomarkers in mental illness.
“The weaknesses of the two systems are both debatable and unavoidable [43, 44] given the lack of any objective pathophysiological compass and the limited specificity of most currently available biomarkers in mental illness [45].”
From Mental, behavioral and neurodevelopmental disorders in the ICD-11: an international perspective on key changes and controversies (2020) · cites this paper as [45]
Cites
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722 · see citing works
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859 · see citing works
Quantifying heterogeneity in a meta‐analysis
Higgins, J. P. T., & Thompson, S. G. (2002) · Statistics in Medicine · doi:10.1002/sim.1186 · Cited by 36,763 · see citing works
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.06.005 · Cited by 19,575 · see citing works
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505 · see citing works
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641 · cited 2× · see citing works
“It has been demonstrated that underpowered studies are more likely to report spurious results, and thus undermine the reliability of a specific field (Button et al. 2013).”
BackgroundWhy Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · see citing works
Assessing heterogeneity in meta-analysis: Q statistic or I² index?
Huedo-Medina, T. B., Sánchez-Meca, J., Marín-Martínez, F., & Botella, J. (2006) · Psychological Methods · doi:10.1037/1082-989x.11.2.193 · Cited by 4,256 · see citing works
A Re-Evaluation of Random-Effects Meta-Analysis
Higgins, J. P. T., Thompson, S. G., & Spiegelhalter, D. J. (2009) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2008.00552.x · Cited by 2,569 · cited 2× · see citing works
“Although this statistic was not reported in the included references, Higgins et al. suggest that one of the main roles of meta-analytic reviews is to quantify the level of uncertainty in a specific subject or field (Higgins et al. 2009).”
BackgroundResultPower and sample size calculations
Dupont, W. D., & Plummer, W. D. (1990) · Controlled Clinical Trials · doi:10.1016/0197-2456(90)90005-m · Cited by 2,112 · see citing works
Cuthbert, B. N. (2014) · World Psychiatry · doi:10.1002/wps.20087 · Cited by 1,377 · see citing works
Berk, M., Kapczinski, F., Andreazza, A. C., Dean, O. M., Giorlando, F., Maes, M., Yücel, M., Gama, C. S., Dodd, S., Dean, B., et al. (2011) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2010.10.001 · Cited by 1,178 · see citing works
Self-citationCommentary: Heterogeneity in meta-analysis should be expected and appropriately quantified
Higgins, J. P. T. (2008) · International Journal of Epidemiology · doi:10.1093/ije/dyn204 · Cited by 995 · see citing works
Kapur, S., Phillips, A. G., & Insel, T. R. (2012) · Molecular Psychiatry · doi:10.1038/mp.2012.105 · Cited by 980 · see citing works
An exploratory test for an excess of significant findings
Ioannidis, J. P., & Trikalinos, T. A. (2007) · Clinical Trials · doi:10.1177/1740774507079441 · Cited by 751 · see citing works
Belbasis, L., Bellou, V., Εvangelou, Ε., Ioannidis, J. P. A., & Tzoulaki, I. (2015) · The Lancet Neurology · doi:10.1016/s1474-4422(14)70267-4 · Cited by 669 · see citing works
Cytokine Alterations in Bipolar Disorder: A Meta-Analysis of 30 Studies
Modabbernia, A., Taslimi, S., Brietzke, E., & Ashrafi, M. (2013) · Biological Psychiatry · doi:10.1016/j.biopsych.2013.01.007 · Cited by 596 · see citing works
Polyakova, M., Stuke, K., Schuemberg, K., Mueller, K., Schoenknecht, P., & Schroeter, M. L. (2015) · Journal of Affective Disorders · doi:10.1016/j.jad.2014.11.044 · Cited by 488 · see citing works
Cytokines in bipolar disorder vs. healthy control subjects: A systematic review and meta-analysis
Munkholm, K., Braüner, J. V., Kessing, L. V., & Vinberg, M. (2013) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2013.05.018 · Cited by 402 · see citing works
Cuijpers, P., Smit, F., Bohlmeijer, E., Hollon, S. D., & Andersson, G. (2010) · British Journal of Psychiatry · doi:10.1192/bjp.bp.109.066001 · Cited by 391 · see citing works
Fernandes, B. S., Gama, C. S., Maria Ceresér, K., Yatham, L. N., Fries, G. R., Colpo, G., de Lucena, D., Kunz, M., Gomes, F. A., & Kapczinski, F. (2011) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2011.03.002 · Cited by 389 · see citing works
Self-citationAn updated meta-analysis of oxidative stress markers in bipolar disorder
Brown, N. C., Andreazza, A. C., & Young, L. T. (2014) · Psychiatry Research · doi:10.1016/j.psychres.2014.04.005 · Cited by 333 · see citing works
The False-positive to False-negative Ratio in Epidemiologic Studies
Ioannidis, J. P. A., Tarone, R., & McLaughlin, J. K. (2011) · Epidemiology · doi:10.1097/ede.0b013e31821b506e · Cited by 321 · see citing works
Biomarkers and clinical staging in psychiatry
McGorry, P. D., Keshavan, M. S., Goldstone, S. D., Amminger, G. P., Allott, K., Berk, M., Lavoie, S., Pantelis, C., Yung, A. R., Wood, S. J., et al. (2014) · World Psychiatry · doi:10.1002/wps.20144 · Cited by 305 · see citing works
Self-citationPhillips, M. L., Chase, H. W., Sheline, Y. I., Etkin, A., Almeida, J. R. C., Deckersbach, T., & Trivedi, M. H. (2015) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2014.14010076 · Cited by 267 · see citing works
Cytokines in bipolar disorder: A systematic review and meta-analysis
Munkholm, K., Vinberg, M., & Vedel Kessing, L. (2013) · Journal of Affective Disorders · doi:10.1016/j.jad.2012.06.010 · Cited by 256 · see citing works
Ioannidis, J. P. A. (2011) · JAMA · doi:10.1001/jama.2011.713 · Cited by 253 · see citing works
Fernandes, B. S., Steiner, J. H., Berk, M., Molendijk, M. L., González-Pinto, A. M., Turck, C. W., Nardin, P., & Ca, G. (2014) · Molecular Psychiatry · doi:10.1038/mp.2014.117 · Cited by 250 · see citing works
Self-citationFrey, B. N., Andreazza, A. C., Houenou, J., Jamain, S., Goldstein, B. I., Frye, M. A., Leboyer, M., Berk, M., Malhi, G. S., Lopez-Jaramillo, C., et al. (2013) · Australian & New Zealand Journal of Psychiatry · doi:10.1177/0004867413478217 · Cited by 234 · see citing works
Self-citationMorning cortisol levels in schizophrenia and bipolar disorder: a meta-analysis
Girshkin, L., Matheson, S. L., Shepherd, A. M., & Green, M. J. (2014) · Psychoneuroendocrinology · doi:10.1016/j.psyneuen.2014.07.013 · Cited by 215 · see citing works
Armitage, R., Trivedi, M. H., Hoffmann, R. F., & Rush, A. J. (1997) · Depression and Anxiety · doi:10.1002/(sici)1520-6394(1997)5:2 · Cited by 169 · see citing works
C-Reactive Protein Alterations in Bipolar Disorder: A Meta-Analysis
Dargél, A. A., Godin, O., Kapczinski, F., Kupfer, D. J., & Leboyer, M. (2015) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.14r09007 · Cited by 155 · cited 1× · see citing works
“Of 212 references selected for full-text review, six references met inclusion criteria (Looney & el-Mallakh, 1997; Modabbernia et al. 2013; Brown et al. 2014; Girshkin et al. 2014; Dargel et al. 2015; Polyakova et al. 2015).”
BackgroundIoannidis, J. P. A. (2013) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2013.03.002 · Cited by 150 · see citing works
Birmaher, B. B., Gill, M. K., Axelson, D., Goldstein, B. I., Goldstein, T. R., Yu, H., Liao, F., Iyengar, S., Diler, R. S., Strober, M. A., et al. (2014) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2014.13121577 · Cited by 130 · see citing works
Towards a classification of biomarkers of neuropsychiatric disease: from encompass to compass
Davis, J., Maes, M., Andreazza, A., McGrath, J. J., Tye, S. J., & Berk, M. (2015) · Molecular Psychiatry · doi:10.1038/mp.2014.139 · Cited by 123 · see citing works
Self-citationDiniz, B. S., Sibille, E., Ding, Y., Tseng, G., Aizenstein, H. J., Lotrich, F., Becker, J. T., Lopez, O. L., Lotze, M. T., Klunk, W. E., et al. (2015) · Molecular Psychiatry · doi:10.1038/mp.2014.76 · Cited by 116 · see citing works
Bias in Associations of Emerging Biomarkers With Cardiovascular Disease
Tzoulaki, I., Siontis, K., Εvangelou, Ε., & Ioannidis, J. P. A. (2013) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2013.3018 · Cited by 108 · see citing works
Lin, P. (2009) · Neuroscience Letters · doi:10.1016/j.neulet.2009.09.044 · Cited by 107 · see citing works
The RDoC program: psychiatry without psyche?
Parnas, J. (2014) · World Psychiatry · doi:10.1002/wps.20101 · Cited by 93 · see citing works
Tsilidis, K. K., Papatheodorou, S., Εvangelou, Ε., & Ioannidis, J. P. A. (2012) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djs437 · Cited by 93 · see citing works
Rosa, A. R., Singh, N., Whitaker, E., de Brito, M., Lewis, A. M., Vieta, E., Churchill, G. C., Geddes, J. R., & Goodwin, G. M. (2014) · Psychological Medicine · doi:10.1017/s0033291714000014 · Cited by 91 · see citing works
Self-citationTwenty-Five Years of Progress: The View from NIMH and NINDS
Insel, T. R., & Landis, S. C. (2013) · Neuron · doi:10.1016/j.neuron.2013.09.041 · Cited by 87 · see citing works
Goldstein, B. I., & Young, L. T. (2013) · Current Psychiatry Reports · doi:10.1007/s11920-013-0425-9 · Cited by 84 · see citing works
Altered Chemokine Levels in the Cerebrospinal Fluid and Plasma of Suicide Attempters
Janelidze, S., Ventorp, F., Erhardt, S., Hansson, O., Minthon, L., Flax, J., Samuelsson, M., Traskman-Bendz, L., & Brundin, L. (2013) · Psychoneuroendocrinology · doi:10.1016/j.psyneuen.2012.09.010 · Cited by 69 · see citing works
A Simple Method for Obtaining Original Data from Published Graphs and Plots
Sistrom, C. L., & Mergo, P. J. (2000) · American Journal of Roentgenology · doi:10.2214/ajr.174.5.1741241 · Cited by 63 · see citing works
Fusar-Poli, P., Radua, J., Frascarelli, M., Mechelli, A., Borgwardt, S., Di Fabio, F., Biondi, M., Ioannidis, J. P. A., & David, S. P. (2014) · Human Brain Mapping · doi:10.1002/hbm.22384 · Cited by 56 · see citing works
Is there an excess of significant findings in published studies of psychotherapy for depression?
Flint, J., Cuijpers, P. J., Horder, J., Koole, S. L., & Munafo, M. R. (2014) · Psychological Medicine · doi:10.1017/s0033291714001421 · Cited by 56 · see citing works
The bipolar maze: a roadmap through translational psychopathology
Vieta, E. (2014) · Acta Psychiatrica Scandinavica · doi:10.1111/acps.12270 · Cited by 54 · see citing works
Self-citationNovel biomarkers for bipolar disorder
Teixeira, A. L., Barbosa, I. G., Machado‐Vieira, R., Rizzo, L. B., Wieck, A., & Bauer, M. E. (2012) · Expert Opinion on Medical Diagnostics · doi:10.1517/17530059.2013.734807 · Cited by 41 · see citing works
Self-citationThe Missing Link between Clinical States and Biomarkers in Mental Disorders
Fava, G. A., Guidi, J., Grandi, S., & Hasler, G. (2014) · Psychotherapy and Psychosomatics · doi:10.1159/000360348 · Cited by 35 · see citing works
Fava, G. A. (2014) · World Psychiatry · doi:10.1002/wps.20108 · Cited by 27 · see citing works
Current State of Biomarkers in Bipolar Disorder
Scola, G., & Andreazza, A. C. (2014) · Current Psychiatry Reports · doi:10.1007/s11920-014-0514-4 · Cited by 27 · see citing works
First, M. B. (2014) · World Psychiatry · doi:10.1002/wps.20107 · Cited by 23 · see citing works
The File-Drawer Problem, Revisited
Young, S. S., & Bang, H. (2004) · Science · doi:10.1126/science.306.5699.1133d · Cited by 22 · see citing works
Murck, H., LAUGHREN, T. P., Lamers, F., Picard, R. W., Walther, S., GOFF, D. C., & Sainati, S. M. (2015) · Europe PMC (PubMed Central) · doi:10.7892/boris.69085 · Cited by 17 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Peripheral cytokine and chemokine alterations in depression: a meta‐analysis of 82 studies
Köhler, C. A., Freitas, T. H., Maes, M., de Andrade, N. Q., Liu, C. S., Fernandes, B. S., Stubbs, B., Solmi, M., Veronese, N., Herrmann, N., et al. (2017) · Acta Psychiatrica Scandinavica · doi:10.1111/acps.12698 · Cited by 1,443 · see citing works
Self-citationWhat causes psychosis? An umbrella review of risk and protective factors
Radua, J., Ramella‐Cravaro, V., Ioannidis, J. P. A., Reichenberg, A., Phiphopthatsanee, N., Amir, T., Yenn Thoo, H., Oliver, D., Davies, C., Morgan, C., et al. (2018) · World Psychiatry · doi:10.1002/wps.20490 · Cited by 553 · see citing works
Köhler, C. A., Freitas, T. H., Stubbs, B., Maes, M., Solmi, M., Veronese, N., de Andrade, N. Q., Morris, G., Fernandes, B. S., Brunoni, A. R., et al. (2017) · Molecular Neurobiology · doi:10.1007/s12035-017-0632-1 · Cited by 449 · see citing works
Self-citationKöhler, C. A., Evangelou, E., Stubbs, B., Solmi, M., Veronese, N., Belbasis, L., Bortolato, B., Melo, M. C. A., Coelho, C. A., Fernandes, B. S., et al. (2018) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2018.05.020 · Cited by 302 · see citing works
Self-citationKim, J. Y., Son, M. J., Son, C. Y., Radua, J., Eisenhut, M., Gressier, F., Koyanagi, A., Carvalho, A. F., Stubbs, B., Solmi, M., et al. (2019) · The Lancet Psychiatry · doi:10.1016/s2215-0366(19)30181-6 · Cited by 271 · see citing works
Self-citationLack of evidence to favor specific preventive interventions in psychosis: a network meta‐analysis
Davies, C., Cipriani, A., Ioannidis, J. P. A., Radua, J., Stahl, D., Provenzani, U., McGuire, P., & Fusar‐Poli, P. (2018) · World Psychiatry · doi:10.1002/wps.20526 · Cited by 229 · see citing works
Pinto, J. V., Moulin, T. C., & Amaral, O. B. (2017) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2017.10.001 · Cited by 103 · see citing works
Cognitive Impairment in Euthymic Pediatric Bipolar Disorder: A Systematic Review and Meta-Analysis
Elias, L. R., Miskowiak, K. W., Vale, A. M. O., Köhler, C. A., Kjærstad, H. L., Stubbs, B., Kessing, L. V., Vieta, E., Maes, M., Goldstein, B. I., et al. (2017) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1016/j.jaac.2017.01.008 · Cited by 104 · see citing works
Self-citationBrownstein, D. J., Salagre, E., Köhler, C., Stubbs, B., Vian, J., Pereira, C., Chavarria, V., Karmakar, C., Turner, A., Quevedo, J., et al. (2018) · Australian & New Zealand Journal of Psychiatry · doi:10.1177/0004867417721654 · Cited by 46 · see citing works
Self-citationDiagnostic biomarkers for obsessive-compulsive disorder: A reasonable quest or ignis fatuus?
Fullana, M. A., Abramovitch, A., Via, E., López-Sola, C., Goldberg, X., Reina, N., Fortea, L., Solanes, A., Buckley, M. J., Ramella-Cravaro, V., et al. (2020) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2020.08.008 · Cited by 44 · see citing works
Self-citationStein, D. J., Szatmari, P., Gaebel, W., Berk, M., Vieta, E., Maj, M., de Vries, Y. A., Roest, A. M., de Jonge, P., Maercker, A., et al. (2020) · BMC Medicine · doi:10.1186/s12916-020-1495-2 · Cited by 264 · see citing works
Self-citation
- Authors
- Carvalho, A. F., Köhler, C. A., Fernandes, B. S., Quevedo, J., Miskowiak, K. W., Brunoni, A. R., Machado-Vieira, R., Maes, M., Vieta, E., & Berk, M.
- Affiliations
- Universidade Federal do Ceará (BR)
- Universidade Federal do Rio Grande do Sul (BR)
- Universidade do Extremo Sul Catarinense (BR)
- Copenhagen University Hospital (DK)
- Universidade de São Paulo (BR)
- Barwon Health (AU)
- Centro de Investigación Biomédica en Red de Salud Mental (ES)
- Corresponding author
- Carvalho, A. F.
- ORCID
- Venue
- Psychological Medicine
- Publisher
- Cambridge University Press (CUP)
- Published
- 2016-05-19
- Volume
- 46
- Issue
- 11
- Pages
- 2287-2297
- ISSN
- 0033-2917 (ISSN-L)
- 1469-8978
- OpenAlex
- W2400490165
- Cited by
- 63
- References
- 55
- Citations by year
- 2026: 1 · 2025: 4 · 2024: 5 · 2023: 5 · 2022: 3 · 2021: 2 · 2020: 8 · 2019: 6 · 2018: 11 · 2017: 15 · 2016: 3
- FWCI
- 1.67
- Type
- journal article
- Language
- en
- Open access
- green
- figshare.com
- Licence
- Locations
- Psychological Medicine (publishedVersion)
- PubMed (publishedVersion)
- Figshare (submittedVersion, cc-by)
- Topics
- Health Sciences › Medicine › Psychiatry and Mental health › Bipolar Disorder and Treatment
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Medicine › Psychiatry and Mental health › Schizophrenia research and treatment
- Keywords
- bipolar disorder, peripheral biomarkers, publication bias, selective reporting, umbrella review of meta-analyses, soluble interleukin-2 receptor, morning cortisol, statistical power
- MeSH terms
- Bipolar Disorder, Humans, Biomarkers, Publication Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
