Assessing and adjusting for publication bias in the relationship between anxiety and the error-related negativity
Open accessAbstract
Many clinical neuroscience investigations have suggested that trait anxiety is associated with increased neural reactivity to mistakes in the form of an event-related potential called the error-related negativity (ERN). Several recent meta-analyses indicated that the anxiety-ERN association was of a small-to-medium effect size, however, these prior investigations did not comprehensively adjust effect sizes for publication bias. Here, in an updated meta-analysis (k = 58, N = 3819), we found support for an uncorrected effect size of r = -0.19, and applied a range of methods to test for and correct publication bias (trim-and-fill, PET, PEESE, Peters' test, three-parameter selection model). The majority of bias-correction methods suggested that the correlation between anxiety and the ERN is non-zero, but smaller than the uncorrected effect size (average adjusted effect size: r = -0.12, range: r = -0.05 to -0.18). Moderation analyses also revealed more robust effects for clinical anxiety and anxious samples characterised by worry, however, it should be noted that these larger effects were also associated with elevated indicators of publication bias relative to the overall analysis. Mixed anxiety and sub-clinical anxiety were not associated with the amplitude of the ERN. Our results suggest that the anxiety-ERN relationship survives multiple corrections for publication bias, albeit not among all sub-types and populations of anxiety. Nevertheless, only 50% of the studies included in our analysis reported significant results, indicating that future research exploring the anxiety-ERN relationship would benefit from increased statistical power.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Heterogeneity in the magnitude of the anxiety/OCD-ERN association has been identified.
“Importantly, however, heterogeneity in the magnitude of the anxiety/OCD-ERN association has been identified (for review: Saunders and Inzlicht, 2020).”
From The Error-Related Negativity (ERN) in Anxiety and Obsessive-Compulsive Disorder (OCD): A Call for Further Investigation of Task Parameters in the Flanker Task (2021) · cites this paper as Saunders and Inzlicht, 2020
The magnitude of the anxiety/OCD-ERN effects is influenced by publication bias.
“Further, a recent meta-analysis found that the magnitude of these effects is influenced by publication bias (Saunders and Inzlicht, 2020), suggesting that findings may be even more heterogeneous than indicated by the published literature.”
From The Error-Related Negativity (ERN) in Anxiety and Obsessive-Compulsive Disorder (OCD): A Call for Further Investigation of Task Parameters in the Flanker Task (2021) · cites this paper as Saunders and Inzlicht, 2020
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641 · cited 1× · see citing works
“One recent analysis of over 1119 cognitive neuroscience papers reported that studies typically obtained 12% power to detect small effects, and 44% power to detect medium-sized effects (Szucs & Ionidis, 2017, see also Button et al., 2013).”
BackgroundThe Existence of Publication Bias and Risk Factors for Its Occurrence
Dickersin, K. (1990) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1990.03440100097014 · Cited by 1,417 · see citing works
Stanley, T. D., & Doucouliagos, H. (2014) · Research Synthesis Methods · doi:10.1002/jrsm.1095 · Cited by 916 · see citing works
The global burden of anxiety disorders in 2010
Baxter, A. J., Vos, T., Scott, K. M., Ferrari, A. J., & Whiteford, H. A. (2014) · Psychological Medicine · doi:10.1017/s0033291713003243 · Cited by 768 · see citing works
Comorbidity of anxiety and depression in children and adolescents
Brady, E. U., & Kendall, P. C. (1992) · Psychological Bulletin · doi:10.1037/0033-2909.111.2.244 · Cited by 748 · see citing works
<i>p</i> -Curve and Effect Size: Correcting for Publication Bias Using Only Significant Results
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691614553988 · Cited by 717 · cited 1× · see citing works
“Given widespread concern that both funnel-plot asymmetry and trim-and-fill under-correct for publication bias (Carter et al., 2019; Idris, 2012; Simonsohn et al., 2014).”
BackgroundAction-Monitoring Dysfunction in Obsessive-Compulsive Disorder
Gehring, W. J., Himle, J., & Nisenson, L. G. (2000) · Psychological Science · doi:10.1111/1467-9280.00206 · Cited by 670 · see citing works
Correcting for Bias in Psychology: A Comparison of Meta-Analytic Methods
Carter, E. C., Schönbrodt, F. D., Gervais, W. M., & Hilgard, J. (2019) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245919847196 · Cited by 588 · see citing works
Pigott, T. D. (2012) · doi:10.1007/978-1-4614-2278-5 · Cited by 453 · see citing works
On the relationship between anxiety and error monitoring: a meta-analysis and conceptual framework
Moser, J. S., Moran, T. P., Schroder, H. S., Donnellan, M. B., & Yeung, N. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00466 · Cited by 445 · see citing works
McShane, B. B., Böckenholt, U., & Hansen, K. T. (2016) · Perspectives on Psychological Science · doi:10.1177/1745691616662243 · Cited by 398 · see citing works
Natural selection and the regulation of defenses
Nesse, R. M. (2005) · Evolution and Human Behavior · doi:10.1016/j.evolhumbehav.2004.08.002 · Cited by 365 · see citing works
Performance monitoring and error significance in patients with obsessive-compulsive disorder
Endrass, T., Schuermann, B., Kaufmann, C., Spielberg, R., Kniesche, R., & Kathmann, N. (2010) · Biological Psychology · doi:10.1016/j.biopsycho.2010.02.002 · Cited by 206 · see citing works
Weinberg, A., Klein, D. N., & Hajcak, G. (2012) · Journal of Abnormal Psychology · doi:10.1037/a0028270 · Cited by 182 · see citing works
Error-related brain activity in the age of RDoC: A review of the literature
Weinberg, A., Dieterich, R., & Riesel, A. (2015) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2015.02.029 · Cited by 179 · cited 2× · see citing works
“The ERN has been implicated in the pathogenesis of anxiety (Meyer, 2017), and has been identified as a candidate trait vulnerability marker for clinical anxiety (Olvet and Hajcak, 2008; Weinberg et al., 2015).”
BackgroundThe erring brain: Error‐related negativity as an endophenotype for OCD—A review and meta‐analysis
Riesel, A. (2019) · Psychophysiology · doi:10.1111/psyp.13348 · Cited by 170 · see citing works
Moser, J. S., Moran, T. P., Kneip, C., Schroder, H. S., & Larson, M. J. (2016) · Psychophysiology · doi:10.1111/psyp.12509 · Cited by 96 · see citing works
Flagged first, then by citations
Research works printed but not linked: 16
- Becker, B.J. (2005). Failsafe N or File-Drawer Number. In H.R. Rothstein, A.J. Sutton, & M. Borenstein (EDs.) Publication Bias in Meta-Analysis: Prevention, Assessment and Adjustments. John Wiley & Sons.
- Carbine, K. A., Lindsey, H. M., Rodeback, R. E., & Larson, M. J. (2019). Quantifying evidential value and selective reporting in recent and 10-year past psychophysiological literature: A pre-registered P-curve analysis. International Journal of Psychophysiology, 142, 33-49.
- Carter, E. C., Kofler, L. M., Forster, D. E., & McCullough, M. E. (2015). A series of meta-analytic tests of the depletion effect: self-control does not seem to rely on a limited resource. Journal of Experimental Psychology: General, 144, 796.
- Cavanagh, J. F., & Shackman, A. J. (2015). Frontal midline theta reflects anxiety and cognitive control: meta-analytic evidence. Journal of Physiology-Paris, 109, 3-15.
- Elkins-Brown, N., Saunders, B., & Inzlicht, M. (2018). The misattribution of emotions and the error-related negativity: a Registered Report. Cortex, 109, 124-140.
- Endrass, T., Riesel, A., Kathmann, N., & Buhlmann, U. (2014). Performance monitoring in obsessive-compulsive disorder and social anxiety disorder. Journal of Abnormal Psychology, 123, 705.
- Inzlicht, M., & Al-Khindi, T. (2012). ERN and the placebo: A misattribution approach to studying the arousal properties of the error-related negativity. Journal of Experimental Psychology: General, 141, 799.
- Inzlicht, M., Gervais, W., & Berkman, E. (2015) Bias-Correction Techniques Alone Cannot Determine Whether Ego Depletion is Different from Zero: Commentary on Carter, Kofler, Forster, & McCullough, 2015. Available at SSRN: https://ssrn.com/abstract=2659409 or http://dx.doi.org/10.2139/ssrn.2659409 · DOI · Paper page
- Meyer, A. (2017). A biomarker of anxiety in children and adolescents: A review focusing on the error-related negativity (ERN) and anxiety across development. Developmental cognitive neuroscience, 27, 58-68.
- Olvet, D. M., & Hajcak, G. (2008). The error-related negativity (ERN) and psychopathology: toward an endophenotype. Clinical Psychology Review, 28, 1343-1354.
- Peters, J. L., Sutton, A. J., Jones, D. R., Abrams, K. R., & Rushton, L. (2006). Comparison of two methods to detect publication bias in meta-analysis. Jama, 295(6), 676-680.
- Riesel, A., Endrass, T., Kaufmann, C., & Kathmann, N. (2011). Overactive error-related brain activity as a candidate endophenotype for obsessive-compulsive disorder: evidence from unaffected first-degree relatives. American Journal of Psychiatry, 168, 317-324.
- Rothstein, H. R., Sutton, A. J., & Borenstein, M. (Eds.). (2006). Publication bias in meta-analysis: Prevention, assessment and adjustments. John Wiley & Sons
- Saunders, B., Lin, H., Milyavskaya, M., & Inzlicht, M. (2017). The emotive nature of conflict monitoring in the medial prefrontal cortex. International Journal of Psychophysiology, 119, 31-40.
- Spielberger, C. D. (1989). State-Trait Anxiety Inventory: Bibliography (2nd ed.). Palo Alto, CA: Consulting Psychologists Press.
- Tamnes, C. K., Walhovd, K. B., Torstveit, M., Sells, V. T., & Fjell, A. M. (2013). Performance monitoring in children and adolescents: A review of developmental changes in the error- related negativity and brain maturation. Developmental Cognitive Neuroscience, 6, 1-13.
Cited by
Sorted by citations
Riesel, A., Härpfer, K., Thoma, L., Kathmann, N., & Klawohn, J. (2023) · Psychophysiology · doi:10.1111/psyp.14164 · Cited by 15 · see citing works
Gloe, L. M., & Louis, C. C. (2021) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2021.779083 · Cited by 7 · see citing works
- Authors
- Saunders, B., & Inzlicht, M.
- Affiliations
- University of Dundee (GB)
- University of Toronto (CA)
- Corresponding author
- Saunders, Blair
- Venue
- International Journal of Psychophysiology
- Publisher
- Elsevier BV
- Published
- 2020-09
- Volume
- 155
- Pages
- 87-98
- ISSN
- 0167-8760 (ISSN-L)
- OpenAlex
- W2954827429
- Cited by
- 50
- References
- 17
- Citations by year
- 2026: 4 · 2025: 7 · 2024: 6 · 2023: 9 · 2022: 11 · 2021: 9 · 2020: 2 · 2019: 1
- FWCI
- 1.13
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- International Journal of Psychophysiology (publishedVersion)
- PubMed (publishedVersion)
- Discovery Research Portal (University of Dundee) (submittedVersion, cc-by-nc-nd)
- PsyArXiv (OSF Preprints) (submittedVersion)
- PsyArXiv (OSF Preprints) (submittedVersion, cc-by)
- OSF Preprints (OSF Preprints) (submittedVersion)
- OSF Preprints (OSF Preprints) (submittedVersion, cc-by)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Keywords
- trait anxiety, error-related negativity, publication bias, bias correction, event-related potentials, clinical anxiety, subclinical anxiety, worry, statistical power
- MeSH terms
- Anxiety, Anxiety Disorders, Electroencephalography, Evoked Potentials, Humans, Publication Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
