The DOI matches its Crossref record; 53 of the 53 references it cites were found in a registry or an index.
Unreliability as a threat to understanding psychopathology: The cautionary tale of attentional bias.
Open accessAbstract
The use of unreliable measures constitutes a threat to our understanding of psychopathology, because advancement of science using both behavioral and biologically oriented measures can only be certain if such measurements are reliable. Two pillars of the National Institute of Mental Health's portfolio-the Research Domain Criteria (RDoC) initiative for psychopathology and the target engagement initiative in clinical trials-cannot succeed without measures that possess the high reliability necessary for tests involving mediation and selection based on individual differences. We focus on the historical lack of reliability of attentional bias measures as an illustration of how reliability can pose a threat to our understanding. Our own data replicate previous findings of poor reliability for traditionally used scores, which suggests a serious problem with the ability to test theories regarding attentional bias. This lack of reliability may also suggest problems with the assumption (in both theory and the formula for the scores) that attentional bias is consistent and stable across time. In contrast, measures accounting for attention as a dynamic process in time show good reliability in our data. The field is sorely in need of research reporting findings and reliability for attentional bias scores using multiple methods, including those focusing on dynamic processes over time. We urge researchers to test and report reliability of all measures, considering findings of low reliability not just as a nuisance but as an opportunity to modify and improve upon the underlying theory. Full assessment of reliability of measures will maximize the possibility that RDoC (and psychological science more generally) will succeed. (PsycINFO Database Record
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Although in a separate study, ketamine failed to illustrate a similar impact on attentional bias towards negative cues 128 , the use of a reaction time measure (the dot-probe task) with notoriously poor psychometric properties 136-138 may have impeded the ability to sensitively detect changes over time.”
From Neuroplasticity in cognitive and psychological mechanisms of depression: an integrative model (2020) · cites this paper as [138]
In the citing paper
“Our ability to identify individually relevant brain and behavioural changes depends on a high level of measurement accuracy, combined with appropriate statistical modelling, neither of which is widely used in RCTs.38”
From Precision clinical trials: a framework for getting to precision medicine for neurobehavioural disorders (2021) · cites this paper as [38]
In the citing paper
“Our ability to identify individually relevant brain and behavioural changes depends on a high level of measurement accuracy, combined with appropriate statistical modelling, neither of which is widely used in RCTs.38”
From Precision clinical trials: a framework for getting to precision medicine for neurobehavioural disorders (2021) · cites this paper as [38]
In the citing paper
“One research challenge is the limited reliability of AB indices derived from manual response times, which may reflect measurement problems and the variability of anxiety-related ABs, as discussed earlier [19,56-58,98].”
From Anxiety and Threat-Related Attention: Cognitive-Motivational Framework and Treatment (2018) · cites this paper as [56]
In the citing paper
“One research challenge is the limited reliability of AB indices derived from manual response times, which may reflect measurement problems and the variability of anxiety-related ABs, as discussed earlier [19,56-58,98].”
From Anxiety and Threat-Related Attention: Cognitive-Motivational Framework and Treatment (2018) · cites this paper as [56]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Statistical power analyses using G∗Power 3.1: tests for correlation and regression analyses
Faul, F., Erdfelder, E., Buchner, A., & Lang, A.-G. (2009) · Behavior Research Methods · doi:10.3758/brm.41.4.1149 · Cited by 37,615
ggplot2: Elegant Graphics for Data Analysis
Wickham, H. (2009) · doi:10.1007/978-0-387-98141-3 · Cited by 14,447
Threat-related attentional bias in anxious and nonanxious individuals: a meta-analytic study.
Bar-Haim, Y., Lamy, D., Pergamin, L., Bakermans-Kranenburg, M. J., & van IJzendoorn, M. H. (2007) · Psychological Bulletin · doi:10.1037/0033-2909.133.1.1 · Cited by 3,721
Cole, D. A., & Maxwell, S. E. (2003) · Journal of Abnormal Psychology · doi:10.1037/0021-843x.112.4.558 · Cited by 3,605
How to improve R&D productivity: the pharmaceutical industry's grand challenge
Paul, S. M., Mytelka, D. S., Dunwiddie, C. T., Persinger, C. C., Munos, B. H., Lindborg, S. R., & Schacht, A. L. (2010) · Nature Reviews Drug Discovery · doi:10.1038/nrd3078 · Cited by 3,566
Structured clinical interview for DSM-IV axis I disorders : SCID-I : clinical version : scoresheet
First, M. B., Spitzer, R. L., Gibbon, M., & Williams, J. B. W. (1997) · American Psychiatric Press eBooks · Cited by 3,101
Balota, D. A., Yap, M. J.-M., Hutchison, K. A., Cortese, M. J., Kessler, B., Loftis, B., Neely, J. H., Nelson, D. L., Simpson, G. B., & Rebecca Treiman, A. (2007) · Behavior Research Methods · doi:10.3758/bf03193014 · Cited by 2,758
Attentional bias in emotional disorders.
MacLeod, C., Mathews, A., & Tata, P. R. (1986) · Journal of Abnormal Psychology · doi:10.1037//0021-843x.95.1.15 · Cited by 2,553
Producing high-dimensional semantic spaces from lexical co-occurrence
Lund, K., & Burgess, C. (1996) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03204766 · Cited by 1,875
Borsboom, D., Mellenbergh, G. J., & van Heerden, J. (2004) · Psychological Review · doi:10.1037/0033-295x.111.4.1061 · Cited by 1,733
Flagged first, then by citations
Research works printed but not linked: 2
- Harvey AG Watkins E Mansell W 2004 Cognitive behavioural processes across psychological disorders New York Oxford University Press
- Nunnally JD Bernstein IH 1994 Psychometric theory 3 New York McGraw-Hill
Cited by
Sorted by citations
Items 301–312 of 312
Basanovic, J., Winer, E. S., & Dunn, B. D. (2026) · Behaviour Research and Therapy · doi:10.1016/j.brat.2026.105069
Mazidi, M., MacLeod, C., Ranjbar, S., Myles, O., & Grafton, B. (2026) · Behavior Research Methods · doi:10.3758/s13428-026-03045-6
Psychometric evaluation of interpretation bias measures for pain
Duijzings, M., Sharpe, L., Ryckeghem, D. M. L. V., Notebaert, L., Coutts‐Bain, D., Pickup, B., Colagiuri, B., & Todd, J. (2026) · Journal of Pain · doi:10.1016/j.jpain.2026.106349
Hagen, H. S., Klonteig, S., Røssberg, J. I., Jonassen, R., Landrø, N. I., & Bø, R. (2026) · Cognitive Affective & Behavioral Neuroscience · doi:10.3758/s13415-026-01466-6
Knauth, K., Brands, A. M. A., Peters, J., & Korkmaz, D. (2025) · OSF Preprints (OSF Preprints)
Zlotnick, E., Sorka, H., Barzilay, S., & Huppert, J. D. (2026) · Clinical Psychological Science · doi:10.1177/21677026261459024
Self-citationNo association of childhood maltreatment experiences and attention to facial emotions in men
Hoepfel, D., Jantz, J., Günther, V., Koelkebeck, K., Kersting, A., & Suslow, T. (2026) · Scientific Reports · doi:10.1038/s41598-026-65088-1
Wright, L., Li, L., Boafo, A., Johnson, D., Atallah, S., Rodriguez, C., Troller‐Renfree, S. V., Sheridan, M. A., McLaughlin, K. A., & Wade, M. (2026) · Clinical Child and Family Psychology Review · doi:10.1007/s10567-026-00584-4
Yin, M., Qiu, B., & Zhang, W. (2026) · Current Psychology · doi:10.1007/s12144-026-09985-8
Wang, K., Qiu, F., & Sun, J. (2026) · International Journal of Sport and Exercise Psychology · doi:10.1080/1612197x.2026.2740801
Smith, K. E., Smith, A., & Page, K. (2026) · International Journal of Eating Disorders · doi:10.1002/eat.70209
Metadata
- Authors
- Rodebaugh, T. L., Scullin, R. B., Langer, J. K., Dixon, D. J., Huppert, J. D., Bernstein, A., Zvielli, A., & Lenze, E. J.
- Affiliations
- Hebrew University of Jerusalem (IL)
- University of Haifa (IL)
- ORCID
- Venue
- Journal of Abnormal Psychology
- Published
- 2016-08
- Volume
- 125
- Issue
- 6
- Pages
- 840-851
- ISSN
- 0021-843X (ISSN-L)
- 1939-1846
- PMC
- PMC4980228
- OpenAlex
- W2465300579
- Cited by
- 312
- References
- 53
- Citations by year
- 2026: 21 · 2025: 24 · 2024: 28 · 2023: 39 · 2022: 41 · 2021: 40 · 2020: 37 · 2019: 24 · 2018: 33 · 2017: 22 · 2016: 2
- FWCI
- 38.33
- Type
- journal article
- Language
- en
- Update notices
- correction (2016-09-01)
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Journal of Abnormal Psychology (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Funders
- National Institutes of Health (UL1TR000448)
- National Institute of Mental Health (R21MH090308, R01MH070547)
- National Center for Research Resources (UL1RR024992)
- National Center for Advancing Translational Sciences
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Mental Health Research Topics
- Keywords
- attentional bias, measurement reliability, psychopathology, Research Domain Criteria, target engagement, replication, individual differences, reliability reporting
- MeSH terms
- Attentional Bias, Mental Disorders, Humans, Individuality, Models, Psychological, National Institute of Mental Health (U.S.), Research Design, United States, Reproducibility of Results
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
