The psychometric upgrade psychophysiology needs
Open accessThe DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Abstract
Although biological measurements are constrained by the same fundamental psychometric principles as self‐report measurements, these essential principles are often neglected in most fields of neuroscience, including psychophysiology. Potential reasons for this neglect could include a lack of understanding of appropriate measurement theory or a lack of accessible software for psychometric analysis. Generalizability theory is a flexible and multifaceted measurement theory that is well suited to handling the nuances of psychophysiological data, such as the often unbalanced number of trials and intraindividual variability of scores of event‐related brain potential (ERP) data. The ERP Reliability Analysis Toolbox (ERA Toolbox) was designed for psychophysiologists and is tractable software that can support the routine evaluation of psychometrics using generalizability theory. Psychometrics can guide task refinement, data‐processing decisions, and selection of candidate biomarkers for clinical trials. The present review provides an extensive treatment of additional psychometric characteristics relevant to studies of psychophysiology, including validity and validation, standardization, dimensionality, and measurement invariance. Although the review focuses on ERPs, the discussion applies broadly to psychophysiological measures and beyond. The tools needed to rigorously assess psychometric reliability and validate psychophysiological measures are now readily available. With the profound implications that psychophysiological research can have on understanding brain‐behavior relationships and the identification of biomarkers, there is simply too much at stake to ignore the crucial processes of evaluating psychometric reliability and validity.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Putnick, D. L., & Bornstein, M. H. (2016) · Developmental Review · doi:10.1016/j.dr.2016.06.004 · Cited by 4,369
The neural basis of error detection: Conflict monitoring and the error-related negativity
Yeung, N., Botvinick, M. M., & Cohen, J. D. (2004) · Psychological Review · doi:10.1037/0033-295x.111.4.931 · Cited by 1,924
Falkenstein, M., Hohnsbein, J., Hoormann, J., & Blanke, L. (1991) · Electroencephalography and Clinical Neurophysiology · doi:10.1016/0013-4694(91)90062-9 · Cited by 1,706
The reliability paradox: Why robust cognitive tasks do not produce reliable individual differences
Hedge, C., Powell, G., & Sumner, P. (2018) · Behavior Research Methods · doi:10.3758/s13428-017-0935-1 · Cited by 1,457
Beyond Power Calculations: Assessing Type S (Sign) and Type M (Magnitude) Errors
Gelman, A., & Carlin, J. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691614551642 · Cited by 1,343
Why We (Usually) Don't Have to Worry About Multiple Comparisons
Gelman, A., Hill, J., & Yajima, M. (2012) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2011.618213 · Cited by 1,274
Nieuwenhuis, S., Ridderinkhof, K. R., Blom, J., Band, G. P. H., & Kok, A. (2001) · Psychophysiology · doi:10.1111/1469-8986.3850752 · Cited by 1,170
Brennan, R. L. (2001) · doi:10.1007/978-1-4757-3456-0 · Cited by 1,121
THEORY OF GENERALIZABILITY: A LIBERALIZATION OF RELIABILITY THEORY†
Cronbach, L. J., Rajaratnam, N., & Gleser, G. C. (1963) · British Journal of Statistical Psychology · doi:10.1111/j.2044-8317.1963.tb00206.x · Cited by 737
The NIMH research domain criteria initiative: Background, issues, and pragmatics
Kozak, M. J., & Cuthbert, B. N. (2016) · Psychophysiology · doi:10.1111/psyp.12518 · Cited by 655
Cited by
Clayson, P. E., Mcdonald, J. B., Park, B., Holbrook, A., Baldwin, S. A., Riesel, A., & Larson, M. J. (2025) · Psychophysiology · doi:10.1111/psyp.14496 · Cited by 21
Self-citationHolbrook, A., Park, B., Baldwin, S. A., Riesel, A., Larson, M. J., & Clayson, P. (2025) · Psychophysiology · doi:10.1111/psyp.70042 · Cited by 6
Self-citationLoTemplio, S., Silcox, J., Strayer, D. L., & Payne, B. R. (2026) · Psychophysiology · doi:10.1111/psyp.70216 · Cited by 3
Metadata
- Authors
- Clayson, P. E.
- Venue
- Psychophysiology
- Publisher
- Wiley
- Published
- 2024-01-16
- Volume
- 61
- Issue
- 3
- Pages
- e14522-e14522
- ISSN
- 0048-5772 (ISSN-L)
- 1469-8986
- OpenAlex
- W4390929125
- Cited by
- 39
- References
- 28
- FWCI
- 11.50
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The psychometric upgrade psychophysiology needs. https://app.cochat.ai/papers/10.1111/psyp.14522
- The paper
- Clayson, P. E. (2024). The psychometric upgrade psychophysiology needs. Psychophysiology. https://doi.org/10.1111/psyp.14522
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.