The DOI matches its Crossref record; 46 of the 47 references it cites were found in a registry or an index.
Methods to split cognitive task data for estimating split-half reliability: A comprehensive review and systematic assessment
Open accessPublished version of a preprint
Abstract
Estimating the reliability of cognitive task datasets is commonly done via split-half methods. We review four methods that differ in how the trials are split into parts: a first-second half split, an odd-even trial split, a permutated split, and a Monte Carlo-based split. Additionally, each splitting method could be combined with stratification by task design. These methods are reviewed in terms of the degree to which they are confounded with four effects that may occur in cognitive tasks: effects of time, task design, trial sampling, and non-linear scoring. Based on the theoretical review, we recommend Monte Carlo splitting (possibly in combination with stratification by task design) as being the most robust method with respect to the four confounds considered. Next, we estimated the reliabilities of the main outcome variables from four cognitive task datasets, each (typically) scored with a different non-linear algorithm, by systematically applying each splitting method. Differences between methods were interpreted in terms of confounding effects inflating or attenuating reliability estimates. For three task datasets, our findings were consistent with our model of confounding effects. Evidence for confounding effects was strong for time and task design and weak for non-linear scoring. When confounding effects occurred, they attenuated reliability estimates. For one task dataset, findings were inconsistent with our model but they may offer indicators for assessing whether a split-half reliability estimate is appropriate. Additionally, we make suggestions on further research of reliability estimation, supported by a compendium R package that implements each of the splitting methods reviewed here.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures

Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“In this sample, Monte Carlo splitting (Pronk et al., 2022) with N = 500 replications revealed split-half correlations r = 0.75 for alerting x Flanker interaction [Flanker costs no cue - Flanker costs double cue ] , and r > 0.98 for remaining ANT measures of interest.”
From Externally orienting cues improve cognitive control in OCD (2024) · cites this paper as Pronk et al., 2022
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A guideline of selecting and reporting intraclass correlation coefficients for reliability research
Koo, T. K., & Li, M. Y. (2016) · Journal of Chiropractic Medicine · doi:10.1016/j.jcm.2016.02.012 · Cited by 31,709
Intraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 24,532
Measuring individual differences in implicit cognition: the implicit association test
Greenwald, A. G., McGhee, D. E., & Schwartz, J. L. K. (1998) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.74.6.1464 · Cited by 9,210
What is coefficient alpha? An examination of theory and applications
Cortina, J. M. (1993) · Journal of Applied Psychology · doi:10.1037/0021-9010.78.1.98 · Cited by 8,810 · cited 1×
“Traditionally, alpha is estimated by fitting a model of essential tau-equivalence on item-level data; each item has the same loading on the true score but has a unique mean and error variance (Cortina, 1993).”
MethodUnderstanding and using the Implicit Association Test: I. An improved scoring algorithm.
Greenwald, A. G., Nosek, B. A., & Banaji, M. R. (2003) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.85.2.197 · Cited by 5,815
Better Bootstrap Confidence Intervals
Efron, B. (1987) · Journal of the American Statistical Association · doi:10.1080/01621459.1987.10478410 · Cited by 3,364
Bhattacharyya, A. (1943) · Medical Entomology and Zoology · Cited by 2,002
The reliability paradox: Why robust cognitive tasks do not produce reliable individual differences
Hedge, C., Powell, G., & Sumner, P. (2018) · Behavior Research Methods · doi:10.3758/s13428-017-0935-1 · Cited by 1,457
Corrections for extreme proportions and their biasing effects on estimated values ofd′
Hautus, M. J. (1995) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03203619 · Cited by 894
Coefficient Alpha and the Reliability of Composite Measurements
Novick, M. R., & Lewis, C. (1967) · Psychometrika · doi:10.1007/bf02289400 · Cited by 466
Cited by
Individual differences in computational psychiatry: A review of current challenges
Karvelis, P., Paulus, M. P., & Diaconescu, A. O. (2023) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2023.105137 · Cited by 143
Improving the Reliability of Cognitive Task Measures: A Narrative ReviewPreprint
Zorowitz, S., & Niv, Y. (2022) · PsyArXiv (OSF Preprints) · doi:10.1016/j.bpsc.2023.02.004 · Cited by 114
Comparative Test of Cronbach's Alpha Reliability Coefficient, Kr-20, Kr-21, And Split-Half Method
Setyaedhi, H. S. (2024) · Journal of Education Research and Evaluation · doi:10.23887/jere.v8i1.68164 · Cited by 89
Testing the Biophilia theory: Automatic approach tendencies towards nature
Schiebel, T., Gallinat, J., & Kühn, S. (2021) · Journal of Environmental Psychology · doi:10.1016/j.jenvp.2021.101725 · Cited by 78
Charting Competence: A Holistic Scale for Measuring Proficiency in Artificial Intelligence Literacy
Yuan, C. W., Tsai, H.-Y. S., & Chen, Y.-T. (2024) · Journal of Educational Computing Research · doi:10.1177/07356331241261206 · Cited by 73
Metadata
- Authors
- Pronk, T., Molenaar, D., Wiers, R. W., & Murre, J.
- Affiliations
- University of Amsterdam (NL)
- Corresponding author
- Pronk, Thomas
- Venue
- Psychonomic Bulletin & Review
- Published
- 2021-06-07
- Volume
- 29
- Issue
- 1
- Pages
- 44-54
- ISSN
- 1069-9384 (ISSN-L)
- 1531-5320
- PMC
- PMC8858277
- OpenAlex
- W3164368159
- Cited by
- 123
- References
- 47
- Citations by year
- 2026: 14 · 2025: 34 · 2024: 40 · 2023: 26 · 2022: 7 · 2021: 2
- FWCI
- 3.68
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Psychonomic Bulletin & Review (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- UvA-DARE (University of Amsterdam) (submittedVersion, cc-by)
- PsyArXiv (OSF Preprints) (submittedVersion)
- PsyArXiv (OSF Preprints) (submittedVersion, cc-by)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Cognitive Abilities and Testing
- Keywords
- split-half reliability, confounding, task design, reliability estimation, R package
- MeSH terms
- Algorithms, Cognition, Humans, Monte Carlo Method, Reproducibility of Results
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Methods to split cognitive task data for estimating split-half reliability: A comprehensive review and systematic assessment. https://app.cochat.ai/papers/10.3758/s13423-021-01948-3
- The paper
- Pronk, T., Molenaar, D., Wiers, R. W., & Murre, J. (2022). Methods to split cognitive task data for estimating split-half reliability: A comprehensive review and systematic assessment. Psychonomic Bulletin & Review. https://doi.org/10.3758/s13423-021-01948-3
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.