The DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Reaction time in differential and developmental research: A review and commentary on the problems and alternatives.
Open accessAbstract
Reaction time is believed to be a good indicator of the speed and efficiency of mental processes and is a ubiquitous variable in the behavioral sciences. Despite this popularity, there are numerous issues associated with using reaction time (RT), specifically in differential and developmental research. Here, we identify and focus on two main problems-unreliability and sensitivity to speed-accuracy interactions. The use of difference scores is a primary factor that leads to many RT measures having demonstrably low reliability, and RT measures in general often do not properly account for speed-accuracy interactions. Both factors jeopardize the validity and interpretability of results based on RT. Here, we evaluate conceptually and empirically how these issues affect individual differences research. Although the empirical evidence we provide are primarily within the domains of attention control and task switching, we highlight examples from various other areas of psychological inquiry. We also discuss many of the statistical and methodological alternatives available to researchers conducting correlational studies. Ultimately, we encourage researchers comparing individuals of differing cognitive and developmental levels to strongly consider using these alternatives in lieu of RT, specifically RT difference scores. (PsycINFO Database Record (c) 2019 APA, all rights reserved).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“One complication is that the testing effect is a difference score between a condition that receives retrieval practice and one that does not, and difference scores are inherently poor in reliability (Draheim et al. 2019).”
From Practicing Retrieval Facilitates Learning (2021) · cites this paper as Draheim et al. 2019
In the citing paper
“However, treating RT and accuracy as distinct outcomes does not appropriately account for the speed-accuracy trade-off that exists between the two parameters24,25.”
From Delayed goal-directed processing underlies inhibitory control challenges in adult ADHD (2026) · cites this paper as [24]
Citations
Cites
Studies of interference in serial verbal reactions
Stroop, J. R. (1935) · Journal of Experimental Psychology · doi:10.1037/h0054651 · Cited by 18,622
Effects of noise letters upon the identification of a target letter in a nonsearch task
Eriksen, B. A., & Eriksen, C. W. (1974) · Perception & Psychophysics · doi:10.3758/bf03203267 · Cited by 7,230
The processing-speed theory of adult age differences in cognition
Salthouse, T. A. (1996) · Psychological Review · doi:10.1037/0033-295x.103.3.403 · Cited by 5,902
Testing the efficiency and independence of attentional networks
Fan, J., McCandliss, B. D., Sommer, T., Raz, A., & Posner, M. I. (2002) · Journal of Cognitive Neuroscience · doi:10.1162/089892902317361886 · Cited by 3,717
The Sources of Four Commonly Reported Cutoff Criteria: What Did They Really Say?
Lance, C. E., Butts, M. M., & Michels, L. C. (2006) · Organizational Research Methods · doi:10.1177/1094428105284919 · Cited by 2,053
Control and interference in task switching—A review
Kiesel, A., Steinhauser, M., Wendt, M., Falkenstein, M., Jost, K., Philipp, A. M., & Koch, I. (2010) · Psychological Bulletin · doi:10.1037/a0019842 · Cited by 1,485
The reliability paradox: Why robust cognitive tasks do not produce reliable individual differences
Hedge, C., Powell, G., & Sumner, P. (2018) · Behavior Research Methods · doi:10.3758/s13428-017-0935-1 · Cited by 1,457
Long-term reduction in implicit race bias: A prejudice habit-breaking intervention
Devine, P. G., Forscher, P. S., Austin, A. J., & Cox, W. T. L. (2012) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2012.06.003 · Cited by 1,259
Executive Attention, Working Memory Capacity, and a Two-Factor Theory of Cognitive Control
Engle, R. W., & Kane, M. J. (2003) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(03)44005-x · Cited by 1,104
Self-citationWorking Memory and Executive Attention: A Revisit
Engle, R. W. (2018) · Perspectives on Psychological Science · doi:10.1177/1745691617720478 · Cited by 339
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Items 201–260 of 260
Individual differences in working memory and the benefit of retrieval practice
Fordyce, A. L., Redick, T. S., Bedwell, J. P., & Karpicke, J. D. (2025) · Journal of Memory and Language · doi:10.1016/j.jml.2025.104664 · Cited by 3
HD-tDCS improves conflict processing and general behavioral stability
Liu, R., Wang, M., Liu, Q., Jing, X.-J., Hao, Q., Yu, H., & Wang, Y.-F. (2025) · Scientific Reports · doi:10.1038/s41598-025-15337-6 · Cited by 1
“Interpretation of reliability indices: values below 0.70 are considered problematic, those between 0.70 and 0.80 are deemed borderline, and values exceeding 0.80 are considered acceptable 54 .”
InfluentialCognitive Control Decision-Making Dynamics Across Internalizing and Externalizing Symptoms in Youth
Warren, S. L., Malaiya, R. K., Drake, O., Tang, A., & Chandra, N. K. (2025) · Research on Child and Adolescent Psychopathology · doi:10.1007/s10802-025-01351-9 · Cited by 1
“…susceptible to speed-accuracy interactions that are not readily identifiable (e.g., emphasis on speed or accuracy domain may differ across individuals and developmental levels), and variations in these interactions across subjects can threaten score reliability and validity (Draheim et al., 2019).”
Moving beyond button presses to enhance the reliability of congruency tasks
Erb, C. D., Morrison, S., & Nicholson-Brown, A. (2025) · Behavior Research Methods · doi:10.3758/s13428-025-02740-0 · Cited by 3
“Longstanding questions regarding the reliability of experimental tasks targeting conflict processing have attracted renewed interest in recent years (Draheim et al., 2019; Enkavi et al., 2019; Hedge et al., 2018), leading some to question whether the field is experiencing a reliability crisis…”
InfluentialFlanking age: Multilingualism and its role in shaping cognitive decline and neural dynamics
Elin, K., Gallo, F., Gabrielsen, A., Voits, T., Rothman, J., & DeLuca, V. (2025) · NeuroImage · doi:10.1016/j.neuroimage.2025.121312 · Cited by 1
“Draheim et al. (2019) note that difference scores, such as the CE used in this study, are less reliable than original performance measures on congruent and incongruent trials.”
Parameterizing Individual Differences in Fraction and Decimal Arithmetic
Braithwaite, D. W., & Rafferty, A. N. (2025) · Cognitive Science · doi:10.1111/cogs.70065 · Cited by 1
Effect of Stroop Interference on Cognitive Reaction Time among College Students in Lipa City
Guino, K. C., Torres, G. D., Canlas, M. M., Palmes, M., Lina, E. G., Garcia, B., Braza, D., Villapando, A. E., Serafica, J., & Muria, N. M. (2025) · Psychology and Education: A Multidisciplinary Journal · doi:10.70838/pemj.350106 · Cited by 1
Stasielowicz, L. (2024) · Psychological Reports · doi:10.1177/00332941241308517 · Cited by 2
“However, one could consider reaction times in future laboratory studies and account for the potential speed-accuracy trade-off (Bruyer & Brysbaert, 2011; Draheim et al., 2019; Vandierendonck, 2017).”
Wooten, T., Sansevere, K. S., Siqueira, S., McWilliams, T., Peach, S., Hussey, E. K., Brunyé, T. T., & Ward, N. (2024) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2024.112388 · Cited by 2
Osborne, K., Zhang, W., Gupta, T., Farrens, J. L., Geiger, M., Kraus, B., Krugel, C., Nusslock, R., Kappenman, E., & Mittal, V. A. (2023) · Journal of Psychopathology and Clinical Science · doi:10.1037/abn0000862 · Cited by 1
“However, as there are a number of methodological issues involved in the use of difference scores in individual differences research (Draheim et al., 2019; Meyer et al., 2017), these analyses are reported in the online supplemental materials.”
Franco-Martínez, A., Vicente-Conesa, F., Shanks, D. R., & Vadillo, M. A. (2025) · Journal of Memory and Language · doi:10.1016/j.jml.2025.104621 · Cited by 2
Does Cognitive Bias Modification Reduce Alcohol Consumption?
Jones, A., & Field, M. (2021) · Springer eBooks · doi:10.1007/978-3-030-66941-6_22 · Cited by 4
Hauenstein, C. E., Kim, E., Phillips, A., Montanez, A. M., Gordon, B., & Smith, D. M. (2026) · Cognition · doi:10.1016/j.cognition.2026.106769
Na, R., Qi, S.-X., Liu, J.-Y., Qi, J.-H., Feng, Y., Liu, T., Wu, X., & Jiang, Y.-P. (2026) · Behavioral Science · doi:10.3390/bs16111999
How visual and auditory task loads differently impair the advantage of multimodal warnings
Peng, X., Liang, Y., Wang, F., Wang, X., Rong, S., & Zhang, J. (2026) · Ergonomics · doi:10.1080/00140139.2026.2751023
Davis, E. E., Davis, S. W., Olsen, R. K., & Campbell, K. L. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.10.05.756834
“Instead of using raw difference scores, which are known to be more variable than the individual means that go into them (Draheim et al., 2019) and to account for age differences in baseline RT, we regressed mean RTs for intact pairs onto rearranged pairs across participants for each block type and…”
Chang, C., Chu, C.-L., Kuo, Y.-L., Chueh, T., & Hung, T. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1879513
“However, these aggregate measures may fail to capture differences in speed–accuracy trade-offs (SAT) or response strategies (Draheim et al., 2019; Heitz, 2014).”
Chang, C., Chu, C.-L., Deng, Y.-J., & Hung, T.-M. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1960942
Nakamura, M., Soares, S. M. P., Roggen, A., Diaz, M. T., & Rossi, E. (2026) · Bilingualism: Language and Cognition · doi:10.1017/s136672892610176x
Lacko, D., Kubelková, A., Oľhová, S., Klocek, A., & Graf, S. (2026) · British Journal of Social Psychology · doi:10.1111/bjso.70136
Peters, A., Waters, N. E., Grammer, J. K., & Isbell, E. (2026) · Journal of Experimental Child Psychology · doi:10.1016/j.jecp.2026.106544
Mashburn, C. A., Clarke, E. M., Pearcey, S. M., Pogue, E. D., Saleem, B., & Holliday, E. D. (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2026.1901554
Self-citationPlanning-related hubs and brain network organization during motor decision-making
Cano, L. A., Albarracín, A. L., Fernández, E., & Farfán, F. (2026) · Brain Structure and Function · doi:10.1007/s00429-026-03174-w
“Previous Previous studies (Cunnington et al. 2002; Draheim et al. 2019; Paraskevopoulou et al. 2021) have reported heterogeneous and sometimes contradictory findings based on reaction time (RT), highlighting the need for methodological approaches capable of capturing the network-level dynamics…”
Martell, S. G. (2026) · Measurement Interdisciplinary Research and Perspectives · doi:10.1080/15366367.2026.2716787
Turning cognitive science into game-based biomarkers: Acute stress detection with tunnel runner
Markovitch, B., Kamps, J. C. C., Orij, J. C., Markopoulos, P., & Birk, M. V. (2026) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2026.101176 · Cited by 1
Huang, X., Wang, L., & Jiang, Y. (2026) · British Journal of Psychology · doi:10.1111/bjop.70104
A Mouse-Tracking Paradigm for Value-Driven Attentional Capture
Chen, Y., Geng, F., Gong, M., Zhou, H. J., & Hu, Y. (2026) · Behavioral Sciences · doi:10.3390/bs16081273
“First, these measures primarily reflect the end-point outcome of attentional selection, whereas in naturalistic settings, multiple stimuli compete for limited attentional resources in a continuous and dynamic manner (Draheim et al., 2019; see Stankovi´c, 2025, for a review).”
Developing a psychological test battery to measure cognition in daily life
Aschenbrenner, A. J., Xi, M., Cohn, J., Simon, D. A., & Jackson, J. J. (2026) · Behavior Research Methods · doi:10.3758/s13428-026-03128-4
Pulido, J. H. P. (2026) · Child Neuropsychology · doi:10.1080/09297049.2026.2700210
Zakirov, F., Hosseini, K., Morazzani, A. G., LaPlace, L., Pettit, J. W., & Buzzell, G. A. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.06.29.734887
“This discrepancy further highlights that raw behavioral measures of post-error adjustments (e.g., PES) are not specific regarding the cognitive processes they actually reflect (Draheim et al., 2019; Dutilh et al., 2019; White & Kitchen, 2022).”
Tao, Y., Li, H., Fu, H., Zhang, X., Yu, X., & Yu, A. (2026) · Journal of Intelligence · doi:10.3390/jintelligence14070140
“Additionally, compared to traditional interference indices that frequently suffer from low reliability in developmental populations (Draheim et al., 2019), utilizing the mean reaction time provides a more robust indicator of the global shift toward proactive control.”
A downside of conceptual metaphor: metaphoric alignments of black and white
Durgin, F. H., Diop, S. M., Lewis-Owona, J., & Eaton, O. (2026) · Cognitive Research Principles and Implications · doi:10.1186/s41235-026-00739-5
“Some argue that this is merely a normal consequence of using highly variable reaction times as a measure (Draheim et al., 2019; Hedge et al., 2018).”
Thomas, S., & Dionísio, A. (2026) · Acta Psychologica · doi:10.1016/j.actpsy.2026.107242
Subdomains of executive function correlate with accuracy on a change detection task
Norman, A., Conduit, R., Laycock, R., & Robinson, S. R. (2026) · Psychonomic Bulletin & Review · doi:10.3758/s13423-026-02937-0
“Consequently, some researchers have opted to utilise accuracy-based outcomes for cognitive tasks rather than RT, as they tend to be more reliable and improve intercorre-lations between other tasks and latent factor scores (Draheim et al., 2019, 2021; Snijder et al., 2024).”
Lacko, D., Prošek, T., Dostál, A., Lázníčková, A., Čeněk, J., Šašinka, Č., & Graf, S. (2026) · Behavior Research Methods · doi:10.3758/s13428-026-03055-4
“First introduced by Donders in the nineteenth century, RTs remain central to understanding decision-making dynamics, attentional allocation, and cognitive load (Draheim et al., 2019).”
Roth, T. S., Hirskyj-Douglas, I., & Perea-García, J. (2026) · Animal Cognition · doi:10.1007/s10071-026-02069-x
Modulations of the P3b effect as a function of bilingual language experience
León, N. E. R. D., Bortfeld, H., & Backer, K. (2026) · Scientific Reports · doi:10.1038/s41598-026-47785-z
Landwehr, I., Landwehr, K., & Schmidt, A. F. (2026) · Archives of Sexual Behavior · doi:10.1007/s10508-026-03449-7
“This unreliability aligns with known limitations of difference scores (Thomas & Zumbo, 2012), particularly concerning RT (Draheim et al., 2019).”
Paying attention: cost of cumulative life stress is for older adults only
Sloane, G., Russo, R., Marshall, A. C., & Cooper, N. R. (2026) · Neuropsychology, development, and cognition. Section B, Aging, neuropsychology and cognition · doi:10.1080/13825585.2026.2642770
“This is to account for general ability to detect targets in an RSVP (Arnell & Stubitz, 2010; MacLean & Arnell, 2012) and to address the speed for accuracy trade off seen in older adults (Draheim et al., 2019).”
The Impact of Interference Rate on the Test-Retest Reliability of the Stroop Test
Liao, X., Sun, Y., Tian, Z., Li, Y., Fan, Y.-M., & Li, X.-Q. (2026) · Quarterly Journal of Experimental Psychology · doi:10.1177/17470218261426913
“However, since performance across these two trial types is often highly correlated, such difference scores tend to exhibit low test–retest reliability (Draheim et al., 2019, 2021).”
Keskin, M., Seçer, E., & Kaya, D. (2026) · Physiotherapy Research International · doi:10.1002/pri.70171
“On the other hand, this performance‐related physical fitness parameter is one of the crucial measurements of physical health and is related to the speed factor, which can be important in daily living activities or sporting skills (Draheim et al. 2019).”
A novel task for measuring numerical bias among adults.
Hochman, S., Ben-Shachar, M. S., Kadosh, R. C., & Henik, A. (2026) · Cognition · doi:10.1016/j.cognition.2026.106439
Vasta, N., Mulatti, C., & Treccani, B. (2026) · Psychological Research · doi:10.1007/s00426-025-02224-y
“In order to clarify these issues, we systematically reviewed studies that investigated whether the Stroop task’s typically assessed via test–retest reliability 1 (cf., Draheim et al., 2019 , 2021 ; Hedge et al., 2018; Paap & Sawi, 2016 ).”
Nakamura, M., & Rossi, E. (2026) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2026.109359
Borji, R., Affes, S., Laatar, R., Sahli, S., & Rebai, H. (2025) · American Journal on Intellectual and Developmental Disabilities · doi:10.1080/19447515.2025.2578153
Impact of Pranayama on Audiovisual Reaction Time among Young Adults
Priya, N., Sindhu, R., Chakraborty, H., Vinay, A. V., & Sreeshma, P. (2025) · Medical Journal of Dr D Y Patil Vidyapeeth · doi:10.4103/mjdrdypu.mjdrdypu_1084_24
Computerized attention measure: a web-based open-source test battery to evaluate students’ attention
Pinelli, M., Palluel, E., Dessus, P., Joët, G., Eichenlaub, J., Brugniaux, J. V., & Portrat, S. (2025) · Frontiers in Education · doi:10.3389/feduc.2025.1540817
Coyne, J. T., Jamison, L., Strong, K., Sibley, C., Foroughi, C., & Melick, S. (2025) · Cognitive Research · doi:10.1186/s41235-025-00660-3
“Research in individual differences in attention control has received increased interest since a pair of recent papers identified measurement issues related to response time difference scores which limited the reliability of traditional measures (Hedge et al., 2018, Draheim et al., 2019).”
To Tigger from Pooh, “There's still work to do”: A response to Donath et al.
Beker, J. C., Dorahy, M. J., & Cording, J. R. (2025) · Clinical Psychology Review · doi:10.1016/j.cpr.2025.102629
Verbal response latency in turn-taking as a measure of language processing in mother-child dialogue
Smith, N. A., Batten, R. N., Kelley, E. S., Ispa, J., & McMurray, B. (2025) · Language Learning and Development · doi:10.1080/15475441.2025.2510223
“In addition to differences between individuals, response latency measures within individuals have been essential to cognitive psychology’s attempts to understand the processing underlying complex tasks (Draheim et al., 2019; Just & Carpenter, 1992).”
Analyzing Grade 11 Students’ Performance in General Mathematics Using CEM Assessment Results
Gorosin, R., & Valenzuela, D. J. (2025) · Psychology and Education A Multidisciplinary Journal · doi:10.70838/pemj.370403
Pezzin, A. C., & da Silva, M. (2025) · Revista Científica Multidisciplinar Núcleo do Conhecimento · doi:10.32749/nucleodoconhecimento.com.br/saude/audiometria-tonal
Rabi, R., Chow, R., Grange, J. A., Hasher, L., Alain, C., & Anderson, N. D. (2024) · Neuropsychology, development, and cognition. Section B, Aging, neuropsychology and cognition · doi:10.1080/13825585.2024.2442786
“This approach does not account for individual differences in speed-accuracy trade-offs (Draheim et al., 2019; Hedge et al., 2022), which also change with aging (Forstmann et al., 2011; Salthouse, 1979).”
Assessment of sleepiness role in working memory and whole-body reaction time
Khandan, M., Ebrahimi, A., Zakerian, S. A., Zamanlu, M., & Koohpaei, A. (2024) · Work · doi:10.1177/10519815241290416
“18 Reaction time, considered a marker of mental processing speed and ef fi ciency, is also in fl uenced by sleep deprivation.”
Inducing inhibition affects propagation of uncertainty and checking behaviors
Adler, H., Eitan, R., & Linkovski, O. (2024) · Current Psychology · doi:10.1007/s12144-024-06704-z
“RT is a common variable, but some studies suggested that accuracy outcomes have excellent reliability, convergent validity, and predictive validity (Draheim et al., 2019, 2021; Wickelgren, 1977).”
Willy, W., Telussa, A. S., & Datusanantyo, R. A. (2024) · Intisari Sains Medis · doi:10.15562/ism.v15i1.1990
Age-related differences in the removal of information from working memory
Mei, S., Gao, Y., He, W., Jarrold, C., & Wang, T. (2024) · Journal of Experimental Child Psychology · doi:10.1016/j.jecp.2023.105836
Attentional Control and Performance Across Increasing Degrees of Unreliable Automation
(2020) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1071181320641294
“Attentional control was measured using a custom, PC-based version of the anti-saccade task based on Draheim, Mashburn, Martin, and Engle (2019).”
MethodThe Effects of Increasing Degree of Unreliable Automation on Older Adults' Performance
Textor, C., & Pak, R. (2020) · Interacción · doi:10.1007/978-3-030-50232-4_23
“15 accuracy rather than reaction time (Draheim et al. 2019).”
MethodAnalysis of Digital Phenotype for Classification of Self-Reported Anxiety
de Meneses, D. A., BONATO, L. A. B., MENESES, D. A. D., Tavares, J. J. C., & Lopez, L. C. S. (2026) · Anais da Academia Brasileira de Ciências · doi:10.1590/0001-3765202620250023
Metadata
- Authors
- Draheim, C., Mashburn, C. A., Martin, J. D., & Engle, R. W.
- Affiliations
- Georgia Institute of Technology (US)
- Venue
- Psychological Bulletin
- Published
- 2019-05
- Volume
- 145
- Issue
- 5
- Pages
- 508-535
- ISSN
- 0033-2909 (ISSN-L)
- 1939-1455
- OpenAlex
- W2923932768
- Cited by
- 322
- References
- 0
- Citations by year
- 2026: 46 · 2025: 42 · 2024: 59 · 2023: 55 · 2022: 48 · 2021: 35 · 2020: 31 · 2019: 5
- FWCI
- 17.30
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Psychological Bulletin (publishedVersion, other-oa)
- PubMed (publishedVersion)
- Funders
- Office of Naval Research (N00014-12-1-0406; N00014-12-1-1011)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Cognitive Abilities and Testing
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Keywords
- reaction time, individual differences, developmental research, speed-accuracy trade-off, reliability statistics, difference scores, attentional control, task switching, alternative measures, cognitive development
- MeSH terms
- Humans, Reaction Time, Reproducibility of Results, Behavioral Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
