Short Report: Reaction Time Analysis with Outlier Exclusion: Bias Varies with Sample Size
The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Abstract
To remove the influence of spuriously long response times, many investigators compute “restricted means”, obtained by throwing out any response time more than 2.0, 2.5, or 3.0 standard deviations from the overall sample average. Because reaction time distributions are skewed, however, the computation of restricted means introduces a bias: the restricted mean underestimates the true average of the population of response times. This problem may be very serious when investigators compare restricted means across conditions with different numbers of observations, because the bias increases with sample size. Simulations show that there is substantial differential bias when comparing conditions with fewer than 10 observations against conditions with more than 20. With strongly skewed distributions and a cutoff of 3.0 standard deviations, differential bias can influence comparisons of conditions with even more observations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Numerical recipes in C: the art of scientific computing
(1993) · Choice Reviews Online · doi:10.5860/choice.30-5638 · Cited by 17,933
Hawkins, D. M. (1980) · doi:10.1007/978-94-015-3994-4 · Cited by 2,982
Barnett, V., Lewis, T., & Abeles, F. (1979) · Physics Today · doi:10.1063/1.2995737 · Cited by 2,824
Statistics for the social sciences
Hays, W. L. (1973) · Medical Entomology and Zoology · Cited by 2,081
(1986) · Cited by 2,022
Numerical recipes: The art of scientific computing
DAVIDROPER, L. (1987) · Bulletin of Mathematical Biology · doi:10.1016/s0092-8240(87)90019-x · Cited by 1,424
Buckland, W. R. (1979) · Journal of the Operational Research Society · doi:10.1057/jors.1979.165 · Cited by 562
Anscombe, F. J. (1960) · Technometrics · doi:10.1080/00401706.1960.10489888 · Cited by 478
Retrieval processes in recognition memory.
Ratcliff, R., & Murdock, B. B. (1976) · Psychological Review · doi:10.1037//0033-295x.83.3.190 · Cited by 465
Towards a model of the mind's eye's movement.
Jonides, J. (1980) · Canadian Journal of Psychology / Revue canadienne de psychologie · doi:10.1037/h0081031 · Cited by 269
A warning about median reaction time.
Miller, J. (1988) · Journal of Experimental Psychology: Human Perception and Performance · doi:10.1037//0096-1523.14.3.539 · Cited by 180
Self-citationSubjective probability and choice reaction time
de Klerk, L. F. W., & Oppe, S. (1970) · Acta Psychologica · doi:10.1016/0001-6918(70)90136-8 · Cited by 8
Cited by
Leys, C., Ley, C., Klein, O., Bernard, P., & Licata, L. (2013) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2013.03.013 · Cited by 4,147
A Solution to the Effect of Sample Size on Outlier Elimination
Van Selst, M., & Jolicoeur, P. (1994) · The Quarterly Journal of Experimental Psychology Section A · doi:10.1080/14640749408401131 · Cited by 829
Climate change fingerprints in recent European plant phenology
Menzel, A., Yuan, Y., Matiu, M., Sparks, T., Scheifinger, H., Gehrig, R., & Estrella, N. (2020) · Global Change Biology · doi:10.1111/gcb.15000 · Cited by 387
Effects of truncation on reaction time analysis.
Ulrich, R., & Miller, J. (1994) · Journal of Experimental Psychology: General · doi:10.1037/0096-3445.123.1.34 · Cited by 363
Self-citationProcessing Limitations in Children With Specific Language Impairment: The Role of Executive Function
Im-Bolter, N., Johnson, J., & Pascual-Leone, J. (2006) · Child Development · doi:10.1111/j.1467-8624.2006.00976.x · Cited by 339
Metadata
- Authors
- Miller, J.
- Venue
- The Quarterly Journal of Experimental Psychology Section A
- Publisher
- SAGE Publications
- Published
- 1991-11
- Volume
- 43
- Issue
- 4
- Pages
- 907-912
- ISSN
- 0272-4987 (ISSN-L)
- 1464-0740
- OpenAlex
- W2229267816
- Cited by
- 382
- References
- 12
- FWCI
- 1.78
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.