On the use of counterbalanced designs in cognitive research: A suggestion for a better and more powerful analysis.
Abstract
Counterbalanced designs are ubiquitous in cognitive psychology. Researchers, however, rarely perform optimal analyses of these designs and, as a result, reduce the power of their experiments. In the context of a simple priming experiment, several idealized data sets are used to illustrate the possible costs of ignoring counterbalancing, and recommendations are made for more appropriate analyses. These recommendations apply to assessment of both reliability of effects over subjects and reliability of effects over stimulus items.
Cites (6)
- Statistical principles in experimental design.Winer, B. J. · 1962 · McGraw-Hill Book Company
- Experimental Design: Procedures for the Behavioral SciencesKirk, Roger · 2013
- The language-as-fixed-effect fallacy: A critique of language statistics in psychological researchClark, Herbert H. · 1973 · Journal of Verbal Learning and Verbal Behavior
- Research design and statistical analysis2011 · Choice Reviews Online
- Review of Experimental Design: Procedures for the Behavioral Sciences. 2nd ed.Hays, William L. · 1983 · Contemporary Psychology: A Journal of Reviews
- Power of Analysis of Variance Test Procedures for Certain Incompletely Specified Models, IHelen Bozivich, T. A. Bancroft, H. O. Hartley · 1956 · The Annals of Mathematical Statistics
Cited by (22)
- How Many Participants Do We Have to Include in Properly Powered Experiments? A Tutorial of Power Analysis with Reference TablesBrysbaert, Marc · 2019 · Journal of Cognition
- The Effect of Implied Orientation Derived from Verbal Context on Picture RecognitionStanfield, Robert A., Zwaan, Rolf A. · 2001 · Psychological Science
- Your Highness: Vertical Positions as Perceptual Symbols of Power.Schubert, Thomas W. · 2005 · Journal of Personality and Social Psychology
- Rapid adaptation to foreign-accented EnglishClarke, Constance M., Garrett, Merrill F. · 2004 · The Journal of the Acoustical Society of America
- Lexical information drives perceptual learning of distorted speech: evidence from the comprehension of noise-vocoded sentencesDavis, Matthew H., Johnsrude, Ingrid S., Hervais-Adelman, Alexis, Taylor, Karen, McGettigan, Carolyn · 2005 · Journal of Experimental Psychology General
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: On the use of counterbalanced designs in cognitive research: A suggestion for a better and more powerful analysis. https://app.cochat.ai/papers/10.1037//0278-7393.21.3.785
- The paper
- Pollatsek, A., & Well, A. D. (1995). On the use of counterbalanced designs in cognitive research: A suggestion for a better and more powerful analysis. Journal of Experimental Psychology Learning Memory and Cognition. https://doi.org/10.1037//0278-7393.21.3.785
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.