On the use of counterbalanced designs in cognitive research: A suggestion for a better and more powerful analysis.

Alexander Pollatsek, Arnold D. Well·1995·Journal of Experimental Psychology Learning Memory and Cognition

Abstract

Counterbalanced designs are ubiquitous in cognitive psychology. Researchers, however, rarely perform optimal analyses of these designs and, as a result, reduce the power of their experiments. In the context of a simple priming experiment, several idealized data sets are used to illustrate the possible costs of ignoring counterbalancing, and recommendations are made for more appropriate analyses. These recommendations apply to assessment of both reliability of effects over subjects and reliability of effects over stimulus items.

Cites (6)

  1. Statistical principles in experimental design.
    Winer, B. J. · 1962 · McGraw-Hill Book Company
  2. Experimental Design: Procedures for the Behavioral Sciences
    Kirk, Roger · 2013
  3. The language-as-fixed-effect fallacy: A critique of language statistics in psychological research
    Clark, Herbert H. · 1973 · Journal of Verbal Learning and Verbal Behavior
  4. Research design and statistical analysis
    2011 · Choice Reviews Online
  5. Review of Experimental Design: Procedures for the Behavioral Sciences. 2nd ed.
    Hays, William L. · 1983 · Contemporary Psychology: A Journal of Reviews
  6. Power of Analysis of Variance Test Procedures for Certain Incompletely Specified Models, I
    Helen Bozivich, T. A. Bancroft, H. O. Hartley · 1956 · The Annals of Mathematical Statistics

Show all 6 references

Cited by (22)

  1. How Many Participants Do We Have to Include in Properly Powered Experiments? A Tutorial of Power Analysis with Reference Tables
    Brysbaert, Marc · 2019 · Journal of Cognition
  2. The Effect of Implied Orientation Derived from Verbal Context on Picture Recognition
    Stanfield, Robert A., Zwaan, Rolf A. · 2001 · Psychological Science
  3. Your Highness: Vertical Positions as Perceptual Symbols of Power.
    Schubert, Thomas W. · 2005 · Journal of Personality and Social Psychology
  4. Rapid adaptation to foreign-accented English
    Clarke, Constance M., Garrett, Merrill F. · 2004 · The Journal of the Acoustical Society of America
  5. Lexical information drives perceptual learning of distorted speech: evidence from the comprehension of noise-vocoded sentences
    Davis, Matthew H., Johnsrude, Ingrid S., Hervais-Adelman, Alexis, Taylor, Karen, McGettigan, Carolyn · 2005 · Journal of Experimental Psychology General

See all 399

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: On the use of counterbalanced designs in cognitive research: A suggestion for a better and more powerful analysis. https://app.cochat.ai/papers/10.1037//0278-7393.21.3.785
The paper
Pollatsek, A., & Well, A. D. (1995). On the use of counterbalanced designs in cognitive research: A suggestion for a better and more powerful analysis. Journal of Experimental Psychology Learning Memory and Cognition. https://doi.org/10.1037//0278-7393.21.3.785

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.