Power, Dominance, and Constraint: A Note on the Appeal of Different Design Traditions
Open accessThe DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Abstract
The recent field-wide emphasis on power has brought the number of participants used in psychological experiments into focus. Social psychology typically follows a tradition in which many participants perform a small number of trials each; in psychophysics, the tradition is to include only a few participants, who perform many trials each; and the tradition in cognitive psychology falls in between, balancing the number of participants and trials. We ask whether it is better to add trials or to add participants if one wishes to increase power. The answer is straightforward—greatest power is achieved by using more people, and the gain from adding people is greater than the gain from adding trials. In light of these results, the design parameters in the social psychology tradition seem ideal. Yet there are conditions in which one may trade people for trials with only a minor decrement in power. Under these conditions, the limiting factor is the trial-to-trial variability rather than the variability across people in the population. These conditions are highly plausible, and we present a theoretical argument as to why. We think that most cognitive effects are characterized by stochastic dominance; that is, everyone’s true effect is in the same direction. For example, it is plausible that when performing the Stroop task, all people truly identify congruent colors faster than incongruent ones. When dominance holds, small mean effects imply a small degree of variability across the population. It is this degree of homogeneity, the consequence of dominance, that licenses the design parameters of the cognitive psychology and psychophysics traditions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007) · Behavior Research Methods · doi:10.3758/bf03193146 · Cited by 65,971
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641 · cited 1×
“Indeed, it is an indicator that researchers may be massaging variability to produce significance (Button et al., 2013; Gelman & Loken, 2014; Ioannidis, 2005).”
BackgroundOn the ability to inhibit thought and action: a theory of an act of control
Logan, G. D., & Cowan, W. B. (1984) · Psychological Review · doi:10.1037/0033-295x.91.3.295 · Cited by 2,590
The statistical power of abnormal-social psychological research: A review.
Cohen, J. (1962) · The Journal of Abnormal and Social Psychology · doi:10.1037/h0045186 · Cited by 1,824
Modeling Response Times for Two-Choice Decisions
Ratcliff, R., & Rouder, J. N. (1998) · Psychological Science · doi:10.1111/1467-9280.00067 · Cited by 1,478
Self-citationThe Statistical Crisis in Science
Gelman, A., & Loken, E. (2014) · American Scientist · doi:10.1511/2014.111.460 · Cited by 963
Szucs, D., & Ioannidis, J. P. A. (2017) · PLOS Biology · doi:10.1371/journal.pbio.2000797 · Cited by 761 · cited 1×
“One long-standing, salient concern is that too many experiments are underpowered (Cohen, 1962; Maxwell, 2004; Szucs & Ioannidis, 2017).”
Background
Cited by
Brysbaert, M. (2019) · Journal of Cognition · doi:10.5334/joc.72 · Cited by 996
How many words do we read per minute? A review and meta-analysis of reading rate
Brysbaert, M. (2019) · Journal of Memory and Language · doi:10.1016/j.jml.2019.104047 · Cited by 553
Small is beautiful: In defense of the small-N design
Smith, P. L., & Little, D. R. (2018) · Psychonomic Bulletin & Review · doi:10.3758/s13423-018-1451-8 · Cited by 456
Parsons, S., Kruijt, A.-W., & Fox, E. (2019) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245919879695 · Cited by 425
A psychometrics of individual differences in experimental tasks
Rouder, J. N., & Haaf, J. M. (2019) · Psychonomic Bulletin & Review · doi:10.3758/s13423-018-1558-y · Cited by 303
Self-citation
Metadata
- Authors
- Rouder, J. N., & Haaf, J. M.
- Venue
- Advances in Methods and Practices in Psychological Science
- Publisher
- SAGE Publications
- Published
- 2018-02-08
- Volume
- 1
- Issue
- 1
- Pages
- 19-26
- ISSN
- 2515-2459 (ISSN-L)
- 2515-2467
- OpenAlex
- W2787074765
- Cited by
- 85
- References
- 7
- FWCI
- 14.19
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Power, Dominance, and Constraint: A Note on the Appeal of Different Design Traditions. https://app.cochat.ai/papers/10.1177/2515245917745058
- The paper
- Rouder, J. N., & Haaf, J. M. (2018). Power, Dominance, and Constraint: A Note on the Appeal of Different Design Traditions. Advances in Methods and Practices in Psychological Science. https://doi.org/10.1177/2515245917745058
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.