Optimal design in psychological research.
Abstract
Psychologists often do not consider the optimality of their research designs. However, increasing costs of using inefficient designs requires psychologists to adopt more efficient designs and to use more powerful analysis strategies. Common designs with many factor levels and equal allocations of observations are often inefficient for the specific questions most psychologists want to answer. Happenstance allocations determined by random sampling are usually even more inefficient and some common analysis strategies can exacerbate the inefficiency. By selecting treatment levels and allocating observations optimally, psychologists can greatly increase the efficiency and statistical power of their research designs. A few heuristic design principles can produce much more efficient designs than are often used. Experimental researchers outside psychology often carefully consider the efficiency of their research designs. For example, the high costs of conducting large-scale experiments with industrial processes has motivated the search for designs that are optimally efficient. As a consequence, a substantial literature on optimal design has developed outside psychology. In contrast, psychologists have not been as constrained by costs, so their research designs have been based on tradition and computational ease. Most psychologists are unaware of the literature on optimal research design; this topic receives little or no attention in popular textbooks on methods and statistics in psychology. Experimental design textbooks offer little if any advice on how many levels of the independent variables to use or on how to allocate observations across those levels to obtain optimal efficiency. When advice is offered, it is usually based on heuristics derived from experience rather than statistical principles. The purpose of this article is to review the basic concepts of optimal design and to illustrate how a few
Cites (6)
- High-speed scanning in human memorySternberg, Saul · 1966 · Science
- Statistical difficulties of detecting interactions and moderator effects.McClelland, Gary H., Judd, Charles M. · 1993 · Psychological Bulletin
- The statistical power of abnormal-social psychological research: A review.Cohen, Jacob · 1962 · The Journal of Abnormal and Social Psychology
- Things I have learned (so far).Cohen, Jacob · 1990 · American Psychologist
- The Philosophy of Multiple ComparisonsTukey, John W. · 1991 · Statistical Science
- Do studies of statistical power have an effect on the power of studies?Sedlmeier, Peter, Gigerenzer, Gerd · 1989 · Psychological Bulletin
Cited by (16)
- Effect size measures for mediation models: Quantitative strategies for communicating indirect effects.Preacher, Kristopher J., Kelley, Ken · 2011 · Psychological Methods
- Verbal working memory and sentence comprehensionCaplan, David, Waters, Gloria S. · 1999 · Behavioral and Brain Sciences
- Many Labs 2: Investigating variation in replicability across samples and settingsKlein, Richard A., Vianello, Michelangelo, Hasselman, Fred, Adams, Byron G., Adams, Reginald B., Alper, Sinan, Aveyard, Mark, Axt, Jordan R., Babalola, Mayowa T., Bahník, Štěpán et al. · 2018 · Advances in Methods and Practices in Psychological Science
- Statistical power and optimal design in experiments in which samples of participants respond to samples of stimuli.Westfall, Jacob, Kenny, David A., Judd, Charles M. · 2014 · Journal of Experimental Psychology: General
- Structural Equation Modeling: Strengths, Limitations, and MisconceptionsTomarken, Andrew J., Waller, Niels G. · 2005 · Annual Review of Clinical Psychology
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Optimal design in psychological research. https://app.cochat.ai/papers/10.1037//1082-989x.2.1.3
- The paper
- McClelland, G. H. (1997). Optimal design in psychological research. Psychological Methods. https://doi.org/10.1037//1082-989x.2.1.3
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.