CoChat
Sign Up
Cited by 431Optimal Experimental Design MethodsPsychology Research and BibliometricsStatistical Methods and Bayesian Inference

Design of experiments with multiple independent variables: A resource management perspective on complete and reduced factorial designs.

Open access
Collins, Linda M., Dziak, John J., Li, Runze·2009·Psychological Methods
DOI 10.1037/a0015826PMC PMC2796056
Open this paper in CoChat — free account

Abstract

An investigator who plans to conduct an experiment with multiple independent variables must decide whether to use a complete or reduced factorial design. This article advocates a resource management perspective on making this decision, in which the investigator seeks a strategic balance between service to scientific objectives and economy. Considerations in making design decisions include whether research questions are framed as main effects or simple effects; whether and which effects are aliased (confounded) in a particular design; the number of experimental conditions that must be implemented in a particular design and the number of experimental subjects the design requires to maintain the desired level of statistical power; and the costs associated with implementing experimental conditions and obtaining experimental subjects. In this article 4 design options are compared: complete factorial, individual experiments, single factor, and fractional factorial. Complete and fractional factorial designs and single-factor designs are generally more economical than conducting individual experiments on each factor. Although relatively unfamiliar to behavioral scientists, fractional factorial designs merit serious consideration because of their economy and versatility.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1037/a0015826

Cites

55 of 55 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Experimental approaches for optimisation include full or fractional factorial experiments, sequential multiple-assignment randomised trials, and system identification techniques.

    “The experimental approaches used for optimization include full or fractional factorial experiments 38, 39, the sequential multiple-assignment randomized trial (SMART), 40 and system identification techniques 41, 42.”

    From Evaluating Digital Health Interventions (2016) · cites this paper as [40]

  • Adaptive implementation can allocate available resources optimally and can switch tactics when a strategy is failing.

    “Such an adaptive approach to enhancing implementation can make the best allocation of available resources (31) and change its approach if a strategy is failing.”

    From An Overview of Research and Evaluation Designs for Dissemination and Implementation (2017) · cites this paper as [31]

  • In individual experiments and comparative treatment designs, interactions among treatment components cannot be estimated, although such interactions exist and may affect results.

    “A key limitation to both the individual experiments and comparative treatment approaches is that interactions among treatment components cannot be estimated, even though these interactions remain present (33) and potentially have an impact on the results.”

    From The Multiphase Optimization Strategy for Engineering Effective Tobacco Use Interventions (2011) · cites this paper as [33]

  • There are pervasive and enduring misconceptions about experimental design that may lead some grant reviewers to assume that examining individual intervention components with acceptable statistical power is not feasible.

    “However, there are some pervasive and enduring misconceptions about experimental design (discussion of these may be found in (33) and (35)) that may lead some reviewers of grant proposals to make the unwarranted assumption that examination of individual intervention components with acceptable statistical power is not feasible.”

    From The Multiphase Optimization Strategy for Engineering Effective Tobacco Use Interventions (2011) · cites this paper as [33]

  • Innovative trial designs including the multiphase optimization strategy and the sequential multiple assignment randomized trial allow rigorous, systematic testing of a combination of approaches within a single trial.

    “innovative trial designs 31 that allow rigorous, systematic ways of testing a combination of approaches within a single trial including the multiphase optimization strategy and the sequential multiple assignment randomized trial. 32,33”

    From A Research Agenda for Advancing Non-pharmacological Management of Chronic Musculoskeletal Pain: Findings from a VHA State-of-the-art Conference (2018) · cites this paper as [31]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 6

  1. Allore H, Peduzzi P, Han L, Tinetti M. Using the SAS system for experimental designs for multicomponent interventionsin medicine (No. 127-31) SAS white paper. 2006 see www2.sas.com/proceedings/sugi31/127-31.pdf.
  2. Hays WL. Statistics. Orlando, Florida: Harcourt Brace & Company; 1994. [Google Scholar]
  3. Insightful Corporation. S-PLUS® 8 for Windows® user's guide. Seattle, WA: Insightful Corporation; 2007. [Google Scholar]
  4. McLean LD. Phantom classrooms. The School Review. 1966;74:139–149. [Google Scholar]
  5. Ryan BF, Joiner BL, Cryer JD. Minitab handbook. 5th. Belmont, CA: Duxbury/Thomson; 2004. [Google Scholar]
  6. SAS Institute Inc. SAS/QC® 9.1 user's guide. Cary, NC: Author; 2004. [Google Scholar]

Cited by

Sorted by citations

Related

Authors
Collins, L. M., Dziak, J. J., & Li, R.
Affiliations
Pennsylvania State University (US)
Corresponding author
Collins, Linda M.
ORCID
Venue
Psychological Methods
Publisher
American Psychological Association (APA)
Published
2009
Volume
14
Issue
3
Pages
202-224
ISSN
  • 1082-989X (ISSN-L)
  • 1939-1463
DOI
10.1037/a0015826
PMC
PMC2796056
OpenAlex
W2153313117
Cited by
431
References
55
Citations by year
2026: 27 · 2025: 29 · 2024: 25 · 2023: 30 · 2022: 34 · 2021: 35 · 2020: 91 · 2019: 38 · 2018: 30 · 2017: 17 · 2016: 19 · 2015: 12 · 2014: 16 · 2013: 14 · 2012: 8
FWCI
6.77
Type
journal article
Language
en
Open access
Locations
  • Psychological Methods (publishedVersion)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
Funders
National Institute on Drug Abuse (P50 DA10075; K05 DA018206)
Topics
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Optimal Experimental Design Methods
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
Keywords
factorial design, fractional factorial design, resource management, confounding, statistical power, main effects, simple effects
MeSH terms
Anxiety, Choice Behavior, Costs and Cost Analysis, Factor Analysis, Statistical, Humans, Practice, Psychological, Psychometrics, Relaxation Therapy, Research Design, Speech, Multivariate Analysis

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review