A tutorial on adaptive design optimization
Open accessThe DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Abstract
Experimentation is ubiquitous in the field of psychology and fundamental to the advancement of its science, and one of the biggest challenges for researchers is designing experiments that can conclusively discriminate the theoretical hypotheses or models under investigation. The recognition of this challenge has led to the development of sophisticated statistical methods that aid in the design of experiments and that are within the reach of everyday experimental scientists. This tutorial paper introduces the reader to an implementable experimentation methodology, dubbed Adaptive Design Optimization, that can help scientists to conduct “smart” experiments that are maximally informative and highly efficient, which in turn should accelerate scientific discovery in psychology and beyond.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Kass, R. E., & Raftery, A. E. (1995) · Journal of the American Statistical Association · doi:10.1080/01621459.1995.10476572 · Cited by 12,558
Modeling by shortest data description
Rissanen, J. (1978) · Automatica · doi:10.1016/0005-1098(78)90005-5 · Cited by 6,011
An introduction to MCMC for machine learning.
Andrieu, C., de Freitas, N., Doucet, A., & Jordan, M. I. (2003) · Machine Learning · doi:10.1023/a:1020281327116 · Cited by 2,451
On a measure of the information provided by an experiment
Lindley, D. V. (1956) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177728069 · Cited by 1,509
Adaptive approximate Bayesian computation
Beaumont, M. A., Cornuet, J.-M., Marin, J.-M., & Robert, C. P. (2009) · Biometrika · doi:10.1093/biomet/asp052 · Cited by 614
Doing Bayesian Data Analysis: A Tutorial with R and BUGS
Barry, J. (2011) · Europe’s Journal of Psychology · doi:10.5964/ejop.v7i4.163 · Cited by 563 · cited 1×
“It is beyond the scope of this tutorial to review this vast literature, and instead, the reader is directed to consult Bayesian textbooks (e.g., Gelman, Carlin, Stern, & Rubin, 2004; Gill, 2007; Kruschke, 2010; Robert, 2001).”
BackgroundWixted, J. T., & Ebbesen, E. B. (1991) · Psychological Science · doi:10.1111/j.1467-9280.1991.tb00175.x · Cited by 513 · cited 2×
“Through years of experimentation with humans (and animals), a handful of the models have proven to be superior to the rest of the field (Wixted and Ebbesen, 1991), although more recent research suggests that they are not complete (Oberauer and Lewandowsky, 2008).”
BackgroundAPPLICATION OF COMPUTERIZED ADAPTIVE TESTING TO EDUCATIONAL PROBLEMS
WEISS, D. J., & KINGSBURY, G. G. (1984) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.1984.tb01040.x · Cited by 501
A tutorial introduction to the minimum description length principle
Grunwald, P. (2004) · arXiv · doi:10.48550/arxiv.math/0406077 · Cited by 293
Systems biology: experimental design
Kreutz, C., & Timmer, J. (2009) · The FEBS Journal · doi:10.1111/j.1742-4658.2008.06843.x · Cited by 257
Cited by
A tutorial on Gaussian process regression: modelling, exploring, and exploiting functions
Schulz, E., Speekenbrink, M., & Krause, A. (2018) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2018.03.001 · Cited by 1,735
Comparing models of contrast gain using psychophysical experiments
DiMattina, C. (2016) · Journal of Vision · doi:10.1167/16.9.1 · Cited by 12
Designing optimal behavioral experiments using machine learning
Valentin, S., Kleinegesse, S., Bramley, N. R., Seriès, P., Gutmann, M. U., & Lucas, C. G. (2024) · eLife · doi:10.7554/elife.86224 · Cited by 8
Metadata
- Authors
- Myung, J. I., Cavagnaro, D. R., & Pitt, M. A.
- Venue
- Journal of Mathematical Psychology
- Publisher
- Elsevier BV
- Published
- 2013-06
- Volume
- 57
- Issue
- 3-4
- Pages
- 53-67
- ISSN
- 0022-2496 (ISSN-L)
- OpenAlex
- W2082413592
- Cited by
- 129
- References
- 10
- FWCI
- 9.52
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.