Optimal experimental design for model discrimination.
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
Models of a psychological process can be difficult to discriminate experimentally because it is not easy to determine the values of the critical design variables (e.g., presentation schedule, stimulus structure) that will be most informative in differentiating them. Recent developments in sampling-based search methods in statistics make it possible to determine these values and thereby identify an optimal experimental design. After describing the method, it is demonstrated in 2 content areas in cognitive psychology in which models are highly competitive: retention (i.e., forgetting) and categorization. The optimal design is compared with the quality of designs used in the literature. The findings demonstrate that design optimization has the potential to increase the informativeness of the experimental method.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Context theory of classification learning
Medin, D. L., & Schaffer, M. M. (1978) · Psychological Review · doi:10.1037/0033-295x.85.3.207 · Cited by 3,061 · cited 2×
“In contrast, exemplar theories prescribe that we encode and store in memory all information about every encountered instance (Medin & Schaffer, 1978).”
BackgroundMethodInformation Theory and an Extension of the Maximum Likelihood Principle
Akaike, H. (1992) · Springer Series in Statistics · doi:10.1007/978-1-4612-0919-5_38 · Cited by 3,028 · cited 2×
“Obviously, design optimality can also be defined with respect to other methods of model selection such as the Akaike Information Criterion (AIC: Akaike, 1973) and the Bayes factor (Kass & Raftery, 1995).”
MethodFisher information and stochastic complexity
Rissanen, J. J. (1996) · IEEE Transactions on Information Theory · doi:10.1109/18.481776 · Cited by 814
Wixted, J. T., & Ebbesen, E. B. (1991) · Psychological Science · doi:10.1111/j.1467-9280.1991.tb00175.x · Cited by 513 · cited 6×
“To determine the suitability of an experiment for addressing a theoretical question of interest, it would be ideal if one could perform a power analysis on the experimental design itself, and thereby improve statistical inference.”
BackgroundMethodInfluentialWixted, J. T., & Ebbesen, E. B. (1997) · Memory & Cognition · doi:10.3758/bf03211316 · Cited by 237
Cited by
A tutorial on Gaussian process regression: modelling, exploring, and exploiting functions
Schulz, E., Speekenbrink, M., & Krause, A. (2018) · Journal of Mathematical Psychology · doi:10.1016/j.jmp.2018.03.001 · Cited by 1,735
VBA: A Probabilistic Treatment of Nonlinear Models for Neurobiological and Behavioural Data
Daunizeau, J., Adam, V., & Rigoux, L. (2014) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003441 · Cited by 385
Bayesian adaptive estimation of the contrast sensitivity function: The quick CSF method
Lesmes, L. A. (2010) · Journal of Vision · doi:10.1167/10.3.17 · Cited by 333
A meta-analytic review of two modes of learning and the description-experience gap.
Wulff, D. U., Mergenthaler-Canseco, M., & Hertwig, R. (2018) · Psychological Bulletin · doi:10.1037/bul0000115 · Cited by 271
An optimal experimental design approach to model discrimination in dynamic biochemical systems
Skanda, D., & Lebiedz, D. (2010) · Bioinformatics · doi:10.1093/bioinformatics/btq074 · Cited by 42
Metadata
- Authors
- Myung, J. I., & Pitt, M. A.
- Venue
- Psychological Review
- Published
- 2009-07
- Volume
- 116
- Issue
- 3
- Pages
- 499-518
- ISSN
- 0033-295X (ISSN-L)
- 1939-1471
- OpenAlex
- W2068036624
- Cited by
- 140
- References
- 5
- FWCI
- 11.73
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.