The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
Painfree and accurate Bayesian estimation of psychometric functions for (potentially) overdispersed data
Open accessAbstract
The psychometric function describes how an experimental variable, such as stimulus strength, influences the behaviour of an observer. Estimation of psychometric functions from experimental data plays a central role in fields such as psychophysics, experimental psychology and in the behavioural neurosciences. Experimental data may exhibit substantial overdispersion, which may result from non-stationarity in the behaviour of observers. Here we extend the standard binomial model which is typically used for psychometric function estimation to a beta-binomial model. We show that the use of the beta-binomial model makes it possible to determine accurate credible intervals even in data which exhibit substantial overdispersion. This goes beyond classical measures for overdispersion-goodness-of-fit-which can detect overdispersion but provide no method to do correct inference for overdispersed data. We use Bayesian inference methods for estimating the posterior distribution of the parameters of the psychometric function. Unlike previous Bayesian psychometric inference methods our software implementation-psignifit 4-performs numerical integration of the posterior within automatically determined bounds. This avoids the use of Markov chain Monte Carlo (MCMC) methods typically requiring expert knowledge. Extensive numerical tests show the validity of the approach and we discuss implications of overdispersion for experimental design. A comprehensive MATLAB toolbox implementing the method is freely available; a python implementation providing the basic capabilities is also available.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (2003) · doi:10.1201/9780429258480 · Cited by 27,276
(2005) · Statistics for Biology and Health · doi:10.1007/0-387-27255-0_9 · Cited by 20,133
Bootstrap methods: another look at the jackknife
Efron, B. (1979) · The Annals of Statistics · doi:10.1214/aos/1176344552 · Cited by 17,602
Convergence Properties of the Nelder--Mead Simplex Method in Low Dimensions
Lagarias, J. C., Reeds, J. A., Wright, M. H., & Wright, P. E. (1998) · SIAM Journal on Optimization · doi:10.1137/s1052623496303470 · Cited by 7,447
Transformed Up-Down Methods in Psychoacoustics
Levitt, H. (1971) · The Journal of the Acoustical Society of America · doi:10.1121/1.1912375 · Cited by 5,477
Probability Theory: The Logic of Science
Jaynes, E. T. (2003) · doi:10.1017/cbo9780511790423 · Cited by 4,045
Modern Applied Statistics with S-Plus.
Venables, W. N., & Ripley, B. D. (1996) · Biometrics · doi:10.2307/2532871 · Cited by 2,994
An Adaptive Metropolis Algorithm
Haario, H., Saksman, E., & Tamminen, J. (2001) · Bernoulli · doi:10.2307/3318737 · Cited by 2,631
A practical solution to the pervasive problems ofp values
Wagenmakers, E.-J. (2007) · Psychonomic Bulletin & Review · doi:10.3758/bf03194105 · Cited by 2,490
Quest: A Bayesian adaptive psychometric method
Watson, A. B., & Pelli, D. G. (1983) · Perception & Psychophysics · doi:10.3758/bf03202828 · Cited by 2,410
The psychometric function: I. Fitting, sampling, and goodness of fit
Wichmann, F. A., & Hill, N. J. (2001) · Perception & Psychophysics · doi:10.3758/bf03194544 · Cited by 2,358
Self-citationCollett, D. (1991) · Springer eBooks · doi:10.1007/978-1-4899-4475-7 · Cited by 1,936
Scientific method: Statistical errors
Nuzzo, R. (2014) · Nature · doi:10.1038/506150a · Cited by 1,536
SEQUENTIAL ESTIMATION OF POINTS ON A PSYCHOMETRIC FUNCTION
Wetherill, G. B., & Levitt, H. (1965) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/j.2044-8317.1965.tb00689.x · Cited by 1,196
Numerical integration using sparse grids
Gerstner, T., & Griebel, M. (1998) · Numerical Algorithms · doi:10.1023/a:1019129717644 · Cited by 923
The psychometric function: II. Bootstrap-based confidence intervals and sampling
Wichmann, F. A., & Hill, N. J. (2001) · Perception & Psychophysics · doi:10.3758/bf03194545 · Cited by 810
Self-citationExtra-Binomial Variation in Logistic Linear Models
Williams, D. A. (1982) · Applied Statistics · doi:10.2307/2347977 · Cited by 798
Bayesian adaptive estimation of psychometric slope and threshold
Kontsevich, L. L., & Tyler, C. W. (1999) · Vision Research · doi:10.1016/s0042-6989(98)00285-5 · Cited by 668
Adaptive psychophysical procedures
Treutwein, B. (1995) · Vision Research · doi:10.1016/0042-6989(95)00016-s · Cited by 648
Robust misinterpretation of confidence intervals
Hoekstra, R., Morey, R. D., Rouder, J. N., & Wagenmakers, E.-J. (2014) · Psychonomic Bulletin & Review · doi:10.3758/s13423-013-0572-3 · Cited by 425
Forced-choice staircases with fixed step sizes: asymptotic and small-sample properties
Garcı́a-Pérez, M. A. (1998) · Vision Research · doi:10.1016/s0042-6989(97)00340-4 · Cited by 423
(2015) · Elsevier eBooks · doi:10.1016/c2012-0-00477-2 · Cited by 416
Likelihood approximation by numerical integration on sparse grids
Heiss, F., & Winschel, V. (2008) · Journal of Econometrics · doi:10.1016/j.jeconom.2007.12.004 · Cited by 374
Modern Applied Statistics with S-PLUS
James, D. A., Venables, W. N., & Ripley, B. D. (1996) · Technometrics · doi:10.2307/1268908 · Cited by 350
Prentice, R. L. (1986) · Journal of the American Statistical Association · doi:10.1080/01621459.1986.10478275 · Cited by 319
Fitting the psychometric function
Treutwein, B., & Strasburger, H. (1999) · Perception & Psychophysics · doi:10.3758/bf03211951 · Cited by 311
Inference for psychometric functions in the presence of nonstationary behavior
Frund, I., Haenel, N. V., & Wichmann, F. A. (2011) · Journal of Vision · doi:10.1167/11.6.16 · Cited by 261
Self-citationQuantifying the effect of intertrial dependence on perceptual decisions
Frund, I., Wichmann, F. A., & Macke, J. H. (2014) · Journal of Vision · doi:10.1167/14.7.9 · Cited by 234
Self-citationConsistency of auditory detection judgments.
Green, D. M. (1964) · Psychological Review · doi:10.1037/h0044520 · Cited by 170
Stimulus selection in adaptive psychophysical procedures
Green, D. M. (1990) · The Journal of the Acoustical Society of America · doi:10.1121/1.399058 · Cited by 168
Bayesian inference for psychometric functions
Kuss, M., Jäkel, F., & Wichmann, F. A. (2005) · Journal of Vision · doi:10.1167/5.5.8 · Cited by 158
Self-citationFOSTER, D. H., & Bischof, W. F. (1991) · Psychological Bulletin · doi:10.1037//0033-2909.109.1.152 · Cited by 153
Jäkel, F., & Wichmann, F. A. (2006) · Journal of Vision · doi:10.1167/6.11.13 · Cited by 147
Self-citationNonindependence of successive responses in measurements of the visual threshold.
Verplanck, W. S., COLLIER, G. H., & Cotton, J. W. (1952) · Journal of Experimental Psychology · doi:10.1037/h0054948 · Cited by 118
Studies of Psychophysical Methods for Measuring Visual Thresholds*†
Blackwell, H. R. (1952) · Journal of the Optical Society of America · doi:10.1364/josa.42.000606 · Cited by 112
Confidence intervals for the parameters of psychometric functions
Maloney, L. T. (1990) · Perception & Psychophysics · doi:10.3758/bf03205977 · Cited by 90
Shen, Y., & Richards, V. M. (2012) · The Journal of the Acoustical Society of America · doi:10.1121/1.4733540 · Cited by 82
Roles of the upper and lower bodies in direction discrimination of point-light walkers
Takahashi, K., Fukuda, H., Ikeda, H., Doi, H., Watanabe, K., Ueda, K., & Shinohara, K. (2011) · Journal of Vision · doi:10.1167/11.14.8 · Cited by 58
Bootstrap estimates of the statistical accuracy of thresholds obtained from psychometric functions.
FOSTER, D. H., & Bischof, W. F. (1997) · PubMed · Cited by 54
O'Regan, J. K., & Humbert, R. (1989) · Perception & Psychophysics · doi:10.3758/bf03210858 · Cited by 50
Contrast discrimination with sinusoidal gratings of different spatial frequency
Bird, C. M., Henning, G. B., & Wichmann, F. A. (2002) · Journal of the Optical Society of America A · doi:10.1364/josaa.19.001267 · Cited by 50
Self-citationAnalysis of response sequences in the setting of a psychophysical experiment.
Senders, V. L., & Sowards, A. (1952) · PubMed · Cited by 47
The Role of Parametric Assumptions in Adaptive Bayesian Estimation.
Alcalá‐Quintana, R., & García-Pérez, M. Á. (2004) · Psychological Methods · doi:10.1037/1082-989x.9.2.250 · Cited by 45
Bootstrap variance estimators for the parameters of small-sample sensory-performance functions
FOSTER, D. H., & Bischof, W. F. (1987) · Biological Cybernetics · doi:10.1007/bf00338826 · Cited by 45
Analysis of Response Sequences in the Setting of a Psychophysical Experiment
Senders, V. L., & Sowards, A. (1952) · The American Journal of Psychology · doi:10.2307/1418758 · Cited by 42
Henning, G. B., Millar, R. W., & Hill, N. J. (2000) · Journal of the Optical Society of America A · doi:10.1364/josaa.17.001147 · Cited by 17
Cited by
Cortical layer-specific critical dynamics triggering perception
Marshel, J. H., Kim, Y. S., Machado, T. A., Quirin, S., Benson, B., Kadmon, J., Raja, C., Chibukhchyan, A., Ramakrishnan, C., Inoue, M., et al. (2019) · Science · doi:10.1126/science.aaw5202 · Cited by 652
Towards foveated rendering for gaze-tracked virtual reality
Patney, A., Salvi, M., Kim, J., Kaplanyan, A., Wyman, C., Benty, N., Luebke, D., & Lefohn, A. (2016) · ACM Transactions on Graphics · doi:10.1145/2980179.2980246 · Cited by 491
Prins, N., & Kingdom, F. A. A. (2018) · Frontiers in Psychology · doi:10.3389/fpsyg.2018.01250 · Cited by 429
Comprehension of acoustically degraded speech in Alzheimer’s disease and primary progressive aphasia
Jiang, J., Johnson, J. C. S., Requena-Komuro, M.-C., Benhamou, E., Sivasathiaseelan, H., Chokesuwattanaskul, A., Nelson, A., Nortley, R., Weil, R. S., Volkmer, A., et al. (2023) · Brain · doi:10.1093/brain/awad163 · Cited by 28
Experimentally disambiguating models of sensory cue integration
Scarfe, P. (2022) · Journal of Vision · doi:10.1167/jov.22.1.5 · Cited by 17
Metadata
- Authors
- Schütt, H. H., Harmeling, S., Macke, J. H., & Wichmann, F. A.
- Venue
- Vision Research
- Publisher
- Elsevier BV
- Published
- 2016-05
- Volume
- 122
- Pages
- 105-123
- ISSN
- 0042-6989 (ISSN-L)
- OpenAlex
- W2307270163
- Cited by
- 402
- References
- 46
- FWCI
- 19.80
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.