Bayesian active probabilistic classification for psychometric field estimation
Open accessAbstract
Psychometric functions are typically estimated by fitting a parametric model to categorical subject responses. Procedures to estimate unidimensional psychometric functions (i.e., psychometric curves) have been subjected to the most research, with modern adaptive methods capable of quickly obtaining accurate estimates. These capabilities have been extended to some multidimensional psychometric functions (i.e., psychometric fields) that are easily parameterizable, but flexible procedures for general psychometric field estimation are lacking. This study introduces a nonparametric Bayesian psychometric field estimator operating on subject queries sequentially selected to improve the estimate in some targeted way. This estimator implements probabilistic classification using Gaussian processes trained by active learning. The accuracy and efficiency of two different actively sampled estimators were compared to two non-actively sampled estimators for simulations of one of the simplest psychometric fields in common use: the pure-tone audiogram. The actively sampled methods achieved estimate accuracy equivalent to the non-actively sampled methods with fewer observations. This trend held for a variety of audiogram phenotypes representative of the range of human auditory perception. Gaussian process classification is a general estimation procedure capable of extending to multiple input variables and response classes. Its success with a two-dimensional psychometric field informed by binary subject responses holds great promise for extension to complex perceptual models currently inaccessible to practical estimation.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Optimization by Simulated Annealing
Kirkpatrick, S., Gelatt, C. D., & Vecchi, M. P. (1983) · Science · doi:10.1126/science.220.4598.671 · Cited by 45,134 · see citing works
Elements of Information Theory
Cover, T. M., & Thomas, J. A. (2001) · Wiley eBooks · doi:10.1002/0471200611 · Cited by 37,139 · see citing works
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970 · see citing works
Gaussian Processes for Machine Learning
Rasmussen, C. E., & Williams, C. K. I. (2005) · doi:10.7551/mitpress/3206.001.0001 · Cited by 11,705 · see citing works
Transformed Up-Down Methods in Psychoacoustics
Levitt, H. (1971) · The Journal of the Acoustical Society of America · doi:10.1121/1.1912375 · Cited by 5,505 · see citing works
(2009) · Minds at UW (University of Wisconsin) · Cited by 4,791 · see citing works
Pattern recognition and machine learning
(2007) · Choice Reviews Online · doi:10.5860/choice.44-5091 · Cited by 2,782 · see citing works
Brochu, E., Cora, V. M., & de Freitas, N. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1012.2599 · Cited by 2,568 · see citing works
Quest: A Bayesian adaptive psychometric method
Watson, A. B., & Pelli, D. G. (1983) · Perception & Psychophysics · doi:10.3758/bf03202828 · Cited by 2,398 · see citing works
The psychometric function: I. Fitting, sampling, and goodness of fit
Wichmann, F. A., & Hill, N. J. (2001) · Perception & Psychophysics · doi:10.3758/bf03194544 · Cited by 2,359 · see citing works
Near-optimal sensor placements in Gaussian processes
Guestrin, C., Krause, A., & Singh, A. (2005) · International Conference on Machine Learning · doi:10.1145/1102351.1102385 · Cited by 1,906 · see citing works
Bayesian Experimental Design: A Review
Chaloner, K. M., & Verdinelli, I. (1995) · Statistical Science · doi:10.1214/ss/1177009939 · Cited by 1,843 · see citing works
A sequential algorithm for training text classifiersPreprint
Lewis, D., & Gale, W. A. (1994) · arXiv (Cornell University) · doi:10.5555/188490.188495 · Cited by 1,838 · see citing works
Gulick, W. L., & von Békésy, G. (1963) · The American Journal of Psychology · doi:10.2307/1419812 · Cited by 1,734 · see citing works
Discharge Patterns of Single Fibers in the Cat's Auditory Nerve
Nelson, S. B., Watanabe, T., c., T. E., et al. (1966) · Defense Technical Information Center (DTIC) · Cited by 1,546 · see citing works
Expectation Propagation for approximate Bayesian inferencePreprint
Minka, T. P. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.2294 · Cited by 1,391 · see citing works
Elements of psychophysics, 1860.
Fechner, G. T. (1948) · Readings in the history of psychology. · doi:10.1037/11304-026 · Cited by 1,307 · see citing works
Auditory-nerve response from cats raised in a low-noise chamber
Liberman, M. C. (1978) · The Journal of the Acoustical Society of America · doi:10.1121/1.381736 · Cited by 1,210 · see citing works
A Sequential Algorithm for Training Text Classifiers
Lewis, D. D., & Gale, W. A. (1994) · SIGIR ’94 · doi:10.1007/978-1-4471-2099-5_1 · Cited by 1,208 · see citing works
Heterogeneous Uncertainty Sampling for Supervised Learning
Lewis, D. D., & Catlett, J. (1994) · Elsevier eBooks · doi:10.1016/b978-1-55860-335-6.50026-x · Cited by 1,174 · see citing works
PEST: Efficient Estimates on Probability Functions
Taylor, M. M., & Creelman, C. D. (1967) · The Journal of the Acoustical Society of America · doi:10.1121/1.1910407 · Cited by 1,173 · see citing works
Preferred Method For Clinical Determination Of Pure-Tone Thresholds
Carhart, R. T., & Jerger, J. F. (1959) · Journal of Speech and Hearing Disorders · doi:10.1044/jshd.2404.330 · Cited by 1,125 · see citing works
Gaussian Processes for Regression
Williams, C. K. I., & Rasmussen, C. E. (1995) · Aston Publications Explorer (Aston University) · Cited by 1,121 · see citing works
Elements of Information Theory
Vitetta, G. M., Taylor, D. P., Colavolpe, G., Pancaldi, F., & Martin, P. A. (2013) · doi:10.1002/9781118576618.ch7 · Cited by 999 · see citing works
Adaptive procedures in psychophysical research
Leek, M. R. (2001) · Perception & Psychophysics · doi:10.3758/bf03194543 · Cited by 824 · see citing works
The psychometric function: II. Bootstrap-based confidence intervals and sampling
Wichmann, F. A., & Hill, N. J. (2001) · Perception & Psychophysics · doi:10.3758/bf03194545 · Cited by 809 · see citing works
Bayesian classification with Gaussian processes
Williams, C. K. I., & Barber, D. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.735807 · Cited by 764 · see citing works
Prediction with Gaussian Processes: From Linear Regression to Linear Prediction and Beyond
Williams, C. K. I. (1998) · Springer eBooks · doi:10.1007/978-94-011-5014-9_23 · Cited by 684 · see citing works
Bayesian adaptive estimation of psychometric slope and threshold
Kontsevich, L. L., & Tyler, C. W. (1999) · Vision Research · doi:10.1016/s0042-6989(98)00285-5 · Cited by 681 · see citing works
Book Review: Pattern Recognition and Machine Learning
Nasrabadi, N. M. (2007) · Journal of Electronic Imaging · doi:10.1117/1.2819119 · Cited by 636 · see citing works
Adaptive psychophysical procedures
Treutwein, B. (1995) · Vision Research · doi:10.1016/0042-6989(95)00016-x · Cited by 630 · see citing works
Algorithm 247: Radical-inverse quasi-random point sequence
Halton, J. H. (1964) · Communications of the ACM · doi:10.1145/355588.365104 · Cited by 609 · see citing works
Ockham's Razor and Bayesian Analysis
Jefferys, W. H., & Berger, J. O. (1992) · American Scientist · Cited by 541 · see citing works
Automatic model construction with Gaussian processes
Duvenaud, D. (2014) · Apollo (University of Cambridge) · doi:10.17863/cam.14087 · Cited by 508 · see citing works
Bayesian Active Learning for Classification and Preference Learning
Houlsby, N., Huszár, F., Ghahramani, Z., & Lengyel, M. (2011) · arXiv · doi:10.48550/arxiv.1112.5745 · Cited by 507 · cited 2× · see citing works
“…is another uncertainty sampling approach that seeks the sample point producing the largest decrease in expected posterior entropy; this approach has been shown to yield the sample for which individual configurations of an underlying latent function most disagree about the outcome (Houlsby et al .”
BackgroundMeasuring, estimating, and understanding the psychometric function: A commentary
Klein, S. A. (2001) · Perception & Psychophysics · doi:10.3758/bf03194552 · Cited by 489 · see citing works
Probability and Random Processes for Electrical and Computer Engineers
Gubner, J. A. (2006) · Cambridge University Press eBooks · doi:10.1017/cbo9780511813610 · Cited by 450 · see citing works
King-Smith, P. E., Grigsby, S. S., Vingrys, A. J., et al. (1994) · Vision Research · doi:10.1016/0042-6989(94)90039-6 · Cited by 445 · see citing works
A new generation of algorithms for computerized threshold perimetry, SITA
Bengtsson, B., Olsson, J., Heijl, A., et al. (1997) · Acta Ophthalmologica Scandinavica · doi:10.1111/j.1600-0420.1997.tb00392.x · Cited by 431 · see citing works
Maximum likelihood estimation: The best PEST
Pentland, A. (1980) · Perception & Psychophysics · doi:10.3758/bf03204398 · Cited by 294 · see citing works
Age changes in pure-tone hearing thresholds in a longitudinal study of normal human aging
Brant, L. J., & Fozard, J. L. (1990) · The Journal of the Acoustical Society of America · doi:10.1121/1.399731 · Cited by 231 · see citing works
Classifying Human Audiometric Phenotypes of Age-Related Hearing Loss from Animal Models
Dubno, J. R., Eckert, M. A., Lee, F., et al. (2013) · Journal of the Association for Research in Otolaryngology · doi:10.1007/s10162-013-0396-x · Cited by 220 · see citing works
Bayesian inference for psychometric functions
Kuß, M., Jäkel, F., & Wichmann, F. A. (2005) · Journal of Vision · doi:10.1167/5.5.8 · Cited by 159 · see citing works
QUEST+: A general multidimensional Bayesian adaptive psychometric method
Watson, A. B. (2017) · Journal of Vision · doi:10.1167/17.3.10 · Cited by 148 · see citing works
Validity of Automated Threshold Audiometry
Mahomed‐Asmail, F., Swanepoel, D. W., Eikelboom, R. H., & Soer, M. (2013) · Ear and Hearing · doi:10.1097/01.aud.0000436255.53747.a4 · Cited by 141 · see citing works
Hybrid adaptive procedure for estimation of psychometric functions
Hall, J. L. (1981) · The Journal of the Acoustical Society of America · doi:10.1121/1.385912 · Cited by 138 · see citing works
Hearing Assessment—Reliability, Accuracy, and Efficiency of Automated Audiometry
Swanepoel, D. W., Mngemane, S., Molemong, S., et al. (2010) · Telemedicine Journal and e-Health · doi:10.1089/tmj.2009.0143 · Cited by 121 · see citing works
Psychometric Functions for Children’s Detection of Tones in Noise
Allen, P., & Wightman, F. L. (1994) · Journal of Speech Language and Hearing Research · doi:10.1044/jshr.3701.205 · Cited by 106 · see citing works
Lesmes, L. A., Jeon, S.-T., Lu, Z., & Dosher, B. A. (2006) · Vision Research · doi:10.1016/j.visres.2006.04.022 · Cited by 90 · see citing works
Automated Pure-Tone Audiometry: An Analysis of Capacity, Need, and Benefit
Margolis, R. H., & Morgan, D. E. (2008) · American Journal of Audiology · doi:10.1044/1059-0889(2008/07-0047 · Cited by 87 · see citing works
Kingdom, F. A. A., & Prins, N. (2016) · Elsevier eBooks · doi:10.1016/b978-0-12-407156-8.00001-3 · Cited by 82 · see citing works
Shen, Y., & Richards, V. M. (2012) · The Journal of the Acoustical Society of America · doi:10.1121/1.4733540 · Cited by 81 · see citing works
Thresholds for primary auditory fibers using statistically defined criteria
Geisler, C. D., Deng, L. F., & Greenberg, S. (1985) · The Journal of the Acoustical Society of America · doi:10.1121/1.392228 · Cited by 78 · see citing works
Buss, E., Hall, J. W., & Grose, J. H. (2009) · The Journal of the Acoustical Society of America · doi:10.1121/1.3050273 · Cited by 64 · see citing works
Fast, Continuous Audiogram Estimation using Machine Learning
Song, X., Wallace, B. M., Gardner, J. R., Ledbetter, N. M., Weinberger, K. Q., & Barbour, D. (2015) · Ear and Hearing · doi:10.1097/aud.0000000000000186 · Cited by 59 · see citing works
Self-citationBuss, E., Hall, J. W., & Grose, J. H. (2006) · The Journal of the Acoustical Society of America · doi:10.1121/1.2354024 · Cited by 59 · see citing works
Infant psychometric functions for detection: Mechanisms of immature sensitivity
Bargones, J. Y., Werner, L. A., & Marean, G. C. (1995) · The Journal of the Acoustical Society of America · doi:10.1121/1.414446 · Cited by 58 · see citing works
Fausti, S. A., Frey, R. H., Henry, J. A., et al. (1990) · PubMed · Cited by 44 · see citing works
Psychometric function estimation by probabilistic classification
Song, X. D., Garnett, R., & Barbour, D. L. (2017) · The Journal of the Acoustical Society of America · doi:10.1121/1.4979594 · Cited by 27 · see citing works
Self-citationActive learning of neural response functions with Gaussian processes
Park, M., Horwitz, G. D., & Pillow, J. W. (2011) · Cited by 25 · see citing works
Classification of audiograms by sequential testing using a dynamic Bayesian procedure
Özdamar, Ö., Eilers, R. E., Miskiel, E., et al. (1990) · The Journal of the Acoustical Society of America · doi:10.1121/1.400114 · Cited by 23 · see citing works
(2016) · doi:10.1016/c2012-0-01278-1 · Cited by 21 · see citing works
Fast adaptive estimation of multidimensional psychometric functions
DiMattina, C. (2015) · Journal of Vision · doi:10.1167/15.9.5 · Cited by 20 · see citing works
Bayesian adaptive estimation of the auditory filter
Shen, Y., & Richards, V. M. (2013) · The Journal of the Acoustical Society of America · doi:10.1121/1.4812856 · Cited by 19 · see citing works
Robust and efficient Bayesian adaptive psychometric function estimation
Doire, C. S. J., Brookes, M., & Naylor, P. A. (2017) · The Journal of the Acoustical Society of America · doi:10.1121/1.4979580 · Cited by 16 · see citing works
Flow visualization on a low aspect ratio wing by liquid crystal thermography
Zharkova, G. M., Dovgal, A. V., Kovrizhina, V. N., et al. (2002) · Journal of Visualization · doi:10.1007/bf03182597 · Cited by 9 · see citing works
Bayesian Adaptive Estimation: A Theoretical Review
Kujala, J. V. (2011) · Advanced series on mathematical psychology · doi:10.1142/9789814368018_0006 · Cited by 3 · see citing works
Objective Bayesian Analysis in Acoustics - Ning Xiang and Cameron Fackler
Farrell, D. (2015) · Acoustics Today · Cited by 3 · see citing works
Bayesian learning methods for neural coding
Park, M. J. (2013) · Texas ScholarWorks (Texas Digital Library) · Cited by 1 · see citing works
Flagged first, then by citations
Research works printed but not linked: 4
- Gardner J Malkomes G Garnett R Weinberger KQ Barbour D Cunningham JP 2015a Bayesian active model selection with an application to automated audiometry Adv Neural Inf Process Syst Morgan Kaufmann Publishers Inc San Francisco, CA 2377 2385
- Gardner JM Song XD Cunningham JP Barbour DL Weinberger KQ 2015b Psychophysical testing with Bayesian active learning Uncertain Artif Intell Morgan Kaufmann Publishers Inc San Francisco, CA 286 295
- Katz J Medwetsky L Burkhard R Hood L 2009 Handbook of Clinical Audiology Lippincott Williams & Wilkins
- Osborne MA Garnett R Roberts SJ 2009 Gaussian processes for global optimization 3rd international conference on learning and intelligent optimization (LION3) 1 15
Cited by
Sorted by citations
Lenze, E. J., Nicol, G. E., Barbour, D. L., Kannampallil, T., Wong, A. W. K., Piccirillo, J., Drysdale, A. T., Sylvester, C. M., Haddad, R., Miller, J. P., et al. (2021) · Journal of Psychiatry and Neuroscience · doi:10.1503/jpn.200042 · Cited by 43 · see citing works
Self-citation
- Authors
- Song, X. D., Sukesan, K. A., & Barbour, D. L.
- Affiliations
- Washington University in St. Louis (US)
- Corresponding author
- Barbour, Dennis L.
- Venue
- Attention, Perception, & Psychophysics
- Published
- 2017-12-18
- Volume
- 80
- Issue
- 3
- Pages
- 798-812
- ISSN
- 1943-3921 (ISSN-L)
- 1943-393X
- PMC
- PMC5839980
- OpenAlex
- W2777779805
- Cited by
- 22
- References
- 69
- Citations by year
- 2025: 2 · 2024: 7 · 2023: 3 · 2022: 1 · 2021: 4 · 2020: 1 · 2018: 3 · 2017: 1
- FWCI
- 0.82
- Type
- journal article
- Language
- en
- Update notices
- correction (2018-06-27)
- Open access
- bronze
- doi.org
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Attention Perception & Psychophysics (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Funders
- Center for Integration of Medicine and Innovative Technology
- National Center for Advancing Translational Sciences (UL1 TR002345)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Gaussian Processes and Bayesian Inference
- Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Keywords
- active learning (machine learning), Bayesian estimation, Gaussian process classification, adaptive sampling, pure-tone audiogram, auditory perception, perceptual models
- MeSH terms
- Auditory Perception, Bayes Theorem, Hearing Tests, Humans, Psychometrics, Models, Statistical, Normal Distribution
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
