The DOI matches its DataCite record; 2 of the 2 references it cites were found in a registry or an index.
Multi-Class Gaussian Process Classification Made Conjugate: Efficient Inference via Data Augmentation
Open accessAbstract
We propose a new scalable multi-class Gaussian process classification approach building on a novel modified softmax likelihood function. The new likelihood has two benefits: it leads to well-calibrated uncertainty estimates and allows for an efficient latent variable augmentation. The augmented model has the advantage that it is conditionally conjugate leading to a fast variational inference method via block coordinate ascent updates. Previous approaches suffered from a trade-off between uncertainty calibration and speed. Our experiments show that our method leads to well-calibrated uncertainty estimates and competitive predictive performance while being up to two orders faster than the state of the art.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“Bian al. [513] | et | 2020 | CNN | Segmentation | Cardiovascular and retinal OCT | 2 | UESM (Uncertainty Estima- tion and Segmentation Mod- ule) + UCE (Uncertainty- aware Cross Entropy) loss”
From A review of uncertainty quantification in deep learning: Techniques, applications and challenges (2021) · cites this paper as [513]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Bian et al. [513] introduced an uncertainty-aware domain alignment approach to tackle the domain shift issue in the cross-domain UDA (Unsupervised Domain Adaptation) task.”
From A review of uncertainty quantification in deep learning: Techniques, applications and challenges (2021) · cites this paper as [513]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
On calibration of modern neural networks
Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017) · arXiv · doi:10.48550/arxiv.1706.04599 · Cited by 3,435
Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-day Readmission
Caruana, R., Lou, Y., Gehrke, J., Koch, P., Sturm, M., & Elhadad, N. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2788613 · Cited by 1,685
Research works printed but not linked: 36
- Albert, J. H. and Chib, S. (1993). Bayesian analysis of binary and polychotomous response data. Journal of the American Statistical Association , 88(422):669-679.
- Arthur, D. and Vassilvitskii, S. (2007). k-means++: The advantages of careful seeding. In Proceedings of the eighteenth annual ACM-SIAM symposium on Discrete algorithms , pages 1027-1035. Society for Industrial and Applied Mathematics.
- Blei, D. M., Kucukelbir, A., and McAuliffe, J. D. (2017). Variational inference: A review for statisticians. Journal of the American Statistical Association .
- Bojarski, M., Testa, D. D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L. D., Monfort, M., Muller, U., Zhang, J., Zhang, X., Zhao, J., and Zieba, K. (2016). End to end learning for self-driving cars. CoRR , abs/1604.07316. · arXiv
- Buchholz, A., Wenzel, F., and Mandt, S. (2018). Quasimonte carlo variational inference. In International Conference on Machine Learning , pages 667-676.
- Češnovar, R. and Štrumbelj, E. (2017). Bayesian lasso and multinomial logistic regression on gpu. PLOS ONE , 12(6):1-17.
- Chai, K. M. A. (2012). Variational multinomial logit gaussian process. Journal of Machine Learning Research , 13:1745-1808.
- De G. Matthews, A. G., Van Der Wilk, M., Nickson, T., Fujii, K., Boukouvalas, A., León-Villagrá, P., Ghahramani, Z., and Hensman, J. (2017). Gpflow: A gaussian process library using tensorflow. J. Mach. Learn. Res. , 18(1):1299-1304.
- Donner, C. and Opper, M. (2017). The inverse Ising problem in continuous time: A latent variable approach. Physical Review E , 96(6):062104.
- Donner, C. and Opper, M. (2018). Efficient Bayesian Inference for a Gaussian Process Density Model. Conference on Uncertainty in Artificial Intelligence (UAI) , pages 1-10.
- Girolami, M. and Rogers, S. (2006). Variational bayesian multinomial probit regression with gaussian process priors. Neural Computation , 18(8):1790-1817.
- Hensman, J., Fusi, N., and Lawrence, N. D. (2013). Gaussian processes for big data. In Conference on Uncertainty in Artificial Intellegence .
- Hensman, J. and Matthews, A. (2015). Scalable Variational Gaussian Process Classification. AISTATS .
- Hensman, J., Matthews, A., Filippone, M., and Ghahramani, Z. (2015). MCMC for variationally sparse gaussian processes. NIPS .
- Hernández-Lobato, D., Hernández-Lobato, J. M., and Dupont, P. (2011). Robust multi-class gaussian process classification. In Advances in neural information processing systems , pages 280-288.
- Hoffman, M. D., Blei, D. M., Wang, C., and Paisley, J. (2013). Stochastic Variational Inference. JMLR .
- Izmailov, P., Novikov, A., and Kropotov, D. (2018). Scalable gaussian processes with billions of inducing inputs via tensor train decomposition. AISTATS .
- Kim, H.-C. and Ghahramani, Z. (2006). Bayesian gaussian process classification with the em-ep algorithm. IEEE Trans. Pattern Anal. Mach. Intell. , 28(12):1948-1959.
- Kingma, D. P. and Ba, J. (2015). Adam: A method for stochastic optimization. Proceedings of the 3rd International Conference on Learning Representations .
- Linderman, S. W., Johnson, M. J., and Adams, R. P. (2015). Dependent multinomial models made easy: Stick-breaking with the polya-gamma augmentation. NIPS .
- Mandt, S., Hoffman, M., and Blei, D. (2016). A Variational Analysis of Stochastic Gradient Algorithms. ICML .
- Naeini, M. P., Cooper, G., and Hauskrecht, M. (2015). Obtaining well calibrated probabilities using bayesian binning. In Twenty-Ninth AAAI Conference on Artificial Intelligence .
- Owen, A. (1998). Monte Carlo extension of quasi-Monte Carlo. 1998 Winter Simulation Conference. Proceedings (Cat. No.98CH36274) , 1(1):571-577.
- Polson, N. G., Scott, J. G., and Windle, J. (2013). Bayesian inference for logistic models using pólya-gamma latent variables. Journal of the American Statistical Association , 108(504):1339-1349.
- Rasmussen, C. E. and Williams, C. K. I. (2005). Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning) . The MIT Press.
- Riihimäki, J., Jylänki, P., and Vehtari, A. (2013). Nested expectation propagation for gaussian process classification. J. Mach. Learn. Res. , 14(1):75-109.
- Ruiz, F. J. R., Titsias, M. K., Dieng, A. B., and Blei, D. M. (2018). Augment and reduce: Stochastic inference for large categorical distributions. ICML .
- Salimbeni, H., Eleftheriadis, S., and Hensman, J. (2018). Natural gradients in practice: Non-conjugate variational inference in gaussian process models. AISTATS .
- Titsias, M. (2016). One-vs-each approximation to softmax for scalable estimation of probabilities. In Advances in Neural Information Processing Systems , pages 41614169.
- Titsias, M. K. (2009). Variational learning of inducing variables in sparse gaussian processes. In In Artificial Intelligence and Statistics 12 , pages 567-574.
- Villacampa-Calvo, C. and Hernández-Lobato, D. (2017). Scalable multi-class gaussian process classification using expectation propagation. ICML .
- Walker, S. G. (2011). Posterior sampling when the normalizing constant is unknown. Communications in Statistics-Simulation and Computation R © , 40(5):784792.
- Wenzel, F., Galy-Fajou, T., Donner, C., Kloft, M., and Opper, M. (2019). Efficient gaussian process classification using polya-gamma data augmentation. AAAI .
- Williams, C. K. I. and Barber, D. (1998). Bayesian classification with gaussian processes. IEEE Transactions on Pattern Analysis and Machine Intelligence , 20:13421351.
- Xiong, H., Wu, J., and Liu, L. (2010). Classification with classoverlapping: A systematic study. In Proceedings of the 1st International Conference on E-Business Intelligence (ICEBI2010), . Atlantis Press.
- Xiong, W., Droppo, J., Huang, X., Seide, F., Seltzer, M., Stolcke, A., Yu, D., and Zweig, G. (2016). Achieving human parity in conversational speech recognition. CoRR , abs/1610.05256. · arXiv
Cited by
Sorted by citations
A review of uncertainty quantification in deep learning: Techniques, applications and challenges
Abdar, M., Pourpanah, F., Hussain, S., Rezazadegan, D., Liu, L., Ghavamzadeh, M., Fieguth, P., Cao, X., Khosravi, A., Acharya, U. R., et al. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.05.008 · Cited by 3,111
Metadata
- Authors
- Galy-Fajou, T., Wenzel, F., Donner, C., & Opper, M.
- Affiliations
- Technische Universität Berlin (DE)
- Humboldt-Universität zu Berlin (DE)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2019
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1905.09670
- OpenAlex
- W2946610432
- Cited by
- 18
- References
- 2
- Citations by year
- 2024: 2 · 2023: 4 · 2022: 2 · 2021: 6 · 2020: 3 · 2019: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Gaussian Processes and Bayesian Inference
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- conjugate priors, uncertainty calibration, variational inference, block coordinate ascent, scalable inference
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
