Understanding Failures in Out-of-Distribution Detection with Deep Generative Models
Open accessAbstract
Deep generative models (DGMs) seem a natural fit for detecting out-of-distribution (OOD) inputs, but such models have been shown to assign higher probabilities or densities to OOD images than images from the training distribution. In this work, we explain why this behavior should be attributed to model misestimation. We first prove that no method can guarantee performance beyond random chance without assumptions on which out-distributions are relevant. We then interrogate the typical set hypothesis, the claim that relevant out-distributions can lie in high likelihood regions of the data distribution, and that OOD detection should be defined based on the data distribution's typical set. We highlight the consequences implied by assuming support overlap between in- and out-distributions, as well as the arbitrariness of the typical set for OOD detection. Our results suggest that estimation error is a more plausible explanation than the misalignment between likelihood-based OOD detection and out-distributions of interest, and we illustrate how even minimal estimation error can lead to OOD detection failures, yielding implications for future work in deep generative modeling and OOD detection.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Learning an OOD model is difficult and may be impossible in certain situations.
“Recent research shows that learning an OOD model is a difficult problem to solve and may even prove to be impossible in certain situations (Zhang et al., 2021;Fang et al., 2022).”
From Novelty Detection in Reinforcement Learning with World Models (2023) · cites this paper as Zhang et al., 2021
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
No citation lists for this paper
No citation lists for this paper yet — checked again daily.
References 28 · Cited by 42
Research works printed but not linked: 26
- Amodei, D., Olah, C., Steinhardt, J., Christiano, P. F., Schulman, J., and Man´ e, D. Concrete problems in ai safety. ArXiv , 2016.
- Bishop, C. M. Novelty detection and neural network validation. IEE Proceedings-Vision, Image and Signal processing , 141(4):217-222, 1994.
- Chen, X., Mishra, N., Rohaninejad, M., and Abbeel, P. Pixelsnail: An improved autoregressive generative model. In ICML , volume abs/1712.09763, 2018. · arXiv
- Choi, H., Jang, E., and Alemi, A. A. Waic, but why? generative ensembles for robust anomaly detection. ArXiv , 2018.
- Dinh, L., Krueger, D., and Bengio, Y. Nice: Non-linear independent components estimation. ArXiv , 2015.
- Dinh, L., Sohl-Dickstein, J., and Bengio, S. Density estimation using real nvp. ArXiv , abs/1605.08803, 2017. · arXiv
- Hendrycks, D. and Gimpel, K. A baseline for detecting misclassified and out-of-distribution examples in neural networks. In ICLR , 2017.
- Hendrycks, D., Mazeika, M., and Dietterich, T. G. Deep anomaly detection with outlier exposure. In ICLR , 2019.
- Jerfel, G., Wang, S. L., Fannjiang, C., Heller, K. A., Ma, Y., and Jordan, M. Variational refinement for importance sampling using the forward kullback-leibler divergence. In Third Symposium on Advances in Approximate Bayesian Inference , 2021.
- Kingma, D. P. and Dhariwal, P. Glow: Generative flow with invertible 1x1 convolutions. In NeurIPS , 2018.
- Kirichenko, P., Izmailov, P., and Wilson, A. Why normalizing flows fail to detect out-of-distribution data. In NeurIPS , 2020.
- Le Lan, C. and Dinh, L. Perfect density models cannot guarantee anomaly detection. In NeurIPS I Can't Believe It's Not Better Workshop , 2020.
- Liang, S., Li, Y., and Srikant, R. Enhancing the reliability of out-of-distribution image detection in neural networks. In ICLR , 2018.
- Liu, Z., Luo, P., Wang, X., and Tang, X. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV) , December 2015.
- Maaløe, L., Fraccaro, M., Li´ evin, V ., and Winther, O. Biva: A very deep hierarchy of latent variables for generative modeling. In NeurIPS , 2019.
- Nalisnick, E., Matsukawa, A., Teh, Y., G¨ or¨ ur, D., and Lakshminarayanan, B. Do deep generative models know what they don't know? In ICLR , 2019a.
- Nalisnick, E., Matsukawa, A., Teh, Y. W., and Lakshminarayanan, B. Detecting out-of-distribution inputs to deep generative models using typicality. In NeurIPS Workshop on Bayesian Deep Learning , 2019b.
- Neyman, J. and Pearson, E. On the problem of the most efficient tests of statistical hypotheses. Philosophical Transactions of the Royal Society A , 231:289-337, 1933.
- Oord, A., Kalchbrenner, N., Espeholt, L., Kavukcuoglu, K., Vinyals, O., and Graves, A. Conditional image generation with pixelcnn decoders. In NeurIPS , 2016a.
- Oord, A., Kalchbrenner, N., and Kavukcuoglu, K. Pixel recurrent neural networks. In ICML , 2016b.
- Ren, J., Liu, P. J., Fertig, E., Snoek, J., Poplin, R., DePristo, M. A., Dillon, J. V., and Lakshminarayanan, B. Likelihood ratios for out-of-distribution detection. In NeurIPS , 2019.
- Ruff, L., G¨ ornitz, N., Deecke, L., Siddiqui, S., Vandermeulen, R. A., Binder, A., M¨ uller, E., and Kloft, M. Deep one-class classification. In ICML , 2018.
- Schirrmeister, R. T., Zhou, Y., Ball, T., and Zhang, D. Understanding anomaly detection with deep invertible networks through hierarchies of distributions and features. In NeurIPS , 2020.
- Sch¨ olkopf, B., Williamson, R. C., Smola, A., Shawe-Taylor, J., and Platt, J. C. Support vector method for novelty detection. In NeurIPS , 1999.
- Serr` a, J., ´ Alvarez, D., G´ omez, V., Slizovskaia, O., N´ u˜ nez, J. F., and Luque, J. Input complexity and out-ofdistribution detection with likelihood-based generative models. In ICLR , 2020.
- Wang, Z., Dai, B., Wipf, D., and Zhu, J. Further analysis of outlier detection with deep generative models. In NeurIPS , 2020.
- Authors
- Zhang, L. H., Goldstein, M., & Ranganath, R.
- Affiliations
- New York University (US)
- ORCID
- Ranganath, Rajesh
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2021
- Volume
- 139
- Pages
- 12427-12436
- arXiv
- 2107.06908
- PMC
- PMC9295254
- OpenAlex
- W3180957062
- Cited by
- 42
- References
- 28
- Citations by year
- 2025: 3 · 2024: 15 · 2023: 15 · 2022: 6 · 2021: 2
- FWCI
- 3.36
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- International Conference on Machine Learning
- PubMed
- Funders
- National Science Foundation (1514422, 1922658)
- DeepMind
- National Institutes of Health (1922658)
- National Heart, Lung, and Blood Institute (R01HL148248)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Anomaly Detection Techniques and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- out-of-distribution detection, deep generative models, typical sets, estimation error
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
