Lagging Inference Networks and Posterior Collapse in Variational Autoencoders
Open accessThe DOI matches its OpenAlex record; 18 of the 18 references it cites were found in a registry or an index.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 2×
“For images, we use a ResNet (He et al., 2016) encoder and a 13-layer Gated PixelCNN (van den Oord et al., 2016) decoder.”
MethodHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1×
“This phenomenon is more common when the generator p θ ( x | z ) is parametrized with a strong autoregressive neural network, for example, an LSTM (Hochreiter & Schmidhuber, 1997) on text or a PixelCNN (van den Oord et al., 2016) on images.”
MethodAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067 · cited 1×
“Variational autoencoders (VAEs) (Kingma & Welling, 2014) represent a popular combination of a deep latent variable model (shown in Figure 1(a)) and an accompanying variational learning technique.”
Background(2017) · International Conference on Learning Representations · Cited by 3,248
Human-level concept learning through probabilistic program induction
Lake, B. M., Salakhutdinov, R., & Tenenbaum, J. B. (2015) · Science · doi:10.1126/science.aab3050 · Cited by 2,817 · cited 3×
“To better visualize the sys-tem difference on OMNIGLOT dataset, for OMNIGLOT figure we ignore some β -VAE baselines that are not competitive.”
MethodGenerating Sentences from a Continuous Space
Bowman, S. R., Vilnis, L., Vinyals, O., Dai, A., Jozefowicz, R., & Bengio, S. (2016) · Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning · doi:10.18653/v1/k16-1002 · Cited by 2,061 · cited 4×
“Posterior collapse in VAEs is first detailed in (Bowman et al., 2016) where they combine a LSTM decoder with VAE for text modeling.”
BackgroundMethodInfluentialConditional Image Generation with PixelCNN Decoders
van den Oord, A., Kalchbrenner, N., Vinyals, O., Espeholt, L., Graves, A., & Kavukcuoglu, K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.05328 · Cited by 1,833 · cited 3×
“This phenomenon is more common when the generator p θ ( x | z ) is parametrized with a strong autoregressive neural network, for example, an LSTM (Hochreiter & Schmidhuber, 1997) on text or a PixelCNN (van den Oord et al., 2016) on images.”
MethodStochastic variational inference
Hoffman, M. D., Blei, D. M., Wang, C., & Paisley, J. W. (2013) · Journal of Machine Learning Research · doi:10.5555/2567709.2502622 · Cited by 1,333 · cited 5×
“…of using inference networks (also called amortized inference) to train deep generative models over traditional locally stochastic variational inference (Hoffman et al., 2013) is that they share parameters over all data samples, amortizing computational cost and allowing for efficient training.”
BackgroundMethodInfluentialImportance Weighted Autoencoders
Burda, Y., Grosse, R., & Salakhutdinov, R. (2015) · arXiv · doi:10.48550/arxiv.1509.00519 · Cited by 644
InfoVAE: Information Maximizing Variational Autoencoders
Zhao, S., Song, J., & Ermon, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.02262 · Cited by 328 · cited 22×
“We use 32-dimensional z and optimize ELBO objective with SGD for text and Adam (Kingma & Ba, 2015) for images.”
BackgroundInfluentialChen, X., Kingma, D. P., Salimans, T., Duan, Y., Dhariwal, P., Schulman, J., Sutskever, I., & Abbeel, P. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.02731 · Cited by 226 · cited 4×
“, 2017), using lossy input (Chen et al., 2017), utilizing different latent variable connections (Dieng et al.”
BackgroundMethodInfluentialA Hybrid Convolutional Variational Autoencoder for Text Generation
Semeniuta, S., Severyn, A., & Barth, E. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1066 · Cited by 210 · cited 4×
“Thus, many proposed solutions to posterior collapse focus on weakening the generator by replacing it with a non-recurrent alternative (Yang et al., 2017; Semeniuta et al., 2017) or modifying the training objective (Zhao et al.”
BackgroundMethodInfluentialTopicRNN: A Recurrent Neural Network with Long-Range Semantic Dependency
Dieng, A. B., Wang, C., Gao, J., & Paisley, J. W. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01702 · Cited by 175 · cited 1×
“…et al., 2018), deploying less powerful de-coders (Yang et al., 2017; Semeniuta et al., 2017), using lossy input (Chen et al., 2017), utilizing different latent variable connections (Dieng et al., 2017; 2018; Park et al., 2018), or changing the prior (Tomczak & Welling, 2018; Xu & Durrett, 2018).”
MethodSpherical Latent Spaces for Stable Variational Autoencoders
Xu, J., & Durrett, G. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1480 · Cited by 170 · cited 1×
“…et al., 2018), deploying less powerful de-coders (Yang et al., 2017; Semeniuta et al., 2017), using lossy input (Chen et al., 2017), utilizing different latent variable connections (Dieng et al., 2017; 2018; Park et al., 2018), or changing the prior (Tomczak & Welling, 2018; Xu & Durrett, 2018).”
MethodImproving Variational Inference with Inverse Autoregressive Flow
Kingma, D. P., Salimans, T., Józefowicz, R., Chen, X., Sutskever, I., & Welling, M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.04934 · Cited by 164
A Hierarchical Latent Structure for Variational Conversation Modeling
Park, Y., Cho, J., & Kim, G. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-1162 · Cited by 109 · cited 2×
“…et al., 2018), deploying less powerful de-coders (Yang et al., 2017; Semeniuta et al., 2017), using lossy input (Chen et al., 2017), utilizing different latent variable connections (Dieng et al., 2017; 2018; Park et al., 2018), or changing the prior (Tomczak & Welling, 2018; Xu & Durrett, 2018).”
BackgroundMethod
Cited by
Zhu, Y., Zabaras, N., Koutsourelakis, P.-S., & Perdikaris, P. (2019) · Journal of Computational Physics · doi:10.1016/j.jcp.2019.05.024 · Cited by 1,030
Bond-Taylor, S., Leach, A., Long, Y., & Willcocks, C. G. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3116668 · Cited by 703
CogView: Mastering Text-to-Image Generation via Transformers
Ding, M., Yang, Z., Hong, W., Zheng, W., Zhou, C., Yin, D., Lin, J., Zou, X., Shao, Z., Yang, H., et al. (2021) · arXiv · doi:10.48550/arxiv.2105.13290 · Cited by 450
Diffusion-based Generation, Optimization, and Planning in 3D Scenes
Huang, S., Wang, Z., Li, P., Jia, B., Liu, T., Zhu, Y., Liang, W., & Zhu, S.-C. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01607 · Cited by 192
Unsupervised Paraphrasing by Simulated Annealing
Liu, X., Mou, L., Meng, F., Zhou, H., Zhou, J., & Song, S. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.28 · Cited by 75
Metadata
- Authors
- He, J., Spokoyny, D., Neubig, G., & Berg-Kirkpatrick, T.
- Venue
- arXiv (Cornell University)
- Publisher
- Cornell University
- Published
- 2019-01-16
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1901.05534
- OpenAlex
- W2910135751
- Cited by
- 189
- References
- 18
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.