Improved Contrastive Divergence Training of Energy Based Models
Open accessAbstract
Contrastive divergence is a popular method of training energy-based models, but is known to have difficulties with training stability. We propose an adaptation to improve contrastive divergence training by scrutinizing a gradient term that is difficult to calculate and is often left out for convenience. We show that this gradient term is numerically significant and in practice is important to avoid training instabilities, while being tractable to estimate. We further highlight how data augmentation and multi-scale processing can be used to improve model robustness and generation quality. Finally, we empirically evaluate stability of model architectures and show improved performance on a host of benchmarks and use cases,such as image generation, OOD detection, and compositional generation.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Training energy-based policies often requires negative sampling to estimate an intractable normalization constant, which is known to cause training instability.
“Training energy-based policies often requires negative sampling to estimate an intractable normalization constant, which is known to cause training instability [11, 12].”
From Diffusion Policy: Visuomotor Policy Learning via Action Diffusion (2023) · cites this paper as [11]
Section 2 (p. 2) says the EBM partition function cannot be reliably computed and that training uses sampling-based approximations. The Introduction (p. 1) reports instability in contrastive-divergence training. The paper discusses energy-based models rather than specifically energy-based policies, and it does not establish that negative sampling to estimate the normalization constant is itself the cause of the instability. (LLM, conf=0.85)
Experimental — accuracy not yet measuredInaccuracy of negative sampling is known to cause training instability for EBMs.
“In practice, the inaccuracy of negative sampling is known to cause training instability for EBMs [11, 48].”
From Diffusion Policy: Visuomotor Policy Learning via Action Diffusion (2023) · cites this paper as [11]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
No citation lists for this paper
No citation lists for this paper yet — checked again daily.
References 56 · Cited by 36
Research works printed but not linked: 25
- Jan Beirlant, E. Dudewicz, L. Gyor, and E.C. Meulen. Nonparametric entropy estimation: An overview. International Journal of Mathematical and Statistical Sciences , 6, 01 1997.
- Bo Dai, Zhen Liu, Hanjun Dai, Niao He, Arthur Gretton, Le Song, and Dale Schuurmans. Exponential family estimation via adversarial dynamics embedding. In Advances in Neural Information Processing Systems , pp. 10979-10990, 2019.
- Yilun Du, Shuang Li, and Igor Mordatch. Compositional visual generation with energy based models. In Advances in Neural Information Processing Systems , 2020a.
- Ruiqi Gao, Yang Lu, Junpei Zhou, Song-Chun Zhu, and Ying Nian Wu. Learning generative convnets via multigrid modeling and sampling. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pp. 9155-9164, 2018.
- Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron Courville. Improved training of wasserstein gans. In NIPS , 2017.
- Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In Advances in Neural Information Processing Systems , pp. 6626-6637, 2017.
- Geoffrey E Hinton. Products of experts. International Conference on Artificial Neural Networks , 1999.
- Geoffrey E Hinton. Training products of experts by minimizing contrastive divergence. Neural Comput. , 14(8): 1771-1800, 2002.
- Aapo Hyv¨ arinen. Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research , 6(Apr):695-709, 2005.
- Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for improved quality, stability, and variation. In ICLR , 2017.
- Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR , 2015.
- LF Kozachenko and Nikolai N Leonenko. Sample estimate of the entropy of a random vector. Problemy Peredachi Informatsii , 23(2):9-16, 1987.
- Siwei Lyu. Unifying non-maximum likelihood learning objectives with minimum kl contraction. In Advances in Neural Information Processing Systems , pp. 64-72, 2011.
- Radford M Neal. Mcmc using hamiltonian dynamics. Handbook of Markov Chain Monte Carlo , 2(11), 2011.
- Erik Nijkamp, Mitch Hill, Song-Chun Zhu, and Ying Nian Wu. Learning non-convergent non-persistent short-run mcmc toward energy-based model. In Advances in Neural Information Processing Systems , pp. 5232-5242, 2019b.
- Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. In ICLR , 2016.
- Prajit Ramachandran, Barret Zoph, and Quoc V. Le. Searching for activation functions, 2018. URL https:// openreview.net/forum?id=SkBYYyZRZ .
- Ruslan Salakhutdinov and Geoffrey E. Hinton. Deep boltzmann machines. In David A. Van Dyk and Max Welling (eds.), AISTATS , volume 5 of JMLR Proceedings , pp. 448-455. JMLR.org, 2009. URL http://www.jmlr.org/proceedings/ papers/v5/salakhutdinov09a.html .
- Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. In NIPS , 2016.
- Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. In Advances in Neural Information Processing Systems , pp. 1191811930, 2019.
- Arash Vahdat, Evgeny Andriyash, and William Macready. Undirected graphical models as approximate posteriors. In Hal Daum´ e III and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Learning , volume 119 of Proceedings of Machine Learning Research , pp. 9680-9689. PMLR, 13-18 Jul 2020. URL http://proceedings.mlr.press/v119/ vahdat20a.html .
- Aaron van den Oord, Nal Kalchbrenner, Lasse Espeholt, Oriol Vinyals, Alex Graves, et al. Conditional image generation with pixelcnn decoders. In NIPS , 2016.
- Aaron Van Oord, Nal Kalchbrenner, and Koray Kavukcuoglu. Pixel recurrent neural networks. In ICML , 2016.
- Ramakrishna Vedantam, Ian Fischer, Jonathan Huang, and Kevin Murphy. Generative models of visually grounded imagination. In ICLR , 2018.
- Jianwen Xie, Yang Lu, Song-Chun Zhu, and Yingnian Wu. A theory of generative convnet. In International Conference on Machine Learning , pp. 2635-2644, 2016.
- Authors
- Du, Y., Li, S., Tenenbaum, J., & Mordatch, I.
- Affiliations
- Massachusetts Institute of Technology (US)
- University of Washington (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2020
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2012.01316
- OpenAlex
- W3107355992
- Cited by
- 36
- References
- 56
- Citations by year
- 2025: 8 · 2024: 7 · 2023: 12 · 2022: 3 · 2021: 5 · 2020: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- contrastive divergence, energy-based models, training stability, gradient terms, data augmentation, multi-scale processing, model robustness, image generation, out-of-distribution detection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
