Generative Modeling by Estimating Gradients of the Data Distribution

Song, Yang; Ermon, Stefano

doi:10.48550/arxiv.1907.05600

PreprintCited by 1,453Generative Adversarial Networks and Image SynthesisAdversarial Robustness in Machine LearningModel Reduction and Neural Networks

Generative Modeling by Estimating Gradients of the Data Distribution

Open access

Song, Yang, Ermon, Stefano·2019·arXiv

DOI 10.48550/arxiv.1907.05600arXiv 1907.05600

Open this paper in CoChat — free account Open PDF

Abstract

We introduce a new generative model where samples are produced via Langevin dynamics using gradients of the data distribution estimated with score matching. Because gradients can be ill-defined and hard to estimate when the data resides on low-dimensional manifolds, we perturb the data with different levels of Gaussian noise, and jointly estimate the corresponding scores, i.e., the vector fields of gradients of the perturbed data distribution for all noise levels. For sampling, we propose an annealed Langevin dynamics where we use gradients corresponding to gradually decreasing noise levels as the sampling process gets closer to the data manifold. Our framework allows flexible model architectures, requires no sampling during training or the use of adversarial methods, and provides a learning objective that can be used for principled model comparisons. Our models produce samples comparable to GANs on MNIST, CelebA and CIFAR-10 datasets, achieving a new state-of-the-art inception score of 8.87 on CIFAR-10. Additionally, we demonstrate that our models learn effective representations via image inpainting experiments.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

DataCite · 10.48550/arxiv.1907.05600

Cites

10 of 10 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Diffusion Policy can express arbitrary normalizable distributions, including multimodal action distributions.
“By learning the gradient of the action score function [46] and performing Stochastic Langevin Dynamics sampling on this gradient field, Diffusion policy can express arbitrary normalizable distributions [32], which includes multimodal action distributions”
From Diffusion Policy: Visuomotor Policy Learning via Action Diffusion (2023) · cites this paper as [46]
Section 2.2 (p. 3) says Langevin dynamics can sample from a probability density using its score, but exact sampling requires limiting step size and runtime and holds only under regularity conditions. Sections 4 and 4.3 (pp. 5–7) address multimodal distributions and demonstrate sampling from a two-mode Gaussian mixture. The paper does not establish the unqualified claim that arbitrary normalizable distributions can be expressed. (LLM, conf=0.84)
Experimental — accuracy not yet measured
Diffusion Policy sidesteps estimating the normalization constant Z by modeling the score function, so neither its inference nor training involves evaluating Z, making training more stable.
“Diffusion Policy and DDPMs sidestep the issue of estimating Z ( a , θ ) altogether by modeling the score function Song and Ermon (2019) of the same action distribution in Eq 6:”
From Diffusion policy: Visuomotor policy learning via action diffusion (2025) · cites this paper as Song and Ermon (2019)
Section 2.1 (p. 2) says a score network can estimate the gradient of the log data distribution without first estimating the distribution itself, supporting the score-modeling rationale for avoiding a normalization constant. Section 4.2 (p. 6) describes score-based training and sampling, but the paper does not assess Diffusion Policy or establish that avoiding Z makes its training more stable. (LLM, conf=0.83)
Experimental — accuracy not yet measured
Diffusion Policy can express multimodal action distributions.
“Expressing multimodal action distributions. By learning the gradient of the action score function Song and Ermon (2019) and performing Stochastic Langevin Dynamics sampling on this gradient field, Diffusion policy can express arbitrary normalizable distributions Neal et al. (2011), which includes multimodal action distributions, a well-known challenge for policy learning.”
From Diffusion policy: Visuomotor policy learning via action diffusion (2025) · cites this paper as Song and Ermon (2019)
Diffusion Policy can express arbitrary normalizable distributions, which includes multimodal action distributions.
“By learning the gradient of the action score function Song and Ermon (2019) and performing Stochastic Langevin Dynamics sampling on this gradient field, Diffusion policy can express arbitrary normalizable distributions Neal et al. (2011), which includes multimodal action distributions, a well-known challenge for policy learning.”
From Diffusion Policy: Visuomotor Policy Learning via Action Diffusion (2023) · cites this paper as Song and Ermon (2019)
A certain parameterization of diffusion models reveals an equivalence with denoising score matching over multiple noise levels during training and with annealed Langevin dynamics during sampling.
“In addition, we show that a certain parameterization of diffusion models reveals an equivalence with denoising score matching over multiple noise levels during training and with annealed Langevin dynamics during sampling (Section 3.2) [55, 61].”
From Denoising Diffusion Probabilistic Models (2020) · cites this paper as [55]

NCSN's sampler coefficients are set by hand post-hoc, and their training procedure is not guaranteed to directly optimize a quality metric of their sampler.
“In contrast, NCSN's sampler coefficients are set by hand post-hoc, and their training procedure is not guaranteed to directly optimize a quality metric of their sampler.”
From Denoising Diffusion Probabilistic Models (2020) · cites this paper as [55]
The model architecture, forward process definition, and prior differ from NCSN in subtle but important ways that improve sample quality.
“Our model architecture, forward process definition, and prior differ from NCSN [55, 56] in subtle but important ways that improve sample quality, and, notably, we directly train our sampler as a latent variable model rather than adding it after training post-hoc.”
From Denoising Diffusion Probabilistic Models (2020) · cites this paper as [55]
The ε-prediction reverse process parameterization establishes a connection between diffusion models and denoising score matching over multiple noise levels with annealed Langevin dynamics for sampling.
“Our ϵ -prediction reverse process parameterization establishes a connection between diffusion models and denoising score matching over multiple noise levels with annealed Langevin dynamics for sampling [55, 56].”
From Denoising Diffusion Probabilistic Models (2020) · cites this paper as [55]
NCSN omits the scaling factor.
“Diffusion models scale down the data with each forward process step (by a √ 1 -β t factor) so that variance does not grow when adding noise, thus providing consistently scaled inputs to the neural net reverse process. NCSN omits this scaling factor.”
From Denoising Diffusion Probabilistic Models (2020) · cites this paper as [55]
Diffusion models have log likelihoods better than the large estimates annealed importance sampling has been reported to produce for energy based models and score matching.
“(our models do, however, have log likelihoods better than the large estimates annealed importance sampling has been reported to produce for energy based models and score matching [11, 55]).”
From Denoising Diffusion Probabilistic Models (2020) · cites this paper as [55]
Section 4.3 (p. 6) mentions annealed importance sampling only as an inspiration for the paper’s sampling method. Section 6 (p. 9) discusses energy-based models and score matching, but gives no AIS log-likelihood estimates or comparison with diffusion models. Searches for those estimates and comparisons found no supporting passage. (LLM, conf=0.75)
Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404 · see citing works
A Style-Based Generator Architecture for Generative Adversarial Networks
Karras, T., Laine, S., & Aila, T. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00453 · Cited by 11,714 · see citing works
GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., & Hochreiter, S. (2017) · arXiv · doi:10.48550/arxiv.1706.08500 · Cited by 8,365 · see citing works
WaveNet: a generative model for raw audio
Oord, A. V. D., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1609.03499 · Cited by 4,911 · see citing works
Training Products of Experts by Minimizing Contrastive Divergence
Hinton, G. E. (2002) · Neural Computation · doi:10.1162/089976602760128018 · Cited by 4,879 · cited 2× · see citing works
“, the evidence lower bound used in variational auto-encoders [29], contrastive divergence in energy-based models [21]) for training.”
BackgroundMethod
Large Scale GAN Training for High Fidelity Natural Image Synthesis
Brock, A., Donahue, J., & Simonyan, K. (2018) · arXiv · doi:10.48550/arxiv.1809.11096 · Cited by 3,717 · see citing works
RefineNet: Multi-path Refinement Networks for High-Resolution Semantic Segmentation
Lin, G., Milan, A., Shen, C., & Reid, I. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.549 · Cited by 3,287 · see citing works
Generative adversarial imitation learning
Ho, J., & Ermon, S. (2016) · arXiv · doi:10.48550/arxiv.1606.03476 · Cited by 1,942 · see citing works
Self-citation
f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization
Nowozin, S., Cseke, B., & Tomioka, R. (2016) · arXiv · doi:10.48550/arxiv.1606.00709 · Cited by 1,183 · see citing works
A Connection Between Score Matching and Denoising Autoencoders
Vincent, P. (2011) · Neural Computation · doi:10.1162/neco_a_00142 · Cited by 1,011 · see citing works

Flagged first, then by citations

Research works printed but not linked: 54

[2][2] M. Arjovsky, S. Chintala, and L. Bottou. Wasserstein generative adversarial networks. In D. Precup and Y. W. Teh, editors, Proceedings of the 34th International Conference on Machine Learning , volume 70 of Proceedings of Machine Learning Research , pages 214-223, International Convention Centre, Sydney, Australia, 06-11 Aug 2017. PMLR. -
[3][3] M. Belkin and P. Niyogi. Laplacian eigenmaps for dimensionality reduction and data representation. Neural computation , 15(6):1373-1396, 2003. -
[4][4] Y. Bengio, L. Yao, G. Alain, and P. Vincent. Generalized denoising auto-encoders as generative models. In Advances in neural information processing systems , pages 899-907, 2013. -
[5][5] F. Bordes, S. Honari, and P. Vincent. Learning to generate samples from noise through infusion training. arXiv preprint arXiv:1703.06975 , 2017. - · arXiv
[7][7] B. Chandra and R. K. Sharma. Adaptive noise schedule for denoising autoencoder. In International conference on neural information processing , pages 535-542. Springer, 2014. -
[8][8] L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille. Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE transactions on pattern analysis and machine intelligence , 40(4):834-848, 2017. -
[9][9] T. Chen, E. Fox, and C. Guestrin. Stochastic gradient hamiltonian monte carlo. In International conference on machine learning , pages 1683-1691, 2014. -
[10][10] Z. Dai, Z. Yang, F. Yang, W. W. Cohen, and R. R. Salakhutdinov. Good semi-supervised learning that requires a bad gan. In Advances in neural information processing systems , pages 6510-6520, 2017. -
[11][11] L. Dinh, D. Krueger, and Y. Bengio. Nice: Non-linear independent components estimation. arXiv preprint arXiv:1410.8516 , 2014. - · arXiv
[12][12] Y. Du and I. Mordatch. Implicit generation and generalization in energy-based models. arXiv preprint arXiv:1903.08689 , 2019. - · arXiv
[13][13] V. Dumoulin, J. Shlens, and M. Kudlur. A learned representation for artistic style. In International Conference on Learning Representations 2017 , 2017. -
[14][14] K. J. Geras and C. Sutton. Scheduled denoising autoencoders. arXiv preprint arXiv:1406.3269 , 2014. - · arXiv
[15][15] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In Advances in neural information processing systems , pages 2672-2680, 2014. -
[16][16] A. G. A. P. Goyal, N. R. Ke, S. Ganguli, and Y. Bengio. Variational walkback: Learning a transition operator as a stochastic recurrent net. In Advances in Neural Information Processing Systems , pages 4392-4402, 2017. -
[17][17] A. Graves. Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850 , 2013. - · arXiv
[18][18] I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, and A. C. Courville. Improved training of wasserstein gans. In Advances in Neural Information Processing Systems , pages 5767-5777, 2017. -
[19][19] M. Gutmann and A. Hyvärinen. Noise-contrastive estimation: A new estimation principle for unnormalized statistical models. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics , pages 297-304, 2010. -
[23][23] X. Huang and S. Belongie. Arbitrary style transfer in real-time with adaptive instance normalization. In Proceedings of the IEEE International Conference on Computer Vision , pages 1501-1510, 2017. -
[24][24] A. Hyvärinen. Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research , 6(Apr):695-709, 2005. -
[25][25] T. Karras, T. Aila, S. Laine, and J. Lehtinen. Progressive growing of GANs for improved quality, stability, and variation. In International Conference on Learning Representations , 2018. -
[27][27] D. Kingma and Y. LeCun. Regularized estimation of image statistics by score matching. In Advances in Neural Information Processing Systems 23: 24th Annual Conference on Neural Information Processing Systems 2010, NIPS 2010 , 2010. -
[28][28] D. P. Kingma, S. Mohamed, D. J. Rezende, and M. Welling. Semi-supervised learning with deep generative models. In Advances in neural information processing systems , pages 3581-3589, 2014. -
[29][29] D. P. Kingma and M. Welling. Auto-encoding variational bayes. In ICLR , 2014. -
[30][30] S. Kirkpatrick, C. D. Gelatt, and M. P. Vecchi. Optimization by simulated annealing. SCIENCE , 220(4598):671-680, 1983. -
[33][33] Q. Liu, J. Lee, and M. Jordan. A kernelized stein discrepancy for goodness-of-fit tests. In International Conference on Machine Learning , pages 276-284, 2016. -
[34][34] Z. Liu, P. Luo, X. Wang, and X. Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV) , 2015. -
[35][35] K. Miyasawa. An empirical bayes estimator of the mean of a normal population. Bull. Inst. Internat. Statist , 38(181-188):1-2, 1961. -
[36][36] T. Miyato, T. Kataoka, M. Koyama, and Y. Yoshida. Spectral normalization for generative adversarial networks. In International Conference on Learning Representations , 2018. -
[37][37] R. M. Neal. Annealed importance sampling. Statistics and computing , 11(2):125-139, 2001. -
[38][38] R. M. Neal. Mcmc using hamiltonian dynamics. arXiv preprint arXiv:1206.1901 , 2012. - · arXiv
[39][39] E. Nijkamp, M. Hill, T. Han, S.-C. Zhu, and Y. N. Wu. On the anatomy of mcmc-based maximum likelihood learning of energy-based models. arXiv preprint arXiv:1903.12370 , 2019. - · arXiv
[41][41] G. Ostrovski, M. G. Bellemare, A. van den Oord, and R. Munos. Count-based exploration with neural density models. In Proceedings of the 34th International Conference on Machine Learning-Volume 70 , pages 2721-2730. JMLR. org, 2017. -
[42][42] G. Ostrovski, W. Dabney, and R. Munos. Autoregressive quantile networks for generative modeling. In ICML , volume 80 of Proceedings of Machine Learning Research , pages 39333942. PMLR, 2018. -
[43][43] M. Raphan and E. P. Simoncelli. Learning to be bayesian without supervision. In Advances in neural information processing systems , pages 1145-1152, 2007. -
[44][44] M. Raphan and E. P. Simoncelli. Least squares estimation without priors or supervision. Neural computation , 23(2):374-420, 2011. -
[45][45] S. Ravuri, S. Mohamed, M. Rosca, and O. Vinyals. Learning implicit generative models with the method of learned moments. In J. Dy and A. Krause, editors, Proceedings of the 35th International Conference on Machine Learning , volume 80 of Proceedings of Machine Learning Research , pages 4314-4323, Stockholmsmässan, Stockholm Sweden, 10-15 Jul 2018. PMLR. -
[47][47] S. T. Roweis and L. K. Saul. Nonlinear dimensionality reduction by locally linear embedding. science , 290(5500):2323-2326, 2000. -
[48][48] T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen. Improved techniques for training gans. In Advances in neural information processing systems , pages 2234-2242, 2016. -
[49][49] S. Saremi, A. Mehrjou, B. Schölkopf, and A. Hyvärinen. Deep energy estimator networks. arXiv preprint arXiv:1805.08306 , 2018. - · arXiv
[50][50] J. Sohl-Dickstein, P. Battaglino, and M. R. DeWeese. Minimum probability flow learning. arXiv preprint arXiv:0906.4779 , 2009. - · arXiv
[51][51] J. Sohl-Dickstein, E. Weiss, N. Maheswaranathan, and S. Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning , pages 2256-2265, 2015. -
[52][52] J. Song, S. Zhao, and S. Ermon. A-nice-mc: Adversarial training for mcmc. In Advances in Neural Information Processing Systems , pages 5140-5150, 2017. -
[53][53] Y. Song, S. Garg, J. Shi, and S. Ermon. Sliced score matching: A scalable approach to density and score estimation. In Proceedings of the Thirty-Fifth Conference on Uncertainty in Artificial Intelligence, UAI 2019, Tel Aviv, Israel, July 22-25, 2019 , page 204, 2019. -
[54][54] Y. Song, T. Kim, S. Nowozin, S. Ermon, and N. Kushman. Pixeldefend: Leveraging generative models to understand and defend against adversarial examples. In International Conference on Learning Representations , 2018. -
[55][55] B. K. Sriperumbudur, K. Fukumizu, A. Gretton, B. Schölkopf, and G. R. Lanckriet. On integral probability metrics, \ phi-divergences and binary classification. arXiv preprint arXiv:0901.2698 , 2009. - · arXiv
[56][56] C. M. Stein. Estimation of the mean of a multivariate normal distribution. The annals of Statistics , pages 1135-1151, 1981. -
[57][57] D. Ulyanov, A. Vedaldi, and V. Lempitsky. Instance normalization: The missing ingredient for fast stylization. arXiv preprint arXiv:1607.08022 , 2016. - · arXiv
[59][59] A. Van den Oord, N. Kalchbrenner, L. Espeholt, O. Vinyals, A. Graves, et al. Conditional image generation with pixelcnn decoders. In Advances in neural information processing systems , pages 4790-4798, 2016. -
[60][60] A. Van Den Oord, N. Kalchbrenner, and K. Kavukcuoglu. Pixel recurrent neural networks. In Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48 , ICML'16, pages 1747-1756. JMLR.org, 2016. -
[62][62] M. Welling and Y. W. Teh. Bayesian learning via stochastic gradient langevin dynamics. In Proceedings of the 28th international conference on machine learning (ICML-11) , pages 681-688, 2011. -
[63][63] L. Wenliang, D. Sutherland, H. Strathmann, and A. Gretton. Learning deep kernels for exponential family densities. In International Conference on Machine Learning , pages 6737-6746, 2019. -
[64][64] F. Yu and V. Koltun. Multi-scale context aggregation by dilated convolutions. In International Conference on Learning Representations (ICLR) , 2016. -
[65][65] F. Yu, V. Koltun, and T. Funkhouser. Dilated residual networks. In Computer Vision and Pattern Recognition (CVPR) , 2017. -
[66][66] Q. Zhang and L. Zhang. Convolutional adaptive denoising autoencoders for hierarchical feature extraction. Frontiers of Computer Science , 12(6):1140-1148, 2018.

Sorted by citations

Denoising Diffusion Probabilistic Models
Ho, J., Jain, A., & Abbeel, P. (2020) · arXiv · doi:10.48550/arxiv.2006.11239 · Cited by 8,712 · see citing works
Diffusion Models Beat GANs on Image Synthesis
Dhariwal, P., & Nichol, A. (2021) · arXiv · doi:10.48550/arxiv.2105.05233 · Cited by 3,362 · see citing works
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676 · see citing works
Self-citation
Scalable Diffusion Models with Transformers
Peebles, W., & Xie, S. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00387 · Cited by 2,284 · see citing works
Score-Based Generative Modeling through Stochastic Differential Equations
Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., & Poole, B. (2020) · arXiv · doi:10.48550/arxiv.2011.13456 · Cited by 1,953 · see citing works
Self-citation
Image Super-Resolution Via Iterative Refinement
Saharia, C., Ho, J., Chan, W., Salimans, T., Fleet, D. J., & Norouzi, M. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3204461 · Cited by 1,829 · see citing works
Align Your Latents: High-Resolution Video Synthesis with Latent Diffusion Models
Blattmann, A., Rombach, R., Ling, H., Dockhorn, T., Kim, S. W., Fidler, S., & Kreis, K. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.02161 · Cited by 724 · see citing works
Diffusion policy: Visuomotor policy learning via action diffusion
Chi, C., Xu, Z., Feng, S., Cousineau, E., Du, Y., Burchfiel, B., Tedrake, R., & Song, S. (2025) · The International Journal of Robotics Research · doi:10.1177/02783649241273668 · Cited by 613 · see citing works
Diffusion Policy: Visuomotor Policy Learning via Action Diffusion
Chi, C., Feng, S., Du, Y., Xu, Z., Cousineau, E., Burchfiel, B., & Song, S. (2023) · Robotics: Science and Systems XIX · doi:10.15607/rss.2023.xix.026 · Cited by 530 · see citing works
Deep multimodal data fusion
Zhao, F., Zhang, C., & Geng, B. (2024) · ACM Computing Surveys · doi:10.1145/3649447 · Cited by 480 · see citing works
MaskGIT: Masked Generative Image Transformer
Chang, H., Zhang, H., Jiang, L., Liu, C., & Freeman, W. T. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01103 · Cited by 446 · see citing works
DDFM: Denoising Diffusion Model for Multi-Modality Image Fusion
Zhao, Z., Bai, H., Zhu, Y., Zhang, J., Xu, S., Zhang, Y., Zhang, K., Meng, D., Timofte, R., & Van Gool, L. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00742 · Cited by 431 · see citing works
Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospects
Zhang, K., Wen, Q., Zhang, C., Cai, R., Jin, M., Liu, Y., Zhang, J. Y., Liang, Y., Pang, G., Song, D., et al. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3387317 · Cited by 295 · see citing works
Learning Gradient Fields for Shape Generation
Cai, R., Yang, G., Averbuch-Elor, H., Hao, Z., Belongie, S., Snavely, N., & Hariharan, B. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58580-8_22 · Cited by 225 · see citing works
Score-Based Point Cloud Denoising
Luo, S., & Hu, W. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00454 · Cited by 209 · see citing works
VideoGPT: Video Generation using VQ-VAE and Transformers
Yan, W., Zhang, Y., Abbeel, P., & Srinivas, A. (2021) · arXiv · doi:10.48550/arxiv.2104.10157 · Cited by 156 · see citing works
Antigen-Specific Antibody Design and Optimization with Diffusion-Based Generative Models for Protein Structures
Luo, S., Su, Y., Peng, X., Wang, S., Peng, J., & Ma, J. (2022) · doi:10.1101/2022.07.10.499510 · Cited by 151 · see citing works
Diffusion Models, Image Super-Resolution, and Everything: A Survey
Moser, B. B., Shanbhag, A. S., Raue, F., Frolov, S., Palacio, S., & Dengel, A. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3476671 · Cited by 117 · see citing works
Machine Learning Methods for Weather Forecasting: A Survey
Zhang, H., Liu, Y., Zhang, C., & Li, N. (2025) · Atmosphere · doi:10.3390/atmos16010082 · Cited by 113 · see citing works
Denoising Pretraining for Semantic Segmentation
Brempong, E. A., Kornblith, S., Chen, T., Parmar, N., Minderer, M., & Norouzi, M. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00462 · Cited by 120 · see citing works
Wavelet-Improved Score-Based Generative Model for Medical Imaging
Wu, W., Wang, Y., Liu, Q., Wang, G., & Zhang, J. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2023.3325824 · Cited by 105 · see citing works
ProteinSGM: Score-based generative modeling for <i>de novo</i> protein design
Lee, J. S., Kim, J., & Kim, P. M. (2022) · doi:10.1101/2022.07.13.499967 · Cited by 94 · see citing works
Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learning
Wang, Z., Hunt, J. J., & Zhou, M. (2022) · arXiv · doi:10.48550/arxiv.2208.06193 · Cited by 83 · see citing works
“Diffusion-based generative models (Ho et al., 2020; Sohl-Dickstein et al., 2015; Song & Ermon, 2019) assume p θ ( x 0 ) := p θ ( x 0: T ) d x 1: T , where x 1 , . . . , x T are latent variables of the same dimensionality as the data x 0 ∼ p ( x 0 ) .”
BackgroundMethod
Token Merging: Your ViT But Faster
Bolya, D., Fu, C.-Y., Dai, X., Zhang, P., Feichtenhofer, C., & Hoffman, J. (2022) · arXiv · doi:10.48550/arxiv.2210.09461 · Cited by 140 · see citing works
“And luckily for us, exactly such a task has recently become very popular: text-conditioned diffusion (Sohl-Dickstein et al., 2015; Song & Ermon, 2019).”
Background
LEGO-Net: Learning Regular Rearrangements of Objects in Rooms
Wei, Q. A., Ding, S., Park, J. J., Sajnani, R., Poulenard, A., Sridhar, S., & Guibas, L. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01825 · Cited by 60 · see citing works
Image captioning by diffusion models: a survey
Daneshfar, F., Bartani, A., & Lotfi, P. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.109288 · Cited by 49 · see citing works
Generating High Fidelity Data from Low-density Regions using Diffusion Models
Sehwag, V., Hazirbas, C., Gordo, A., Ozgenel, F., & Ferrer, C. C. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01120 · Cited by 41 · see citing works
Stochastic Image Denoising by Sampling from the Posterior Distribution
Kawar, B., Vaksman, G., & Elad, M. (2021) · 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) · doi:10.1109/iccvw54120.2021.00213 · Cited by 42 · see citing works
MRI Super-Resolution With Partial Diffusion Models
Zhao, K., Pang, K., Hung, A. L. Y., Zheng, H., Yan, R., & Sung, K. (2025) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3483109 · Cited by 36 · see citing works
Early Detection of Alzheimer’s Disease Using Generative Models: A Review of GANs and Diffusion Models in Medical Imaging
Alam, M. M., & Latifi, S. (2025) · doi:10.20944/preprints202506.1854.v1 · Cited by 17 · see citing works
Diffusion Policy: Visuomotor Policy Learning via Action Diffusion
Chi, C., Xu, Z., Feng, S., Cousineau, E., Du, Y., Burchfiel, B., Tedrake, R., & Song, S. (2023) · arXiv · doi:10.48550/arxiv.2303.04137 · Cited by 12 · see citing works
“By learning the gradient of the action score function (Song and Ermon, 2019) and performing Stochastic Langevin Dynamics sampling on this gradient ﬁ eld, Diffusion policy can express arbitrary normalizable distributions (Neal, 2011), which includes multimodal action distributions, a well-known…”
PixArt-$α$: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis
Chen, J., Yu, J., Ge, C., Yao, L., Xie, E., Wu, Y., Wang, Z., Kwok, J., Luo, P., Lu, H., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.00426 · Cited by 30 · see citing works
“…A.1.1 D ENOISING DIFFUSION PROBABILISTIC MODELS Diffusion models (Ho et al., 2020; Sohl-Dickstein et al., 2015) and score-based generative models (Song & Ermon, 2019; Song et al., 2021) have emerged as highly successful approaches for image generation, surpassing previous generative models such…”
MethodInfluential
Efficient neural decoding based on multimodal training
Wang, Y. (2024) · Brain Sciences · doi:10.3390/brainsci14100988 · Cited by 3 · see citing works
A survey on neuro-mimetic deep learning via predictive coding
Salvatori, T., Mali, A., Buckley, C. L., Lukasiewicz, T., Rao, R. P. N., Friston, K., & Ororbia, A. (2026) · Neural Networks · doi:10.1016/j.neunet.2025.108161 · Cited by 2 · see citing works
Video Diffusion Models: A Survey
Melnik, A., Ljubljanac, M., Lu, C., Yan, Q., Ren, W., & Ritter, H. (2024) · arXiv · doi:10.48550/arxiv.2405.03150 · Cited by 4 · see citing works

Authors: Song, Y., & Ermon, S.
Affiliations: Stanford University (US)
ORCID: Song, Yang
Ermon, Stefano
Venue: arXiv
Publisher: arXiv
Published: 2019
Volume: 32
Pages: 11895-11907
ISSN: 2331-8422 (ISSN-L)
DOI: 10.48550/arxiv.1907.05600
arXiv: 1907.05600
OpenAlex: W2959300817
Cited by: 1,453
References: 10
Citations by year: 2026: 9 · 2025: 346 · 2024: 539 · 2023: 313 · 2022: 73 · 2021: 101 · 2020: 64 · 2019: 6 · 2018: 2
Type: posted content
Language: en
Open access: green
arxiv.org
Licence: arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations: arXiv (Cornell University) (submittedVersion)
arXiv (Cornell University)
Funders: National Science Foundation (1651565, 1522054, 1733686)
Toyota Research Institute
Office of Naval Research (N00014-19, FA9550, N00014-19-1-2145, N00014)
Air Force Office of Scientific Research (FA9550-, FA9550-19-1-0024)
Topics: Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Model Reduction and Neural Networks
Keywords: generative models, score matching, Langevin dynamics, gradient estimation, data manifold, noise perturbation, sampling, image generation, image inpainting, model comparison

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Generative Modeling by Estimating Gradients of the Data Distribution

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Self-check

Cites

Cited by

Record

About these checks