A Variational Perspective on Diffusion-Based Generative Models and Score Matching
Open accessThe DOI matches its OpenAlex record; 18 of the 18 references it cites were found in a registry or an index.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 4×
“We train the model on MNIST (LeCun et al., 1998) and CIFAR10 (Krizhevsky et al.”
MethodInfluentialLarge Scale GAN Training for High Fidelity Natural Image Synthesis
Brock, A., Donahue, J., & Simonyan, K. (2018) · arXiv · doi:10.48550/arxiv.1809.11096 · Cited by 3,717
Diffusion Models Beat GANs on Image Synthesis
Dhariwal, P., & Nichol, A. (2021) · arXiv · doi:10.48550/arxiv.2105.05233 · Cited by 3,362
Python for scientific computing
Oliphant, T. E. (2007) · Computing in Science & Engineering · doi:10.1109/mcse.2007.58 · Cited by 3,303
Stochastic Differential Equations
Øksendal, B. (2003) · Universitext · doi:10.1007/978-3-642-14394-6_5 · Cited by 3,081
Image Super-Resolution Via Iterative Refinement
Saharia, C., Ho, J., Chan, W., Salimans, T., Fleet, D. J., & Norouzi, M. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3204461 · Cited by 1,829 · cited 2×
“This line of work has been successfully applied to modeling high dimensional natural images (Dhariwal & Nichol, 2021; Saharia et al., 2021), audio (Kong et al.”
MethodImage Super-Resolution via Iterative Refinement
Saharia, C., Ho, J., Chan, W., Salimans, T., Fleet, D. J., & Norouzi, M. (2021) · arXiv · doi:10.48550/arxiv.2104.07636 · Cited by 1,829
Generative Modeling by Estimating Gradients of the Data Distribution
Song, Y., & Ermon, S. (2019) · arXiv · doi:10.48550/arxiv.1907.05600 · Cited by 1,453 · cited 2×
“Song & Ermon (2019); Song et al. (2021) propose to remedy this by multiplying the DSM loss by σ 2 s /g 2 (assuming g is a scalar for simplicity), so that the target has constant magnitude on average E [ || σ s s θ + (cid:15) || 2 ] , which would result in a biased gradient estimate with much…”
BackgroundA Connection Between Score Matching and Denoising Autoencoders
Vincent, P. (2011) · Neural Computation · doi:10.1162/neco_a_00142 · Cited by 1,011 · cited 2×
“A few alternative losses can be used, which are all equal to one another up to a constant, including implicit score matching (Hyvärinen & Dayan, 2005, ISM), sliced score matching (Song et al., 2020, SSM), and denoising score matching (Vincent, 2011, DSM).”
MethodFFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models
Grathwohl, W., Chen, R. T. Q., Bettencourt, J., Sutskever, I., & Duvenaud, D. (2018) · arXiv · doi:10.48550/arxiv.1810.01367 · Cited by 796
(2005) · doi:10.5555/1046920.1088696 · Cited by 756
A stochastic estimator of the trace of the influence matrix for laplacian smoothing splines
Hutchinson, M. F. (1990) · Communications in Statistics - Simulation and Computation · doi:10.1080/03610919008812866 · Cited by 610 · cited 2×
“Computing the divergence term ∇ · µ directly can be expensive, but it can be efficiently estimated using the Hutchinson trace estimator (Hutchinson, 1989) along with reverse-mode automatic differentiation, similar to Grathwohl et al. (2018).”
MethodDiffWave: A Versatile Diffusion Model for Audio Synthesis
Kong, Z., Ping, W., Huang, J., Zhao, K., & Catanzaro, B. (2020) · arXiv · doi:10.48550/arxiv.2009.09761 · Cited by 569
Reverse-time diffusion equation models
Anderson, B. D. O. (1982) · Stochastic Processes and their Applications · doi:10.1016/0304-4149(82)90051-5 · Cited by 565 · cited 1×
“Time-reversal of diffusion processes Plenty of works have studied the reverse-time diffusion processes (2), including Anderson (1982); Föllmer (1985); Elliott & Anderson (1985); Haussmann & Pardoux (1986).”
BackgroundImproved Techniques for Training Score-Based Generative Models
Song, Y., & Ermon, S. (2020) · arXiv · doi:10.48550/arxiv.2006.09011 · Cited by 482 · cited 4×
“…generative models Our work lays a theoretical foundation for Song et al. (2021), which recognizes that conditional denoising score matching (Song & Ermon, 2019, 2020) and discrete-time diffusion-based generative models (Sohl-Dickstein et al., 2015; Goyal et al., 2017; Ho et al., 2020) can…”
BackgroundMethodInfluentialKingma, D. P., Salimans, T., Poole, B., & Ho, J. (2021) · arXiv · doi:10.48550/arxiv.2107.00630 · Cited by 374
Learning Gradient Fields for Shape Generation
Cai, R., Yang, G., Averbuch-Elor, H., Hao, Z., Belongie, S., Snavely, N., & Hariharan, B. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58580-8_22 · Cited by 225
Cited by
Diffusion Models in Vision: A Survey
Croitoru, F.-A., Hondru, V., Ionescu, R. T., & Shah, M. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3261988 · Cited by 1,859
Kingma, D. P., Salimans, T., Poole, B., & Ho, J. (2021) · arXiv · doi:10.48550/arxiv.2107.00630 · Cited by 374
Score-based Generative Modeling in Latent Space
Vahdat, A., Kreis, K., & Kautz, J. (2021) · arXiv · doi:10.48550/arxiv.2106.05931 · Cited by 224
SegDiff: Image Segmentation with Diffusion Probabilistic Models
Amit, T., Shaharbany, T., Nachmani, E., & Wolf, L. (2021) · arXiv · doi:10.48550/arxiv.2112.00390 · Cited by 179
“A variational framework for the likelihood estimation of diffusion models was introduced by Huang et al. [21].”
MethodDOLCE: A Model-Based Probabilistic Diffusion Framework for Limited-Angle CT Reconstruction
Liu, J., Anirudh, R., Thiagarajan, J. J., He, S., Mohan, K. A., Kamilov, U. S., & Kim, H. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00963 · Cited by 106
“Despite being discovered independently, both classes are often referred to as diffusion models due to their similarity [36, 44].”
Background
Metadata
- Authors
- Huang, C.-W., Lim, J. H., & Courville, A. C.
- Venue
- arXiv (Cornell University)
- Publisher
- Cornell University
- Published
- 2021-06-05
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2106.02808
- OpenAlex
- W3169386841
- Cited by
- 39
- References
- 18
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.