A Connection Between Score Matching and Denoising Autoencoders
Abstract
Denoising autoencoders have been previously shown to be competitive alternatives to restricted Boltzmann machines for unsupervised pretraining of each layer of a deep architecture. We show that a simple denoising autoencoder training criterion is equivalent to matching the score (with respect to the data) of a specific energy-based model to that of a nonparametric Parzen density estimator of the data. This yields several useful insights. It defines a proper probabilistic model for the denoising autoencoder technique, which makes it in principle possible to sample from them or rank examples by their energy. It suggests a different way to apply score matching that is related to learning to denoise and does not require computing second derivatives. It justifies the use of tied weights between the encoder and decoder and suggests ways to extend the success of denoising autoencoders to a larger family of energy-based models.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
An energy-based model trained with denoising score matching can effectively purify attacked images within just a few steps.
“Adaptive Denoising Purification (ADP) [273] demonstrates that an EBM trained with denoising score matching [240] can effectively purify attacked images within just a few steps.”
From Diffusion Models: A Comprehensive Survey of Methods and Applications (2024) · cites this paper as [240]
Cites
Advances in neural information processing systems 7
Cios, K. J., & Shields, M. E. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,089 · see citing works
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336 · see citing works
Representation Learning: A Review and New Perspectives
Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001 · see citing works
Self-citationExtracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436 · see citing works
Self-citationTraining Products of Experts by Minimizing Contrastive Divergence
Hinton, G. E. (2002) · Neural Computation · doi:10.1162/089976602760128018 · Cited by 4,879 · cited 3× · see citing works
“1Specifically SM (Hyvärinen, 2005), minimum velocity learning (Movellan, 2008), and certain forms of contrastive divergence (Hinton, 2002; Welling & Hinton, 2002) are all recast as minimizing the Kullback-Leibler divergence between the data distribution and the distribution obtained after running, for infinitesimal ti…”
BackgroundMethodGreedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841 · see citing works
(2010) · Journal of Machine Learning Research · doi:10.5555/1756006.1953039 · Cited by 4,051 · see citing works
Self-citationDuane, S., Kennedy, A. D., Pendleton, B. J., & Roweth, D. (1987) · Physics Letters B · doi:10.1016/0370-2693(87)91197-x · Cited by 3,920 · see citing works
Why Does Unsupervised Pre-training Help Deep Learning?
Erhan, D., Bengio, Y., Courville, A. C., Manzagol, P.-A., Vincent, P., & Samy Bengio, A. (2010) · Cited by 2,180 · see citing works
Self-citationAdvances in Neural Information Processing Systems 18
Murray, I., Mackay, D., Zoubin Ghahramani, A., & Skilling, J. (2006) · Cited by 1,927 · see citing works
Advances in Neural Information Processing Systems 21
Griffiths, T. L., Lucas, C. G., Williams, J. J., & Kalish, M. L. (2009) · neural information processing systems · Cited by 1,854 · see citing works
Proceedings of the 26th Conference on Uncertainty in Artificial Intelligence (UAI 2010)
Adams, R. P., Dahl, G. E., & Murray, I. (2010) · Cited by 943 · see citing works
Estimation of Non-Normalized Statistical Models by Score Matching
Hyvärinen, A. (2005) · doi:10.5555/1046920.1088696 · Cited by 756 · see citing works
Exponential Family Harmoniums with an Application to Information Retrieval
Welling, M., Rosen‐Zvi, M., & Hinton, G. E. (2004) · Cited by 423 · see citing works
Some extensions of score matching
Hyvärinen, A. (2006) · Computational Statistics & Data Analysis · doi:10.1016/j.csda.2006.09.003 · Cited by 174 · see citing works
Inductive Principles for Restricted Boltzmann Machine Learning
Marlin, B. M., Swersky, K., Chen, B., & de Freitas, N. (2010) · Cited by 133 · see citing works
Sutton, C. A., & Jordan, M. I. (2010) · Cited by 97 · see citing works
A New Learning Algorithm for Mean Field Boltzmann Machines
Welling, M., & Hinton, G. E. (2002) · Lecture notes in computer science · doi:10.1007/3-540-46084-5_57 · Cited by 76 · see citing works
Learning Continuous Attractors in Recurrent Networks
Seung, H. S. (1997) · Cited by 76 · see citing works
Interpretation and Generalization of Score MatchingPreprint
Lyu, S. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1205.2629 · Cited by 68 · see citing works
Hyvärinen, A. (2007) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2007.895819 · Cited by 52 · see citing works
Regularized estimation of image statistics by Score Matching
Kingma, D. P., & LeCun, Y. (2010) · neural information processing systems · Cited by 52 · see citing works
Minimum Probability Flow LearningPreprint
Sohl‐Dickstein, J., Battaglino, P., & DeWeese, M. R. (2009) · arXiv (Cornell University) · doi:10.48550/arxiv.0906.4779 · Cited by 45 · see citing works
Optimal Approximation of Signal Priors
Hyvärinen, A. (2008) · Neural Computation · doi:10.1162/neco.2008.10-06-384 · Cited by 22 · see citing works
Memoires associatives distribuees: Une comparaison (Distributed associative memories: A comparison)
Gallinari, P., LeCun, Y., Thiria, S., & Soulié, F. F. (1987) · Cited by 15 · see citing works
Inductive principles for learning Restricted Boltzmann Machines
Swersky, K. (2010) · cIRcle (University of British Columbia) · doi:10.14288/1.0051929 · Cited by 9 · see citing works
Contrastive Divergence in Gaussian Diffusions
Movellan, J. R. (2008) · Neural Computation · doi:10.1162/neco.2008.01-07-430 · Cited by 5 · see citing works
Flagged first, then by citations
2 of the 29 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Representation Learning: A Review and New Perspectives
Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001 · see citing works
Self-citationDenoising Diffusion Probabilistic Models
Ho, J., Jain, A., & Abbeel, P. (2020) · arXiv · doi:10.48550/arxiv.2006.11239 · Cited by 8,712 · see citing works
A survey of deep neural network architectures and their applications
Liu, W., Wang, Z., Liu, X., Zeng, N., Liu, Y., & Alsaadi, F. E. (2017) · Neurocomputing · doi:10.1016/j.neucom.2016.12.038 · Cited by 3,239 · see citing works
Denoising Diffusion Implicit Models
Song, J., Meng, C., & Ermon, S. (2020) · arXiv · doi:10.48550/arxiv.2010.02502 · Cited by 2,507 · see citing works
Practical Recommendations for Gradient-Based Training of Deep Architectures
Bengio, Y. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-35289-8_26 · Cited by 2,259 · see citing works
Diffusion Models in Vision: A Survey
Croitoru, F.-A., Hondru, V., Ionescu, R. T., & Shah, M. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3261988 · Cited by 1,965 · see citing works
Score-Based Generative Modeling through Stochastic Differential Equations
Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., & Poole, B. (2020) · arXiv · doi:10.48550/arxiv.2011.13456 · Cited by 1,953 · see citing works
Image Super-Resolution Via Iterative Refinement
Saharia, C., Ho, J., Chan, W., Salimans, T., Fleet, D. J., & Norouzi, M. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3204461 · Cited by 1,829 · see citing works
Deep Learning: Methods and Applications
Deng, L. (2014) · now publishers, Inc. eBooks · doi:10.1561/9781601988157 · Cited by 1,785 · see citing works
Diffusion Models: A Comprehensive Survey of Methods and Applications
Yang, L., Zhang, Z., Song, Y., Hong, S., Xu, R., Zhao, Y., Zhang, W., Cui, B., & Yang, M.-H. (2024) · ACM Computing Surveys · doi:10.1145/3626235 · Cited by 1,704 · see citing works
Domain Adaptation for Large-Scale Sentiment Classification: A Deep Learning Approach
Glorot, X., Bordes, A., & Bengio, Y. (2011) · Cited by 1,569 · see citing works
Generative Modeling by Estimating Gradients of the Data Distribution
Song, Y., & Ermon, S. (2019) · arXiv · doi:10.48550/arxiv.1907.05600 · Cited by 1,453 · see citing works
Deep learning modelling techniques: Current progress, applications, advantages, and challenges
Ahmed, S. F., Alam, M. S. B., Hassan, M., Rozbu, M. R., Ishtiak, T., Rafa, N., Mofijur, M., Shawkat Ali, A. B. M., & Gandomi, A. H. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10466-8 · Cited by 1,079 · see citing works
Neural Networks and Deep Learning
Theodoridis, S. (2015) · Elsevier eBooks · doi:10.1016/b978-0-12-801522-3.00018-5 · Cited by 942 · see citing works
Diffusion models in medical imaging: A comprehensive survey
Kazerouni, A., Aghdam, E. K., Heidari, M., Azad, R., Fayyaz, M., Hacihaliloglu, I., & Merhof, D. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102846 · Cited by 850 · see citing works
Imagic: Text-Based Real Image Editing with Diffusion Models
Kawar, B., Zada, S., Lang, O., Tov, O., Chang, H., Dekel, T., Mosseri, I., & Irani, M. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00582 · Cited by 813 · see citing works
Bond-Taylor, S., Leach, A., Long, Y., & Willcocks, C. G. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3116668 · Cited by 735 · see citing works
Align Your Latents: High-Resolution Video Synthesis with Latent Diffusion Models
Blattmann, A., Rombach, R., Ling, H., Dockhorn, T., Kim, S. W., Fidler, S., & Kreis, K. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.02161 · Cited by 724 · see citing works
A Survey on Generative Diffusion Models
Cao, H., Tan, C., Gao, Z., Xu, Y., Chen, G., Heng, P., & Li, S. Z. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3361474 · Cited by 679 · see citing works
ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models
Choi, J., Kim, S., Jeong, Y., Gwon, Y., & Yoon, S. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01410 · Cited by 665 · see citing works
Deep Learning of Representations: Looking Forward
Bengio, Y. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-39593-2_1 · Cited by 558 · see citing works
Restoring Vision in Adverse Weather Conditions With Patch-Based Denoising Diffusion Models
Özdenizci, O., & Legenstein, R. A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3238179 · Cited by 483 · see citing works
Improved Techniques for Training Score-Based Generative Models
Song, Y., & Ermon, S. (2020) · arXiv · doi:10.48550/arxiv.2006.09011 · Cited by 482 · see citing works
A deep learning approach to structured signal recovery
Mousavi, A., Patel, A. B., & Baraniuk, R. G. (2015) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton.2015.7447163 · Cited by 479 · see citing works
Deep Anomaly Detection Using Geometric TransformationsPreprint
Golan, I., & El‐Yaniv, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.10917 · Cited by 479 · see citing works
A review of deep learning with special emphasis on architectures, applications and recent trends
Sengupta, S., Basak, S., Saikia, P., Paul, S., Tsalavoutis, V. A., Atiah, F. D., Ravi, V., & Peters, A. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.105596 · Cited by 468 · see citing works
Unsupervised Feature Learning and Deep Learning: A Review and New Perspectives
Bengio, Y., Courville, A. C., & Vincent, P. (2012) · Cited by 455 · see citing works
Self-citationScore-based diffusion models for accelerated MRI
Chung, H., & Ye, J. C. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102479 · Cited by 436 · see citing works
WaveGrad: Estimating Gradients for Waveform GenerationPreprint
Chen, N., Zhang, Y., Zen, H., Weiss, R. J., Norouzi, M., & Chan, W. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.00713 · Cited by 410 · see citing works
Kingma, D. P., Salimans, T., Poole, B., & Ho, J. (2021) · arXiv · doi:10.48550/arxiv.2107.00630 · Cited by 374 · see citing works
Amortised MAP Inference for Image Super-resolutionPreprint
Sønderby, C. K., Caballero, J., Theis, L., Shi, W., & Huszár, F. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.04490 · Cited by 364 · see citing works
Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generation
Wang, H., Du, X., Li, J., Yeh, R. A., & Shakhnarovich, G. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01214 · Cited by 355 · see citing works
Auto-Encoders in Deep Learning—A Review with New Perspectives
Chen, S., & Guo, W. (2023) · Mathematics · doi:10.3390/math11081777 · Cited by 341 · see citing works
Structured Denoising Diffusion Models in Discrete State-SpacesPreprint
Austin, J., Johnson, D. D., Ho, J., Tarlow, D., & van den Berg, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.03006 · Cited by 340 · see citing works
Deep Structured Energy Based Models for Anomaly Detection
Zhai, S., Cheng, Y., Lu, W., & Zhang, Z. (2016) · arXiv · doi:10.48550/arxiv.1605.07717 · Cited by 334 · see citing works
What Regularized Auto-Encoders Learn from the Data Generating DistributionPreprint
Alain, G., & Bengio, Y. (2013) · Cited by 319 · see citing works
High-resolution image reconstruction with latent diffusion models from human brain activity
Takagi, Y., & Nishimoto, S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01389 · Cited by 314 · see citing works
Compositional Visual Generation with Composable Diffusion Models
Liu, N., Li, S., Du, Y., Torralba, A. B., & Tenenbaum, J. B. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19790-1_26 · Cited by 302 · see citing works
Generalized Denoising Auto-Encoders as Generative Models
Bengio, Y., Yao, L., Alain, G., & Vincent, P. (2013) · arXiv · doi:10.48550/arxiv.1305.6663
Self-citationSolving Inverse Problems in Medical Imaging with Score-Based Generative ModelsPreprint
Song, Y. S., Shen, L., Xing, L., & Ermon, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.08005 · Cited by 287 · see citing works
ICA with Reconstruction Cost for Efficient Overcomplete Feature Learning
Le, Q. V., Karpenko, A., Ngiam, J., & Ng, A. Y. (2011) · Cited by 283 · see citing works
Speech Enhancement and Dereverberation With Diffusion-Based Generative Models
Richter, J., Welker, S., Lemercier, J.-M., Lay, B., & Gerkmann, T. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3285241 · Cited by 256 · see citing works
PhysDiff: Physics-Guided Human Motion Diffusion Model
Yuan, Y., Song, J., Iqbal, U., Vahdat, A., & Kautz, J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01467 · Cited by 246 · see citing works
Pylearn2: a machine learning research libraryPreprint
Goodfellow, I., Warde-Farley, D., Lamblin, P., Dumoulin, V., Mirza, M., Pascanu, R., Bergstra, J., Bastien, F., & Bengio, Y. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1308.4214 · Cited by 245 · see citing works
Regularization by Denoising: Clarifications and New Interpretations
Reehorst, E. T., & Schniter, P. (2018) · IEEE Transactions on Computational Imaging · doi:10.1109/tci.2018.2880326 · Cited by 241 · see citing works
DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanism
Liu, J., Li, C., Ren, Y., Chen, F., & Zhao, Z. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21350 · Cited by 236 · see citing works
Learning Gradient Fields for Shape Generation
Cai, R., Yang, G., Averbuch-Elor, H., Hao, Z., Belongie, S., Snavely, N., & Hariharan, B. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58580-8_22 · Cited by 225 · see citing works
Score-based Generative Modeling in Latent Space
Vahdat, A., Kreis, K., & Kautz, J. (2021) · arXiv · doi:10.48550/arxiv.2106.05931 · Cited by 224 · see citing works
On the Expressive Power of Deep Architectures
Bengio, Y., & Delalleau, O. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-24477-3_1 · Cited by 209 · see citing works
Perception Prioritized Training of Diffusion Models
Choi, J., Lee, J., Shin, C., Kim, S., Kim, H., & Yoon, S. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01118 · Cited by 205 · see citing works
Deep Learning for Time Series Forecasting: Advances and Open Problems
Casolaro, A., Capone, V., Iannuzzo, G., & Camastra, F. (2023) · Information · doi:10.3390/info14110598 · Cited by 199 · see citing works
Maximum Likelihood Training of Score-Based Diffusion ModelsPreprint
Song, Y. S., Durkan, C., Murray, I., & Ermon, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.09258 · Cited by 197 · see citing works
Diffusion-based Generation, Optimization, and Planning in 3D Scenes
Huang, S., Wang, Z., Li, P., Jia, B., Liu, T., Zhu, Y., Liang, W., & Zhu, S.-C. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01607 · Cited by 192 · see citing works
On the Expressive Power of Deep Architectures
Bengio, Y., & Delalleau, O. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-24412-4_3 · Cited by 182 · see citing works
One-Step Diffusion with Distribution Matching Distillation
Yin, T., Gharbi, M., Zhang, R., Shechtman, E., Durand, F., Freeman, W. T., & Park, T. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00632 · Cited by 179 · see citing works
Predicting equilibrium distributions for molecular systems with deep learning
Zheng, S., He, J., Liu, C., Shi, Y., Lu, Z., Feng, W., Ju, F., Wang, J., Zhu, J., Min, Y., et al. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-024-00837-3 · Cited by 178 · see citing works
A Theory of Generative ConvNetPreprint
Xie, J., Lu, Y., Zhu, S. C., & Wu, Y. N. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.03264 · Cited by 176 · see citing works
ECG signal enhancement based on improved denoising auto-encoder
Xiong, P., Wang, H., Liu, M., Zhou, S., Hou, Z., & Liu, X. (2016) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2016.02.015 · Cited by 174 · see citing works
BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language ModelPreprint
Wang, A., & Cho, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.04094 · Cited by 174 · see citing works
Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series ForecastingPreprint
Rasul, K., Seward, C., Schuster, I., & Vollgraf, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.12072 · Cited by 174 · see citing works
Diffusion Model-Based Image Editing: A Survey
Huang, Y., Huang, J., Liu, Y., Yan, M., Lv, J., Liu, J., Xiong, W., Zhang, H., Cao, L., & Chen, S. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3541625 · Cited by 168 · see citing works
Deep AutoRegressive NetworksPreprint
Gregor, K., Danihelka, I., Mnih, A., Blundell, C., & Wierstra, D. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1310.8499 · Cited by 166 · see citing works
Scheduled denoising autoencodersPreprint
Geras, K. J., & Sutton, C. A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1406.3269 · Cited by 165 · see citing works
Image Denoising: The Deep Learning Revolution and Beyond—A Survey Paper
Elad, M., Kawar, B., & Vaksman, G. (2023) · SIAM Journal on Imaging Sciences · doi:10.1137/23m1545859 · Cited by 164 · see citing works
Sobolev Training for Neural NetworksPreprint
Czarnecki, W. M., Osindero, S., Jaderberg, M., Świrszcz, G., & Pascanu, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.04859 · Cited by 164 · see citing works
Diffusion models in bioinformatics and computational biology
Guo, Z., Liu, J., Wang, Y., Chen, M., Wang, D., Xu, D., & Cheng, J. (2023) · Nature Reviews Bioengineering · doi:10.1038/s44222-023-00114-9 · Cited by 163 · see citing works
Your Diffusion Model is Secretly a Zero-Shot Classifier
Li, A. C., Prabhudesai, M., Duggal, S., Brown, E. V., & Pathak, D. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00210 · Cited by 159 · see citing works
Conditional Image Generation with Score-Based Diffusion ModelsPreprint
Batzolis, G., Stanczuk, J., Schönlieb, C., & Etmann, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.13606 · Cited by 157 · see citing works
Solving 3D Inverse Problems Using Pre-Trained 2D Diffusion Models
Chung, H., Ryu, D., McCann, M. T., Klasky, M., & Ye, J. C. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02159 · Cited by 156 · see citing works
State of the Art on Diffusion Models for Visual Computing
Po, R., Yifan, W., Golyanik, V., Aberman, K., Barron, J. T., Bermano, A. H., Chan, E. R., Dekel, T., Holynski, A., Kanazawa, A., et al. (2024) · Computer Graphics Forum · doi:10.1111/cgf.15063 · Cited by 156 · see citing works
MR Image Denoising and Super-Resolution Using Regularized Reverse Diffusion
Chung, H., Lee, E. S., & Ye, J. C. (2022) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2022.3220681 · Cited by 155 · see citing works
Collaborative Diffusion for Multi-Modal Face Generation and Editing
Huang, Z., Chan, K. C. K., Jiang, Y., & Liu, Z. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00589 · Cited by 155 · see citing works
Deep Computation Model for Unsupervised Feature Learning on Big Data
Zhang, Q., Yang, L. T., & Chen, Z. (2015) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2015.2497705 · Cited by 152 · see citing works
Versatile Diffusion: Text, Images and Variations All in One Diffusion Model
Xu, X., Wang, Z., Zhang, E., Wang, K., & Shi, H. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00713 · Cited by 150 · see citing works
Wavelet Diffusion Models are fast and scalable Image Generators
Phung, H., Dao, Q. T., & Tran, A. N. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00983 · Cited by 145 · see citing works
At the Dawn of Generative AI Era: A Tutorial-cum-Survey on New Frontiers in 6G Wireless Intelligence
Çelik, A., & Eltawil, A. M. (2024) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2024.3362271 · Cited by 145 · see citing works
Dimensionality Reduction in Surrogate Modeling: A Review of Combined Methods
Hou, C. K. J., & Behdinan, K. (2022) · Data Science and Engineering · doi:10.1007/s41019-022-00193-5 · Cited by 142 · see citing works
WaveDM: Wavelet-Based Diffusion Models for Image Restoration
Huang, Y., Huang, J., Liu, J., Yan, M., Yu, D., Lv, J., Chen, C., & Chen, S. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3359769 · Cited by 142 · see citing works
Opportunities and challenges of diffusion models for generative AI
Chen, M., Song, M., Fan, J., & Wang, M. (2024) · National Science Review · doi:10.1093/nsr/nwae348 · Cited by 141 · see citing works
UNIT-DDPM: UNpaired Image Translation with Denoising Diffusion Probabilistic ModelsPreprint
Sasaki, H., Willcocks, C. G., & Breckon, T. P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.05358 · Cited by 140 · see citing works
Diffusion models for time-series applications: a survey
Lin, L., Li, Z., Li, R., Li, X., & Gao, J. (2023) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.2300310 · Cited by 139 · see citing works
Sliced Score Matching: A Scalable Approach to Density and Score EstimationPreprint
Song, Y. S., Garg, S., Shi, J., & Ermon, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.07088 · Cited by 134 · see citing works
How to Train Your Energy-Based ModelsPreprint
Song, Y., & Kingma, D. P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.03288 · Cited by 132 · see citing works
StoRM: A Diffusion-Based Stochastic Regeneration Model for Speech Enhancement and Dereverberation
Lemercier, J.-M., Richter, J., Welker, S., & Gerkmann, T. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3294692 · Cited by 131 · see citing works
Toward High-Quality HDR Deghosting With Conditional Diffusion Models
Yan, Q., Hu, T., Sun, Y., Tang, H., Zhu, Y., Dong, W. F., Gool, L. V., & Zhang, Y. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3326293 · Cited by 131 · see citing works
MIMO Channel Estimation Using Score-Based Generative Models
Arvinte, M., & Tamir, J. I. (2022) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2022.3220784 · Cited by 129 · see citing works
Deep Learning of Representations: Looking ForwardPreprint
Bengio, Y. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1305.0445 · Cited by 128 · see citing works
How Auto-Encoders Could Provide Credit Assignment in Deep Networks via Target PropagationPreprint
Bengio, Y. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1407.7906 · Cited by 127 · see citing works
Knowledge Distillation in Iterative Generative Models for Improved Sampling SpeedPreprint
Luhman, E., & Luhman, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.02388 · Cited by 125 · see citing works
Distributed Compressive Sensing: A Deep Learning Approach
Palangi, H., Ward, R. K., & Deng, L. (2016) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2016.2557301 · Cited by 122 · see citing works
Score-based generative models for calorimeter shower simulation
Mikuni, V. M., & Nachman, B. P. (2022) · Physical review. D/Physical review. D. · doi:10.1103/physrevd.106.092009 · Cited by 121 · see citing works
Denoising Pretraining for Semantic Segmentation
Brempong, E. A., Kornblith, S., Chen, T., Parmar, N., Minderer, M., & Norouzi, M. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00462 · Cited by 120 · see citing works
GenSLMs: Genome-scale language models reveal SARS-CoV-2 evolutionary dynamics
Zvyagin, M., Brace, A., Hippe, K., Deng, Y., Zhang, B., Bohorquez, C. O., Clyde, A., Kale, B., Perez-Rivera, D., Ma, H., et al. (2023) · The International Journal of High Performance Computing Applications · doi:10.1177/10943420231201154 · Cited by 120 · see citing works
GenSLMs: Genome-scale language models reveal SARS-CoV-2 evolutionary dynamics
Zvyagin, M., Brace, A., Hippe, K., Deng, Y., Zhang, B., Bohorquez, C. O., Clyde, A., Kale, B., Perez-Rivera, D., Ma, H., et al. (2022) · doi:10.1101/2022.10.10.511571 · Cited by 120 · see citing works
Tallman, T. N., & Smyl, D. (2020) · Smart Materials and Structures · doi:10.1088/1361-665x/abb352 · Cited by 120 · see citing works
Diffusion Models, Image Super-Resolution, and Everything: A Survey
Moser, B. B., Shanbhag, A. S., Raue, F., Frolov, S., Palacio, S., & Dengel, A. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3476671 · Cited by 117 · see citing works
Deep Learners Benefit More from Out-of-Distribution Examples
Bengio, Y., Bastien, F., Bergeron, A., Boulanger-Lewandowski, N., Breuel, T. M., Chherawala, Y., Cissé, M., Côté, M., Erhan, D., Eustache, J., et al. (2011) · Cited by 117 · see citing works
A Morphology Focused Diffusion Probabilistic Model for Synthesis of Histopathology Images
Moghadam, P. A., Dalen, S. V., Martin, K. C., Lennerz, J. K., Yip, S. S. F., Farahani, H., & Bashashati, A. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00204 · Cited by 117 · see citing works
DiffusioNeRF: Regularizing Neural Radiance Fields with Denoising Diffusion Models
Wynn, J., & Turmukhambetov, D. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00407 · Cited by 111 · see citing works
Diffusion Models in Low-Level Vision: A Survey
He, C., Shen, Y., Fang, C., Xiao, F., Tang, L., Zhang, Y., Zuo, W., Guo, Z., & Li, X. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3545047 · Cited by 109 · see citing works
- Authors
- Vincent, P.
- Affiliations
- Université de Montréal (CA)
- Corresponding author
- Vincent, Pascal
- Venue
- Neural Computation
- Publisher
- MIT Press
- Published
- 2011-07
- Volume
- 23
- Issue
- 7
- Pages
- 1661-1674
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2013035813
- Cited by
- 1,011
- References
- 29
- Citations by year
- 2026: 131 · 2025: 254 · 2024: 198 · 2023: 126 · 2022: 46 · 2021: 59 · 2020: 52 · 2019: 29 · 2018: 20 · 2017: 22 · 2016: 20 · 2015: 14 · 2014: 9 · 2013: 14 · 2012: 9
- FWCI
- 11.49
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Neural Computation (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Signal Denoising Methods
- Keywords
- score matching, denoising autoencoder, energy-based models, unsupervised pre-training, restricted Boltzmann machine, probabilistic models, deep learning architectures
- MeSH terms
- Computer Simulation, Models, Statistical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review