The DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
Extracting and composing robust features with denoising autoencoders
Abstract
Previous work has shown that the difficulties in learning deep generative or discriminative models can be overcome by an initial unsupervised learning step that maps inputs to useful intermediate representations. We introduce and motivate a new training principle for unsupervised learning of a representation based on the idea of making the learned representations robust to partial corruption of the input pattern. This approach can be used to train autoencoders, and these denoising autoencoders can be stacked to initialize deep architectures. The algorithm can be motivated from a manifold learning and information theoretic perspective or from a generative model perspective. Comparative experiments clearly show the surprising advantage of corrupting the input of autoencoders on a pattern classification benchmark suite. 1.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Proposition 3.2.If the cross entropy score comparison becomes nonnegative when introducing the vector x t , then 4 defines a decision boundary in the latent space over the measure of robustness to partial destruction of the input (Vincent et al., 2008;Srivastava et al., 2014), i.e.:”
From Novelty Detection in Reinforcement Learning with World Models (2023) · cites this paper as Vincent et al., 2008
In the citing paper
“Proposition 3.2 (restated) If the cross entropy score comparison becomes nonnegative when introducing the vector x t , then 4 defines a decision boundary in the latent space over the measure of robustness to partial destruction of the input (Vincent et al., 2008;Srivastava et al., 2014), i.e:”
From Novelty Detection in Reinforcement Learning with World Models (2023) · cites this paper as Vincent et al., 2008
In the citing paper
“Our sub-sampling technique is somewhat similar to denoising [31, 18] where a subset of inputs are 'partially destructed”
From Multimodal Generative Models for Scalable Weakly-Supervised Learning (2018) · cites this paper as [31]
In the citing paper
“For the same clinical task, a similar workflow has been adapted by researchers, but applying denoising AEs (DAEs) [91] instead [35, 36].”
From Multimodal deep learning for biomedical data fusion: a review (2022) · cites this paper as [91]
In the citing paper
“This architecture allows us to obtain useful and robust features in the presence of unstable environments such as IoT systems, in which edge devices may succumb to physical disturbances at the control and communication layers.”
From Deep learning-enabled anomaly detection for IoT systems (2023) · cites this paper as [14]
In the citing paper
“While the traditional autoencoder can enhance the detection of most AI-powered applications by abstracting and mapping an input to an intermediate representation, the traditional autoencoder is not able to obtain robust and useful features in heterogeneous environments [14,15].”
From Deep learning-enabled anomaly detection for IoT systems (2023) · cites this paper as [14]
In the citing paper
“The denoising autoencoder can be used to address these problems.”
From Deep learning-enabled anomaly detection for IoT systems (2023) · cites this paper as [14]
In the citing paper
“In other words, the pre-training process adopted by our model allows the DNN to have better initialization of the parameters to be used during applying the backpropagation algorithm and fine tuning processes [14,15].”
From Deep learning-enabled anomaly detection for IoT systems (2023) · cites this paper as [14]
In the citing paper
“While RBMs can be used to learn”
From Deep learning-enabled anomaly detection for IoT systems (2023) · cites this paper as [14]
In the citing paper
“This is mainly due to the fact that the RBM does not take into consideration noisy inputs during the pre-training process [14].”
From Deep learning-enabled anomaly detection for IoT systems (2023) · cites this paper as [14]
In the citing paper
“While RBMs can be used to learn”
From Deep learning-enabled anomaly detection for IoT systems (2023) · cites this paper as [14]
In the citing paper
“This can be interpreted due to the fact that the traditional autoencoder does not work properly when applied in heterogeneous and noisy environments such as IoT systems [14,15].”
From Deep learning-enabled anomaly detection for IoT systems (2023) · cites this paper as [14]
In the citing paper
“This proves that DAE can not only effectively deal with the noise in the input, but also perform well in dimensionality reduction for high-dimensional data.”
From EEG-based emotion recognition with autoencoder feature fusion and MSC-TimesNet model (2026) · cites this paper as Vincent et al. 2008
In the citing paper
“The results show that as the noise level increases, the filter gradually learns more discriminative features.”
From EEG-based emotion recognition with autoencoder feature fusion and MSC-TimesNet model (2026) · cites this paper as Vincent et al. 2008
In the citing paper
“Inspired by Denoising Auto-Encoder (DAE) [3, 44], we add perturbations to feature tokens, guiding the model to learn knowledge of normal samples by the denoising task.”
From A Unified Model for Multi-Class Anomaly Detection (2022) · cites this paper as [44]
In the citing paper
“On one hand, the applications of edge computing urgently need the powerful processing capabilities of deep learning to handle various complicated scenarios, such as video analytics [9], transportation control [10], etc.”
From Deep learning for edge computing applications: A State-of-the-Art survey (2020) · cites this paper as [10]
In the citing paper
“Similarly, Decoder Denoising Pretraining (DDeP) [19] integrates diffusion models with denoising autoencoders [241] and delivers promising results on label-efficient semantic segmentation.”
From Diffusion Models: A Comprehensive Survey of Methods and Applications (2024) · cites this paper as [241]
In the citing paper
“Deep Neural Network for Graph Representations (DNGR) [59] uses a stacked denoising autoencoder [108] to encode and decode the PPMI matrix via multi-layer perceptrons.”
From A Comprehensive Survey on Graph Neural Networks (2021) · cites this paper as [108]
In the citing paper
“Vincent et al. [41] used denoising autoencoders for robust feature learning, Kingma and Welling [42] introduced variational autoencoders for generative modeling, and Sakurada and Yairi [43] employed autoencoders for anomaly detection in time-series data.”
From Reducing Label Dependency in Human Activity Recognition with Wearables: From Supervised Learning to Novel Weakly Self-Supervised Approaches (2025) · cites this paper as [41]
Citations
Cites
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
Neural networks and physical systems with emergent collective computational abilities.
Hopfield, J. J. (1982) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.79.8.2554 · Cited by 19,270
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries
Elad, M., & Aharon, M. (2006) · IEEE Transactions on Image Processing · doi:10.1109/tip.2006.881969 · Cited by 5,321
Learning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 5,050
Self-citationGreedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841
Self-citationTraining with Noise is Equivalent to Tikhonov Regularization
Bishop, C. M. (1995) · Neural Computation · doi:10.1162/neco.1995.7.1.108 · Cited by 1,325
Efficient Learning of Sparse Representations with an Energy-Based Model
Ranzato, M., Poultney, C., Chopra, S., & LeCun, Y. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0147 · Cited by 1,106
Fields of Experts: A Framework for Learning Image Priors
Roth, S., & Black, M. J. (2005) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2005.160 · Cited by 1,087
Cited by
Sorted by citations
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Self-citationBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839
Metadata
- Authors
- Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A.
- Affiliations
- Université de Montréal (CA)
- Venue
- Proceedings of the 25th international conference on Machine learning - ICML '08
- Publisher
- ACM Press
- Published
- 2008
- Pages
- 1096-1103
- OpenAlex
- W2025768430
- Cited by
- 7,436
- References
- 22
- Citations by year
- 2026: 270 · 2025: 532 · 2024: 622 · 2023: 668 · 2022: 685 · 2021: 834 · 2020: 837 · 2019: 798 · 2018: 613 · 2017: 511 · 2016: 371 · 2015: 294 · 2014: 178 · 2013: 102 · 2012: 50
- FWCI
- 17.08
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Machine Learning (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face and Expression Recognition
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Keywords
- denoising autoencoder, robust feature extraction, unsupervised feature learning, stacked autoencoder, manifold learning, information theory, pattern classification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
