The DOI matches its Crossref record; 72 of the 72 references it cites were found in a registry or an index.
A practical tutorial on autoencoders for nonlinear feature fusion: Taxonomy, models, software and guidelines
Open accessAbstract
Many of the existing machine learning algorithms, both supervised and unsupervised, depend on the quality of the input characteristics to generate a good model. The amount of these variables is also important, since performance tends to decline as the input dimensionality increases, hence the interest in using feature fusion techniques, able to produce feature sets that are more compact and higher level. A plethora of procedures to fuse original variables for producing new ones has been developed in the past decades. The most basic ones use linear combinations of the original variables, such as PCA (Principal Component Analysis) and LDA (Linear Discriminant Analysis), while others find manifold embeddings of lower dimensionality based on non-linear combinations, such as Isomap or LLE (Linear Locally Embedding) techniques. More recently, autoencoders (AEs) have emerged as an alternative to manifold learning for conducting nonlinear feature fusion. Dozens of AE models have been proposed lately, each with its own specific traits. Although many of them can be used to generate reduced feature sets through the fusion of the original ones, there also AEs designed with other applications in mind. The goal of this paper is to provide the reader with a broad view of what an AE is, how they are used for feature fusion, a taxonomy gathering a broad range of models, and how they relate to other classical techniques. In addition, a set of didactic guidelines on how to choose the proper AE for a given task is supplied, together with a discussion of the software tools available. Finally, two case studies illustrate the usage of AEs with datasets of handwritten digits and breast cancer.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
Multilayer feedforward networks are universal approximators
Hornik, K., Stinchcombe, M., & White, H. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90020-8 · Cited by 21,728
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
On Information and Sufficiency
Kullback, S., & Leibler, R. A. (1951) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729694 · Cited by 20,193
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570
A logical calculus of the ideas immanent in nervous activity
McCulloch, W. S., & Pitts, W. (1943) · The Bulletin of Mathematical Biophysics · doi:10.1007/bf02478259 · Cited by 18,566
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS
FISHER, R. A. (1936) · Annals of Eugenics · doi:10.1111/j.1469-1809.1936.tb02137.x · Cited by 15,227
Nonlinear Dimensionality Reduction by Locally Linear Embedding
Roweis, S. T., & Saul, L. K. (2000) · Science · doi:10.1126/science.290.5500.2323 · Cited by 14,965
Bellman, R. E. (2010) · doi:10.1515/9781400835386 · Cited by 14,588
A Global Geometric Framework for Nonlinear Dimensionality Reduction
Tenenbaum, J. B., Silva, V. D., & Langford, J. C. (2000) · Science · doi:10.1126/science.290.5500.2319 · Cited by 13,585
Representation Learning: A Review and New Perspectives
Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001
LIII. On lines and planes of closest fit to systems of points in space
Pearson, K. (1901) · The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science · doi:10.1080/14786440109462720 · Cited by 11,934
Backpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077
Peng, H., Long, F., & Ding, C. (2005) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2005.159 · Cited by 10,544
A Stochastic Approximation Method
Robbins, H., & Monro, S. (1951) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729586 · Cited by 9,643
Analysis of a complex of statistical variables into principal components.
Hotelling, H. (1933) · Journal of Educational Psychology · doi:10.1037/h0071325 · Cited by 9,633
Nonlinear Component Analysis as a Kernel Eigenvalue Problem
Schölkopf, B., Smola, A., & Müller, K.-R. (1998) · Neural Computation · doi:10.1162/089976698300017467 · Cited by 8,084
Laplacian Eigenmaps for Dimensionality Reduction and Data Representation
Belkin, M., & Niyogi, P. (2003) · Neural Computation · doi:10.1162/089976603321780317 · Cited by 7,621
Emergence of simple-cell receptive field properties by learning a sparse code for natural images
Olshausen, B. A., & Field, D. J. (1996) · Nature · doi:10.1038/381607a0 · Cited by 5,694
Training Products of Experts by Minimizing Contrastive Divergence
Hinton, G. E. (2002) · Neural Computation · doi:10.1162/089976602760128018 · Cited by 4,879 · cited 1×
“Instead, Contrastive Divergence [110] is used to train an RBM. RBMs are an alternative to AEs for greedy layer-wise initialization of weights in ANNs including AEs.”
MethodGreedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841
A Learning Algorithm for Continually Running Fully Recurrent Neural Networks
Williams, R. J., & Zipser, D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.2.270 · Cited by 4,369
The Handbook of brain theory and neural networks
(1996) · Choice Reviews Online · doi:10.5860/choice.33-4484 · Cited by 3,870
Multi-column deep neural networks for image classification
Ciresan, D., Meier, U., & Schmidhuber, J. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6248110 · Cited by 3,720
Sparse coding with an overcomplete basis set: A strategy employed by V1?
Olshausen, B. A., & Field, D. J. (1997) · Vision Research · doi:10.1016/s0042-6989(97)00169-7 · Cited by 3,642
Feature selection for classification
DASH, M., & LIU, H. (1997) · Intelligent Data Analysis · doi:10.1016/s1088-467x(97)00008-5 · Cited by 3,574
A Nonlinear Mapping for Data Structure Analysis
Sammon, J. W. (1969) · IEEE Transactions on Computers · doi:10.1109/t-c.1969.222678 · Cited by 3,380
A few useful things to know about machine learning
Domingos, P. (2012) · Communications of the ACM · doi:10.1145/2347736.2347755 · Cited by 3,326 · cited 2×
“The most common are as follows: Feature engineering [19].”
BackgroundNonlinear principal component analysis using autoassociative neural networks
Kramer, M. A. (1991) · AIChE Journal · doi:10.1002/aic.690370209 · Cited by 3,037
The Vanishing Gradient Problem During Learning Recurrent Neural Nets and Problem Solutions
Hochreiter, S. (1998) · International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems · doi:10.1142/s0218488598000094 · Cited by 2,939
Singular value decomposition and least squares solutions
Golub, G. H., & Reinsch, C. (1970) · Numerische Mathematik · doi:10.1007/bf02163027 · Cited by 2,929 · cited 1×
“The procedure to curate biological databases is very expensive, so usually machine learning methods such as SVD ( Singular Value Decomposition ) [145] are applied to help in the process.”
MethodThe JPEG still picture compression standard
Wallace, G. K. (1992) · IEEE Transactions on Consumer Electronics · doi:10.1109/30.125072 · Cited by 2,851
Multidimensional Scaling: I. Theory and Method
Torgerson, W. S. (1952) · Psychometrika · doi:10.1007/bf02288916 · Cited by 2,126
Unsupervised Learning of Video Representations using LSTMs
Srivastava, N., Mansimov, E., & Salakhutdinov, R. (2015) · arXiv · doi:10.48550/arxiv.1502.04681 · Cited by 1,861
Unsupervised feature selection using feature similarity
Mitra, P., Murthy, C. A., & Pal, S. K. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.990133 · Cited by 1,460
Neural networks and principal component analysis: Learning from examples without local minima
Baldi, P., & Hornik, K. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90014-2 · Cited by 1,383
Auto-association by multilayer perceptrons and singular value decomposition
Bourlard, H., & Kamp, Y. (1988) · Biological Cybernetics · doi:10.1007/bf00332918 · Cited by 1,345
Salakhutdinov, R., & Hinton, G. (2009) · International Journal of Approximate Reasoning · doi:10.1016/j.ijar.2008.11.006 · Cited by 1,246
A survey on deep learning for big data
Zhang, Q., Yang, L. T., Chen, Z., & Li, P. (2018) · Information Fusion · doi:10.1016/j.inffus.2017.10.006 · Cited by 1,226
Efficient Learning of Sparse Representations with an Energy-Based Model
Ranzato, M., Poultney, C., Chopra, S., & LeCun, Y. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0147 · Cited by 1,106
Extended Boolean information retrieval
Salton, G., Fox, E. A., & Wu, H. (1983) · Communications of the ACM · doi:10.1145/182.358466 · Cited by 1,054
Hinton, G. (2009) · Scholarpedia · doi:10.4249/scholarpedia.5947 · Cited by 885
Stacked Sparse Autoencoder (SSAE) for Nuclei Detection on Breast Cancer Histopathology Images
Xu, J., Xiang, L., Liu, Q., Gilmore, H., Wu, J., Tang, J., & Madabhushi, A. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2015.2458702 · Cited by 854
Breast Cancer Diagnosis and Prognosis Via Linear Programming
Mangasarian, O. L., Street, W. N., & Wolberg, W. H. (1995) · Operations Research · doi:10.1287/opre.43.4.570 · Cited by 843
Sparse Feature Learning for Deep Belief Networks
Ranzato, M., Boureau, Y.-L., & LeCun, Y. (2007) · neural information processing systems · Cited by 744
Medical Image Denoising Using Convolutional Denoising Autoencoders
Gondara, L. (2016) · 2016 IEEE 16th International Conference on Data Mining Workshops (ICDMW) · doi:10.1109/icdmw.2016.0041 · Cited by 737
Multimodal Deep Autoencoder for Human Pose Recovery
Hong, C., Yu, J., Wan, J., Tao, D., & Wang, M. (2015) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2487860 · Cited by 568
Tutorial on practical tips of the most influential data preprocessing algorithms in data mining
García, S., Luengo, J., & Herrera, F. (2016) · Knowledge-Based Systems · doi:10.1016/j.knosys.2015.12.006 · Cited by 310
Self-citationA Survey of Decision Fusion and Feature Fusion Strategies for Pattern Classification
Mangai, U., Samanta, S., Das, S., & Chowdhury, P. (2010) · IETE Technical Review · doi:10.4103/0256-4602.64604 · Cited by 302
Survey on distance metric learning and dimensionality reduction in data mining
Wang, F., & Sun, J. (2015) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-014-0356-z · Cited by 168
Hashing Techniques: A Survey and Taxonomy
Chi, L., & Zhu, X. (2018) · ACM Computing Surveys · doi:10.1145/3047307 · Cited by 161
Knowledge Matters: Importance of Prior Information for OptimizationPreprint
Gülçehre, Ç., & Bengio, Y. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.4083 · Cited by 140
Distributed optimization of deeply nested systemsPreprint
Carreira-Perpiñán, M. Á., & Wang, W. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1212.5921 · Cited by 134
Replicator Neural Networks for Universal Optimal Source Coding
Hecht-Nielsen, R. (1995) · Science · doi:10.1126/science.269.5232.1860 · Cited by 134
Representation learning via Dual-Autoencoder for recommendation
Zhuang, F., Zhang, Z., Qian, M., Shi, C., Xie, X., & He, Q. (2017) · Neural Networks · doi:10.1016/j.neunet.2017.03.009 · Cited by 115
Algorithms for manifold learning
Cayton, L. (2005) · Cited by 101
Multimodal video classification with stacked contractive autoencoders
Liu, Y., Xiao-qing, F., & Zhou, Z. (2015) · Signal Processing · doi:10.1016/j.sigpro.2015.01.001 · Cited by 96
Laplacian Auto-Encoders: An explicit learning of nonlinear data manifold
Jia, K., Sun, L., Gao, S., Song, Z. P., & Shi, B. E. (2015) · Neurocomputing · doi:10.1016/j.neucom.2015.02.023 · Cited by 79
Lightweight Lossy Compression of Biometric Patterns via Denoising Autoencoders
Testa, D. D., & Rossi, M. (2015) · IEEE Signal Processing Letters · doi:10.1109/lsp.2015.2476667 · Cited by 77
Whispered Speech Recognition Using Deep Denoising Autoencoder and Inverse Filtering
Grozdić, Đ., & Jovičić, S. T. (2017) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2017.2738559 · Cited by 72
Embedding with Autoencoder Regularization
Yu, W., Zeng, G., Luo, P., Zhuang, F., He, Q., & Shi, Z. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-40994-3_14 · Cited by 65
Unsupervised feature extraction with autoencoder trees
İrsoy, O., & Alpaydın, E. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.02.075 · Cited by 60
Fake Twitter followers detection by denoising autoencoder
Castellini, J., Poggioni, V., & Sorbi, G. (2017) · IEEE/WIC/ACM International Conference on Web Intelligence · doi:10.1145/3106426.3106489 · Cited by 51
On building ensembles of stacked denoising auto-encoding classifiers and their further improvement
Alvear-Sandoval, R. F., & Figueiras-Vidal, A. R. (2017) · Information Fusion · doi:10.1016/j.inffus.2017.03.008 · Cited by 49
Using Autoencoders for Mammogram Compression
Tan, C. C., & Eswaran, C. (2009) · Journal of Medical Systems · doi:10.1007/s10916-009-9340-3 · Cited by 38
Vision-Based Pose Estimation From Points With Unknown Correspondences
Zhou, H., Zhang, T., & Lu, W. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2014.2329765 · Cited by 23
Autoencoder-based holographic image restoration
Shimobaba, T., Endo, Y., Hirayama, R., et al. (2017) · Applied Optics · doi:10.1364/ao.56.000f27 · Cited by 23
Cited by
Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.10045 · Cited by 11,321
Self-citationBarredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
Self-citationAutoencoders and their applications in machine learning: a survey
Berahmand, K., Daneshfar, F., Salehi, E. S., Li, Y., & Xu, Y. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-023-10662-6 · Cited by 593
Deep Learning for Medical Anomaly Detection – A Survey
Fernando, T., Gammulle, H., Denman, S., Sridharan, S., & Fookes, C. (2022) · ACM Computing Surveys · doi:10.1145/3464423 · Cited by 399
Górriz, J. M., Ramírez, J., Ortíz, A., Martínez-Murcia, F. J., Segovia, F., Suckling, J., Leming, M., Zhang, Y.-D., Álvarez-Sánchez, J. R., Bologna, G., et al. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.05.078 · Cited by 319
Self-citation
Metadata
- Authors
- Charte, D., Charte, F., García, S., del Jesus, M. J., & Herrera, F.
- Venue
- Information Fusion
- Publisher
- Elsevier BV
- Published
- 2018-11
- Volume
- 44
- Pages
- 78-96
- ISSN
- 1566-2535 (ISSN-L)
- OpenAlex
- W2778685442
- Cited by
- 273
- References
- 72
- FWCI
- 14.76
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.