Nonlinear principal component analysis using autoassociative neural networks
Abstract
Nonlinear principal component analysis is a novel technique for multivariate data analysis, similar to the well‐known method of principal component analysis. NLPCA, like PCA, is used to identify and remove correlations among problem variables as an aid to dimensionality reduction, visualization, and exploratory data analysis. While PCA identifies only linear correlations between variables, NLPCA uncovers both linear and nonlinear correlations, without restriction on the character of the nonlinearities present in the data. NLPCA operates by training a feedforward neural network to perform the identity mapping, where the network inputs are reproduced at the output layer. The network contains an internal “bottleneck” layer (containing fewer nodes than input or output layers), which forces the network to develop a compact representation of the input data, and two additional hidden layers. The NLPCA method is demonstrated using time‐dependent, simulated batch reaction data. Results show that NLPCA successfully reduces dimensionality and produces a feature space map resembling the actual distribution of the underlying system parameters.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Kramer (1991) used neural networks to extend PCA to the extraction of nonlinear relationships in a dataset, defining nonlinear principal component analysis (NLPCA).
“Since the 1980s, models of neural networks have been developed and Kramer (1991) used them to extend PCA abilities to the extraction of nonlinear relationships in a dataset, which defines the nonlinear principal component analysis (NLPCA).”
From Sea surface temperature patterns in the Tropical Atlantic: Principal component analysis and nonlinear principal component analysis (2017) · cites this paper as Kramer (1991)
The fundamental deep learning architectures and methods (autoencoders, ResNet, Siamese networks, TCNs, and contrastive loss functions) have been extensively tested and widely adopted in prior studies, providing a strong and reliable foundation for the research.
“These methods have been extensively tested and widely adopted in prior studies, providing a strong and reliable foundation for our research.”
From Reducing Label Dependency in Human Activity Recognition with Wearables: From Supervised Learning to Novel Weakly Self-Supervised Approaches (2025) · cites this paper as [36]
Cites
System Identification: Theory for the User
Lennart Ljung, A. (1987) · Cited by 17,690 · see citing works
Learning Internal Representations by Error Propagation
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1985) · doi:10.21236/ada164453 · Cited by 16,501 · see citing works
Parallel Distributed Processing: Explorations in the Microstructure of Cognition: Foundations
Rumelhart, D. E., McClelland, J. L., & AU (1986) · doi:10.7551/mitpress/5236.001.0001 · Cited by 15,686 · see citing works
Approximation by superpositions of a sigmoidal function
Cybenko, G. (1989) · Mathematics of Control, Signals, and Systems · doi:10.1007/bf02551274 · Cited by 13,883 · see citing works
Pattern classification and scene analysis
Duda, R. O., & Hart, P. E. (1973) · CERN Document Server (European Organization for Nuclear Research) · Cited by 12,895 · see citing works
Partial least-squares regression: a tutorial
Geladi, P. L. M., & Kowalski, B. R. (1986) · Analytica Chimica Acta · doi:10.1016/0003-2670(86)80028-9 · Cited by 7,016 · see citing works
A Learning Algorithm for Boltzmann Machines
Ackley, D., Hinton, G. E., & Sejnowski, T. (1985) · Cognitive Sciences · doi:10.1016/b978-0-08-051581-6.50053-2 · Cited by 4,462 · see citing works
A learning algorithm for Boltzmann machines
Ackley, D. H., Hinton, G. E., & Sejnowski, T. J. (1985) · Cognitive Science · doi:10.1207/s15516709cog0901_7 · Cited by 3,402 · see citing works
Beyond Regression : "New Tools for Prediction and Analysis in the Behavioral Sciences
Werbos, P. J. (1974) · Medical Entomology and Zoology · Cited by 2,772 · see citing works
Beyond regression : new fools for prediction and analysis in the behavioral sciences
Werbos, P. J. (1974) · Medical Entomology and Zoology · Cited by 2,746 · see citing works
Simplified neuron model as a principal component analyzer
Oja, E. (1982) · Journal of Mathematical Biology · doi:10.1007/bf00275687 · Cited by 2,309 · see citing works
Optimal unsupervised learning in a single-layer linear feedforward neural network
Sanger, T. D. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90044-0 · Cited by 1,506 · see citing works
Neural networks and principal component analysis: Learning from examples without local minima
Baldi, P., & Hornik, K. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90014-2 · Cited by 1,383 · see citing works
Pattern Classification and Scene Analysis.
Grenander, U., Duda, R. O., & Hart, P. E. (1974) · Journal of the American Statistical Association · doi:10.2307/2286028 · Cited by 1,018 · see citing works
Learning distributed representations of concepts.
Hinton, G. E. (1989) · eScholarship (California Digital Library) · Cited by 815 · see citing works
Fast-learning variations on back propagation: an empirical study.
Fahlman, S. E. (1989) · Medical Entomology and Zoology · Cited by 724 · see citing works
A simple procedure for pruning back-propagation trained neural networks
Karnin, E. D. (1990) · IEEE Transactions on Neural Networks · doi:10.1109/72.80236 · Cited by 672 · see citing works
Application of the Karhunen-Loève Expansion to Feature Selection and Ordering
Fukunaga, K., & Koontz, W. L. G. (1970) · IEEE Transactions on Computers · doi:10.1109/t-c.1970.222918 · Cited by 537 · see citing works
Skeletonization: A Technique for Trimming the Fat from a Network via Relevance Assessment
Mozer, M. C., & Smolensky, P. (1988) · neural information processing systems · Cited by 530 · see citing works
Artificial neural network models of knowledge representation in chemical engineering
Hoskins, J. C., & Himmelblau, D. M. (1988) · Computers & Chemical Engineering · doi:10.1016/0098-1354(88)87015-7 · Cited by 416 · see citing works
A neural network methodology for process fault diagnosis
Venkatasubramanian, V., & Chan, K. Y. (1989) · AIChE Journal · doi:10.1002/aic.690351210 · Cited by 394 · see citing works
Baggaley, A. R. (1981) · The Personnel and Guidance Journal · doi:10.1002/j.2164-4918.1981.tb00489.x · Cited by 372 · see citing works
Modular learning in neural networks
Ballard, D. H. (1987) · National Conference on Artificial Intelligence · Cited by 362 · see citing works
Improvement of the backpropagation algorithm for training neural networks
Leonard, J. A., & Kramer, M. A. M. (1990) · Computers & Chemical Engineering · doi:10.1016/0098-1354(90)87070-6 · Cited by 287 · see citing works
Self-citationAdaptive network for optimal linear feature extraction
Földiák, P. (1989) · doi:10.1109/ijcnn.1989.118615 · Cited by 275 · see citing works
A unified approach to physical property estimation using multivariate statistical techniques
Joback, K. G. (1984) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 175 · see citing works
Pattern Recognition and Artificial Intelligence
El‐Yacoubi, M. A., Granger, E., Yuen, P. C., Pal, U., & Vincent, N. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-09037-0 · Cited by 161 · see citing works
A Neural Network Structure for System Identification
Haesloop, D., & Holt, B. R. (1990) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc.1990.4791170 · Cited by 49 · see citing works
Mapping Techniques for Exploratory Pattern Analysis
Siedlecki, W., Siedlecka, K., & Sklansky, J. (1988) · Machine intelligence and pattern recognition · doi:10.1016/b978-0-444-87137-4.50025-4 · Cited by 8 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570 · see citing works
Nonlinear Dimensionality Reduction by Locally Linear Embedding
Roweis, S. T., & Saul, L. K. (2000) · Science · doi:10.1126/science.290.5500.2323 · Cited by 14,965 · see citing works
A Global Geometric Framework for Nonlinear Dimensionality Reduction
Tenenbaum, J. B., Silva, V. D., & Langford, J. C. (2000) · Science · doi:10.1126/science.290.5500.2319 · Cited by 13,585 · see citing works
Anomaly Detection Using Autoencoders with Nonlinear Dimensionality Reduction
Sakurada, M., & Yairi, T. (2014) · Proceedings of the MLSDA 2014 2nd Workshop on Machine Learning for Sensory Data Analysis · doi:10.1145/2689746.2689747 · Cited by 1,438 · see citing works
Drug discovery with explainable artificial intelligence
Jiménez-Luna, J., Grisoni, F., & Schneider, G. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-00236-4 · Cited by 1,063 · see citing works
Federated learning on non-IID data: A survey
Zhu, H., Xu, J., Liu, S., & Jin, Y. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.07.098 · Cited by 1,062 · see citing works
A Survey on the Explainability of Supervised Machine Learning
Burkart, N., & Huber, M. F. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.12228 · Cited by 1,008 · see citing works
Effects of environmental and operational variability on structural health monitoring
Sohn, H. (2007) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2006.1935 · Cited by 883 · see citing works
Learning and inference in the brain
Friston, K. (2003) · Neural Networks · doi:10.1016/j.neunet.2003.06.005 · Cited by 724 · see citing works
Generalized Out-of-Distribution Detection: A Survey
Yang, J., Zhou, K., Li, Y., & Liu, Z. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02117-4 · Cited by 884 · see citing works
Advances and opportunities in machine learning for process data analytics
Qin, S. J., & Chiang, L. H. (2019) · Computers & Chemical Engineering · doi:10.1016/j.compchemeng.2019.04.003 · Cited by 351 · see citing works
A survey on deep learning-based non-invasive brain signals: recent advances and new frontiers
Zhang, X., Yao, L., Wang, X., Monaghan, J., McAlpine, D., & Zhang, Y. (2021) · Journal of Neural Engineering · doi:10.1088/1741-2552/abc902 · Cited by 296 · see citing works
PHOTOMETRIC SUPERNOVA CLASSIFICATION WITH MACHINE LEARNING
Lochner, M., McEwen, J. D., Peiris, H. V., Lahav, O., & Winter, M. K. (2016) · The Astrophysical Journal Supplement Series · doi:10.3847/0067-0049/225/2/31 · Cited by 200 · see citing works
DeepMicro: Deep representation learning for disease prediction based on microbiome data
Oh, M., & Zhang, L. (2020) · Scientific Reports · doi:10.1038/s41598-020-63159-5 · Cited by 190 · see citing works
Structural health monitoring research under varying temperature condition: a review
Han, Q., Ma, Q., Xu, J., & Liu, M. (2021) · Journal of Civil Structural Health Monitoring · doi:10.1007/s13349-020-00444-x · Cited by 190 · see citing works
Modal identification of bridges under varying environmental conditions: Temperature and wind effects
Li, H., Li, S., Ou, J., & Li, H. (2009) · Structural Control and Health Monitoring · doi:10.1002/stc.319 · Cited by 168 · see citing works
DISCRETE- vs. CONTINUOUS-TIME NONLINEAR SIGNAL PROCESSING OF Cu ELECTRODISSOLUTION DATA
RICO-MARTÍNEZ, R., KRISCHER, K., KEVREKIDIS, I. G., KUBE, M. C., & HUDSON, J. L. (1992) · Chemical Engineering Communications · doi:10.1080/00986449208936084 · Cited by 144 · see citing works
Machine-learning-based anomaly detection in optical fiber monitoring
Abdelli, K., Cho, J. Y., Azendorf, F., Griesser, H., Tropschug, C., & Pachnicke, S. (2022) · Journal of Optical Communications and Networking · doi:10.1364/jocn.451289 · Cited by 149 · see citing works
Challenges in the Development of Soft Sensors for Bioprocesses: A Critical Review
Brunner, V., Siegl, M., Geier, D., & Becker, T. (2021) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2021.722202 · Cited by 100 · see citing works
Villar, V. A., Hosseinzadeh, G., Berger, E., Ntampaka, M., Jones, D. O., Challis, P., Chornock, R., Drout, M. R., Foley, R. J., Kirshner, R. P., et al. (2020) · The Astrophysical Journal · doi:10.3847/1538-4357/abc6fd · Cited by 92 · see citing works
Population-based heteropolymer design to mimic protein mixtures
Ruan, Z., Li, S., Grigoropoulos, A., Amiri, H., Hilburg, S. L., Chen, H., Jayapurna, I., Jiang, T., Gu, Z., Alexander-Katz, A., et al. (2023) · Nature · doi:10.1038/s41586-022-05675-0 · Cited by 64 · see citing works
A model of egocentric to allocentric understanding in mammalian brains
Uria, B., Ibarz, B., Banino, A., Zambaldi, V., Kumaran, D., Hassabis, D., Barry, C., & Blundell, C. (2020) · doi:10.1101/2020.11.11.378141 · Cited by 48 · see citing works
Kohiyama, M., Oka, K., & Yamashita, T. (2020) · Structural Control and Health Monitoring · doi:10.1002/stc.2552 · Cited by 38 · see citing works
Policy manifold search: exploring the manifold hypothesis for diversity-based neuroevolution
Rakicevic, N., Cully, A., & Kormushev, P. (2021) · Proceedings of the Genetic and Evolutionary Computation Conference · doi:10.1145/3449639.3459320 · Cited by 32 · see citing works
Cadini, F., Lomazzi, L., Ferrater Roca, M., Sbarufatti, C., & Giglio, M. (2022) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2021.108048 · Cited by 25 · see citing works
Handling Missing Sensors in Topology-Aware IoT Applications with Gated Graph Neural Network
Liu, S., Yao, S., Huang, Y., Liu, D., Shao, H., Zhao, Y., Li, J., Wang, T., Wang, R., Yang, C., et al. (2020) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3411818 · Cited by 20 · see citing works
Entezami, A., De Michele, C., Arslan, A. N., & Behkamal, B. (2022) · Sensors · doi:10.3390/s22134964 · Cited by 19 · see citing works
Predicting chemotherapy response using a variational autoencoder approach
Wei, Q., & Ramsey, S. A. (2021) · BMC Bioinformatics · doi:10.1186/s12859-021-04339-6 · Cited by 19 · see citing works
Thorn, A. (2022) · Current Opinion in Structural Biology · doi:10.1016/j.sbi.2022.102368 · Cited by 16 · see citing works
In-context Autoencoder for Context Compression in a Large Language Model
Ge, T., Hu, J., Wang, L., Wang, X., Chen, S.-Q., & Wei, F. (2023) · arXiv · doi:10.48550/arxiv.2307.06945 · Cited by 11 · see citing works
“Like a typical autoencoder (Kramer, 1991), ICAE consists of an encoder and a decoder.”
MethodInterpretable representation learning for 3D multi-piece intracellular structures using point clouds
Vasan, R., Ferrante, A. J., Borensztejn, A., Frick, C. L., Garrison, P., Gaudreault, N., Mogre, S. S., Mohammed, F. S., Morris, B., Pires, G. G., et al. (2025) · Nature Methods · doi:10.1038/s41592-025-02729-9 · Cited by 7 · see citing works
Sheng, T., & Huber, M. (2025) · Sensors · doi:10.3390/s25134032 · Cited by 5 · see citing works
Kenfack, C. S., Mkankam, F. K., Alory, G., Penhoat, Y. D., Hounkonnou, M. N., Vondou, D. A., & Jr., B. G. N. (2017) · Terrestrial, Atmospheric and Oceanic Sciences · doi:10.3319/tao.2016.08.29.01 · Cited by 5 · see citing works
Sheng, T., & Huber, M. (2025) · 2025 International Conference on Machine Learning and Applications (ICMLA) · doi:10.1109/icmla66185.2025.00049
- Authors
- Kramer, M. A.
- Affiliations
- Massachusetts Institute of Technology (US)
- Corresponding author
- Kramer, Mark A.
- ORCID
- Kramer, Mark A.
- Venue
- AIChE Journal
- Publisher
- Wiley
- Published
- 1991-02
- Volume
- 37
- Issue
- 2
- Pages
- 233-243
- ISSN
- 0001-1541 (ISSN-L)
- 1547-5905
- OpenAlex
- W2122538988
- Cited by
- 3,037
- References
- 29
- Citations by year
- 2026: 68 · 2025: 171 · 2024: 250 · 2023: 279 · 2022: 256 · 2021: 322 · 2020: 229 · 2019: 98 · 2018: 72 · 2017: 75 · 2016: 65 · 2015: 60 · 2014: 63 · 2013: 79 · 2012: 81
- FWCI
- 9.61
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- AIChE Journal (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Engineering › Control and Systems Engineering › Fault Detection and Control Systems
- Physical Sciences › Chemistry › Analytical Chemistry › Spectroscopy and Chemometric Analyses
- Keywords
- nonlinear principal component analysis, autoencoder, dimensionality reduction, nonlinear correlation, feedforward neural networks, bottleneck layer, feature space mapping, multivariate analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
