Latent variable models in the era of industrial big data: Extension and beyond
Open accessThe DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
Abstract
A rich supply of data and innovative algorithms have made data-driven modeling a popular technique in modern industry. Among various data-driven methods, latent variable models (LVMs) and their counterparts account for a major share and play a vital role in many industrial modeling areas. LVM can be generally divided into statistical learning-based classic LVM and neural networks-based deep LVM (DLVM). We first discuss the definitions, theories and applications of classic LVMs in detail, which serves as both a comprehensive tutorial and a brief application survey on classic LVMs. Then we present a thorough introduction to current mainstream DLVMs with emphasis on their theories and model architectures, soon afterwards provide a detailed survey on industrial applications of DLVMs. The aforementioned two types of LVM have obvious advantages and disadvantages. Specifically, classic LVMs have concise principles and good interpretability, but their model capacity cannot address complicated tasks. Neural networks-based DLVMs have sufficient model capacity to achieve satisfactory performance in complex scenarios, but it comes at sacrifices in model interpretability and efficiency. Aiming at combining the virtues and mitigating the drawbacks of these two types of LVMs, as well as exploring non-neural-network manners to build deep models, we propose a novel concept called lightweight deep LVM (LDLVM). After proposing this new idea, the article first elaborates the motivation and connotation of LDLVM, then provides two novel LDLVMs, along with thorough descriptions on their principles, architectures and merits. Finally, outlooks and opportunities are discussed, including important open questions and possible research directions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Measuring Economic Policy Uncertainty*
Baker, S. R., Bloom, N., & Davis, S. J. (2016) · The Quarterly Journal of Economics · doi:10.1093/qje/qjw024 · Cited by 12,458
McLachlan, G., & Peel, D. (2000) · Wiley Series in Probability and Statistics · doi:10.1002/0471721182 · Cited by 10,991
PLS-regression: a basic tool of chemometrics
Wold, S., Sjöström, M., & Eriksson, L. (2001) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/s0169-7439(01)00155-1 · Cited by 10,084 · cited 1×
“Among them, PLS-Regression is the most widely used method, for more details about it, please refer to [84].”
MethodExtracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436
"General Intelligence," Objectively Determined and Measured
Spearman, C. (1904) · The American Journal of Psychology · doi:10.2307/1412107 · Cited by 6,207
Greedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841 · cited 1×
“Another deep structure of AE is called stacked AE (SAE) [136, 137, 138], which shares the same structure with the original deep AE but has a different pre-training approach.”
Background(2010) · Journal of Machine Learning Research · doi:10.5555/1756006.1953039 · Cited by 4,051
Probabilistic Principal Component Analysis
Tipping, M. E., & Bishop, C. M. (1999) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00196 · Cited by 3,679
Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review
Rawat, W., & Wang, Z. (2017) · Neural Computation · doi:10.1162/neco_a_00990 · Cited by 3,503 · cited 1×
“Such as convolutional neural networks (CNN) in image recognition [42], recurrent neural networks (RNN) and its variant long short-term memory (LSTM) [43] networks in natural language processing [44, 45], deep generative models like generative adversarial network (GAN) in data generation [46], and deep reinforcement le…”
BackgroundA learning algorithm for Boltzmann machines
Ackley, D. H., Hinton, G. E., & Sejnowski, T. J. (1985) · Cognitive Science · doi:10.1207/s15516709cog0901_7 · Cited by 3,402
A plant-wide industrial process control problem
Downs, J. J., & Vogel, E. F. (1993) · Computers & Chemical Engineering · doi:10.1016/0098-1354(93)80018-i · Cited by 3,348 · cited 1×
“The superiority of the proposed model are presented in [209] though evaluations on the TE process [210] (a benchmark for fault detection).”
MethodNICE: Non-linear Independent Components Estimation
Dinh, L., Krueger, D., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1410.8516 · Cited by 1,493 · cited 1×
“Popular augmentation approaches include sample interpolation [218], GAN [219, 220], flow-based model [221], etc.”
MethodA Survey of Machine and Deep Learning Methods for Internet of Things (IoT) Security
Al-Garadi, M. A., Mohamed, A., Al-Ali, A. K., Du, X., Ali, I., & Guizani, M. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2988293 · Cited by 1,392 · cited 1×
“The industry community has already developed many DL-based methods for modeling [28, 49, 50, 51, 52, 53].”
MethodThe "Wake-Sleep" Algorithm for Unsupervised Neural Networks
Hinton, G. E., Dayan, P., Frey, B. J., & Neal, R. M. (1995) · Science · doi:10.1126/science.7761831 · Cited by 1,070
Data Mining and Analytics in the Process Industry: The Role of Machine Learning
Ge, Z., Song, Z., Ding, S. X., & Huang, B. (2017) · IEEE Access · doi:10.1109/access.2017.2756872 · Cited by 922 · cited 1×
“Intelligent algorithms and models in industry have great significance in reducing production cost, improving production quality, ensuring plant safety, and promoting smart manufacturing [4, 5, 6].”
BackgroundSelf-citationShin, H.-C., Orton, M. R., Collins, D. J., Doran, S. J., & Leach, M. O. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.277 · Cited by 488 · cited 1×
“Another deep structure of AE is called stacked AE (SAE) [136, 137, 138], which shares the same structure with the original deep AE but has a different pre-training approach.”
BackgroundA Multi-Agent Reinforcement Learning-Based Data-Driven Method for Home Energy Management
Xu, X., Jia, Y., Xu, Y., Xu, Z., Chai, S., & Lai, C. S. (2020) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2020.2971427 · Cited by 420 · cited 2×
“In the past two decades, a lot of data-driven modeling and analyzing methods have been proposed for different industrial applications, such as semiconductor manufacturing [15, 16], injection molding [17, 18], smart grid [19, 20], wind energy forecasting [21, 22], predictive maintenance [23, 24], prognostics health man…”
BackgroundEstimating common trends in multivariate time series using dynamic factor analysis
Zuur, A. F., Fryer, R. J., Jolliffe, I. T., Dekker, R., & Beukema, J. J. (2003) · Environmetrics · doi:10.1002/env.611 · Cited by 311
Cited by
Perera, Y. S., Ratnaweera, D. A. A. C., Dasanayaka, C. H., & Abeykoon, C. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.105988 · Cited by 286
Mostert, C. M., Udeh-Momoh, C., Winkler, A. S., McLaughlin, C., Eyre, H., Salama, M., Ranchod, K., Trepel, D., Vradenburg, G., Hynes, W., et al. (2025) · eBioMedicine · doi:10.1016/j.ebiom.2025.105950 · Cited by 42
Artificial intelligence for dementia prevention
Newby, D., Orgeta, V., Marshall, C. R., Lourida, I., Albertyn, C. P., Tamburin, S., Raymont, V., Veldsman, M., Koychev, I., Bauermeister, S., et al. (2023) · Alzheimer's & Dementia · doi:10.1002/alz.13463 · Cited by 42
Metadata
- Authors
- Kong, X., Jiang, X., Zhang, B., Yuan, J., & Ge, Z.
- Venue
- Annual Reviews in Control
- Publisher
- Elsevier BV
- Published
- 2022
- Volume
- 54
- Pages
- 167-199
- ISSN
- 1367-5788 (ISSN-L)
- OpenAlex
- W4300960307
- Cited by
- 121
- References
- 22
- FWCI
- 10.76
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.