The DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
Independent Factor Analysis
Abstract
We introduce the independent factor analysis (IFA) method for recovering independent hidden sources from their observed mixtures. IFA generalizes and unifies ordinary factor analysis (FA), principal component analysis (PCA), and independent component analysis (ICA), and can handle not only square noiseless mixing but also the general case where the number of mixtures differs from the number of sources and the data are noisy. IFA is a two-step procedure. In the first step, the source densities, mixing matrix, and noise covariance are estimated from the observed data by maximum likelihood. For this purpose we present an expectation-maximization (EM) algorithm, which performs unsupervised learning of an associated probabilistic model of the mixing situation. Each source in our model is described by a mixture of gaussians; thus, all the probabilistic calculations can be performed analytically. In the second step, the sources are reconstructed from the observed data by an optimal nonlinear estimator. A variational approximation of this algorithm is derived for cases with a large number of sources, where the exact algorithm becomes intractable. Our IFA algorithm reduces to the one for ordinary FA when the sources become gaussian, and to an EM algorithm for PCA in the zero-noise limit. We derive an additional EM algorithm specifically for noiseless IFA. This algorithm is shown to be superior to ICA since it can learn arbitrary source densities from the data. Beyond blind separation, IFA can be used for modeling multidimensional data by a highly constrained mixture of gaussians and as a tool for nonlinear signal encoding.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444
Elements of Information Theory
Cover, T. M., & Thomas, J. A. (2001) · Wiley eBooks · doi:10.1002/0471200611 · Cited by 36,808
An Information-Maximization Approach to Blind Separation and Blind Deconvolution
Bell, A. J., & Sejnowski, T. J. (1995) · Neural Computation · doi:10.1162/neco.1995.7.6.1129 · Cited by 8,986
Independent component analysis, A new concept?
Comon, P. (1994) · Signal Processing · doi:10.1016/0165-1684(94)90029-9 · Cited by 8,205
Brown, R. A., Hastie, T. J., & Tibshirani, R. J. (1991) · Biometrics · doi:10.2307/2532174 · Cited by 8,176
Emergence of simple-cell receptive field properties by learning a sparse code for natural images
Olshausen, B. A., & Field, D. J. (1996) · Nature · doi:10.1038/381607a0 · Cited by 5,694 · cited 1×
“The resulting algorithm was used to derive efficient codes for images and sounds (Lewicki & Olshausen, 1998) and was put forth as a computational model for interpreting neural responses in V1 in the efficient coding framework (Olshausen & Field, 1996, 1997).”
MethodMacKay, D. J. C. (1992) · Neural Computation · doi:10.1162/neco.1992.4.3.415 · Cited by 4,388
Probabilistic Principal Component Analysis
Tipping, M. E., & Bishop, C. M. (1999) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/1467-9868.00196 · Cited by 3,679
Sparse coding with an overcomplete basis set: A strategy employed by V1?
Olshausen, B. A., & Field, D. J. (1997) · Vision Research · doi:10.1016/s0042-6989(97)00169-7 · Cited by 3,642
Journal of the Royal Statistical Society (B)
Fildes, R. (1993) · International Journal of Forecasting · doi:10.1016/0169-2070(93)90088-5 · Cited by 3,437
A Fast Fixed-Point Algorithm for Independent Component Analysis
Hyvärinen, A., & Oja, E. (1997) · Neural Computation · doi:10.1162/neco.1997.9.7.1483 · Cited by 3,391
Hierarchical Mixtures of Experts and the EM Algorithm
Jordan, M. I., & Jacobs, R. A. (1994) · Neural Computation · doi:10.1162/neco.1994.6.2.181 · Cited by 2,673
Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture
Jutten, C., & Herault, J. (1991) · Signal Processing · doi:10.1016/0165-1684(91)90079-x · Cited by 2,521
Friedman, J. H., & Stuetzle, W. (1981) · Journal of the American Statistical Association · doi:10.1080/01621459.1981.10477729 · Cited by 2,216
A View of the Em Algorithm that Justifies Incremental, Sparse, and other Variants
Neal, R. M., & Hinton, G. E. (1998) · Learning in Graphical Models · doi:10.1007/978-94-011-5014-9_12 · Cited by 2,187
A New Learning Algorithm for Blind Signal Separation
Амари, Ш.-И., Cichocki, A. S., & Yang, H. H. (1995) · Cited by 1,904
GTM: The Generative Topographic Mapping
Bishop, C. M., Svensén, M., & Williams, C. K. I. (1998) · Neural Computation · doi:10.1162/089976698300017953 · Cited by 1,388
Equivariant adaptive source separation
Cardoso, J.-F., & Laheld, B. H. (1996) · IEEE Transactions on Signal Processing · doi:10.1109/78.553476 · Cited by 1,349
Huber, P. J. (1985) · The Annals of Statistics · doi:10.1214/aos/1176349519 · Cited by 1,277
Dayan, P., Hinton, G. E., Neal, R. M., & Zemel, R. S. (1995) · Neural Computation · doi:10.1162/neco.1995.7.5.889 · Cited by 1,207
(1993) · Cited by 1,173
Factorial Hidden Markov Models
Ghahramani, Z., & Jordan, M. I. (1997) · Machine Learning · doi:10.1023/a:1007425814087 · Cited by 1,157
The "Wake-Sleep" Algorithm for Unsupervised Neural Networks
Hinton, G. E., Dayan, P., Frey, B. J., & Neal, R. M. (1995) · Science · doi:10.1126/science.7761831 · Cited by 1,070
EM Algorithms for PCA and SPCA
Roweis, S. T. (1997) · neural information processing systems · Cited by 1,018
A Unifying Review of Linear Gaussian Models
Roweis, S., & Ghahramani, Z. (1999) · Neural Computation · doi:10.1162/089976699300016674 · Cited by 869
Infomax and maximum likelihood for blind source separation
Cardoso, J.-F. (1997) · IEEE Signal Processing Letters · doi:10.1109/97.566704 · Cited by 684
EM Algorithms for ML Factor Analysis
Rubin, D. B., & Thayer, D. T. (1982) · Psychometrika · doi:10.1007/bf02293851 · Cited by 603
An Introduction to Latent Variable Models
Everitt, B. S. (1984) · Springer eBooks · doi:10.1007/978-94-009-5564-6 · Cited by 523
Blind separation of sources, part II: Problems statement
Comon, P., Jutten, C., & Herault, J. (1991) · Signal Processing · doi:10.1016/0165-1684(91)90080-3 · Cited by 394
Mean Field Theory for Sigmoid Belief Networks
Saul, L. K., Jaakkola, T., & Jordan, M. I. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.251 · Cited by 327
Probabilistic framework for the adaptation and comparison of image codes
Lewicki, M. S., & Olshausen, B. A. (1999) · Journal of the Optical Society of America A · doi:10.1364/josaa.16.001587 · Cited by 308
Blind separation of convolved sources based on information maximization
Torkkola, K. (2002) · Neural Networks for Signal Processing VI. Proceedings of the 1996 IEEE Signal Processing Society Workshop · doi:10.1109/nnsp.1996.548372 · Cited by 238
Lee, T.-W., Lewicki, M. S., & Sejnowski, T. J. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.879789 · Cited by 218
Blind Separation of Time-Delayed and Convolved Sources
Lee, T.-W. (1998) · Independent Component Analysis · doi:10.1007/978-1-4757-2851-4_4 · Cited by 190
Blind separation of instantaneous mixture of sources via an independent component analysis
Pham, D. T. (1996) · IEEE Transactions on Signal Processing · doi:10.1109/78.542183 · Cited by 185
Maximum likelihood for blind separation and deconvolution of noisy signals using mixture models
Moulines, E., Cardoso, J.-F., & Gassiat, E. (2002) · 1997 IEEE International Conference on Acoustics, Speech, and Signal Processing · doi:10.1109/icassp.1997.604649 · Cited by 176
Maximum Likelihood Blind Source Separation: A Context-Sensitive Generalization of ICA
Pearlmutter, B. A., & Parra, L. C. (1996) · Cited by 143
Independent component analysis for noisy data — MEG data analysis
Ikeda, S., & Toyama, K. (2000) · Neural Networks · doi:10.1016/s0893-6080(00)00071-x · Cited by 139
Blind Source Separation and Deconvolution: The Dynamic Component Analysis Algorithm
Attias, H. T., & Schreiner, C. E. (1998) · Neural Computation · doi:10.1162/089976698300017223 · Cited by 125
Self-citationA unified model for probabilistic principal surfaces
Chang, K., & Ghosh, J. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.899944 · Cited by 122
Adaptive Elastic Models for Hand-Printed Character Recognition
Hinton, G. E., Williams, C. K. I., & Revow, M. D. (1991) · Cited by 84
Independent Component Analysis: A Flexible Nonlinearity and Decorrelating Manifold Approach
Everson, R., & Roberts, S. (1999) · Neural Computation · doi:10.1162/089976699300016043 · Cited by 84
Learning Nonlinear Overcomplete Representations for Efficient Coding
Lewicki, M. S., & B, T. J. S. A. (1997) · Cited by 72
Factorial Learning and the EM Algorithm
Ghahramani, Z. (1994) · Cited by 67
Inferring sparse, overcomplete image codes using an efficient coding framework
Lewicki, M. S., & Olshausen, B. A. (1998) · neural information processing systems · Cited by 42
Learning linear, sparse, factorial codes
Olshausen, B. A. (1996) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 40
The Bayesian Evidence Scheme for Regularizing Probability-Density Estimating Neural Networks
Husmeier, D. (2000) · Neural Computation · doi:10.1162/089976600300014890 · Cited by 28
Cited by
Probabilistic Independent Component Analysis for Functional Magnetic Resonance Imaging
Beckmann, C. F., & Smith, S. M. (2004) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2003.822821 · Cited by 2,715
Natural Image Statistics and Neural Representation
Simoncelli, E. P., & Olshausen, B. A. (2001) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.24.1.1193 · Cited by 2,460
Independent Component Analysis: Principles and Practice
(2001) · Cambridge University Press eBooks · doi:10.1017/cbo9780511624148 · Cited by 1,917
(2003) · Cited by 1,680
GTM: The Generative Topographic Mapping
Bishop, C. M., Svensén, M., & Williams, C. K. I. (1998) · Neural Computation · doi:10.1162/089976698300017953 · Cited by 1,388
Metadata
- Authors
- Attias, H.
- Venue
- Neural Computation
- Publisher
- MIT Press
- Published
- 1999-05-01
- Volume
- 11
- Issue
- 4
- Pages
- 803-851
- ISSN
- 0899-7667 (ISSN-L)
- 1530-888X
- OpenAlex
- W2137969290
- Cited by
- 504
- References
- 47
- FWCI
- 57.65
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.