The DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index.
Information geometry on hierarchy of probability distributions
Abstract
An exponential family or mixture family of probability distributions has a natural hierarchical structure. This paper gives an "orthogonal" decomposition of such a system based on information geometry. A typical example is the decomposition of stochastic dependency among a number of random variables. In general, they have a complex structure of dependencies. Pairwise dependency is easily represented by correlation, but it is more difficult to measure effects of pure triplewise or higher order interactions (dependencies) among these variables. Stochastic dependency is decomposed quantitatively into an "orthogonal" sum of pairwise, triplewise, and further higher order dependencies. This gives a new invariant decomposition of joint entropy. This problem is important for extracting intrinsic interactions in firing patterns of an ensemble of neurons and for estimating its functional connections. The orthogonal decomposition is given in a wide class of hierarchical structures including both exponential and mixture families. As an example, we decompose the dependency in a higher order Markov chain into a sum of those in various lower order Markov chains.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Elements of Information Theory
Cover, T. M., & Thomas, J. A. (2001) · Wiley eBooks · doi:10.1002/0471200611 · Cited by 36,808
Elements of Information Theory
Cover, T. M., & Thomas, J. A. (2005) · doi:10.1002/047174882x · Cited by 13,791
Hapeman, D. E. (1991) · Technometrics · doi:10.1080/00401706.1991.10484817 · Cited by 9,373
Discrete Multivariate Analysis: Theory and Practice.
Beniger, J. R., Bishop, Y. M. M., Feinberg, S. E., Holland, P. W., Light, R. J., & Mosteller, F. (1975) · Contemporary Sociology · doi:10.2307/2063625 · Cited by 5,108
Wiley Series in Probability and Statistics
(2008) · Wiley series in probability and statistics · doi:10.1002/series1345 · Cited by 4,217
Lauritzen, S. L. (1996) · doi:10.1093/oso/9780198522195.001.0001 · Cited by 2,976
Natural Gradient Works Efficiently in Learning
Amari, S.-I. (1998) · Neural Computation · doi:10.1162/089976698300017746 · Cited by 2,681
Self-citationFleiss, J. L., & Agresti, A. (1991) · Journal of the American Statistical Association · doi:10.2307/2290537 · Cited by 2,641
Methods of Information Geometry
Amari, S.-I., & Nagaoka, H. (2007) · Translations of Mathematical Monographs · doi:10.1090/mmono/191 · Cited by 2,204
Self-citationInformation and the Accuracy Attainable in the Estimation of Statistical Parameters
Rao, C. R. (1992) · Springer Series in Statistics · doi:10.1007/978-1-4612-0919-5_16 · Cited by 1,808
Differential-Geometrical Methods in Statistics
Amari, S.-I. (1985) · Lecture Notes in Statistics · doi:10.1007/978-1-4612-5056-2 · Cited by 1,673
$I$-Divergence Geometry of Probability Distributions and Minimization Problems
Csiszar, I. (1975) · The Annals of Probability · doi:10.1214/aop/1176996454 · Cited by 1,645
On the rationale of maximum-entropy methods
Jaynes, E. T. (1982) · Proceedings of the IEEE · doi:10.1109/proc.1982.12425 · Cited by 1,297
Information and Exponential Families in Statistical Theory.
Kingman, J. F. C., & Barndorff-Nielsen, O. (1979) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2344658 · Cited by 1,173
Dynamics of neuronal firing correlation: modulation of "effective connectivity"
Aertsen, A. M., Gerstein, G. L., Habib, M. K., & Palm, G. (1989) · Journal of Neurophysiology · doi:10.1152/jn.1989.61.5.900 · Cited by 1,084
Statistical Decision Rules and Optimal Inference
Čencov, N. (2000) · Translations of Mathematical Monographs · doi:10.1090/mmono/053 · Cited by 522
Information and Exponential Families in Statistical Theory
Mathai, A. M. (1980) · Technometrics · doi:10.1080/00401706.1980.10486149 · Cited by 498
Information geometry of the EM and em algorithms for neural networks
Amari, S.-I. (1995) · Neural Networks · doi:10.1016/0893-6080(95)00003-8 · Cited by 331
Self-citationOn the significance of correlations among neuronal spike trains
Palm, G., Aertsen, A. M. H. J., & Gerstein, G. L. (1988) · Biological Cybernetics · doi:10.1007/bf00336885 · Cited by 250
Pistone, G., & Sempi, C. (1995) · The Annals of Statistics · doi:10.1214/aos/1176324311 · Cited by 232
Geometrical Foundations of Asymptotic Inference
Kass, R. E., & Vos, P. W. (1997) · Wiley Series in Probability and Statistics · doi:10.1002/9781118165980 · Cited by 226
Geometrical Foundations of Asymptotic Inference
Lindsay, B. G., Kass, R. E., & Vos, P. W. (1999) · Journal of the American Statistical Association · doi:10.2307/2670184 · Cited by 216
Statistical inference under multiterminal data compression
Han, T. S., & Amari, S. (1998) · IEEE Transactions on Information Theory · doi:10.1109/18.720540 · Cited by 215
Self-citationNonnegative entropy measures of multivariate symmetric correlations
Han, T. S. (1978) · Information and Control · doi:10.1016/s0019-9958(78)90275-9 · Cited by 210
Differential Geometry and Statistics
Murray, M. K., & Rice, J. W. (2017) · Routledge eBooks · doi:10.1201/9781315141268 · Cited by 201
Information geometry of Boltzmann machines
Amari, S., Kurata, K., & Nagaoka, H. (1992) · IEEE Transactions on Neural Networks · doi:10.1109/72.125867 · Cited by 193
Self-citationDifferential Geometry and Statistics
Murray, M. K., & Rice, J. W. (1993) · Springer eBooks · doi:10.1007/978-1-4899-3306-5 · Cited by 172
Conditional limit theorems under Markov conditioning
Csiszár, I., Cover, T. M., & Choi, B. (1987) · IEEE Transactions on Information Theory · doi:10.1109/tit.1987.1057385 · Cited by 99
Амари, Ш.-И. (1987) · Theory of Computing Systems · doi:10.1007/bf01692059 · Cited by 77
The Exponential Statistical Manifold: Mean Parameters, Orthogonality and Space Transformations
Pistone, G., & Rogantin, M. P. (1999) · Bernoulli · doi:10.2307/3318699 · Cited by 77
Information Geometry of Mean-Field Approximation
Tanaka, T. (2000) · Neural Computation · doi:10.1162/089976600300015213 · Cited by 71
Dualistic geometry of the manifold of higher-order neurons
Амари, Ш.-И. (1991) · Neural Networks · doi:10.1016/0893-6080(91)90040-c · Cited by 69
The relation between information theory and the differential geometry approach to statistics
Campbell, L. L. (1985) · Information Sciences · doi:10.1016/0020-0255(85)90050-7 · Cited by 67
Statistical inference under multiterminal rate restrictions: a differential geometric approach
Амари, Ш.-И., & Han, T. S. (1989) · IEEE Transactions on Information Theory · doi:10.1109/18.32118 · Cited by 64
Ohara, A., Nobuhide, S. U. D. A., & Амари, Ш.-И. (1996) · Linear Algebra and its Applications · doi:10.1016/0024-3795(94)00348-3 · Cited by 62
Fisher information under restriction of Shannon information in multi-terminal situations
Амари, Ш.-И. (1989) · Annals of the Institute of Statistical Mathematics · doi:10.1007/bf00057730 · Cited by 31
Model Dependence in Quantification of Spike Interdependence by Joint Peri-Stimulus Time Histogram
Ito, H., & Tsuji, S. (2000) · Neural Computation · doi:10.1162/089976600300015952 · Cited by 28
Cited by
MAP Estimation Via Agreement on Trees: Message-Passing and Linear Programming
Wainwright, M. J., Jaakkola, T. S., & Willsky, A. S. (2005) · IEEE Transactions on Information Theory · doi:10.1109/tit.2005.856938 · Cited by 654
The Structure of Multi-Neuron Firing Patterns in Primate Retina
Shlens, J., Field, G. D., Gauthier, J. L., Grivich, M. I., Petrusca, D., Sher, A., Litke, A. M., & Chichilnisky, E. J. (2006) · The Journal of Neuroscience · doi:10.1523/jneurosci.1282-06.2006 · Cited by 468
A New Class of Upper Bounds on the Log Partition Function
Wainwright, M. J., Jaakkola, T. S., & Willsky, A. S. (2005) · IEEE Transactions on Information Theory · doi:10.1109/tit.2005.850091 · Cited by 381
Network Information and Connected Correlations
Schneidman, E., Still, S., Berry, M. J., & Bialek, W. (2003) · Physical Review Letters · doi:10.1103/physrevlett.91.238701 · Cited by 317
Bridging the gaps between cameras
Makris, D., Ellis, T., & Black, J. (2004) · Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004. · doi:10.1109/cvpr.2004.1315165 · Cited by 311
Metadata
- Authors
- Amari, S.-I.
- Venue
- IEEE Transactions on Information Theory
- Published
- 2001-07
- Volume
- 47
- Issue
- 5
- Pages
- 1701-1711
- ISSN
- 0018-9448 (ISSN-L)
- OpenAlex
- W2097230740
- Cited by
- 401
- References
- 37
- FWCI
- 12.39
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.