The DOI matches its Crossref record; 67 of the 67 references it cites were found in a registry or an index.
Families of Alpha- Beta- and Gamma- Divergences: Flexible and Robust Measures of Similarities
Open accessAbstract
In this paper, we extend and overview wide families of Alpha-, Beta- and Gamma-divergences and discuss their fundamental properties. In literature usually only one single asymmetric (Alpha, Beta or Gamma) divergence is considered. We show in this paper that there exist families of such divergences with the same consistent properties. Moreover, we establish links and correspondences among these divergences by applying suitable nonlinear transformations. For example, we can generate the Beta-divergences directly from Alpha-divergences and vice versa. Furthermore, we show that a new wide class of Gamma-divergences can be generated not only from the family of Beta-divergences but also from a family of Alpha-divergences. The paper bridges these divergences and shows also their links to Tsallis and Rényi entropies. Most of these divergences have a natural information theoretic interpretation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
On Information and Sufficiency
Kullback, S., & Leibler, R. A. (1951) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729694 · Cited by 20,193
Possible generalization of Boltzmann-Gibbs statistics
Tsallis, C. (1988) · Journal of Statistical Physics · doi:10.1007/bf01016429 · Cited by 9,589
On Measures of Entropy and Information
Rényi, A. (1961) · Project Euclid (Cornell University) · Cited by 4,031
A Measure of Asymptotic Efficiency for Tests of a Hypothesis Based on the sum of Observations
Chernoff, H. (1952) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729330 · Cited by 3,550
Bregman, L. M. (1967) · USSR Computational Mathematics and Mathematical Physics · doi:10.1016/0041-5553(67)90040-7 · Cited by 2,618
An invariant form for the prior probability in estimation problems
Jeffreys, H. (1946) · Proceedings of the Royal Society of London. Series A. Mathematical and Physical Sciences · doi:10.1098/rspa.1946.0056 · Cited by 2,323
Methods of Information Geometry
Amari, S.-I., & Nagaoka, H. (2007) · Translations of Mathematical Monographs · doi:10.1090/mmono/191 · Cited by 2,204
Self-citationDifferential-Geometrical Methods in Statistics
Amari, S.-I. (1985) · Lecture Notes in Statistics · doi:10.1007/978-1-4612-5056-2 · Cited by 1,673
Cichocki, A., Zdunek, R., Phan, A. H., & Amari, S. (2009) · doi:10.1002/9780470747278 · Cited by 1,540
Self-citationClustering with Bregman Divergences
Banerjee, A., Merugu, S., Dhillon, I., & Ghosh, J. (2004) · Proceedings of the 2004 SIAM International Conference on Data Mining · doi:10.1137/1.9781611972740.22 · Cited by 1,472
Multinomial Goodness-Of-Fit Tests
Cressie, N., & Read, T. R. C. (1984) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1984.tb01318.x · Cited by 1,331
A General Class of Coefficients of Divergence of One Distribution from Another
Ali, S. M., & Silvey, S. D. (1966) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1966.tb00626.x · Cited by 1,205
Févotte, C., Bertin, N., & Durrieu, J.-L. (2009) · Neural Computation · doi:10.1162/neco.2008.04-08-771 · Cited by 1,119
Information Theoretic Learning: Renyi's Entropy and Kernel Perspectives
Principe, J. C. (2010) · Information Science and Statistics · doi:10.1007/978-1-4419-1570-2 · Cited by 939
(1967) · Cited by 853
Neue Begründung der Theorie quadratischer Formen von unendlichvielen Veränderlichen.
Hellinger, E. (1909) · Journal für die reine und angewandte Mathematik · doi:10.1515/crll.1909.136.210 · Cited by 804
Goodness-of-fit statistics for discrete multivariate data
(1989) · Choice Reviews Online · doi:10.5860/choice.26-2740 · Cited by 738
Rényi, A. (2007) · Cited by 727
Fast Local Algorithms for Large Scale Nonnegative Matrix and Tensor Factorizations
CICHOCKI, A., & PHAN, A.-H. (2009) · IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences · doi:10.1587/transfun.e92.a.708 · Cited by 575
Self-citationDivergence measures and message passing
Minka, T. P. (2005) · Cited by 477
Theory of statistical inference and information
Vajda, I. (1989) · Medical Entomology and Zoology · Cited by 463
Liese, F., & Vajda, I. (1987) · Cited by 452
On the convexity of some divergence measures based on entropy functions
Burbea, J., & Rao, C. (1982) · IEEE Transactions on Information Theory · doi:10.1109/tit.1982.1056497 · Cited by 423
Generalized Nonnegative Matrix Approximations with Bregman Divergences
Sra, S., & Dhillon, I. S. (2005) · neural information processing systems · Cited by 395
Some inequalities for information divergence and related measures of discrimination
Topsoe, F. (2000) · IEEE Transactions on Information Theory · doi:10.1109/18.850703 · Cited by 325
A Generalized Maximum Entropy Approach to Bregman Co-clustering and Matrix Approximation
Banerjee, A., Dhillon, I. S., Ghosh, J., et al. (2004) · Cited by 303
Axiomatic Characterizations of Information Measures
Csiszár, I. (2008) · Entropy · doi:10.3390/e10030261 · Cited by 282
Burbea, J., & Rao, C. R. (1982) · Journal of Multivariate Analysis · doi:10.1016/0047-259x(82)90065-3 · Cited by 274
Csiszár’s Divergences for Non-negative Matrix Factorization: Family of New Algorithms
Cichocki, A., Zdunek, R., & Amari, S.-I. (2006) · Lecture Notes in Computer Science · doi:10.1007/11679363_5 · Cited by 271
Self-citationRobust parameter estimation with a small bias against heavy contamination
Fujisawa, H., & Eguchi, S. (2008) · Journal of Multivariate Analysis · doi:10.1016/j.jmva.2008.02.004 · Cited by 267
Applications of entropic spanning graphs
Hero, A. O., Ma, B., Michel, O. J. J., & Gorman, J. (2002) · IEEE Signal Processing Magazine · doi:10.1109/msp.2002.1028355 · Cited by 266
$\alpha$-Divergence Is Unique, Belonging to Both $f$-Divergence and Bregman Divergence Classes
Амари, Ш.-И. (2009) · IEEE Transactions on Information Theory · doi:10.1109/tit.2009.2030485 · Cited by 230
Markov Processes and the H-Theorem
Morimoto, T. (1963) · Journal of the Physical Society of Japan · doi:10.1143/jpsj.18.328 · Cited by 216
Information geometry of divergence functions
Амари, Ш.-И., & Cichocki, A. S. (2010) · Bulletin of the Polish Academy of Sciences Technical Sciences · doi:10.2478/v10175-010-0019-1 · Cited by 202
Self-citationOn the Foundations of Information Theory
Rényi, A., & Renyi, A. (1965) · Revue de l'Institut International de Statistique / Review of the International Statistical Institute · doi:10.2307/1401301 · Cited by 199
A Generalized Divergence Measure for Nonnegative Matrix Factorization
Kompass, R. (2007) · Neural Computation · doi:10.1162/neco.2007.19.3.780 · Cited by 191
Information Geometry of U-Boost and Bregman Divergence
Murata, N., Takenouchi, T., Kanamori, T., & Eguchi, S. (2004) · Neural Computation · doi:10.1162/089976604323057452 · Cited by 176
Robust Blind Source Separation by Beta Divergence
Mihoko, M., & Eguchi, S. (2002) · Neural Computation · doi:10.1162/089976602760128045 · Cited by 173
Divergence Function, Duality, and Convex Analysis
Zhang, J. (2004) · Neural Computation · doi:10.1162/08997660460734047 · Cited by 163
Sided and Symmetrized Bregman Centroids
Nielsen, F., & Nock, R. (2009) · IEEE Transactions on Information Theory · doi:10.1109/tit.2009.2018176 · Cited by 163
Non-negative matrix factorization with α-divergence
Cichocki, A. S., Lee, H.-K., Kim, Y., & Choi, S. (2008) · Pattern Recognition Letters · doi:10.1016/j.patrec.2008.02.016 · Cited by 145
Self-citationBoissonnat, J., Nielsen, F., & Nock, R. (2010) · Discrete & Computational Geometry · doi:10.1007/s00454-010-9256-1 · Cited by 142
A generalized maximum entropy approach to bregman co-clustering and matrix approximation
Banerjee, A., Dhillon, I. S., Ghosh, J., Merugu, S., & Modha, D. S. (2004) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1014052.1014111 · Cited by 137
Hilbertian Metrics and Positive Definite Kernels on Probability Measures
Hein, M., & Bousquet, O. (2005) · Cited by 135
Extended SMART Algorithms for Non-negative Matrix Factorization
Cichocki, A. S., Амари, Ш.-И., Zdunek, R., Kompass, R., Hori, G., & He, Z. (2006) · Lecture notes in computer science · doi:10.1007/11785231_58 · Cited by 120
Self-citationIntegration of Stochastic Models by Minimizing α-Divergence
Амари, Ш.-И. (2007) · Neural Computation · doi:10.1162/neco.2007.19.10.2780 · Cited by 117
New Developments in Generalized Information Measures
Taneja, I. J. (1995) · Advances in imaging and electron physics · doi:10.1016/s1076-5670(08)70106-x · Cited by 110
Nonnegative Matrix Factorisations As Probabilistic Inference In Composite Models
Cemgil, A. T., & Févotte, C. (2009) · doi:10.5281/zenodo.41636 · Cited by 106
Non-Negative Tensor Factorization using Alpha and Beta Divergences
Cichocki, A. S., Zdunek, R., Choi, S., Plemmons, R. J., & Амари, Ш.-И. (2007) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2007.367106 · Cited by 98
Self-citationEntropy: The Markov Ordering Approach
Gorban, A. N., Gorban, P. A., & Judge, G. (2010) · Entropy · doi:10.3390/e12051145 · Cited by 83
Additive models, boosting, and inference for generalized divergences
Lafferty, J. (1999) · Conference on Learning Theory · doi:10.1145/307400.307422 · Cited by 78
Dualistic geometry of the manifold of higher-order neurons
Амари, Ш.-И. (1991) · Neural Networks · doi:10.1016/0893-6080(91)90040-c · Cited by 69
Entropy: The Markov Ordering Approach
Gorban, A. N., Gorban, P. A., & Judge, G. (2010) · Cited by 60
Flexible HALS algorithms for sparse non-negative matrix/tensor factorization
Cichocki, A. S., Phan, A. H., & Caiafa, C. F. (2008) · Machine learning for signal processing ... · doi:10.1109/mlsp.2008.4685458 · Cited by 53
Self-citationNovel Multi-layer Non-negative Tensor Factorization with Sparsity Constraints
Cichocki, A. S., Zdunek, R., Choi, S., Plemmons, R. J., & Амари, Ш.-И. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-71629-7_31 · Cited by 47
Self-citationEntropy and Divergence Associated with Power Function and the Statistical Application
Eguchi, S., & Kato, S. (2010) · Entropy · doi:10.3390/e12020262 · Cited by 47
Information Geometry and Its Applications: Convex Function and Dually Flat Manifold
Амари, Ш.-И. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-00826-9_4 · Cited by 39
Robust extraction of local structures by the minimum -divergence method
Mollah, M. N. H., Sultana, N., Minami, M., & Eguchi, S. (2009) · Neural Networks · doi:10.1016/j.neunet.2009.11.011 · Cited by 37
A note on curvature of α-connections of a statistical manifold
Zhang, J. (2007) · Annals of the Institute of Statistical Mathematics · doi:10.1007/s10463-006-0105-1 · Cited by 32
Bayesian invariant measurements of generalization
Zhu, H., & Rohwer, R. (1995) · Neural Processing Letters · doi:10.1007/bf02309013 · Cited by 30
Measurements of Generalisation Based on Information Geometry
Zhu, H., & Rohwer, R. (1997) · Operations research, computer science. Interface series · doi:10.1007/978-1-4615-6099-9_69 · Cited by 20
A generalization of the Kullback–Leibler divergence and its properties
Yamano, T. (2009) · Journal of Mathematical Physics · doi:10.1063/1.3116115 · Cited by 18
Learning α-integration with partially-labeled data
Choi, H., Choi, S., Katake, A. B., & Choe, Y. (2010) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2010.5495025 · Cited by 15
ON EM-LIKE ALGORITHMS FOR MINIMUM DISTANCE ESTIMATION
Eggermont, P. P. B., & LaRiccia, V. N. (1998) · Cited by 12
Divergence Based Online Learning in Vector Quantization
Villmann, T., Haase, S., Schleif, F.-M., & Hammer, B. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-13208-7_60 · Cited by 7
Villmann, T., & Haase, S. (2010) · Cited by 7
Mathematical Aspects of Divergence Based Vector Quantization Using Fr´ echet-Derivatives
Villmann, T., & Haase, S. (2009) · Cited by 5
Cited by
Nonnegative Matrix Factorization: A Comprehensive Review
Wang, Y.-X., & Zhang, Y.-J. (2013) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2012.51 · Cited by 981
Algorithms for Nonnegative Matrix Factorization with the β-Divergence
Févotte, C., & Idier, J. (2011) · Neural Computation · doi:10.1162/neco_a_00168 · Cited by 663
Automatic Relevance Determination in Nonnegative Matrix Factorization with the /spl beta/-Divergence
Tan, V. Y. F., & Fevotte, C. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.240 · Cited by 271
Predictive Coding: a Theoretical and Experimental Review
Millidge, B., Seth, A., & Buckley, C. L. (2021) · arXiv · doi:10.48550/arxiv.2107.12979 · Cited by 106
Indefinite core vector machine
Schleif, F.-M., & Tino, P. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2017.06.003 · Cited by 26
Metadata
- Authors
- Cichocki, A., & Amari, S.-I.
- Venue
- Entropy
- Publisher
- MDPI AG
- Published
- 2010-06-14
- Volume
- 12
- Issue
- 6
- Pages
- 1532-1568
- ISSN
- 1099-4300 (ISSN-L)
- OpenAlex
- W2150070703
- Cited by
- 459
- References
- 67
- FWCI
- 13.63
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.