The DOI matches its Crossref record.
Divergence measures based on the Shannon entropy
Abstract
A novel class of information-theoretic divergence measures based on the Shannon entropy is introduced. Unlike the well-known Kullback divergences, the new measures do not require the condition of absolute continuity to be satisfied by the probability distributions involved. More importantly, their close relationship with the variational distance and the probability of misclassification error are established in terms of bounds. These bounds are crucial in many applications of divergence measures. The measures are also well characterized by the properties of nonnegativity, finiteness, semiboundedness, and boundedness.>
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cited by
Comaniciu, D., Ramesh, V., & Meer, P. (2003) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2003.1195991 · Cited by 4,560
An atlas of active enhancers across human cell types and tissues
The FANTOM Consortium, Andersson, R., Gebhard, C., Miguel-Escalada, I., Hoof, I., Bornholdt, J., Boyd, M., Chen, Y., Zhao, X., Schmidl, C., et al. (2014) · Nature · doi:10.1038/nature12787 · Cited by 2,550
Is the Sample Good Enough? Comparing Data from Twitter's Streaming API with Twitter's Firehose
Morstatter, F., Pfeffer, J., Liu, H., & Carley, K. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v7i1.14401 · Cited by 863
Romero, R., Hassan, S. S., Gajer, P., Tarca, A. L., Fadrosh, D. W., Nikita, L., Galuppi, M., Lamont, R. F., Chaemsaithong, P., Miranda, J., et al. (2014) · Microbiome · doi:10.1186/2049-2618-2-4 · Cited by 852
Buganim, Y., Faddah, D. A., Cheng, A. W., Itskovich, E., Markoulaki, S., Ganz, K., Klemm, S. L., van Oudenaarden, A., & Jaenisch, R. (2012) · Cell · doi:10.1016/j.cell.2012.08.023 · Cited by 793
Domain Adaptation Problems: A DASVM Classification Technique and a Circular Validation Strategy
Bruzzone, L., & Marconcini, M. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.57 · Cited by 591
Integrated Morphoelectric and Transcriptomic Classification of Cortical GABAergic Cells
Gouwens, N. W., Sorensen, S. A., Baftizadeh, F., Budzillo, A., Lee, B. R., Jarsky, T., Alfiler, L., Baker, K., Barkan, E., Berry, K., et al. (2020) · Cell · doi:10.1016/j.cell.2020.09.057 · Cited by 538
Li, B., Zhang, W., Guo, C., Xu, H., Li, L., Fang, M., Hu, Y., Zhang, X., Yao, X., Tang, M., et al. (2022) · Nature Methods · doi:10.1038/s41592-022-01480-9 · Cited by 459
Self-citationWomen Also Snowboard: Overcoming Bias in Captioning Models
Hendricks, L. A., Burns, K., Saenko, K., Darrell, T., & Rohrbach, A. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01219-9_47 · Cited by 404
(1999) · Cited by 397
Kraus, M., Feuerriegel, S., & Oztekin, A. (2020) · European Journal of Operational Research · doi:10.1016/j.ejor.2019.09.018 · Cited by 392
Statistically Significant Detection of Linguistic Change
Kulkarni, V., Al-Rfou, R., Perozzi, B., & Skiena, S. (2015) · Proceedings of the 24th International Conference on World Wide Web · doi:10.1145/2736277.2741627 · Cited by 390
On the Jensen–Shannon Symmetrization of Distances Relying on Abstract Means
Nielsen, F. (2019) · Entropy · doi:10.3390/e21050485 · Cited by 257
Quantification of network structural dissimilarities
Schieber, T. A., Carpi, L., Díaz-Guilera, A., Pardalos, P. M., Masoller, C., & Ravetti, M. G. (2017) · Nature Communications · doi:10.1038/ncomms13928 · Cited by 237
A carbon-fiber electrode array for long-term neural recording
Guitchounts, G., Markowitz, J. E., Liberti, W. A., & Gardner, T. J. (2013) · Journal of Neural Engineering · doi:10.1088/1741-2560/10/4/046016 · Cited by 224
Diffusion Distance for Histogram Comparison
Ling, H., & Okada, K. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 1 (CVPR'06) · doi:10.1109/cvpr.2006.99 · Cited by 222
High-order social interactions in groups of mice
Shemesh, Y., Sztainberg, Y., Forkosh, O., Shlapobersky, T., Chen, A., & Schneidman, E. (2013) · eLife · doi:10.7554/elife.00759 · Cited by 213
Ball and rackets: inferring fiber fanning from diffusion-weighted MRI
Sotiropoulos, S. N., Behrens, T. E. J., & Jbabdi, S. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.01.056 · Cited by 156
A survey: Optimization and applications of evidence fusion algorithm based on Dempster–Shafer theory
Zhao, K., Li, L., Chen, Z., Sun, R., Yuan, G., & Li, J. (2022) · Applied Soft Computing · doi:10.1016/j.asoc.2022.109075 · Cited by 150
The Minimal Energetic Requirement of Sustained Awareness after Brain Injury
Stender, J., Mortensen, K. N., Thibaut, A., Darkner, S., Laureys, S., Gjedde, A., & Kupers, R. (2016) · Current Biology · doi:10.1016/j.cub.2016.04.024 · Cited by 131
On Learning Invariant Representation for Domain Adaptation
Zhao, H., Combes, R. T. D., Zhang, K., & Gordon, G. J. (2019) · arXiv · doi:10.48550/arxiv.1901.09453 · Cited by 134
Functional coordination of alternative splicing in the mammalian central nervous system
Fagnani, M., Barash, Y., Ip, J. Y., Misquitta, C., Pan, Q., Saltzman, A. L., Shai, O., Lee, L., Rozenhek, A., Mohammad, N., et al. (2007) · Genome Biology · doi:10.1186/gb-2007-8-6-r108 · Cited by 120
Network medicine framework for identifying drug-repurposing opportunities for COVID-19
Morselli Gysi, D., do Valle, Í., Zitnik, M., Ameli, A., Gan, X., Varol, O., Ghiassian, S. D., Patten, J. J., Davey, R. A., Loscalzo, J., et al. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2025581118 · Cited by 107
Sequences of purchases in credit card data reveal lifestyles in urban populations
Di Clemente, R., Luengo-Oroz, M., Travizano, M., Xu, S., Vaitla, B., & González, M. C. (2018) · Nature Communications · doi:10.1038/s41467-018-05690-8 · Cited by 105
Kunik, V., & Ofran, Y. (2013) · Protein Engineering Design and Selection · doi:10.1093/protein/gzt027 · Cited by 96
Bowman, G. R. (2012) · The Journal of Chemical Physics · doi:10.1063/1.4755751 · Cited by 76
Machine learning action parameters in lattice quantum chromodynamics
Shanahan, P. E., Trewartha, D., & Detmold, W. (2018) · Physical Review D · doi:10.1103/physrevd.97.094506 · Cited by 65
The Effect of Collective Attention on Controversial Debates on Social Media
Garimella, K., De Francisci Morales, G., Gionis, A., & Mathioudakis, M. (2017) · Proceedings of the 2017 ACM on Web Science Conference · doi:10.1145/3091478.3091486 · Cited by 56
“This measure captures the difference between the word distributionsW 1 d andW 2 d , and is based on the Jensen-Shannon divergence [22].”
MethodBehavioral diversity in microbes and low-dimensional phenotypic spaces
Jordan, D., Kuehn, S., Katifori, E., & Leibler, S. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1308282110 · Cited by 52
Escort evolutionary game theory
Harper, M. (2011) · Physica D: Nonlinear Phenomena · doi:10.1016/j.physd.2011.04.008 · Cited by 51
Rossi, L., Torsello, A., & Hancock, E. R. (2015) · Physical Review E · doi:10.1103/physreve.91.022815 · Cited by 46
Improving Natural Language Processing Tasks with Human Gaze-Guided Neural Attention
Sood, E., Tannert, S., Mueller, P., & Bulling, A. (2020) · arXiv · doi:10.48550/arxiv.2010.07891 · Cited by 42
Efficient and secure image encryption algorithm using 2D LIM map and Latin square matrix
Huang, X., Tang, J., & Zhang, Z. (2024) · Nonlinear Dynamics · doi:10.1007/s11071-024-10191-z · Cited by 35
Multiplicity and word sense: evaluating and learning from multiply labeled word sense annotations
Passonneau, R. J., Bhardwaj, V., Salleb-Aouissi, A., & Ide, N. (2012) · Language Resources and Evaluation · doi:10.1007/s10579-012-9188-x · Cited by 34
A 3D structural SARS-CoV-2–human interactome to explore genetic and drug perturbations
Wierbowski, S. D., Liang, S., Liu, Y., Chen, Y., Gupta, S., Andre, N. M., Lipkin, S. M., Whittaker, G. R., & Yu, H. (2021) · Nature Methods · doi:10.1038/s41592-021-01318-w · Cited by 32
Are protein-protein interfaces special regions on a protein’s surface?
Tonddast-Navaei, S., & Skolnick, J. (2015) · The Journal of Chemical Physics · doi:10.1063/1.4937428 · Cited by 31
Crean, R. M., Slusky, J. S. G., Kasson, P. M., & Kamerlin, S. C. L. (2023) · The Journal of Chemical Physics · doi:10.1063/5.0140882 · Cited by 30
Amuse: Human-AI Collaborative Songwriting with Multimodal Inspirations
Kim, Y., Lee, S.-J., & Donahue, C. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713818 · Cited by 23
Fischer-Holzhausen, S., & Röblitz, S. (2022) · Journal of Theoretical Biology · doi:10.1016/j.jtbi.2022.111150 · Cited by 17
Bayram, F., Ahmed, B. S., & Hallin, E. (2024) · Journal of Systems and Software · doi:10.1016/j.jss.2024.112184 · Cited by 16
Machine Learning Algorithm for Wireless Indoor Localization
Abdullah, O. A., & Abdel-Qader, I. (2018) · Machine Learning - Advanced Techniques and Emerging Applications · doi:10.5772/intechopen.74754 · Cited by 14
MS2: Multi-Document Summarization of Medical Studies
DeYoung, J., Beltagy, I., van Zuylen, M., Kuehl, B., & Wang, L. L. (2021) · arXiv · doi:10.48550/arxiv.2104.06486 · Cited by 10
Borgmans, S., Rogge, S. M. J., Vanduyfhuys, L., & Van Speybroeck, V. (2023) · Journal of Chemical Theory and Computation · doi:10.1021/acs.jctc.3c01028 · Cited by 8
Hsueh, S. C. C., Aina, A., Roman, A. Y., Cashman, N. R., Peng, X., & Plotkin, S. S. (2022) · ACS Chemical Neuroscience · doi:10.1021/acschemneuro.1c00567 · Cited by 8
Compact Time-Domain Representation for Logical Access Spoofed Audio
Karo, M., Yeredor, A., & Lapidot, I. (2024) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2023.3341000 · Cited by 6
Murail, S., Sawmynaden, J., Zemirli, A., Jusot, M., Pietrucci, F., Chomilier, J., Tufféry, P., & Stratmann, D. (2025) · Journal of Chemical Theory and Computation · doi:10.1021/acs.jctc.5c01123 · Cited by 6
Understanding the Dark Side of LLMs' Intrinsic Self-Correction
Zhang, Q., Wang, D., Qian, H., Li, Y., Zhang, T., Huang, M., Xu, K., Li, H., Yan, L., & Qiu, H. (2025) · Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2025.acl-long.1314 · Cited by 36
Lee, W.-C., Lee, J.-S., & Suh, J. (2026) · Applied Sciences · doi:10.3390/app16062816 · Cited by 2
Within-Model vs Between-Prompt Variability in Large Language Models for Creative Tasks
Haase, J., Gonnermann-Müller, J., Hanel, P. H. P., Leins, N., Kosch, T., Mendling, J., & Pokutta, S. (2026) · arXiv · doi:10.48550/arxiv.2601.21339
“To quantify the breadth of a model’s semantic range, we computed the Thematic Entropy (Shannon Entropy) of its category distribution: H ( M ) = − c ∈ C p ( c | M ) ln p ( c | M ) , where p ( c | M ) is the proportion of model M ’s ideas falling into category c (Lin, 1991).”
Metadata
- Authors
- Lin, J.
- Venue
- IEEE Transactions on Information Theory
- Published
- 1991
- Volume
- 37
- Issue
- 1
- Pages
- 145-151
- ISSN
- 0018-9448 (ISSN-L)
- OpenAlex
- W2146950091
- Cited by
- 5,140
- References
- 0
- FWCI
- 3.20
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
