The DOI matches its Crossref record; 60 of the 60 references it cites were found in a registry or an index.
Relevance in the Renormalization Group and in Information Theory
Abstract
The analysis of complex physical systems hinges on the ability to extract the relevant degrees of freedom from among the many others. Though much hope is placed in machine learning, it also brings challenges, chief of which is interpretability. It is often unclear what relation, if any, the architecture- and training-dependent learned "relevant" features bear to standard objects of physical theory. Here we report on theoretical results which may help to systematically address this issue: we establish equivalence between the field-theoretic relevance of the renormalization group, and an information-theoretic notion of relevance we define using the information bottleneck (IB) formalism of compression theory. We show analytically that for statistical physical systems described by a field theory the relevant degrees of freedom found using IB compression indeed correspond to operators with the lowest scaling dimensions. We confirm our field theoretic predictions numerically. We study dependence of the IB solutions on the physical symmetries of the data. Our findings provide a dictionary connecting two distinct theoretical toolboxes, and an example of constructively incorporating physical interpretability in applications of deep learning in physics.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb00917.x · Cited by 48,949
Proceedings of the 25th international conference on Machine learning - ICML '08
(2008) · doi:10.1145/1390156 · Cited by 6,860
Crystal Statistics. I. A Two-Dimensional Model with an Order-Disorder Transition
Onsager, L. (1944) · Physical Review · doi:10.1103/physrev.65.117 · Cited by 6,651
Quantum Field Theory and Critical Phenomena
Zinn‐Justin, J. (2002) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780198509233.001.0001 · Cited by 5,374
The renormalization group and the ε expansion
Wilson, K. G. (1974) · Physics Reports · doi:10.1016/0370-1573(74)90023-4 · Cited by 5,224
Information Theory, Inference, and Learning Algorithms
(2004) · Kybernetes · doi:10.1108/03684920410534506 · Cited by 4,937
Infinite conformal symmetry in two-dimensional quantum field theory
Belavin, A. A., Polyakov, A. M., & Zamolodchikov, A. B. (1984) · Nuclear Physics B · doi:10.1016/0550-3213(84)90052-x · Cited by 4,840
The renormalization group: Critical phenomena and the Kondo problem
Wilson, K. G. (1975) · Reviews of Modern Physics · doi:10.1103/revmodphys.47.773 · Cited by 4,522
Di Francesco, P., Mathieu, P., & Sénéchal, D. (1997) · Graduate Texts in Contemporary Physics · doi:10.1007/978-1-4612-2256-9 · Cited by 3,219
Machine learning and the physical sciences
Carleo, G., Cirac, J. I., Cranmer, K. S., Daudet, L., Schuld, M., Tishby, N., Vogt-Maranto, L., & Zdeborová, L. (2019) · Reviews of Modern Physics · doi:10.1103/revmodphys.91.045002 · Cited by 2,673
The information bottleneck method
Tishby, N., Pereira, F. C., & Bialek, W. (2000) · arXiv · doi:10.48550/arxiv.physics/0004057 · Cited by 2,626
Definitions, methods, and applications in interpretable machine learning
Murdoch, W. J., Singh, C., Kumbier, K., Abbasi-Asl, R., & Yu, B. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1900654116 · Cited by 2,167
ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 16
Storkey, A. (2004) · neural information processing systems · Cited by 2,085
Operator content of two-dimensional conformally invariant theories
Cardy, J. L. (1986) · Nuclear Physics B · doi:10.1016/0550-3213(86)90552-3 · Cited by 1,955
Information, Physics, and Computation
Mézard, M., & Montanari, A. (2009) · doi:10.1093/acprof:oso/9780198570837.001.0001 · Cited by 1,913
Scaling and Renormalization in Statistical Physics
Cardy, J. (1996) · Cambridge University Press eBooks · doi:10.1017/cbo9781316036440 · Cited by 1,902
Statistics of the Two-Dimensional Ferromagnet. Part I
Kramers, H. A., & Wannier, G. H. (1941) · Physical Review · doi:10.1103/physrev.60.252 · Cited by 1,866
Deconfined Quantum Critical Points
Senthil, T., Vishwanath, A., Balents, L., Subir Sachdev, A., & Fisher, M. P. A. (2004) · Science · doi:10.1126/science.1091806 · Cited by 1,385
Machine learning & artificial intelligence in the quantum domain: a review of recent progress
Dunjko, V., & Briegel, H. J. (2018) · Reports on Progress in Physics · doi:10.1088/1361-6633/aab406 · Cited by 1,241
Conformal Invariance, Unitarity, and Critical Exponents in Two Dimensions
Friedan, D., Qiu, Z., & Shenker, S. H. (1984) · Physical Review Letters · doi:10.1103/physrevlett.52.1575 · Cited by 1,165
Mutual Information Neural Estimation.
Belghazi, M. I., Baratin, A., Rajeshwar, S., Ozair, S., Bengio, Y., Courville, A. C., & Hjelm, D. (2018) · International Conference on Machine Learning · Cited by 920
The conformal bootstrap: Theory, numerical techniques, and applications
Poland, D., Rychkov, S., & Vichi, A. (2019) · Reviews of Modern Physics · doi:10.1103/revmodphys.91.015002 · Cited by 760
Conformal invariance and universality in finite-size scaling
Cardy, J. L. (1984) · Journal of Physics A: Mathematical and General · doi:10.1088/0305-4470/17/7/003 · Cited by 744
"Irreversibility" of the flux of the renormalization group in a 2D field theory
Zamolodchikov, A. B. (1986) · Cited by 742
Renormalization group theory: Its basis and formulation in statistical physics
Fisher, M. E. (1998) · Reviews of Modern Physics · doi:10.1103/revmodphys.70.653 · Cited by 631
Document clustering using word clusters via the information bottleneck method
Slonim, N., & Tishby, N. (2000) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/345508.345578 · Cited by 480
SO4 SYMMETRY IN A HUBBARD MODEL
YANG, C. N., & ZHANG, S. C. (1990) · Modern Physics Letters B · doi:10.1142/s0217984990000933 · Cited by 400
Infinite conformal symmetry of critical fluctuations in two dimensions
Belavin, A. A., Polyakov, A. M., & Zamolodchikov, A. B. (1984) · Journal of Statistical Physics · doi:10.1007/bf01009438 · Cited by 358
Parameter Space Compression Underlies Emergent Theories and Predictive Models
Machta, B. B., Chachra, R., Transtrum, M. K., & Sethna, J. P. (2013) · Science · doi:10.1126/science.1238723 · Cited by 288
Finite-size scaling and phenomenological renormalization (invited)
Nightingale, P. (1982) · Journal of Applied Physics · doi:10.1063/1.330232 · Cited by 252
Derrida, B., & de Seze, L. (1982) · Journal de physique · doi:10.1051/jphys:01982004303047500 · Cited by 238
Coveney, P. V., Dougherty, E. R., & Highfield, R. R. (2016) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2016.0153 · Cited by 229
Mutual information, neural networks and the renormalization group
Koch-Janusz, M., & Ringel, Z. (2018) · Nature Physics · doi:10.1038/s41567-018-0081-4 · Cited by 220
Self-citationAc-theorem for entanglement entropy
Casini, H., & Huerta, M. (2007) · Journal of Physics A Mathematical and Theoretical · doi:10.1088/1751-8113/40/25/s57 · Cited by 214
Machine learning meets quantum physics
Sarma, S. D., Deng, D.-L., & Duan, L.-M. (2019) · Physics Today · doi:10.1063/pt.3.4164 · Cited by 204
Information Bottleneck for Gaussian Variables
Chechik, G., Globerson, A., Tishby, N., & Weiss, Y. (2003) · Cited by 161
Quantum information and physics: Some future directions
Preskill, J. P. (2000) · Journal of Modern Optics · doi:10.1080/09500340008244031 · Cited by 160
Conformal Invariance and Applications to Statistical Mechanics
Itzykson, C., Saleur, H., & Zuber, J.-B. (1998) · WORLD SCIENTIFIC eBooks · doi:10.1142/0608 · Cited by 152
Yannakoudakis, E. J., Greece, S. 2. A. I. S. C. O. R. A. D. I. I. R. 2. A., Association for Computing Machinery-Digital Library, ACM Special Interest Group on Information Retrieval, & SIGs, A. (2000) · Cited by 115
Past-future information bottleneck in dynamical systems
Creutzig, F., Globerson, A., & Tishby, N. (2009) · Physical Review E · doi:10.1103/physreve.79.041925 · Cited by 76
Relative entropy and proximity of quantum field theories
Balasubramanian, V., Heckman, J. J., & Maloney, A. (2015) · Journal of High Energy Physics · doi:10.1007/jhep05(2015)104 · Cited by 69
Proceedings of the 16th International Conference on Neural Information Processing Systems
Thrun, S., Saul, L. K., & Schölkopf, B. (2003) · Cited by 64
Information Bottleneck Approach to Predictive Inference
Still, S. (2014) · Entropy · doi:10.3390/e16020968 · Cited by 57
Predictive Coding and the Slowness Principle: An Information-Theoretic Approach
Creutzig, F., & Sprekeler, H. (2007) · Neural Computation · doi:10.1162/neco.2008.01-07-455 · Cited by 54
Field theory entropy, the H theorem, and the renormalization group
Gaite, J., & O’Connor, D. (1996) · Physical review. D. Particles, fields, gravitation, and cosmology/Physical review. D. Particles and fields · doi:10.1103/physrevd.54.5163 · Cited by 48
The renormalization group via statistical inference
Bény, C., & Osborne, T. J. (2015) · New Journal of Physics · doi:10.1088/1367-2630/17/8/083005 · Cited by 46
Optimal Renormalization Group Transformation from Information Theory
Lenggenhager, P. M., Gökmen, D. E., Ringel, Z., Huber, S. D., & Koch-Janusz, M. (2020) · Physical Review X · doi:10.1103/physrevx.10.011037 · Cited by 45
Self-citationInformation-geometric approach to the renormalization group
Bény, C., & Osborne, T. J. (2015) · Physical Review A · doi:10.1103/physreva.92.022330 · Cited by 42
The Information Bottleneck and Geometric Clustering
Strouse, D., & Schwab, D. J. (2018) · Neural Computation · doi:10.1162/neco_a_01136 · Cited by 37
Machine-learning iterative calculation of entropy for physical systems
Nir, A., Sela, E., Beck, R., & Bar‐Sinai, Y. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2017042117 · Cited by 36
Geometric Clustering Using the Information Bottleneck Method
Still, S., Bialek, W., & Bottou, L. (2003) · Cited by 34
A Spiking Neuron as Information Bottleneck
Buesing, L., & Maass, W. (2010) · Neural Computation · doi:10.1162/neco.2010.08-09-1084 · Cited by 31
The renormalization group via statistical inference
Bény, C., & Osborne, T. J. (2015) · Institutional Repository of Leibniz Universität Hannover (Leibniz Universität Hannover) · doi:10.15488/371 · Cited by 31
Optimal Renormalization Group Transformation from Information Theory
Lenggenhager, P. M., Gökmen, D. E., Ringel, Z., Huber, S. D., & Koch-Janusz, M. (2020) · Repository for Publications and Research Data (ETH Zurich) · doi:10.3929/ethz-b-000400030 · Cited by 29
Self-citationInformation theory and renormalization group flows
Apenko, S. M. (2011) · Physica A Statistical Mechanics and its Applications · doi:10.1016/j.physa.2011.08.014 · Cited by 28
Annealing and the Rate Distortion Problem
Parker, A. E., Gedeon, T., & Dimitrov, A. G. (2002) · Cited by 16
The Mathematical Structure of Information Bottleneck Methods
Gedeon, T., Parker, A. E., & Dimitrov, A. G. (2012) · Entropy · doi:10.3390/e14030456 · Cited by 15
Coarse-grained distinguishability of field interactions
Bény, C. (2018) · Quantum · doi:10.22331/q-2018-05-24-67 · Cited by 14
Coarse-grained distinguishability of field interactions
Bény, C. (2018) · Institutional Repository of Leibniz Universität Hannover (Leibniz Universität Hannover) · doi:10.15488/3890 · Cited by 9
Research works printed but not linked: 4
- N. Tishby, 2001, Proceedings of the 37th Allerton Conference on Communication, Control and Computation
- A. B. Zamolodchikov, 1986, JETP Lett., 43, 730
- S. Hassanpour, 2017, Proceedings of the 11th International ITG Conference on Systems, Communications and Coding, SCC 2017
- S. Still, 2004, Advances in Neural Information Processing Systems
Metadata
- Authors
- Gordon, A., Banerjee, A., Koch-Janusz, M., & Ringel, Z.
- Venue
- Physical Review Letters
- Publisher
- American Physical Society (APS)
- Published
- 2021-06-17
- Volume
- 126
- Issue
- 24
- ISSN
- 0031-9007
- 1079-7114
- Cited by
- 18
- References
- 60
- Language
- en
- Funders
- Israel Science Foundation (2250/19, 1466/15, 820314)
- Eidgenössische Technische Hochschule Zürich
- Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung
- National Center of Competence in Research Affective Sciences - Emotions in Individual Behaviour and Social Processes
- National Center of Competence in Research Quantum Science and Technology
- H2020 European Research Council (771503)
- H2020 Marie Skłodowska-Curie Actions (896004)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.