Tree tensor networks for generative modeling
Open accessThe DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Abstract
Matrix product states (MPSs), a tensor network designed for one-dimensional quantum systems, were recently proposed for generative modeling of natural data (such as images) in terms of the ``Born machine.'' However, the exponential decay of correlation in MPSs restricts its representation power heavily for modeling complex data such as natural images. In this work, we push forward the effort of applying tensor networks to machine learning by employing the tree tensor network (TTN), which exhibits balanced performance in expressibility and efficient training and sampling. We design the tree tensor network to utilize the two-dimensional prior of the natural images and develop sweeping learning and sampling algorithms which can be efficiently implemented utilizing graphical processing units. We apply our model to random binary patterns and the binary MNIST data sets of handwritten digits. We show that the TTN is superior to MPSs for generative modeling in keeping the correlation of pixels in natural images, as well as giving better log-likelihood scores in standard data sets of handwritten digits. We also compare its performance with state-of-the-art generative models such as variational autoencoders, restricted Boltzmann machines, and PixelCNN. Finally, we discuss the future development of tensor network states in machine learning problems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
On Information and Sufficiency
Kullback, S., & Leibler, R. A. (1951) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729694 · Cited by 20,193
Neural networks and physical systems with emergent collective computational abilities.
Hopfield, J. J. (1982) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.79.8.2554 · Cited by 19,270
Factor graphs and the sum-product algorithm
Kschischang, F. R., Frey, B. J., & Loeliger, H.-A. (2001) · IEEE Transactions on Information Theory · doi:10.1109/18.910572 · Cited by 6,486 · cited 1×
“Fortunately, for acyclic graphs such as the tree-structure factor graph, the second term can be computed exactly by using the sum-product algorithm 36 , a generic message passing algorithm operates on the factor graphs.”
MethodThe density-matrix renormalization group in the age of matrix product states
Schollwöck, U. (2011) · Annals of Physics · doi:10.1016/j.aop.2010.09.012 · Cited by 4,818
Orús, R. (2014) · Annals of Physics · doi:10.1016/j.aop.2014.06.013 · Cited by 2,255
Learning and relearning in Boltzmann machines
Hinton, G. E., & Sejnowski, T. J. (2001) · Graphical Models · doi:10.7551/mitpress/3349.003.0005 · Cited by 1,037
Importance Weighted Autoencoders
Burda, Y., Grosse, R., & Salakhutdinov, R. (2015) · arXiv · doi:10.48550/arxiv.1509.00519 · Cited by 644
Classical simulation of quantum many-body systems with a tree tensor network
Shi, Y.-Y., Duan, L.-M., & Vidal, G. (2006) · Physical Review A · doi:10.1103/physreva.74.022320 · Cited by 577
On the quantitative analysis of deep belief networks
Salakhutdinov, R., & Murray, I. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390266 · Cited by 411
Why Does Deep and Cheap Learning Work So Well?
Lin, H. W., Tegmark, M., & Rolnick, D. (2017) · Journal of Statistical Physics · doi:10.1007/s10955-017-1836-5 · Cited by 393 · cited 1×
“On the PGM side, although the success of machine learning models based PGM in natural images is not completely understood, some arguments support that natural images actually only have sparse long-range correlations, making them much less complex than the arbitrary images 13,27 .”
BackgroundDifferentiable learning of quantum circuit Born machines
Liu, J.-G., & Wang, L. (2018) · Physical Review A · doi:10.1103/physreva.98.062324 · Cited by 279
Self-citationTagliacozzo, L., Evenbly, G., & Vidal, G. (2009) · Physical Review B · doi:10.1103/physrevb.80.235127 · Cited by 261
Information Perspective to Probabilistic Modeling: Boltzmann Machines versus Born Machines
Cheng, S., Chen, J., & Wang, L. (2018) · Entropy · doi:10.3390/e20080583 · Cited by 104 · cited 4×
“As we have motivated in the introduction, TTN generative model can also be treated as one kind of Born machine 13 , that is, the TTN represents a pure quantum state Ψ ( x ), and the p ( x ) is induced from the square of the amplitude of the wavefunction following Born’s rule where Z = x | Ψ ( x ) |…”
BackgroundMethodInfluentialSelf-citation
Cited by
Machine learning and the physical sciences
Carleo, G., Cirac, I., Cranmer, K., Daudet, L., Schuld, M., Tishby, N., Vogt-Maranto, L., & Zdeborová, L. (2019) · Reviews of Modern Physics · doi:10.1103/revmodphys.91.045002 · Cited by 2,417
Differentiable Programming Tensor Networks
Liao, H.-J., Liu, J.-G., Wang, L., & Xiang, T. (2019) · Physical Review X · doi:10.1103/physrevx.9.031041 · Cited by 251
Self-citationTensor Network Contractions: Methods and Applications to Quantum Many-Body Systems
Ran, S.-J., Tirrito, E., Peng, C., Chen, X., Tagliacozzo, L., Su, G., & Lewenstein, M. (2020) · Lecture Notes in Physics · doi:10.1007/978-3-030-34489-4 · Cited by 176
Yao.jl: Extensible, Efficient Framework for Quantum Algorithm Design
Luo, X.-Z., Liu, J.-G., Zhang, P., & Wang, L. (2020) · Quantum · doi:10.22331/q-2020-10-11-341 · Cited by 157
“Besides serving as an efficient simulation backend, the tensor networks are also helpful for quantum machine learning, given the fact that they have already found many applications in various machine learning tasks [104–108].”
MethodSelf-citationSolving quantum statistical mechanics with variational autoregressive networks and quantum circuits
Liu, J.-G., Mao, L., Zhang, P., & Wang, L. (2021) · Machine Learning: Science and Technology · doi:10.1088/2632-2153/aba19d · Cited by 57
Self-citation
Metadata
- Authors
- Cheng, S., Wang, L., Xiang, T., & Zhang, P.
- Venue
- Physical Review B
- Publisher
- American Physical Society (APS)
- Published
- 2019-04-18
- Volume
- 99
- Issue
- 15
- ISSN
- 2469-9950 (ISSN-L)
- 2469-9969
- OpenAlex
- W2910259907
- Cited by
- 122
- References
- 14
- FWCI
- 6.00
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.