The DOI matches its Crossref record; 82 of the 82 references it cites were found in a registry or an index.
Normalizing Flows: An Introduction and Review of Current Methods
Open accessAbstract
Normalizing Flows are generative models which produce tractable distributions where both sampling and density evaluation can be efficient and exact. The goal of this survey article is to give a coherent and comprehensive review of the literature around the construction and use of Normalizing Flows for distribution learning. We aim to provide context and explanation of the models, review current state-of-the-art literature, and identify open questions and promising future directions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
UCI Machine Learning Repository
Asuncion, A. (2007) · Medical Entomology and Zoology · Cited by 25,387
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741
Labaca-Castro, R. (2023) · Machine Learning under Malware Attack · doi:10.1007/978-3-658-40442-0_9 · Cited by 19,272
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
Martin, D., Fowlkes, C., Tal, D., & Malik, J. (2002) · Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001 · doi:10.1109/iccv.2001.937655 · Cited by 8,216
Ordinary Differential Equations
(2017) · Python for Scientists · doi:10.1017/9781108120241.010 · Cited by 7,240
Improved Techniques for Training GANs
Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., & Chen, X. (2016) · arXiv · doi:10.48550/arxiv.1606.03498 · Cited by 6,965
(2017) · International Conference on Machine Learning · Cited by 6,382
Generative Adversarial Networks: An Overview
Creswell, A., White, T., Dumoulin, V., Arulkumaran, K., Sengupta, B., & Bharath, A. A. (2018) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2765202 · Cited by 4,540
Introduction to the modern theory of dynamical systems
(1996) · Choice Reviews Online · doi:10.5860/choice.33-5770 · Cited by 3,592
Deep Unsupervised Learning using Nonequilibrium Thermodynamics
Sohl-Dickstein, J., Weiss, E. A., Maheswaranathan, N., & Ganguli, S. (2015) · arXiv · doi:10.48550/arxiv.1503.03585 · Cited by 2,637
Introduction to the Modern Theory of Dynamical Systems
Katok, A., & Hasselblatt, B. (1995) · Cambridge University Press eBooks · doi:10.1017/cbo9780511809187 · Cited by 2,169
Neural ordinary differential equations
Chen, R. T. Q., Rubanova, Y., Bettencourt, J., & Duvenaud, D. (2018) · arXiv · doi:10.48550/arxiv.1806.07366 · Cited by 2,097
Generating Sentences from a Continuous Space
Bowman, S. R., Vilnis, L., Vinyals, O., Dai, A., Jozefowicz, R., & Bengio, S. (2016) · Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning · doi:10.18653/v1/k16-1002 · Cited by 2,061
Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
Bengio, Y., Léonard, N., & Courville, A. (2013) · arXiv · doi:10.48550/arxiv.1308.3432 · Cited by 1,910
Bayesian Learning via Stochastic Gradient Langevin Dynamics
Welling, M., & Teh, Y. W. (2012) · Cited by 1,551
NICE: Non-linear Independent Components Estimation
Dinh, L., Krueger, D., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1410.8516 · Cited by 1,493
Variational Inference with Normalizing Flows
Rezende, D. J., & Mohamed, S. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.05770 · Cited by 1,253
“Waveglow: a flow-based generative network for speech synthesis,”
Prenger, R., Valle, R., & Catanzaro, B. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683143 · Cited by 1,004
Generative adversarial networks: introduction and outlook
Wang, K., Gou, C., Duan, Y., Lin, Y., Zheng, X., & Wang, F.-Y. (2017) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2017.7510583 · Cited by 806
Spivak, M. (2018) · CRC Press eBooks · doi:10.1201/9780429501906 · Cited by 799
Boltzmann generators: Sampling equilibrium states of many-body systems with deep learning
Noé, F., Olsson, S., Köhler, J., & Wu, H. (2019) · Science · doi:10.1126/science.aaw1147 · Cited by 744
An Introduction to Variational Autoencoders
Kingma, D. P., & Welling, M. (2019) · doi:10.1561/9781680836233 · Cited by 643
A Proposal on Machine Learning via Dynamical Systems
E, W. (2017) · Communications in Mathematics and Statistics · doi:10.1007/s40304-017-0103-z · Cited by 607
Tolstikhin, I., Bousquet, O., Gelly, S., & Schölkopf, B. (2018) · MPG.PuRe (Max Planck Society) · Cited by 602
Invertible Residual NetworksPreprint
Behrmann, J., Grathwohl, W., Chen, R. T. Q., Duvenaud, D., & Jacobsen, J.-H. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.00995 · Cited by 529
MADE: Masked Autoencoder for Distribution Estimation
Germain, M., Gregor, K., Murray, I., & Larochelle, H. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1502.03509 · Cited by 448
A Family of Nonparametric Density Estimation Algorithms
Tabak, E. G., & Turner, C. V. (2013) · Communications on Pure and Applied Mathematics · doi:10.1002/cpa.21423 · Cited by 346
Markov Chain Monte Carlo and Variational Inference: Bridging the Gap
Salimans, T., Kingma, D. P., & Welling, M. (2014) · UvA-DARE (University of Amsterdam) · doi:10.48550/arxiv.1410.6460 · Cited by 343
The Reversible Residual Network: Backpropagation Without Storing Activations
Gomez, A. N., Ren, M., Urtasun, R., & Grosse, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.04585 · Cited by 335
Parallel WaveNet: Fast High-Fidelity Speech Synthesis
van den Oord, A., Li, Y., Babuschkin, I., Simonyan, K., Vinyals, O., Kavukcuoglu, K., van den Driessche, G., Lockhart, E., Cobo, L. C., Stimberg, F., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.10433 · Cited by 330
Müller, T., Mcwilliams, B., Rousselle, F., Gross, M., & Novák, J. (2019) · ACM Transactions on Graphics · doi:10.1145/3341156 · Cited by 321
Normalizing Flows for Probabilistic Modeling and Inference
Papamakarios, G., Nalisnick, E., Rezende, D. J., Mohamed, S., & Lakshminarayanan, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.02762 · Cited by 318
Probabilistic Graphical Models
Koller, D., & Friedman, N. (2009) · Cited by 306
Density estimation by dual ascent of the log-likelihood
Tabak, E. G., & Vanden-Eijnden, E. (2010) · Communications in Mathematical Sciences · doi:10.4310/cms.2010.v8.n1.a11 · Cited by 283
Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion LimitPreprint
Tzen, B., & Raginsky, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.09883 · Cited by 282
GraphNVP: An Invertible Flow Model for Generating Molecular GraphsPreprint
Madhawa, K., Ishiguro, K., Nakago, K., & Abe, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.11600 · Cited by 261
Ho, J., Chen, X., Srinivas, A., Duan, Y., & Abbeel, P. (2019) · arXiv · Cited by 245
Reversible Architectures for Arbitrarily Deep Residual Neural Networks
Chang, B., Meng, L., Haber, E., Ruthotto, L., Begert, D., & Holtham, E. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11668 · Cited by 233
AntisymmetricRNN: A Dynamical System View on Recurrent Neural NetworksPreprint
Chang, B., Chen, M., Haber, E., & Ed, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.09689 · Cited by 194
Equivariant Flow-Based Sampling for Lattice Gauge Theory
Kanwar, G., Albergo, M. S., Boyda, D., Cranmer, K., Hackett, D. C., Racanière, S., Rezende, D. J., & Shanahan, P. E. (2020) · Physical Review Letters · doi:10.1103/physrevlett.125.121601 · Cited by 193
Flow-GAN: Combining Maximum Likelihood and Adversarial Learning in Generative Models
Grover, A., Dhar, M., & Ermon, S. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11829 · Cited by 189
Noise Flow: Noise Modeling With Conditional Normalizing Flows
Abdelhamed, A., Brubaker, M., & Brown, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00326 · Cited by 182
Self-citationScalable Gradients for Stochastic Differential EquationsPreprint
Li, X., Wong, T. L., Chen, R. T. Q., & Duvenaud, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.01328 · Cited by 175
Residual Flows for Invertible Generative Modeling
Chen, R. T. Q., Behrmann, J., Duvenaud, D., & Jacobsen, J.-H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.02735 · Cited by 170
How to train your neural ODE: the world of Jacobian and kinetic regularizationPreprint
Finlay, C., Jacobsen, J.-H., Nurbekyan, L., & Oberman, A. M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.02798 · Cited by 166
Improving Variational Inference with Inverse Autoregressive Flow
Kingma, D. P., Salimans, T., Józefowicz, R., Chen, X., Sutskever, I., & Welling, M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.04934 · Cited by 164
Piecewise Rational Quadratic Interpolation to Monotonic Data
GREGORY, J. A., & Delbourgo, R. (1982) · IMA Journal of Numerical Analysis · doi:10.1093/imanum/2.2.123 · Cited by 157
Sylvester Normalizing Flows for Variational Inference
van den Berg, R., Hasenclever, L., Tomczak, J. M., & Welling, M. (2018) · UvA-DARE (University of Amsterdam) · doi:10.48550/arxiv.1803.05649 · Cited by 157
Improving Variational Auto-Encoders using Householder FlowPreprint
Tomczak, J. M., & Welling, M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.09630 · Cited by 156
Targeted free energy estimation via learned mappings
Wirnsberger, P., Ballard, A. J., Papamakarios, G., Abercrombie, S., Racanière, S., Pritzel, A., Rezende, D. J., & Blundell, C. (2020) · The Journal of Chemical Physics · doi:10.1063/5.0018903 · Cited by 154
Stochastic differential equations (3rd ed.): an introduction with applications
Øksendal, B. (1992) · Springer eBooks · Cited by 136
Normalizing Flows on Tori and SpheresPreprint
Rezende, D. J., Papamakarios, G., Racanière, S., Albergo, M. S., Kanwar, G., Shanahan, P. E., & Cranmer, K. S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.02428 · Cited by 107
Triangular transformations of measures
Bogachev, V. I., Kolesnikov, A. V., & Медведев, К. В. (2005) · Sbornik Mathematics · doi:10.1070/sm2005v196n03abeh000882 · Cited by 105
Equivariant Flows: sampling configurations for multi-body systems with symmetric energiesPreprint
Köhler, J., Klein, L., & Noé, F. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.00753 · Cited by 104
High-Dimensional Probability Estimation with Deep Density ModelsPreprint
Rippel, O., & Adams, R. P. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1302.5125 · Cited by 103
Conditional Density Estimation with Bayesian Normalising FlowsPreprint
Trippe, B. L., & Turner, R. E. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.04908 · Cited by 93
Latent Normalizing Flows for Discrete SequencesPreprint
Ziegler, Z. M., & Rush, A. M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.10548 · Cited by 88
VideoFlow: A Flow-Based Generative Model for Video
Kumar, M. N. A., Babaeizadeh, M., Erhan, D., Finn, C., Levine, S., Dinh, L., & Kingma, D. (2019) · Cited by 84
Sum-of-Squares Polynomial FlowPreprint
Jaini, P., Selby, K. A., & Yu, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.02325 · Cited by 84
Discrete Flows: Invertible Generative Models of Discrete DataPreprint
Tran, D., Vafa, K., Agrawal, K. K., Dinh, L., & Poole, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.10347 · Cited by 77
Pathwise Derivatives Beyond the Reparameterization TrickPreprint
Jankowiak, M., & Obermeyer, F. H. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.01851 · Cited by 75
Unconstrained Monotonic Neural NetworksPreprint
Wehenkel, A., & Louppe, G. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.05164 · Cited by 75
Deep unsupervised learning using nonequilibrium thermodynamics
Sohl‐Dickstein, J. (2024) · International Conference on Machine Learning · doi:10.57702/qrfwyszh · Cited by 61
Improving Exploration in Soft-Actor-Critic with Normalizing Flows PoliciesPreprint
Ward, P., Smofsky, A., & Bose, A. J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.02771 · Cited by 54
Gravitational-wave population inference with deep flow-based generative network
Wong, K. W. K., Contardo, G., & Ho, S. (2020) · Physical review. D/Physical review. D. · doi:10.1103/physrevd.101.123005 · Cited by 53
A RAD approach to deep mixture modelsPreprint
Dinh, L., Sohl‐Dickstein, J., Larochelle, H., & Pascanu, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.07714 · Cited by 45
Continuous-Time Flows for Efficient Inference and Density EstimationPreprint
Chen, C., Li, C., Chen, L., Wang, W., Pu, Y., & Carin, L. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.01179 · Cited by 41
Poincaré Wasserstein AutoencoderPreprint
Ovinnikov, I. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.01427 · Cited by 40
Latent Variable Modelling with Hyperbolic Normalizing FlowsPreprint
Bose, A. J., Smofsky, A., Liao, R., Panangaden, P., & Hamilton, W. L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.06336 · Cited by 39
Semi-Conditional Normalizing Flows for Semi-Supervised LearningPreprint
Atanov, A., Volokhova, A., Ashukha, A., Sosnovik, I., & Dmitry, V. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.00505 · Cited by 36
Learning Discrete Distributions by DequantizationPreprint
Hoogeboom, E., Cohen, T., & Tomczak, J. M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.11235 · Cited by 35
Learning across scales - A multiscale method for Convolution Neural NetworksPreprint
Haber, E., Ruthotto, L., Holtham, E., & Jun, S.-H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.02009 · Cited by 33
Clustering and Classification through Normalizing Flows in Feature Space
Agnelli, J. P., Cadeiras, M., Tabak, E. G., Turner, C. V., & Vanden‐Eijnden, E. (2010) · Multiscale Modeling and Simulation · doi:10.1137/100783522 · Cited by 27
Improving Variational Auto-Encoders using convex combination linear Inverse Autoregressive FlowPreprint
Tomczak, J. M., & Welling, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.02326 · Cited by 26
Deep Diffeomorphic Normalizing FlowsPreprint
Salman, H., Yadollahpour, P., Fletcher, T., & Batmanghelich, K. N. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.03256 · Cited by 25
Riemannian Normalizing Flow on Variational Wasserstein Autoencoder for Text Modeling
Wang, P. Z., & Wang, W. Y. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1025 · Cited by 25
Approximation Capabilities of Neural ODEs and Invertible Residual NetworksPreprint
Zhang, H., Gao, X., Unterman, J., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.12998 · Cited by 25
Convolutional Normalizing FlowsPreprint
Zheng, G., Yang, Y., & Carbonell, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.02255 · Cited by 13
Cited by
A Unifying Review of Deep and Shallow Anomaly Detection
Ruff, L., Kauffmann, J. R., Vandermeulen, R. A., Montavon, G., Samek, W., Kloft, M., Dietterich, T. G., & Muller, K.-R. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3052449 · Cited by 914
Bond-Taylor, S., Leach, A., Long, Y., & Willcocks, C. G. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3116668 · Cited by 703
A Survey on Generative Diffusion Models
Cao, H., Tan, C., Gao, Z., Xu, Y., Chen, G., Heng, P.-A., & Li, S. Z. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3361474 · Cited by 538
Low-Light Image Enhancement with Normalizing Flow
Wang, Y., Wan, R., Yang, W., Li, H., Chau, L.-P., & Kot, A. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i3.20162 · Cited by 533
Generalized Out-of-Distribution Detection: A Survey
Yang, J., Zhou, K., Li, Y., & Liu, Z. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02117-4 · Cited by 884
Metadata
- Authors
- Kobyzev, I., Prince, S. J. D., & Brubaker, M. A.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2021-11-01
- Volume
- 43
- Issue
- 11
- Pages
- 3964-3979
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W2992005611
- Cited by
- 1,235
- References
- 82
- FWCI
- 55.50
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.