Nonlinear 3D cosmic web simulation with heavy-tailed generative adversarial networks

“The generator and discriminator were trained for 150 epochs with learning rates of 1 × 10 − 4 and 5 × 10 − 5 , respectively, using the ADAM optimizer [37].”

“Generative adversarial networks (GANs), originally introduced in [21], are a class of unsupervised models that learn an underlying probability distribution, which we denote p data ( x ), through adversarial training.”

“The authors also acknowledge the use of the following software for visualization and analysis: H5PY, MATPLOTLIB, NBODYKIT [46], NUMPY [56], POWERBOX [57], PYLIANS, and PYTORCH.”

“In fact, [35] argues that a bounded final activation allows the model to more quickly saturate the range of the training distribution.”

“Despite these practicali-ties, GANs with reasonable model capacity and sufficient training data can be trained successfully and produce high-fidelity samples from the underlying data distribution [30, 31].”

“The conditional GAN (cGAN) is a natural extension of the GAN that addresses these cases [28].”

“Most bias reduction techniques in the machine learning literature cite improvement according to neural network based metrics such as the Frechet Inception Distance (FID) [48] and the Inception Score (IS) [49].”

“We also conducted several experiments with the Wasserstein GAN [38] with gradient penalty [39] scheme (WGAN-GP).”

“Despite these practicali-ties, GANs with reasonable model capacity and sufficient training data can be trained successfully and produce high-fidelity samples from the underlying data distribution [30, 31].”

“We also conducted several experiments with the Wasserstein GAN [38] with gradient penalty [39] scheme (WGAN-GP).”

“We base our network architecture on [34], in which the generator upsamples a latent vector z with 3D transpose convolutional layers and the discriminator downsamples with 3D convolutional layers.”

“Natural image data sets, a common focus of GAN applications, also contain tails in pixel space [28], however the application in cosmology is unique because the tails of the voxel PDF often have a disproportionate impact on the relevant summary statistics.”

“While many types of cosmological data have nonlinear properties, we choose to train a GAN to model the dark matter component of the “cosmic web” [33].”

“For this reason, numerical simulations [1] of a representative volume of the universe are required to bridge the gap between the early universe and present day observations.”

“The need for fast and accurate simulations at a cheaper computational cost has led to the development of many “approximate” methods of cosmic structure simulation [3–11], which implement various physical approximations of gravity to decrease the simulation run-time and/or memory requirement while attempting to maintai…”

“[14] S.”

“The need for fast and accurate simulations at a cheaper computational cost has led to the development of many “approximate” methods of cosmic structure simulation [3–11], which implement various physical approximations of gravity to decrease the simulation run-time and/or memory requirement while attempting to maintai…”

“A promising avenue in machine learning literature seeks to embed inductive biases into neural network models, such that they are guaranteed to respect the structures and symmetries of a system [45–47].”

“They have been used to generate accurate weak lensing convergence maps as evaluated through the power spectrum and a variety of Minkowski functionals [23], to model 2D slices of the cosmic density field from 3D N-body simulations [24], and to generate 3D realizations of neutral hydrogen [25].”

“[43] M.”

“In [24], the scaling parameter κ is set to 4.”

“A promising avenue in machine learning literature seeks to embed inductive biases into neural network models, such that they are guaranteed to respect the structures and symmetries of a system [45–47].”

“We calculate sample bispectra using the software package Pylians 7 .”

“One such bias reduction technique is discriminator rejection sampling (DRS) [60].”

“This is similar in concept to [19,20], which used GANs to map dark matter density fields to corresponding halo number count maps and hydrodynamical quantities, respectively.”

“The need for fast and accurate simulations at a cheaper computational cost has led to the development of many “approximate” methods of cosmic structure simulation [3–11], which implement various physical approximations of gravity to decrease the simulation run-time and/or memory requirement while attempting to maintai…”

“Image-to-image mapping techniques that transform initial conditions to the final structures [12–14] or augment dark matter-only simulation results with various physical effects not present in the simulation [15–20] have shown promise.”

“Reference [27] proposes a super-resolution scheme for generating large-scale realizations of the matter density field hierarchically, treating the scalability problem separately from sample accuracy.”

“The ensemble of power spectra also capture super-sample variance from modes larger than 64 Mpc, since our training samples are sub-volumes from larger simulations. ation from GADGET-2 N-body samples.”

“This is similar in concept to [18] and [19], which used GANs to map dark matter density fields to corresponding halo number count maps and hydrodynamical quantities, respectively.”

“lations on the largest scales in order to constrain models like ΛCDM at the subpercent level [2], a task that is”

“With an approximate posterior available, one might then train a normalizing flow to resample the latent space in a decorrelated manner, as has been done with variational autoencoders in [44].”

“They have been used to generate accurate weak lensing convergence maps as evaluated through the power spectrum and a variety of Minkowski functionals [23], to model 2D slices of the cosmic density field from 3D N-body simulations [24], and to generate 3D realizations of neutral hydrogen [25].”

“A similar point is argued in [49], which considers the role of dual data representations for classification.”

“Depending on the data set and purpose of the generative model, it may be possible to optimize the prior distribution non-parametrically as in [36].”

“[18] F.”

“Concurrent work (Feder et al., 2020) uses a Student-t prior to produce unbounded heavy-tailed data, which has similar tail characteristics to our GPD prior.”

“…(Choudhury et al. 2020), generating accurate HI fields and 21 cm maps by means of Generative Adversarial Networks (Zamudio-Fernandez et al. 2019; Feder et al. 2020; List & Lewis 2020), and emulating reionization simulations to extract information about the underlying astrophysical processes, by…”

“When extended from group symmetry to probability structures induced from other operators, our work is also related to GAN-assisted coarse-graining (CG) for molecular dynamics (Durumeric & Voth, 2019) and cosmology (Mustafa et al., 2019; Feder et al., 2020); see the end of Section 4.”

“This strategy has been implemented through GANs with Pareto priors in Feder et al. (2020); Huster et al. (2021), as well as NFs with Student-t priors in Jaini et al. (2020); Laszkiewicz et al. (2022), the latter allowing for marginals with heterogeneous tail behaviors.”

“There thus exists a demand for emulators: frameworks that can generate summary statistics [Heitmann et al., 2009] or the underlying cosmological fields [Feder et al., 2020, Jamieson et al., 2022] conditional on an input cosmological / astrophysical parameter set.”

“Machine learning can also be used to produce density maps from large N-body simulations of dark matter (DM; Rodríguez et al. 2018; Feder et al. 2020) in a computationally cheaper manner, to predict the effects of DM annihilation feedback on gas densities (List et al. 2019), or to infer a mapping…”

“Data scaling is used in the work of Feder et al. (2020), which preserves high density features, combined with a heavy-tailed latent space prior, instead of a standard Gaussian.”

“However, GANs based on Integral Probability Metrics (IPMs), such as the Wasserstein-1 metric, may struggle to learn these distributions without additional tail estimation strategies [18, 24, 1].”

“2018, Feder et al. 2020). Higherorder statistics would include Minkowski functionals such as those used by Mustafa et al. (2019) to assess their weak lensing convergence maps.”

“The other is asymptotic and uses Extreme Value Theory to prove a new theorem in the continuity of the theoretical work of Huster et al. in [9] and the experimental approach of [6].”

“The analysis of the statistical fluctuations of the estimators obtained by considering the empirical counterpart of the above optimization problem is conducted in [28].”

“In [17, 27], a Generalized Pareto Distribution is adopted for the latent distribution µ Z , in contrast to original GANs which use light-tailed distributions such as uniform or Gaussian.”

“Previous studies aimed at accelerating various cosmological simulations of different fields using generative models.”

“Previous studies aimed at accelerating various cosmological simulations of different fields using generative models [2, 3, 4, 5, 6, 7, 8].”

“…indicates that ν GAN produces genuinely novel data, rather than simply replicating the training set, as observed in previous studies ( I. J. Goodfellow et al. 2014; A. C. Rodríguez et al. 2018; M. Mustafa et al. 2019; R. M. Feder et al. 2020; N. Perraudin et al. 2020; A. Tamosiunas et al. 2021 ) .”

“However, GANs based on Integral Probability Metrics (IPMs), such as the Wasserstein-1 metric, may struggle to learn these distributions without additional tail estimation strategies [18, 24, 1].”