Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
PreprintCited by 1

The DOI matches its DataCite record.

MultiScale MeshGraphNets

Fortunato, Meire, Pfaff, Tobias, Wirnsberger, Peter, Pritzel, Alexander, Battaglia, Peter·2022·arXiv
DOI 10.48550/arxiv.2210.00612arXiv 2210.00612
Open this paper in CoChat — free account

Abstract

In recent years, there has been a growing interest in using machine learning to overcome the high cost of numerical simulation, with some learned models achieving impressive speed-ups over classical solvers whilst maintaining accuracy. However, these methods are usually tested at low-resolution settings, and it remains to be seen whether they can scale to the costly high-resolution simulations that we ultimately want to tackle. In this work, we propose two complementary approaches to improve the framework from MeshGraphNets, which demonstrated accurate predictions in a broad range of physical systems. MeshGraphNets relies on a message passing graph neural network to propagate information, and this structure becomes a limiting factor for high-resolution simulations, as equally distant points in space become further apart in graph space. First, we demonstrate that it is possible to learn accurate surrogate dynamics of a high-resolution system on a much coarser mesh, both removing the message passing bottleneck and improving performance; and second, we introduce a hierarchical approach (MultiScale MeshGraphNets) which passes messages on two different resolutions (fine and coarse), significantly improving the accuracy of MeshGraphNets while requiring less computational resources.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2210.00612

Cites

Statistics

Figures

Figure 1 The four update operators on MS-MGN: D ownsample (left), where each node on the low-resolution mesh (orange mesh) receives information from the high-resolution mesh triangle (blue mesh) enclosing the node; H igh-resolution (bottom-middle), where high-resolution nodes are updated by their connected neighbors; L ow-resolution (top-middle), where low-resolution nodes are updated by connected their neighbors; U psample (right), where each high-resolution node receives information from the corresponding low-resolution nodes it updates in the D ownsample update.
Figure 1 The four update operators on MS-MGN: D ownsample (left), where each node on the low-resolution mesh (orange mesh) receives information from the high-resolution mesh triangle (blue mesh) enclosing the node; H igh-resolution (bottom-middle), where high-resolution nodes are updated by their connected neighbors; L ow-resolution (top-middle), where low-resolution nodes are updated by connected their neighbors; U psample (right), where each high-resolution node receives information from the corresponding low-resolution nodes it updates in the D ownsample update.CC-BY · From the PDF
Figure 2 A reference simulation of the Karman vortex street simulated with COMSOL. The colormap shows the x -component of the velocity field. Top: The simulation mesh is not fine enough to resolve all flow features, and the characteristic vortex shedding is suppressed. Bottom: A more expensive simulation on a finer mesh correctly resolves the dynamics. Middle: 'high-accuracy' labels from the high-resolution simulation (bottom) interpolated on the coarse mesh from (top), with vortex-shedding still visible. We use this to generate a training set.
Figure 2 A reference simulation of the Karman vortex street simulated with COMSOL. The colormap shows the x -component of the velocity field. Top: The simulation mesh is not fine enough to resolve all flow features, and the characteristic vortex shedding is suppressed. Bottom: A more expensive simulation on a finer mesh correctly resolves the dynamics. Middle: 'high-accuracy' labels from the high-resolution simulation (bottom) interpolated on the coarse mesh from (top), with vortex-shedding still visible. We use this to generate a training set.CC-BY · From the PDF
Figure 3 Relationship between the minimum mesh edge length ( edge min ) and the number of nodes and edges on the input mesh for the CylinderFlow test dataset.
Figure 3 Relationship between the minimum mesh edge length ( edge min ) and the number of nodes and edges on the input mesh for the CylinderFlow test dataset.CC-BY · From the PDF
Figure 4 Log-log plot of 1-step MSE measured against a highlyresolved 'ground-truth' simulation. We expect the error to converge to 0 with decreased edge length (red curve); MGN initially tracks this curve, but accuracy drops off at small edge lengths. Increasing the number of message passing steps moves the drop-off point, but the fundamental issue remains.
Figure 4 Log-log plot of 1-step MSE measured against a highlyresolved 'ground-truth' simulation. We expect the error to converge to 0 with decreased edge length (red curve); MGN initially tracks this curve, but accuracy drops off at small edge lengths. Increasing the number of message passing steps moves the drop-off point, but the fundamental issue remains.CC-BY · From the PDF
Figure 5 Convergence analysis of MS-MGN. With the same number of message passing steps, MS-MGN exhibits a significantly lower error than the corresponding MGN model. At 25 mps, MSMGN closely tracks the convergence curve of the reference solver (red).
Figure 5 Convergence analysis of MS-MGN. With the same number of message passing steps, MS-MGN exhibits a significantly lower error than the corresponding MGN model. At 25 mps, MSMGN closely tracks the convergence curve of the reference solver (red).CC-BY · From the PDF
Figure 6 Solution accuracy vs. training speed. MS-MGN allows better performance/accuracy tradeoffs at all mesh resolutions.
Figure 6 Solution accuracy vs. training speed. MS-MGN allows better performance/accuracy tradeoffs at all mesh resolutions.CC-BY · From the PDF
Figure 7 MGNbaseline model trained with high-accuracy labels. At edge lengths above 0 . 0016 , the model is more accurate than the reference solver at a given resolution.
Figure 7 MGNbaseline model trained with high-accuracy labels. At edge lengths above 0 . 0016 , the model is more accurate than the reference solver at a given resolution.CC-BY · From the PDF
Figure 8 MS-MGN model trained with high-accuracy labels. At 25 mps, the model is more accurate than the reference across all mesh resolutions.
Figure 8 MS-MGN model trained with high-accuracy labels. At 25 mps, the model is more accurate than the reference across all mesh resolutions.CC-BY · From the PDF
Figure 9 Rollout error for simulation trajectories on highresolution meshes.
Figure 9 Rollout error for simulation trajectories on highresolution meshes.CC-BY · From the PDF
Figure 10 Choice of G l . Uniform grids show higher error than unstructured meshes created from the same distribution as the G h .
Figure 10 Choice of G l . Uniform grids show higher error than unstructured meshes created from the same distribution as the G h .CC-BY · From the PDF
Figure A.1. MS-MGN Processor parameter sweep with 5 seeds per processor. MSE-N is the average N-step mean squared error for the first N steps in the trajectory.
Figure A.1. MS-MGN Processor parameter sweep with 5 seeds per processor. MSE-N is the average N-step mean squared error for the first N steps in the trajectory.CC-BY · From the PDF
Figure A.2. Graph Fourier analysis of the errors between the model prediction and the ground-truth velocity after one rollout step (left) and ten rollout steps (right) for the MGN baseline (solid blue) and one of the MS-MGN models in Fig. A.1 (dashed purple). The comparison is carried out for edge min = 1 . 5 × 10 -3 and comprises 20 different seeds for each model, with shaded regions corresponding to one standard deviation of the error of the mean.
Figure A.2. Graph Fourier analysis of the errors between the model prediction and the ground-truth velocity after one rollout step (left) and ten rollout steps (right) for the MGN baseline (solid blue) and one of the MS-MGN models in Fig. A.1 (dashed purple). The comparison is carried out for edge min = 1 . 5 × 10 -3 and comprises 20 different seeds for each model, with shaded regions corresponding to one standard deviation of the error of the mean.CC-BY · From the PDF
Figure A.3. Illustration of the contributions of selected eigenvalues (indexed by n ) to the absolute error in the x -velocity. The signal varies slowly for small n (top left panel) and very fast for large n (bottom right panel).
Figure A.3. Illustration of the contributions of selected eigenvalues (indexed by n ) to the absolute error in the x -velocity. The signal varies slowly for small n (top left panel) and very fast for large n (bottom right panel).CC-BY · From the PDF
Figure A.4. Initial (left) and final (right) state for different data examples on the 2D Cylinder Flow dataset. The position and size of the obstacle vary in each example, as well as the inflow initial velocity and the mesh resolution.
Figure A.4. Initial (left) and final (right) state for different data examples on the 2D Cylinder Flow dataset. The position and size of the obstacle vary in each example, as well as the inflow initial velocity and the mesh resolution.CC-BY · From the PDF
Num nodes=760, num edges = 2181, edge_min=1.0E-02
Num nodes=760, num edges = 2181, edge_min=1.0E-02CC-BY · From the PDF
Figure A.5. Different mesh resolutions in the CylinderFlow Fixed Obstacle Test Set. On top we see an example of the coarsest resolution on the test set (which is also the same resolution as the coarse mesh on the MS-MGN architecture. On the bottom we see an example for the finest resolution. We note the bottom mesh has almost 30 times more nodes and edges than the top one.
Figure A.5. Different mesh resolutions in the CylinderFlow Fixed Obstacle Test Set. On top we see an example of the coarsest resolution on the test set (which is also the same resolution as the coarse mesh on the MS-MGN architecture. On the bottom we see an example for the finest resolution. We note the bottom mesh has almost 30 times more nodes and edges than the top one.CC-BY · From the PDF

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 23

  1. Allen, K. R., Lopez-Guevara, T., Stachenfeld, K., Sanchez- Gonzalez, A., Battaglia, P., Hamrick, J., and Pfaff, T. Physical design using differentiable learned simulators. arXiv, 2022.
  2. Battaglia, P. W., Hamrick, J. B., Bapst, V., Sanchez- Gonzalez, A., Zambaldi, V., Malinowski, M., Tacchetti, A., Raposo, D., Santoro, A., Faulkner, R., et al. Rela- tional inductive biases, deep learning, and graph networks. arXiv preprint arXiv:1806.01261, 2018. · arXiv
  3. Belbute-Peres, F. d. A., Economon, T. D., and Kolter, J. Z. Combining differentiable PDE solvers and graph neural networks for fluid flow prediction. In Proceedings of the 37th International Conference on Machine Learning ICML 2020, 2020.
  4. Chen, D., Lin, Y., Li, W., Li, P., Zhou, J., and Sun, X. Measuring and relieving the over-smoothing problem for graph neural networks from the topological view. Pro- ceedings of the AAAI Conference on Artificial Intelli- gence, 34, 2020.
  5. Gilmer, J., Schoenholz, S. S., Riley, P. F., Vinyals, O., and Dahl, G. E. Neural message passing for quantum chem- istry. In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pp. 1263-1272, 2017.
  6. Goodrich, C. P., King, E. M., Schoenholz, S. S., Cubuk, E. D., and Brenner, M. P. Designing self-assembling ki- netics with differentiable statistical physics models. Pro- ceedings of the National Academy of Sciences, 118(10): e2024083118, 2021. doi: 10.1073/pnas.2024083118. · DOI
  7. Han, X., Gao, H., Pfaff, T., Wang, J.-X., and Liu, L.-P. Predicting physics in mesh-reduced space with tempo- ral attention. In International Conference on Learning Representations, 2022.
  8. Hanocka, R., Hertz, A., Fish, N., Giryes, R., Fleishman, S., and Cohen-Or, D. Meshcnn: A network with an edge. ACM Trans. Graph., 38(4), jul 2019.
  9. Keisler, R. Forecasting global weather with graph neural networks. arXiv preprint arXiv:2202.07575, 2022. · arXiv
  10. Kochkov, D., Pfaff, T., Sanchez-Gonzalez, A., Battaglia, P., and Clark, B. K. Learning ground states of quantum hamiltonians with graph networks, 2021a. Kochkov, D., Smith, J. A., Alieva, A., Wang, Q., Brenner, M. P., and Hoyer, S. Machine learning-accelerated com- putational fluid dynamics. Proceedings of the National Academy of Sciences, 118(21), 2021b.
  11. Li, Q., Han, Z., and Wu, X.-M. Deeper insights into graph convolutional networks for semi-supervised learning. In Proceedings of the AAAI Conference on Artificial Intelli- gence. AAAI Press, 2018a.
  12. Lino, M., Cantwell, C., Bharath, A. A., and Fotiadis, S. Simulating continuum mechanics with multi-scale graph neural networks. arXiv preprint arXiv:2106.04900, 2021. · arXiv
  13. Mrowca, D., Zhuang, C., Wang, E., Haber, N., Fei-Fei, L., Tenenbaum, J. B., and Yamins, D. L. Flexible neural representation for physics prediction. In Advances in Neural Information Processing Systems, 2018.
  14. Pfaff, T., Fortunato, M., Sanchez-Gonzalez, A., and Battaglia, P. W. Learning mesh-based simulation with graph networks. In 9th International Con- ference on Learning Representations, ICLR 2021, 2021. URL https://openreview.net/forum? id=roNqYL0_XP.
  15. Pope, S. B. Turbulent flows. Cambridge Univ. Press, Cambridge, 2011. URL https://cds.cern.ch/ record/1346971.
  16. Sanchez-Gonzalez, A., Godwin, J., Pfaff, T., Ying, R., Leskovec, J., and Battaglia, P. W. Learning to simulate complex physics with graph networks. In Proceedings of the 37th International Conference on Machine Learning ICML 2020, 2020.
  17. Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M., and Monfardini, G. The graph neural network model. IEEE Transactions on Neural Networks, 20(1):61-80, 2008.
  18. Shuman, D. I., Narang, S. K., Frossard, P., Ortega, A., and Vandergheynst, P. The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains. IEEE signal processing magazine, 30(3):83-98, 2013.
  19. Stachenfeld, K., Fielding, D. B., Kochkov, D., Cranmer, M., Pfaff, T., Godwin, J., Cui, C., Ho, S., Battaglia, P., and Sanchez-Gonzalez, A. Learned coarse models for efficient turbulence simulation. In 10th International Conference on Learning Representations, ICLR 2022, 2022.
  20. Thuerey, N., Weißenow, K., Prantl, L., and Hu, X. Deep learning methods for reynolds-averaged navier-stokes simulations of airfoil flows. AIAA Journal, 58(1):25-36, 2020a. doi: 10.2514/1.J058291. · DOI
  21. Thuerey, N., Weißenow, K., Prantl, L., and Hu, X. Deep learning methods for reynolds-averaged navier-stokes simulations of airfoil flows. AIAA Journal, 58(1):25-36, 2020b.
  22. Ying, R., You, J., Morris, C., Ren, X., Hamilton, W. L., and Leskovec, J. Hierarchical graph representation learn- ing with differentiable pooling. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS'18, 2018.
  23. Yu, F. and Koltun, V. Multi-scale context aggregation by dilated convolutions. In 4th International Conference on Learning Representations, ICLR 2016, 2016.

Cited by

Metadata

Authors
Fortunato, M., Pfaff, T., Wirnsberger, P., Pritzel, A., & Battaglia, P.
Venue
arXiv
Publisher
arXiv
Published
2022
DOI
10.48550/arxiv.2210.00612
arXiv
2210.00612
References
0
Type
Preprint
Licence
creativecommons.org/licenses/by/4.0/legalcode

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.