CoChat
Sign Up
Cited by 184Advanced Graph Neural NetworksMultimodal Machine Learning ApplicationsGraph Theory and Algorithms

The DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.

Multimodal learning with graphs

Open access
Ektefaie, Yasha, Dasoulas, George, Noori, Ayush, Farhat, Maha, Zitnik, Marinka·2023·Nature Machine Intelligence
DOI 10.1038/s42256-023-00624-6PMC PMC10704992
Open this paper in CoChat — free accountOpen PDF

Abstract

Artificial intelligence for graphs has achieved remarkable success in modeling complex systems, ranging from dynamic networks in biology to interacting particle systems in physics. However, the increasingly heterogeneous graph datasets call for multimodal methods that can combine different inductive biases-the set of assumptions that algorithms use to make predictions for inputs they have not encountered during training. Learning on multimodal datasets presents fundamental challenges because the inductive biases can vary by data modality and graphs might not be explicitly given in the input. To address these challenges, multimodal graph AI methods combine different modalities while leveraging cross-modal dependencies using graphs. Diverse datasets are combined using graphs and fed into sophisticated multimodal architectures, specified as image-intensive, knowledge-grounded and language-intensive models. Using this categorization, we introduce a blueprint for multimodal graph learning, use it to study existing methods and provide guidelines to design new models.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s42256-023-00624-6

Cites

0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 9 references

Research works printed but not linked: 121

  1. Greener, J. G., Kandathil, S. M., Moffat, L. & Jones, D. T. A guide to machine learning for biologists. Nat. Rev. Mol. Cell Biol. 23, 40–55 (2022). · DOI · Paper page
  2. Gilmer, J., Schoenholz, S. S., Riley, P. F., Vinyals, O. & Dahl, G. E. Neural message passing for quantum chemistry. In Proc. 34th International Conference on Machine Learning: Proc. Machine Learning Research Vol. 70 (eds Precup, D. & Teh, Y. W.) 1263–1272 (PMLR, 2017).
  3. Sanchez-Gonzalez, A. et al. Graph networks as learnable physics engines for inference and control. In Proc. 35th International Conference on Machine Learning: Proc. Machine Learning Research Vol. 80 (eds Dy, J. & Krause, A.) 4470–4479 (PMLR, 2018).
  4. Sanchez-Gonzalez, A. et al. Learning to simulate complex physics with graph networks. In Proc. 37th International Conference on Machine Learning: Proc. Machine Learning Research Vol. 119 (eds Daumé, H. III & Singh, A.) 8459–8468 (PMLR, 2020).
  5. Liu, Q., Kusner, M. J. & Blunsom, P. A survey on contextual embeddings. Preprint at https://arxiv.org/abs/2003.07278 (2020). · arXiv
  6. Scarselli, F., Gori, M., Tsoi, A. C., Hagenbuchner, M. & Monfardini, G. The graph neural network model. IEEE Trans. Neural Netw. 20, 61–80 (2009). · DOI · Paper page
  7. Kipf, T. N. & Welling, M. Semi-supervised classification with graph convolutional networks. In Proc. 5th International Conference on Learning Representations (2017).
  8. Kipf, T. N. & Welling, M. Variational graph auto-encoders. In NIPS Workshop on Bayesian Deep Learning (2016).
  9. Grover, A., Zweig, A. & Ermon, S. Graphite: iterative generative modeling of graphs. In Proc. 36th International Conference on Machine Learning: Proc. Machine Learning Research Vol. 972 (eds Chaudhuri, K. & Salakhutdinov, R.) 434–2444 (PMLR, 2019).
  10. Guo, X. & Zhao, L. A systematic survey on deep generative models for graph generation. Preprint at https://arxiv.org/abs/2007.06686 (2020). · DOI
  11. Baltrusaitis, T., Ahuja, C. & Morency. L-P. Multimodal machine learning: a survey and taxonomy. IEEE Trans. Pattern Anal. Mach. Intell. 41, 423–443 (2019). · DOI · Paper page
  12. Hong, C., Yu, J., Wan, J., Tao, D. & Wang, M. Multimodal deep autoencoder for human pose recovery. IEEE Trans. Image Process. 24, 5659–5670 (2015). · DOI
  13. Khattar, D., Goud, J. S., Gupta, M. & Varma, V. MVAE: multimodal variational autoencoder for fake news detection. In The World Wide Web Conference 2915–2921 (Association for Computing Machinery, 2019). · DOI
  14. Mao, J., Xu, J., Jing, Y. & Yuille, A. Training and evaluating multimodal word embeddings with large-scale web annotated images. In Proc. 30th International Conference on Neural Information Processing Systems 442–450 (Curran Associates, 2016).
  15. Huang, Y., Lin, J., Zhou, C., Yang, H. & Huang, L. Modality competition: what makes joint training of multi-modal network fail in deep learning? (Provably). In Proc. 39th International Conference on Machine Learning: Proc. Machine Learning Research Vol. 162 (eds Chaudhuri, K. et al.) 9226–9259 (PMLR, 2022).
  16. Xu, P., Zhu, X. & Clifton, D. A. Multimodal learning with transformers: a survey. Preprint at https://arxiv.org/abs/2206.06488 (2022). · arXiv
  17. Bayoudh, K., Knani, R., Hamdaoui, F. & Mtibaa, A. A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets. Vis.Comput. 38, 2939–2970 (2022). · DOI · Paper page
  18. Zhang, C., Yang, Z., He, X. & Deng, L. Multimodal intelligence: representation learning, information fusion, and applications. IEEE J. Sel. Top. Signal Process. 14, 478–493 (2020). · DOI · Paper page
  19. Javaloy, A., Meghdadi, M. & Valera, I. Mitigating modality collapse in multimodal VAEs via impartial optimization. In Proc. 39th International Conference on Machine Learning: Proc. Machine Learning Research Vol. 162 (eds Chaudhuri, K. et al.) 9938–9964 (PMLR, 2022).
  20. Ma, M. et al. SMIL: multimodal learning with severely missing modality. Proc. AAAI Conf. Artif. Intell. 35, 2302–2310 (2021).
  21. Poklukar, P. et al. Geometric multimodal contrastive representation learning. in Proc. Mach. Learn. Res. 162, 17782–17800 (2022).
  22. Zitnik, M. et al. Machine learning for integrating data in biology and medicine: principles, practice, and opportunities. Inf. Fusion 50, 71–91 (2019). · DOI · Paper page
  23. Somnath, V. R., Bunne, C. & Krause, A. Multi-scale representation learning on proteins. Adv. Neural Inf. Process. Syst. 34 (2021).
  24. Walters, W. P. & Barzilay, R. Applications of deep learning in molecule generation and molecular property prediction. Acc. Chem. Res. 54, 263–270 (2021). · DOI
  25. Wang, J., Hu, J., Qian, S., Fang, Q. & Xu, C. Multimodal graph convolutional networks for high quality content recognition. Neurocomputing 412, 42–51 (2020). · DOI
  26. Mai, S., Hu, H. & Xing, S. Modality to modality translation: an adversarial representation learning and graph fusion network for multimodal fusion. Proc. AAAI Conf. Artif. Intell. 34, 164–172 (2020).
  27. Zhang, X., Zeman, M., Tsiligkaridis, T. & Zitnik, M. Graph-guided network for irregularly sampled multivariate time series. In International Conference on Learning Representations (2022).
  28. Zhao, F. & Wang, D. Multimodal Graph Meta Contrastive Learning 3657–3661 (Association for Computing Machinery, 2021). · DOI
  29. Ngiam, J. et al. Multimodal deep learning. In Proc. 28th International Conference on International Conference on Machine Learning 689–696 (Omnipress, 2011).
  30. Aafaq, N., Akhtar, N., Liu, W., Gilani, S. Z. & Mian, A. Spatio-temporal dynamics and semantic attribute enriched visual encoding for video captioning. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition 12487–12496 (IEEE, 2019). · DOI
  31. Fang, Z., Gokhale, T., Banerjee, P., Baral, C. & Yang, Y. Video2Commonsense: generating commonsense descriptions to enrich video captioning. In Proc. 2020 Conference on Empirical Methods in Natural Language Processing 840–860 (Association for Computational Linguistics, 2020). · DOI
  32. Kiros, R., Salakhutdinov, R. & Zemel, R. Multimodal neural language models. In Proc. 31st International Conference on Machine Learning: Proc. Machine Learning Research Vol. 32 (eds Xing, E. P. & Jebara, T.) 595–603 (PMLR, 2014).
  33. Rezaei-Shoshtari, S., Hogan, F. R., Jenkin, M., Meger, D. & Dudek, G. Learning intuitive physics with multimodal generative models. Proc. AAAI Conf. Artif. Intell. 35, 6110–6118 (2021).
  34. Bronstein, M. M., Bruna, J., Cohen, T. & Veličković, P. Geometric deep learning: grids, groups, graphs, geodesics, and gauges. Preprint at https://arxiv.org/abs/2104.13478 (2021). · arXiv
  35. Chen, Y. et al. Graph-based global reasoning networks. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition 433–442 (IEEE, 2019). · DOI
  36. Varga, V. & Lorincz, A. Fast interactive video object segmentation with graph neural networks. In International Joint Conference on Neural Networks 1–10 (IEEE, 2021). · DOI
  37. Liu, Q., Kampffmeyer, M., Jenssen, R. & Salberg, A.-B. Self-constructing graph neural networks to model long-range pixel dependencies for semantic segmentation of remote sensing images. Int. J. Remote Sensing 42, 6184–6208 (2021). · DOI
  38. Zhou, S., Zhang, J., Zuo, W. & Loy, C. C. Cross-scale internal graph neural network for image super-resolution. Adv. Neural Inf. Process. Syst. 33, 3499–3509 (2020).
  39. Mou, C. & Zhang, J. Graph attention neural network for image restoration. In 2021 IEEE International Conference on Multimedia and Expo 1–6 (IEEE, 2021). · DOI
  40. Qi, S., Wang, W., Jia, B., Shen, J. & Zhu, S. C. Learning human-object interactions by graph parsing neural networks. In Computer Vision – EECV 2018 407–423 (Springer, 2018). · DOI
  41. Wang, H., Zheng, W.-s. & Yingbiao, L. Contextual heterogeneous graph network for human–object interaction detection. In Computer Vision—ECCV 2020: Proc. 16th European Conference Part XVII 248–264 (Springer-Verlag, 2020). · DOI
  42. Avelar, P. C., Tavares, A. R., da Silveira, T. T., Jung, C. R. & Lamb, L. C. Superpixel image classification with graph attention networks. In 33rd SIBGRAPI Conference on Graphics, Patterns and Images 203–209 (IEEE Computer Society, 2020). · DOI
  43. Lu, Y., Chen, Y., Zhao, D. & Chen, J. in Advances in Neural Networks: Lecture Notes in Computer Science Vol. 11554 (eds Lu, H. et al.) 97–105 (Springer, 2019). · DOI
  44. Kim, J., Lee, J. K. & Lee, K. M. Deeply-recursive convolutional network for image super-resolution. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 1637–1645 (IEEE, 2016). · DOI
  45. Achanta, R. et al. SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34, 2274–2282 (2012). · DOI
  46. Zeng, H., Liu, Q., Zhang, M., Han, X. & Wang, Y. Semi-supervised hyperspectral image classification with graph clustering convolutional networks. Preprint at https://arxiv.org/abs/2012.10932 (2020). · arXiv
  47. Wan, S. et al. Multiscale dynamic graph convolutional network for hyperspectral image classification. IEEE Trans. Geosc. Remote Sensing 58, 3162–3177 (2019). · DOI
  48. Long, J., Shelhamer, E. & Darrell, T. Fully convolutional networks for semantic segmentation. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 3431–3440 (IEEE, 2015). · DOI · Paper page
  49. Simonyan, K. & Zisserman, A. Very deep convolutional networks for large-scale image recognition. In Proc. 3rd International Conference on Learning Representations (eds Bengio, Y. & LeCun, Y.) (2015).
  50. Knyazev, B., Lin, X., Amer, M. R. & Taylor, G. W. Image classification with hierarchical multigraph networks. In British Machine Vision Conference (2019).
  51. Veličković, P. et al. Graph attention networks. In International Conference on Learning Representations (2018).
  52. alsesia, D., Fracastoro, G. & Magli, E. Deep graph-convolutional image denoising. In IEEE Trans. Image Process. 8226–8237 (2020). · DOI
  53. Bresson, X. & Laurent, T. Residual gated graph ConvNets. Preprint at https://arxiv.org/abs/1711.07553 (2017). · arXiv
  54. Biten, A. F. et al. Scene text visual question answering. In Proc. IEEE/CVF International Conference on Computer Vision 4291–4301 (2019). · DOI
  55. Singh, A. et al. Towards VQA models that can read. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition 8317–8326 (IEEE, 2019). · DOI
  56. Liu, C. et al. Graph structured network for image-text matching. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition 10921–10930 (IEEE, 2020). · DOI
  57. Zhang, F. Z., Campbell, D. & Gould, S. Spatially conditioned graphs for detecting human–object interactions. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition 13319–13327 (IEEE, 2021). · DOI
  58. Ulutan, O., Iftekhar, A. S. M. & Manjunath, B. S. VSGNet: spatial attention network for detecting human object interactions using graph convolutions. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition 13617–13626 (IEEE, 2020). · DOI
  59. Gao, C., Xu, J., Zou, Y. & Huang, J.-B. DRG: Dual relation graph for human–object interaction detection. In Computer Vision—ECCV 2020 (eds Vedaldi, A. et al.) 696–712 (Springer, 2020). · DOI
  60. Zhou, P. & Chi, M. Relation parsing neural network for human–object interaction detection. In Proc. IEEE/CVF International Conference on Computer Vision 843–851 (IEEE, 2019). · DOI
  61. Gao, D., Li, K., Wang, R., Shan, S. & Chen, X. Multi-modal graph neural network for joint reasoning on vision and scene text. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition 12746–12756 (IEEE, 2020). · DOI
  62. Wu, T. et al. GINet: graph interaction network for scene parsing. In Computer Vision—ECCV 2020 (eds Vedaldi, A. et al.) 34–51 (Springer, 2020). · DOI
  63. Wu, L. et al. Graph neural networks for natural language processing: a survey. Preprint at https://arxiv.org/abs/2106.06090 (2021). · arXiv
  64. Vaswani, A. et al. Attention is all you need. Adv. Neural Inf. Process. Syst. 30, 5998–6008 (2017).
  65. Li, I., Li, T., Li, Y., Dong, R. & Suzumura, T. Heterogeneous graph neural networks for multi-label text classification. Preprint at https://arxiv.org/abs/2103.14620 (2021). · arXiv
  66. Huang, L., Ma, D., Li, S., Zhang, X. & Wang, H. Text level graph neural network for text classification. In Proc. 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing 3444–3450 (Association for Computational Linguistics, 2019). · DOI
  67. Zhang, Y. et al. Every document owns its structure: inductive text classification via graph neural networks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics 334–339 (Association for Computational Linguistics, 2020). · DOI
  68. Pan, J., Peng, M. & Zhang, Y. Mention-centered graph neural network for document-level relation extraction. Preprint at https://arxiv.org/abs/2103.08200 (2021). · arXiv
  69. Zhu, H. et al. Graph neural networks with generated parameters for relation extraction. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics 1331–1339 (Association for Computational Linguistics, 2019). · DOI
  70. Guo, Z., Zhang, Y. & Lu, W. Attention guided graph convolutional networks for relation extraction. In Proc. 57th Annual Meeting of the Association for Computational Linguistics 241–251 (Association for Computational Linguistics, 2019). · DOI
  71. Zeng, S., Xu, R., Chang, B. & Li, L. Double graph based reasoning for document-level relation extraction. In Proc. 2020 Conference on Empirical Methods in Natural Language Processing 1630–1640 (Association for Computational Linguistics, 2020). · DOI
  72. Chen, X. et al. Aspect sentiment classification with document-level sentiment preference modeling. In Proc. 58th Annual Meeting of the Association for Computational Linguistics 3667–3677 (Association for Computational Linguistics, 2020). · DOI
  73. Zhang, C., Li, Q. & Song, D. Aspect-based sentiment classification with aspect-specific graph convolutional networks. In Proc. 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing 4568–4578 (Association for Computational Linguistics, 2019). · DOI
  74. Zhang, M. & Qian, T. Convolution over hierarchical syntactic and lexical graphs for aspect level sentiment analysis. In Proc. 2020 Conference on Empirical Methods in Natural Language Processing 3540–3549 (Association for Computational Linguistics, 2020). · DOI
  75. Pouran Ben Veyseh, A. et al. Improving aspect-based sentiment analysis with gated graph convolutional networks and syntax-based regulation. In Findings of the Association for Computational Linguistics: EMNLP 2020 4543–4548 (Association for Computational Linguistics, 2020). · DOI
  76. Shlomi, J., Battaglia, P. & Vlimant, J.-R. Graph neural networks in particle physics. Mach. Learn. Sci. Technol. 2, 021001 (2021). · DOI
  77. Henrion, I. et al. Neural message passing for jet physics. In Deep Learning for Physical Sciences Workshop at the 31st Conference on Neural Information Processing Systems (2017).
  78. Qasim, S. R., Kieseler, J., Iiyama, Y. & Pierini, M. Learning representations of irregular particle-detector geometry with distance-weighted graph networks. Eur. Phys. J. C 79, 608 (2019). · DOI
  79. Mikuni, V. & Canelli, F. ABCnet: an attention-based method for particle tagging. Eur. Phys. J. Plus 135, 463 (2020). · DOI
  80. Ju, X. et al. Graph neural networks for particle reconstruction in high energy physics detectors. Preprint at https://arxiv.org/abs/2003.11603 (2020). · arXiv
  81. Shukla, K., Xu, M., Trask, N. & Karniadakis, G. E. Scalable algorithms for physics-informed neural and graph networks. Data Centric Eng. 3, e24 (2022). · DOI
  82. Seo, S. & Liu, Y. Differentiable physics-informed graph networks. Preprint at https://arxiv.org/abs/1902.02950 (2019). · arXiv
  83. Li, W. & Deka, D. Physics based GNNs for locating faults in power grids. Preprint at https://arxiv.org/abs/2107.02275 (2021). · arXiv
  84. Battaglia, P. W. et al. Relational inductive biases, deep learning, and graph networks. Preprint at https://arxiv.org/abs/1806.01261 (2018). · arXiv
  85. Veličković, P., Ying, R., Padovano, M., Hadsell, R. & Blundell, C. Neural execution of graph algorithms. In International Conference on Learning Representations (2020).
  86. Schuetz, M. J. A., Brubaker, J. K. & Katzgraber, H. G. Combinatorial optimization with physics-inspired graph neural networks. Nat. Mach. Intell. 4, 367–377 (2022). · DOI
  87. Gasteiger, J., Gross, J. & Günnemann, S. Directional message passing for molecular graphs. In International Conference on Learning Representations (2020).
  88. Jørgensen, P. B., Jacobsen, K. W. & Schmidt, M. N. Neural message passing with edge updates for predicting properties of molecules and materials. Preprint at https://arxiv.org/abs/1806.03146 (2018). · arXiv
  89. Gasteiger, J., Yeshwanth, C. & Günnemann, S. Directional message passing on molecular graphs via synthetic coordinates. Adv. Neural Inf. Process. Syst. 34, 15421–15433 (2021).
  90. Liu, M. et al. Fast quantum property prediction via deeper 2D and 3D graph networks. AI for Science Workshop (NeurIPS, 2021).
  91. St. John, P. C., Guan, Y., Kim, Y., Kim, S. & Paton, R. S. Prediction of organic homolytic bond dissociation enthalpies at near chemical accuracy with sub-second computational cost. Nat. Commun. 11, 2328 (2020). · DOI
  92. Pattanaik, L. et al. Message passing networks for molecules with tetrahedral chirality. Preprint at https://arxiv.org/abs/2012.00094 (2020). · arXiv
  93. Fey, M., Yuen, J.-G. & Weichert, F. Hierarchical inter-message passing for learning on molecular graphs. Preprint at https://arxiv.org/abs/2006.12179 (2020). · arXiv
  94. Ariëns, E. Chirality in bioactive agents and its pitfalls. Trends Pharmacol. Sci. 7, 200–205 (1986). · DOI
  95. Guan, Y. et al. Regio-selectivity prediction with a machine-learned reaction representation and on-the-fly quantum mechanical descriptors. Chem. Sci. 12, 2198–2208 (2021). · DOI
  96. Struble, T. J., Coley, C. W. & Jensen, K. F. Multitask prediction of site selectivity in aromatic C–H functionalization reactions. React. Chem. Eng. 5, 896–902 (2020). · DOI
  97. Stuyver, T. & Coley, C. W. Quantum chemistry-augmented neural networks for reactivity prediction: performance, generalizability, and explainability. J. Chem. Phys. 156, 084104 (2022). · DOI
  98. Stokes, J. M. et al. A deep learning approach to antibiotic discovery. Cell 180, 688–702.e13 (2020). · DOI · Paper page
  99. Fu, T. et al. Differentiable scaffolding tree for molecule optimization. In International Conference on Learning Representations (2022).
  100. Mercado, R. et al. Graph networks for molecular design. Mach. Learn. Sci. Technol. 2, 025023 (2021). · DOI
  101. Moon, S., Zhung, W., Yang, S., Lim, J. & Kim, W. Y. PIGMnet: a physics-informed deep learning model toward generalized drug-target interaction predictions. Chem. Sci. 13, 3661–3673 (2022). · DOI
  102. Gainza, P. et al. Deciphering interaction fingerprints from protein molecular surfaces using geometric deep learning. Nat. Methods 17, 184–192 (2020). · DOI
  103. Sanner, M. F., Olson, A. J. & Spehner, J.-C. Reduced surface: an efficient way to compute molecular surfaces. Biopolymers 38, 305–320 (1996). · DOI
  104. Sverrisson, F., Feydy, J., Correia, B. E. & Bronstein, M. M. Fast end-to-end learning on protein surfaces. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition 15272–15281 (IEEE, 2021). · DOI
  105. Feng, Y., You, H., Zhang, Z., Ji, R. & Gao, Y. Hypergraph neural networks. Proc. AAAI Conf. Artif. Intell. 33, 3558–3565 (2019).
  106. Srinivasan, B., Zheng, D. & Karypis, G. Learning over Families of Sets—Hypergraph Representation Learning for Higher Order Tasks 756–764 (SIAM Activity Group on Data Science, 2021). · DOI
  107. Jo, J. et al. Edge representation learning with hypergraphs. Adv. Neural Inf. Process. Syst. 34, 7534–7546 (2021).
  108. Zhang, C., Song, D., Huang, C., Swami, A. & Chawla, N. V. Heterogeneous graph neural network. In Proc. 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining 793–803 (Association for Computing Machinery, 2019). · DOI · Paper page
  109. Chandak, P., Huang, K. & Zitnik, M. Building a knowledge graph to enable precision medicine. Sci. Data 10, 67 (2023). · DOI · Paper page
  110. Lee, S. & Song, B. C. Graph-based knowledge distillation by multi-head attention network. In Proc. British Machine Vision Conference (eds Sidorov, K. & Hicks, Y.) 162.1–162.12 (BMVA, 2019).
  111. Zhou, S. et al. Distilling holistic knowledge with graph neural networks. In Proc. IEEE/CVF International Conference on Computer Vision 10387–10396 (IEEE, 2021). · DOI
  112. Sun, L., Gou, J., Yu, B., Du, L. & Tao, D. Collaborative teacher–student learning via multiple knowledge transfer. Preprint at https://arxiv.org/abs/2101.08471 (2021). · arXiv
  113. Park, W., Kim, D., Lu, Y. & Cho, M. Relational knowledge distillation. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition 3967–3976 (IEEE, 2019). · DOI
  114. Liu, Y. et al. Knowledge distillation via instance relationship graph. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition 7096–7104 (IEEE, 2019). · DOI
  115. Ma, J. et al. Using deep learning to model the hierarchical structure and function of a cell. Nat. Methods 15, 290–298 (2018). · DOI · Paper page
  116. Borisov, V. et al. Deep neural networks and tabular data: a survey. in IEEE Transactions on Neural Networks and Learning Systems https://doi.org/10.1109/TNNLS.2022.3229161 (2022). · DOI
  117. Jiang, D. et al. Could graph neural networks learn better molecular representation for drug discovery? A comparison study of descriptor-based and graph-based models. J. Cheminform. 13, 12 (2021). · DOI · Paper page
  118. Nicholson, D. N. & Greene, C. S. Constructing knowledge graphs and their biomedical applications. Comput. Struct. Biotechnol. J. 18, 1414–1428 (2020). · DOI
  119. Xu, K., Hu, W., Leskovec, J. & Jegelka, S. How powerful are graph neural networks? In International Conference on Learning Representations (2019).
  120. Hamilton, W., Ying, Z. & Leskovec, J. Inductive representation learning on large graphs. Adv. Neural Inf. Process. Syst. 30, 1024–1034 (2017).
  121. Xu, K. et al. Representation learning on graphs with jumping knowledge networks. In Proc. 35th International Conference on Machine Learning: Proc. Machine Learning Research Vol. 80 (eds Dy, J. & Krause, A.) 5453–5462 (PMLR, 2018).

Cited by

Sorted by citations

Metadata

Authors
Ektefaie, Y., Dasoulas, G., Noori, A., Farhat, M., & Zitnik, M.
Affiliations
Harvard University (US)
ORCID
Venue
Nature Machine Intelligence
Publisher
Springer Science and Business Media LLC
Published
2023-04-03
Volume
5
Issue
4
Pages
340-350
ISSN
2522-5839 (ISSN-L)
DOI
10.1038/s42256-023-00624-6
PMC
PMC10704992
OpenAlex
W4362579348
Cited by
166
References
97
Citations by year
2026: 44 · 2025: 72 · 2024: 37 · 2023: 13
FWCI
15.65
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature Machine Intelligence (publishedVersion)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Funders
  • AstraZeneca
  • U.S. Air Force (FA8702‐15‐D‐0001)
  • Harvard Data Science Initiative, Harvard University
  • National Human Genome Research Institute (T32HG002295)
  • National Institute of Allergy and Infectious Diseases (R01AI155765)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Advanced Graph Neural Networks
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
  • Physical Sciences › Computer Science › Computational Theory and Mathematics › Graph Theory and Algorithms
Keywords
graph representation learning, inductive bias, geometric relationships

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review