Heterogeneous Graph Transformer
Open accessAbstract
Recent years have witnessed the emerging success of graph neural networks (GNNs) for modeling structured data. However, most GNNs are designed for homogeneous graphs, in which all nodes and edges belong to the same types, making it infeasible to represent heterogeneous structures. In this paper, we present the Heterogeneous Graph Transformer (HGT) architecture for modeling Web-scale heterogeneous graphs. To model heterogeneity, we design node- and edge-type dependent parameters to characterize the heterogeneous attention over each edge, empowering HGT to maintain dedicated representations for different types of nodes and edges. To handle Web-scale graph data, we design the heterogeneous mini-batch graph sampling algorithm—HGSampling—for efficient and scalable training. Extensive experiments on the Open Academic Graph of 179 million nodes and 2 billion edges show that the proposed HGT model consistently outperforms all the state-of-the-art GNN baselines by 9–21 on various downstream tasks. The dataset and source code of HGT are publicly available at https://github.com/acbull/pyHGT.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
HGNN-based approaches offer enhanced capabilities for detecting complex and evolving node behaviors in heterogeneous systems by incorporating temporal reasoning, type-aware message passing, and schema-guided evolution modeling.
“In contrast, HGNN-based approaches offer enhanced capabilities for detecting complex and evolving node behaviors in heterogeneous systems by incorporating temporal reasoning, type-aware message passing, and schema-guided evolution modeling. [4], [19]”
From A Survey of Heterogeneous Graph Neural Networks for Cybersecurity Anomaly Detection (2025) · cites this paper as [19]
The Introduction and §2.4 describe type-aware attention and message passing using node and edge types; §4.4 shows that HGT learns task-relevant meta paths. The paper does not describe temporal reasoning or schema-guided evolution modeling, and its evaluated tasks (§4.2) do not establish detection of evolving node behaviors. (LLM, conf=0.87)
Experimental — accuracy not yet measuredCybersecurity data is inherently multi-entity, multi-relation, and evolves over time.
“Cybersecurity data is inherently multi-entity, multi-relation, and evolves over time. [76]”
From A Survey of Heterogeneous Graph Neural Networks for Cybersecurity Anomaly Detection (2025) · cites this paper as [76]
The Introduction describes heterogeneous graphs as containing different types of entities and relationships, and §2.1 formalizes node and edge types. The paper’s examples and evaluation concern academic and other general networks; our copy does not establish that cybersecurity data has these properties or that it evolves over time. (LLM, conf=0.82)
Experimental — accuracy not yet measuredHeterogeneous graphs support multiple node and edge types, capturing rich semantic information essential for modeling diverse cybersecurity behaviors.
“In contrast, heterogeneous graphs [18], [19] involve multiple types of nodes and/or edges.”
From A Survey of Heterogeneous Graph Neural Networks for Cybersecurity Anomaly Detection (2025) · cites this paper as [19]
Edges in molecular graphs can be added to indicate pairwise spatial distance between atoms, or directed with bond angles and rotations incorporated.
“Edges can also be added to indicate pairwise spatial distance between two atoms [72], or directed with information on bond angles and rotations incorporated into the molecular graph [84].”
From Graph representation learning in biomedicine and healthcare (2022) · cites this paper as [72]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
Semi-supervised classification with graph convolutional networks
Kipf, T. N., & Welling, M. (2016) · arXiv · doi:10.48550/arxiv.1609.02907 · Cited by 24,853 · see citing works
Decoupled Weight Decay Regularization
Loshchilov, I., & Hutter, F. (2017) · arXiv · doi:10.48550/arxiv.1711.05101 · Cited by 12,522 · see citing works
Veličković, P., Cucurull, G., Casanova, A., Romero, A., Liò, P., & Bengio, Y. (2017) · arXiv · doi:10.48550/arxiv.1710.10903 · Cited by 12,107
Inductive Representation Learning on Large Graphs
Hamilton, W. L., Ying, R., & Leskovec, J. (2017) · arXiv · doi:10.48550/arxiv.1706.02216 · Cited by 10,639 · see citing works
Transformers: State-of-the-Art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2020.emnlp-demos.6 · Cited by 8,741 · see citing works
XLNet: Generalized Autoregressive Pretraining for Language Understanding
Yang, Z., Dai, Z., Yang, Y., Carbonell, J., Salakhutdinov, R., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1906.08237 · Cited by 6,687 · see citing works
Modeling Relational Data with Graph Convolutional Networks
Schlichtkrull, M., Kipf, T. N., Bloem, P., van den Berg, R., Titov, I., & Welling, M. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-93417-4_38 · Cited by 5,570 · see citing works
SGDR: Stochastic Gradient Descent with Warm Restarts
Loshchilov, I., & Hutter, F. (2016) · arXiv · doi:10.48550/arxiv.1608.03983 · Cited by 5,309 · see citing works
Fast Graph Representation Learning with PyTorch GeometricPreprint
Fey, M., & Lenssen, J. E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.02428 · Cited by 3,402 · see citing works
Heterogeneous Graph Attention Network
Wang, X., Ji, H., Shi, C., Wang, B., Ye, Y., Cui, P., & Yu, P. S. (2019) · The World Wide Web Conference · doi:10.1145/3308558.3313562 · Cited by 3,191 · see citing works
Dong, Y., Chawla, N. V., & Swami, A. (2017) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3097983.3098036 · Cited by 2,224 · see citing works
Self-citationLearning to rank for information retrieval
Liu, T. (2010) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/1835449.1835676 · Cited by 1,879 · see citing works
Sun, Y., Han, J., Yan, X., Yu, P. S., & Wu, T. (2011) · Proceedings of the VLDB Endowment · doi:10.14778/3402707.3402736 · Cited by 1,875 · see citing works
Self-citationHeterogeneous Graph Neural Network
Zhang, C., Song, D., Huang, C., Swami, A., & Chawla, N. V. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330961 · Cited by 1,591 · see citing works
FastGCN: Fast Learning with Graph Convolutional Networks via Importance SamplingPreprint
Chen, J., Ma, T., & Xiao, C. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.10247 · Cited by 1,172 · see citing works
An Overview of Microsoft Academic Service (MAS) and Applications
Sinha, A., Shen, Z., Song, Y., Ma, H., Eide, D., Hsu, B.-J. P., & Wang, K. (2015) · International World Wide Web Conference (WWW) · doi:10.1145/2740908.2742839 · Cited by 968 · see citing works
Self-citationGraph Transformer NetworksPreprint
Yun, S., Jeong, M., Kim, R., Kang, J., & Kim, H. J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.06455 · Cited by 747 · see citing works
Mining Heterogeneous Information Networks: Principles and Methodologies
Sun, Y., & Han, J. (2012) · Synthesis lectures on data mining and knowledge discovery · doi:10.2200/s00433ed1v01y201207dmk005 · Cited by 481 · see citing works
Self-citationLearning to Rank for Information Retrieval
Liu, T. (2011) · Springer eBooks · doi:10.1007/978-3-642-14267-3 · Cited by 411 · see citing works
Sun, Y., Norick, B., Han, J., Yan, X., Yu, P. S., & Yu, X. (2012) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2339530.2339738 · Cited by 199 · see citing works
Self-citationLayer-Dependent Importance Sampling for Training Deep and Large Graph Convolutional NetworksPreprint
Zou, D., Hu, Z., Wang, Y., Jiang, S., Sun, Y., & Gu, Q. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.07323 · Cited by 144 · see citing works
Self-citationZhang, F., Liu, X., Tang, J., Dong, Y., Yao, P., Zhang, J., Gu, X., Wang, Y., Shao, B., Li, R., et al. (2019) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3292500.3330785 · Cited by 97 · see citing works
Self-citationLearning to Rank for Information Retrieval and Natural Language Processing
Li, H. (2015) · Synthesis lectures on human language technologies · doi:10.1007/978-3-031-02155-8 · Cited by 64 · see citing works
Learning to Rank for Information Retrieval and Natural Language Processing, Second Edition
Li, H. (2014) · Synthesis lectures on human language technologies · doi:10.2200/s00607ed2v01y201410hlt026 · Cited by 47 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- Yizhou Sun, Jiawei Han, Xifeng Yan, Philip S. Yu, and Tianyi Wu. 2011. Pathsim: Meta path-based top-k similarity search in heterogeneous information networks. In VLDB '11.
- Fanjin Zhang, Xiao Liu, Jie Tang, Yuxiao Dong, Peiran Yao, Jie Zhang, Xiaotao Gu, Yan Wang, Bin Shao, Rui Li, and Kuansan Wang. 2019. OAG: Toward Linking Large-scale Heterogeneous Entity Graphs. In KDD'19.
Cited by
Sorted by citations
Graph neural networks: A review of methods and applications
Zhou, J., Cui, G., Hu, S., Zhang, Z., Yang, C., Liu, Z., Wang, L., Li, C., & Sun, M. (2020) · AI Open · doi:10.1016/j.aiopen.2021.01.001 · Cited by 6,880 · see citing works
V2X-ViT: Vehicle-to-Everything Cooperative Perception with Vision Transformer
Xu, R., Xiang, H., Tu, Z., Xia, X., Yang, M.-H., & Ma, J. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19842-7_7 · Cited by 584 · see citing works
Explainable Artificial Intelligence Applications in Cyber Security: State-of-the-Art in Research
Zhang, Z., Hamadi, H. A., Damiani, E., Yeun, C. Y., & Taher, F. (2022) · IEEE Access · doi:10.1109/access.2022.3204051 · Cited by 450 · see citing works
Graph representation learning in biomedicine and healthcare
Li, M. M., Huang, K., & Zitnik, M. (2022) · Nature Biomedical Engineering · doi:10.1038/s41551-022-00942-x · Cited by 294 · see citing works
“Recent architectures have leveraged dynamic message passing 72,85,86 to deal with evolving and time-varying graphs, as well as few-shot learning 87 or self-supervised strategies 88,89 to deal with graphs that are poorly annotated and that have limited information about labels.”
A foundation model for clinician-centered drug repurposing
Huang, K., Chandak, P., Wang, Q., Havaldar, S., Vaid, A., Leskovec, J., Nadkarni, G. N., Glicksberg, B. S., Gehlenborg, N., & Zitnik, M. (2024) · Nature Medicine · doi:10.1038/s41591-024-03233-x · Cited by 249 · see citing works
“…, diffusion state distance (DSD) 17 , state-of-the-art GNN methods, including relational graph convolutional networks (RGCNs) 18,26 , heterogeneous graph transformer (HGT) 27 and heterogeneous attention networks (HANs) 28 and a natural language-processing model, BioBERT 29 (Supplementary Note 4).”
BackgroundPolygonGNN: Representation Learning for Polygonal Geometries with Heterogeneous Visibility Graph
Yu, D., Hu, Y., Li, Y., & Zhao, L. (2024) · Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3637528.3671738 · Cited by 15 · see citing works
A Survey of Heterogeneous Graph Neural Networks for Cybersecurity Anomaly Detection
Jiang, L., Ryan, R., Li, Q., & Ferdosian, N. (2025) · arXiv · doi:10.48550/arxiv.2510.26307
“[76] Threat behaviors, whether driven by malicious insiders, external intruders, or advanced persistent actors, rarely occur in isolation.”
- Authors
- Hu, Z., Dong, Y., Wang, K., & Sun, Y.
- Affiliations
- University of California, Los Angeles (US)
- Microsoft (United States) (US)
- Venue
- Proceedings of The Web Conference 2020
- Publisher
- ACM
- Published
- 2020-04-20
- Pages
- 2704-2710
- OpenAlex
- W3012871709
- Cited by
- 1,646
- References
- 26
- Citations by year
- 2026: 340 · 2025: 350 · 2024: 346 · 2023: 258 · 2022: 191 · 2021: 129 · 2020: 32
- FWCI
- 57.08
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- International World Wide Web Conference (WWW) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1937599, 1741634, CAREER, 1705169)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Graph Neural Networks
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Graph Theory and Algorithms
- Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
- Keywords
- heterogeneous graph neural networks, graph transformer, scalable training
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
