The DOI matches its Crossref record; 91 of the 91 references it cites were found in a registry or an index.
Exploring the Potential of Large Language Models (LLMs)in Learning on Graphs
Open accessAbstract
Learning on Graphs has attracted immense attention due to its wide real-world applications. The most popular pipeline for learning on graphs with textual node attributes primarily relies on Graph Neural Networks (GNNs), and utilizes shallow text embedding as initial node representations, which has limitations in general knowledge and profound semantic understanding. In recent years, Large Language Models (LLMs) have been proven to possess extensive common knowledge and powerful semantic comprehension abilities that have revolutionized existing workflows to handle text data. In this paper, we aim to explore the potential of LLMs in graph machine learning, especially the node classification task, and investigate two possible pipelines: LLMs-as-Enhancers and LLMs-as-Predictors. The former leverages LLMs to enhance nodes' text attributes with their massive knowledge and then generate predictions through GNNs. The latter attempts to directly employ LLMs as standalone predictors. We conduct comprehensive and systematical studies on these two pipelines under various settings. From comprehensive empirical results, we make original observations and find new insights that open new possibilities and suggest promising directions to leverage LLMs for learning on graphs. Our codes and datasets are available at: https://github.com/CurryTang/Graph-LLM .
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Our review also found that only 6 of the 23 surveyed benchmark studies ( [7], [10], [31], [39], [44], [52]) were peerreviewed at the time of this article, reflecting the early-stage nature of research in this domain.”
From Inadequacies of Large Language Model Benchmarks in the Era of Generative Artificial Intelligence (2026) · cites this paper as [31]
In the citing paper
“HELM [31]: The study acknowledged the challenge of standardizing language model evaluation across diverse scenarios, indicating an understanding of response variability but not fully addressing it through its methodology.”
From Inadequacies of Large Language Model Benchmarks in the Era of Generative Artificial Intelligence (2026) · cites this paper as [31]
In the citing paper
“HELM [31]: The HELM evaluation framework included targeted evaluations and multi-metric measurement across various scenarios, but did not specifically address the tension between helpfulness and harmlessness in its criteria or methodologies, indicating an unresolved challenge in balancing these aspects within the”
From Inadequacies of Large Language Model Benchmarks in the Era of Generative Artificial Intelligence (2026) · cites this paper as [31]
In the citing paper
“The study acknowledged the challenge of accounting for linguistic diversity in benchmarks, but did not provide a solution, exemplified by its focus on English and limited consideration of other languages, which indicates an unaddressed and unresolved inadequacy regarding linguistic differences and embedded logics.”
From Inadequacies of Large Language Model Benchmarks in the Era of Generative Artificial Intelligence (2026) · cites this paper as [31]
In the citing paper
“The study acknowledged the challenge of ensuring uniform evaluation methods across all models, highlighting ongoing efforts to refine and adapt benchmarking practices to maintain relevancy and accuracy in assessments, without explicitly stating that this inadequacy has been fully resolved.”
From Inadequacies of Large Language Model Benchmarks in the Era of Generative Artificial Intelligence (2026) · cites this paper as [31]
In the citing paper
“The study utilized a standardized fewshot prompting adaptation for all models, indicating an awareness of prompt engineering challenges.However, it also highlighted the sensitivity of model performance to prompt formatting and adaptation methods, revealing an ongoing struggle with crafting prompts that accurately assess model capabilities without introducing biases or inaccuracies.”
From Inadequacies of Large Language Model Benchmarks in the Era of Generative Artificial Intelligence (2026) · cites this paper as [31]
In the citing paper
“The study implemented a comprehensive and systematic evaluation approach to benchmarking LLMs, incorporating extensive metrics and scenarios, which inherently necessitated significant computational resources and time for thorough evaluation, thus likely extending iteration time for model testing.The study, while acknowledging the broad and intricate scope of its benchmark, did not specifically”
From Inadequacies of Large Language Model Benchmarks in the Era of Generative Artificial Intelligence (2026) · cites this paper as [31]
In the citing paper
“The study's extensive evaluation framework incorporated metrics such as fairness, bias, and toxicity across a diverse set of scenarios, including multiple languages and cultural contexts, aiming to measure LLMs' performance holistically.However, the study acknowledged its limitations in fully capturing the pluralistic nature of human beliefs and values, due to inherent challenges in standardizing”
From Inadequacies of Large Language Model Benchmarks in the Era of Generative Artificial Intelligence (2026) · cites this paper as [31]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Semi-supervised classification with graph convolutional networks
Kipf, T. N., & Welling, M. (2016) · arXiv · doi:10.48550/arxiv.1609.02907 · Cited by 24,853
Advances in Neural Information Processing Systems 19
(2007) · The MIT Press eBooks · doi:10.52202/085713 · Cited by 16,043
Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Reimers, N., & Gurevych, I. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1410 · Cited by 12,778
Inductive Representation Learning on Large Graphs
Hamilton, W. L., Ying, R., & Leskovec, J. (2017) · arXiv · doi:10.48550/arxiv.1706.02216 · Cited by 10,639
Neural Message Passing for Quantum ChemistryPreprint
Gilmer, J., Schoenholz, S. S., Riley, P. F., Vinyals, O., & Dahl, G. E. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.01212 · Cited by 5,678
Harris, Z. S. (1954) · WORD · doi:10.1080/00437956.1954.11659520 · Cited by 4,807
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2201.11903 · Cited by 4,263
Flagged first, then by citations
Research works printed but not linked: 10
- V. P. Dwivedi, L. Ramp´a'sek, M. Galkin, A. Parviz, G. Wolf, A. T. Luu, and D. Beaini. Long range graph benchmark. In Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2022.
- S. Gui, X. Li, L. Wang, and S. Ji. GOOD: A graph outof- distribution benchmark. In Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2022.
- X. He, X. Bresson, T. Laurent, and B. Hooi. Explanations as features: Llm-based features for text-attributed graphs. arXiv preprint arXiv:2305.19523, 2023. · arXiv
- G. Li, M. M¨uller, B. Ghanem, and V. Koltun. Training graph neural networks with 1000 layers. In International conference on machine learning, pages 6437-- 6449. PMLR, 2021.
- OpenAI. Introducing chatgpt, 2022.
- A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever. Language models are unsupervised multitask learners. 2019.
- N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu. Exploring the limits of transfer learning with a unified textto- text transformer. Journal of Machine Learning Research, 21(140):1--67, 2020.
- T. Sun, Y. Shao, H. Qian, X. Huang, and X. Qiu. Blackbox tuning for language-model-as-a-service. In International Conference on Machine Learning, pages 20841-- 20855. PMLR, 2022.
- G. Cucurull, A. Casanova, A. Romero, P. Li'o, and Y. Bengio. Graph attention networks. In International Conference on Learning Representations, 2018.
- H. Wang, Y. Gao, X. Zheng, P. Zhang, H. Chen, and J. Bu. Graph neural architecture search with gpt-4. arXiv preprint arXiv:2310.01436, 2023. · arXiv
Cited by
Sorted by citations
Items 201–210 of 210
Ying, Y., Jiang, W., Zheng, T., Wang, Y.-Q., Liu, S., Chen, K., & Song, M. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3817937 · Cited by 1
HyMAGE: Semantic-Aware Dynamic Hypergraph Generation
Gu, B., Zeng, J., Zhou, N., Qi, X., & Li, D. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3817897
Wang, Y., Ding, Y., Ye, Y., Zhao, M., Zhang, W., Wang, R., & Li, J. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3817966
Optimizing information retrieval tasks with large language model for data enhancement
Vaidya, C. D., Amudhavel, J., Mishra, P. K., Palekar, V. R., Tikhe, G. N., & Bhure, K. (2026) · Journal of Nonlinear Complex and Data Science · doi:10.1515/jncds-2025-0124
Leveraging gated architectures for enhanced long-range dependency learning in non-attributed graphs
Lee, J., Lee, D., An, C., & Park, E. K. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.116985
Trustworthy LLM–GNN systems: a systematic review and multi-axis taxonomy
Xue, R., He, F., Deng, H., Wang, M., & Zhang, Z. (2026) · Applied Intelligence · doi:10.1007/s10489-026-07458-w
Zhang, Y. (2026) · International Conference on Computer Information Science and Application Technology (CISAT) · doi:10.1109/cisat71030.2026.11682228
Graph Foundation Models: State of the Art and Future Directions
Zhou, A., Li, H., & Chen, L. (2026) · Proceedings of the VLDB Endowment · doi:10.14778/3827998.3828150
Self-citationXu, A., Chang, B., Shu, K., Jian, L., & Song, Y. (2026) · Scientometrics · doi:10.1007/s11192-026-05807-7
GaCLLM: Large Language Model with Graph Convolution for Recommendation
Du, Y., Wang, Z., Sun, Z., Chua, H., Ong, Y.-S., & Zhang, J. (2026) · ACM Transactions on Information Systems · doi:10.1145/3857345
Metadata
- Authors
- Chen, Z., Mao, H., Li, H., Jin, W., Wen, H., Wei, X., Wang, S., Yin, D., Fan, W., Liu, H., & Tang, J.
- Affiliations
- Michigan State University (US)
- Emory University (US)
- Baidu (China) (CN)
- Hong Kong Polytechnic University (HK)
- ORCID
- Venue
- ACM SIGKDD Explorations Newsletter
- Published
- 2024-03-26
- Volume
- 25
- Issue
- 2
- Pages
- 42-61
- ISSN
- 1931-0145 (ISSN-L)
- 1931-0153
- OpenAlex
- W4393277515
- Cited by
- 209
- References
- 57
- Citations by year
- 2026: 57 · 2025: 104 · 2024: 46
- FWCI
- 30.67
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- ACM SIGKDD Explorations Newsletter (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Graph Neural Networks
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Graph Theory and Algorithms
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- large language models, graph neural networks, node classification, text embeddings, semantic understanding, knowledge improvement
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
