Graph Few-shot Class-incremental Learning
The DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index.
Abstract
The ability to incrementally learn new classes is vital to all real-world artificial intelligence systems. A large portion of high-impact applications like social media, recommendation systems, E-commerce platforms, etc. can be represented by graph models. In this paper, we investigate the challenging yet practical problem,Graph Few-shot Class-incremental (Graph FCL) problem, where the graph model is tasked to classify both newly encountered classes and previously learned classes. Towards that purpose, we put forward a Graph Pseudo Incremental Learning paradigm by sampling tasks recurrently from the base classes, so as to produce an arbitrary number of training episodes for our model to practice the incremental learning skill. Furthermore, we design a Hierarchical-Attention-based Graph Meta-learning framework, HAG-Meta from an optimization perspective. We present a task-sensitive regularizer calculated from task-level attention and node class prototypes to mitigate overfitting onto either novel or base classes. To employ the topological knowledge, we add a node-level attention module to adjust the prototype representation. Our model not only achieves greater stability of old knowledge consolidation, but also acquires advantageous adaptability to new knowledge with very limited data samples. Extensive experiments on three real-world datasets, including Amazon-clothing, Reddit, and DBLP, show that our framework demonstrates remarkable advantages in comparison with the baseline and other related state-of-the-art methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Subramanian, A., Tamayo, P., Mootha, V. K., Mukherjee, S., Ebert, B. L., Gillette, M. A., Paulovich, A., Pomeroy, S. L., Golub, T. R., Lander, E. S., et al. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0506580102 · Cited by 53,625
Overcoming catastrophic forgetting in neural networks
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1611835114 · Cited by 7,899
How Powerful are Graph Neural Networks?
Xu, K., Hu, W., Leskovec, J., & Jegelka, S. (2018) · arXiv · doi:10.48550/arxiv.1810.00826 · Cited by 6,078
Li, Z., & Hoiem, D. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2773081 · Cited by 4,202
ArnetMiner: extraction and mining of academic social networks
Tang, J., Zhang, J., Yao, L., Li, J., Zhang, L., & Su, Z. (2008) · Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1401890.1402008 · Cited by 2,231 · cited 2×
“We conduct our experiments 1 on three widely used graph Few-shot learning datasets: Amazon-Clothing [23], DBLP [36], and Reddit [9].”
BackgroundMethodLarge Scale Incremental Learning
Wu, Y., Chen, Y., Wang, L., Ye, Y., Liu, Z., Guo, Y., & Fu, Y. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00046 · Cited by 1,313
Deep Convolutional Networks on Graph-Structured Data
Henaff, M., Bruna, J., & LeCun, Y. (2015) · arXiv · doi:10.48550/arxiv.1506.05163 · Cited by 1,255
Learning a Unified Classifier Incrementally via Rebalancing
Hou, S., Pan, X., Loy, C. C., Wang, Z., & Lin, D. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00092 · Cited by 1,195 · cited 1×
“compared to the base classes, resulting in a severe class-imbalance problem [12, 37, 44].”
BackgroundRiemannian Walk for Incremental Learning: Understanding Forgetting and Intransigence
Chaudhry, A., Dokania, P. K., Ajanthan, T., & Torr, P. H. S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01252-6_33 · Cited by 1,091 · cited 1×
“The other family of methods is regularizationbased methods [2, 18], where various regularizers are proposed to regularize the parameters of a neural network so that more important parameters concerning the previous task can be protected when models are trained on each new task.”
MethodAn Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks
Goodfellow, I. J., Mirza, M., Xiao, D., Courville, A., & Bengio, Y. (2013) · arXiv · doi:10.48550/arxiv.1312.6211 · Cited by 1,089
Measuring Catastrophic Forgetting in Neural Networks
Kemker, R., McClure, M., Abitino, A., Hayes, T., & Kanan, C. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11651 · Cited by 672
Inferring Networks of Substitutable and Complementary Products
McAuley, J., Pandey, R., & Leskovec, J. (2015) · Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2783258.2783381 · Cited by 645 · cited 1×
“We conduct our experiments 1 on three widely used graph Few-shot learning datasets: Amazon-Clothing [23], DBLP [36], and Reddit [9].”
MethodExperience Replay for Continual Learning
Rolnick, D., Ahuja, A., Schwarz, J., Lillicrap, T. P., & Wayne, G. (2018) · arXiv · doi:10.48550/arxiv.1811.11682 · Cited by 529 · cited 1×
“The first family includes replay-based methods [22, 30, 32, 35], which maintain a subset of previous samples, and train models together with samples in the new session.”
MethodFew-Shot Class-Incremental Learning
Tao, X., Hong, X., Chang, X., Dong, S., Wei, X., & Gong, Y. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01220 · Cited by 499
Dual Graph Convolutional Networks for Graph-Based Semi-Supervised Classification
Zhuang, C., & Ma, Q. (2018) · Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW '18 · doi:10.1145/3178876.3186116 · Cited by 423
Few-Shot Incremental Learning with Continually Evolved Classifiers
Zhang, C., Song, N., Lin, G., Zheng, Y., Pan, P., & Xu, Y. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01227 · Cited by 382 · cited 4×
“[31, 46, 47] further extend the situation to the Few-shot Class-incremental Learning (FCL) setting on image domain.”
BackgroundMethodInfluentialEncoder Based Lifelong Learning
Rannen, A., Aljundi, R., Blaschko, M. B., & Tuytelaars, T. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.148 · Cited by 324
Cited by
A survey on few-shot class-incremental learning
Tian, S., Li, L., Li, W., Ran, H., Ning, X., & Tiwari, P. (2024) · Neural Networks · doi:10.1016/j.neunet.2023.10.039 · Cited by 220
S3C: Self-Supervised Stochastic Classifiers for Few-Shot Class-Incremental Learning
Kalla, J., & Biswas, S. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19806-9_25 · Cited by 45
“The works in [11,34,44] used graph-based networks for old classes’ knowledge retention.”
MethodContinual Learning for Generative Retrieval over Dynamic Corpora
Chen, J., Zhang, R., Guo, J., de Rijke, M., Chen, W., Fan, Y., & Cheng, X. (2023) · Proceedings of the 32nd ACM International Conference on Information and Knowledge Management · doi:10.1145/3583780.3614821 · Cited by 40
“Continual learning (CL) has been a longstanding research topic to overcome the catastrophic forgetting problem of previously acquired knowledge, while continuously learning new knowledge from few labeled samples [43].”
Metadata
- Authors
- Tan, Z., Ding, K., Guo, R., & Liu, H.
- Venue
- Proceedings of the Fifteenth ACM International Conference on Web Search and Data Mining
- Publisher
- ACM
- Published
- 2022-02-11
- Pages
- 987-996
- OpenAlex
- W4212857616
- Cited by
- 42
- References
- 17
- FWCI
- 5.69
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.