Feature Distribution Distillation-Based Few Shot Class Incremental Learning

Abstract

This paper focuses on the challenging but practical problem: convolutional neural networks (CNN) based few-shot class-incremental learning (FSCIL), which requires learning new knowledge incrementally from a few examples without forgetting the previously learned knowledge. To balance the memory of the old knowledge and the adaptation to the new knowledge, we propose a feature distribution distillation-based incremental learning framework. First, we freeze the shallow parameters of the network to alleviate the massive forgetting of the learned knowledge. Then, we propose a feature distribution-based knowledge distillation loss to transfer the previously knowledge. Finally, we also generate the classification weights initialization for each new class to improve the adaptation of the network. Extensive experiments show that our proposed method can effectively alleviate the forgetting of the previously knowledge and improve the adaptability to the new knowledge. Our method outperforms other state-of-the-art few-shot class incremental learning methods on MiniImageNet dataset and gets a competitive result on CIFAR-100 dataset.