The DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
Continual learning of context-dependent processing in neural networks
Open accessOverview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“It has been shown that input created from a generative model can indeed assist with learning, such as preventing catastrophic forgetting.149”
From The overfitted brain: Dreams evolved to assist generalization (2021) · cites this paper as [149]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Golub, G. (2012) · doi:10.56021/9781421407944 · Cited by 34,505
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741
An Integrative Theory of Prefrontal Cortex Function
Miller, E. K., & Cohen, J. D. (2001) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.24.1.167 · Cited by 12,803
Neural Mechanisms of Selective Visual Attention
Desimone, R., & Duncan, J. (1995) · Annual Review of Neuroscience · doi:10.1146/annurev.ne.18.030195.001205 · Cited by 8,388
Overcoming catastrophic forgetting in neural networks
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1611835114 · Cited by 7,899
Deep Learning Face Attributes in the Wild
Liu, Z., Luo, P., Wang, X., & Tang, X. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.425 · Cited by 7,747
International Conference on Learning Representations (ICLR 2013)
松尾, 豊. (2013) · 人工知能学会誌 = Journal of Japanese Society for Artificial Intelligence · Cited by 6,286
Research works printed but not linked: 15
- Newell, A. Unified Theories of Cognition (Harvard Univ. Press, 1994).
- Haykin, S. S. Adaptive Filter theory (Pearson Education India, 2008).
- Singhal, S. & Wu, L. Training feed-forward networks with the extended kalman algorithm. In International Conference on Acoustics, Speech, and Signal Processing 1187–1190 (IEEE, 1989).
- Jaeger, H. Controlling recurrent neural networks by conceptors. Preprint at https://arxiv.org/abs/1403.3369 (2014). · arXiv
- Nair, V. & Hinton, G. E. Rectified linear units improve restricted Boltzmann machines. In International Conference on Machine Learning 807–814 (PMLR, 2010).
- Lee, S.-W., Kim, J.-H., Jun, J., Ha, J.-W. & Zhang, B.-T. Overcoming catastrophic forgetting by incremental moment matching. In Advances in Neural Information Processing Systems 4652–4662 (Curran Associates, 2017).
- Zenke, F., Poole, B. & Ganguli, S. Continual learning through synaptic intelligence. In International Conference on Machine Learning 6072–6082 (PMLR, 2017).
- Řehůřek, R. & Sojka, P. Software framework for topic modelling with large corpora. Proc. LREC 2010 Workshop on New Challenges for NLP Frameworks 45–50 (ELRA, 2010).
- Shin, H., Lee, J. K., Kim, J. & Kim, J. Continual learning with deep generative replay. In Advances in Neural Information Processing Systems 2990–2999 (Curran Associates, 2017).
- Yosinski, J., Clune, J., Bengio, Y. & Lipson, H. How transferable are features in deep neural networks? In Advances in Neural Information Processing Systems 3320–3328 (Curran Associates, 2014).
- Hinton, G., Vinyals, O. & Dean, J. Distilling the knowledge in a neural network. Preprint at https://arxiv.org/abs/1503.02531 (2015). · arXiv
- Glorot, X. & Bengio, Y. Understanding the difficulty of training deep feedforward neural networks. In Proc. Thirteenth International Conference on Artificial Intelligence and Statistics 249–256 (Microtome, 2010).
- Nair, V. & Hinton, G. E. Rectified linear units improve restricted boltzmann machines. In Proc. 27th International Conference on Machine Learning (ICML-10) 807–814 (PMLR, 2010).
- Srivastava, R. K., Masci, J., Kazerounian, S., Gomez, F. & Schmidhuber, J. Compete to compute. In Advances in Neural Information Processing Systems 2310–2318 (Curran Associates, 2013).
- Hu, W. et al. Overcoming catastrophic forgetting via model adaptation. In International Conference on Learning Representations (ICLR, 2019).
Cited by
Sorted by citations
Items 101–200 of 313
Wang, X., Liang, Z., Koe, A. S. V., Wu, Q., Zhang, X., Li, H., & Yang, Q. (2022) · International Journal of Intelligent Systems · doi:10.1002/int.22727 · Cited by 11
“Therefore, we extend the scope of the work in References [34,35] to a multiparty training scenario, wherein the training data are distributed instead of being centralized.”
BackgroundRecursive Least-Squares Estimator-Aided Online Learning for Visual Tracking
Gao, J. G., Lu, Y., Qi, X., Kou, Y., Li, B., Li, L., Yu, S., & Hu, W. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3156977 · Cited by 13
“This resembles the emerging continual learning literature [33], [34], [35], [36], which has recently proved valuable in numerous typical supervised learning and reinforcement learning-based tasks where some static models are offline trained based on continual learning with fixed volumes of data.”
BackgroundModeling Inner- and Cross-Task Contrastive Relations for Continual Image Classification
Luo, Y., Cong, R., Liu, X., Ip, H. H. S., & Kwong, S. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3414277 · Cited by 12
“Regularized-based methods added constraints or penalties to the loss function [31], gradient [32], [33], network structure [34], or learning rate [35] during training.”
APM: Adaptive parameter multiplexing for class incremental learning
Gao, J., Xie, T., Li, R., Wang, K., & Zhao, L.-J. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.125135 · Cited by 12
Continual Learning With Neuromorphic Computing: Foundations, Methods, and Emerging Applications
Minhas, M. F., Putra, R. V. W., Awwad, F. R., Hasan, O., & Shafique, M. (2025) · IEEE Access · doi:10.1109/access.2025.3588665 · Cited by 24
“in [212]–[221] and the ones with Replay-based Strategies [90], [91], [222], [223].”
Xun, D., Chen, D., Zhou, Y., Lauschke, V. M., Wang, R., & Wáng, Y. (2022) · iScience · doi:10.1016/j.isci.2022.105506 · Cited by 13
“Emerging methods such as continual learning 33 are also worth to investigating in this context.”
MethodCross Domain Lifelong Learning Based on Task Similarity
Yang, S., & Cai, Z. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3276991 · Cited by 11
Dual memory model for experience-once task-incremental lifelong learning
Ma, G., Jiang, R., Wang, L., & Tang, H. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.07.009 · Cited by 13
Addressing catastrophic forgetting for medical domain expansionPreprint
Gupta, S., Singh, P., Chang, K., Qu, L., Aggarwal, M., Arun, N., Vaswani, A., Raghavan, S., Agarwal, V., Gidwani, M., et al. (2021) · Research Square · doi:10.21203/rs.3.rs-1087025/v1 · Cited by 11
PASS++: A Dual Bias Reduction Framework for Non-Exemplar Class-Incremental Learning
Zhu, F., Zhang, X.-Y., Cheng, Z., & Liu, C. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3568886 · Cited by 13
“…set of experiments using a pre-trained DeiT-S/16 [114] backbone, and compare with more non-exemplar and exemplar-based method baselines such as OWM [65], ADAM [74], HATCIL [81], SLDA [115], L2P [70], A-GEM [116], EEIL [35], GD [38], DER++ [117], HAL [118], FOSTER [18], BEEF [119], MORE [77], ROW…”
BackgroundMethodEnergy Minimum Regularization in Continual Learning
Li, X., Shan, L., Li, M., & Wang, W. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9412744 · Cited by 10
“We first review the method OWM [7], which is the back-bone in our paper.”
MethodInfluentialLiu, Z., Xi, L., Quan, Y., Cheng, C., & Zhu, J. (2024) · IEEE Transactions on Sustainable Energy · doi:10.1109/tste.2024.3488062 · Cited by 10
“To address these challenges, this paper proposes a novel Proximal Brain-Inspired Policy Optimization (PBPO) method, which adapts an orthogonal weight modification algo-rithm inspired by the work presented in [24].”
Shen, J., Ni, W., Xu, Q., Pan, G., & Tang, H. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.112999 · Cited by 10
Cheng, D., Hu, Y., Wang, N., Zhang, D., & Gao, X. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3547916 · Cited by 11
“, EWC [29], LwF [39], IMM [36], PGMA [25], GR [56], ER-MIR [3] RPSnet [51], OWM [75] and AOP [19].”
InfluentialDeepCollaboration: Collaborative Generative and Discriminative Models for Class Incremental Learning
Cui, B., Hu, G., & Yu, S. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i2.16204 · Cited by 10
“A novel orthogonal weight modification algorithm, OWM, was proposed to enable the weights of a network only be modified in the directions orthogonal to the subspace spanned by all previously learned tasks (Zeng et al. 2019).”
MethodModeling learnable electrical synapse for high precision spatio-temporal recognition
Wu, Z., Zhang, Z., Gao, H., Qin, J., Zhao, R., Zhao, G., & Li, G. (2022) · Neural Networks · doi:10.1016/j.neunet.2022.02.006 · Cited by 10
Enhance the old representations’ adaptability dynamically for exemplar-free continual learning
Li, K., Ding, C., Wan, J., & Yu, S. P. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130286 · Cited by 10
Parameter-Efficient continual fine-tuning: A survey
Coleman, E. N., Quarantiello, L., Liu, Z., Yang, Q., Mukherjee, S., Hurtado, J., & Lomonaco, V. (2026) · Neurocomputing · doi:10.1016/j.neucom.2026.134502 · Cited by 28
“In contrast, OWM [155] and OGD [31] offer contrasting strategies: OWM modifies updates to be orthogonal to the previous input space, while OGD preserves old gradient directions and rectifies current ones.”
Efficient Self-Supervised Continual Learning with Progressive Task-Correlated Layer Freezing
Yang, L., Lin, S., Zhang, F., Zhang, J., & Fan, D. (2025) · International Symposium on Quality Electronic Design (ISQED) · doi:10.1109/isqed65160.2025.11014440 · Cited by 9
“However, these SSCL works directly combine the existing SSL frameworks with supervised continual learning techniques (e.g, knowledge distillation, mixup, memory replay, etc,.), while overlooking the substantial training costs inherited from SSL. Compared to Supervised Learning, the training cost of SSL models is notor…”
InfluentialSpiking Neural Predictive Coding for Continual Learning from Data StreamsPreprint
Ororbia, A. G. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.08655 · Cited by 9
“These range from problems in effective implementation in (neuromorphic) hardware meant for real-time computation [10, 46] to challenges in parameter optimization [86] to designing agents that generalize well across tasks and datasets [100, 106].”
BackgroundClass Incremental Learning With Less Forgetting Direction and Equilibrium Point
Wen, H., Qiu, H., Wang, L., Cheng, H., & Li, H. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3477951 · Cited by 10
“Gradient projection methods for continual learning project the gradient of the new task to the orthogonal direction of the old task subspace and can achieve zero forgetting theoretically [48], [49], [50].”
CKDF-V2: Effectively Alleviating Representation Shift for Continual Learning With Small Memory
Li, K., Chen, H., Wan, J., & Yu, S. P. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3569834 · Cited by 9
“Exemplar-free methods mainly include regularization-based [15], [16], [17] and parameter isolation [18], [19] methods.”
Knowledge Distillation in Object Detection: A Survey from CNN to Transformer
Shehzadi, T., Noor, R., Ifza, I., Liwicki, M., Stricker, D., & Afzal, M. Z. (2026) · Sensors · doi:10.3390/s26010292 · Cited by 10
“Kang et al. [194] addresses the issue of catastrophic forgetting in incremental learning [195]—specifically object detection as little work has been done in this domain [117,196]—by proposing a method that uses knowledge distillation both within class as well as between class.”
Harman, F. (2026) · IEEE Access · doi:10.1109/access.2026.3659025 · Cited by 8
“Zeng et al. [73] extended this principle through Orthogonal Weight Modification (OWM), constraining gradient updates to directions orthogonal to previously learned subspaces.”
Class Incremental Learning based on Identically Distributed Parallel One-Class Classifiers
Sun, W., Li, Q., Zhang, J., Wang, W., & Geng, Y. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126579 · Cited by 8
Bio-inspired, task-free continual learning through activity regularization
Lässig, F., Aceituno, P. V., Sorbaro, M., & Grewe, B. F. (2023) · Biological Cybernetics · doi:10.1007/s00422-023-00973-w · Cited by 9
“In another approach, a dedicated system, inspired by the role of the prefrontal cortex, is used to detect contextual information instead (Zeng et al. 2019).”
MethodOnline continual learning through unsupervised mutual information maximization
Hihn, H., & Braun, D. A. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127422 · Cited by 8
On-Device Continual Learning With STT-Assisted-SOT MRAM-Based In-Memory Computing
Zhang, F., Sridharan, A., Hwang, W., Xue, F., Tsai, W., Wang, S. X., & Fan, D. (2024) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2024.3371268 · Cited by 8
“On the other hand, orthogonal-projection-based methods (e.g., [41], [42], [43], [44], and [45]) update the model for each new task in a direction orthogonal to the subspace spanned by the inputs of old tasks.”
MethodExperimental demonstration of quantum continual learning with superconducting qubits
Zhang, C., Lu, Z., Zhao, L., Xu, S., Li, W., Wang, K., Chen, J., Wu, Y., Jin, F., Zhu, X., et al. (2026) · npj Quantum Information · doi:10.1038/s41534-025-01174-y · Cited by 11
Hu, J., Zhang, J., Qi, L., Shi, Y., & Gao, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72983-6_12 · Cited by 9
Hyper-parameter Tuning for Progressive Learning and its Application to Network Cyber Security
Karn, R. R., Ziegler, M. M., Jung, J., & Elfadel, I. M. (2022) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas48785.2022.9937433 · Cited by 8
“Representative algorithms that follow such an approach are synaptic intelligence [1], elastic weight consolidation (EWC) [2], and orthogonal weight modification [10].”
BackgroundA Marr's Three‐Level Analytical Framework for Neuromorphic Electronic Systems
Guo, Y., Zou, X., Hu, Y., Yang, Y., Wang, X., He, Y., Kong, R., Guo, Y., Li, G., Zhang, W., et al. (2021) · Advanced Intelligent Systems · doi:10.1002/aisy.202100054 · Cited by 14
Self-Supervised Learning Aided Class-Incremental Lifelong LearningPreprint
Zhang, S., Shen, G., Huang, J., & Deng, Z.-H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.05882 · Cited by 14
“We optimize F with gradient computed by back propagation (BP) while optimize E and C with OWM algorithm [45].”
BackgroundMethodInfluentialSTSF: Spiking Time Sparse Feedback Learning for Spiking Neural Networks
He, P., Xiao, R., Tang, C., Huang, S., Lv, J., & Tang, H. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3527700 · Cited by 7
“Motivated by this observation, the research community has shown a long-term interest in spiking neural networks (SNNs) due to their biological plausibility, theoretical computational power, and energy efficiency [1], [2], [3].”
Toward Deep Semi-Supervised Continual Learning: A Unified Survey for Scalable and Adaptive AI
Jalal, M. E., & Elmaghraby, A. S. (2025) · IEEE Access · doi:10.1109/access.2025.3556569 · Cited by 8
“For instance, LOGD [112] aligns updates with experience replay to preserve both the input and gradient spaces of previous tasks, while OWM [113] modifies updates to be orthogonal to the input space, reducing interference with past knowledge.”
Joint encoding of stimulus and decision in monkey primary visual cortex
Yang, Y., Klon-Lipok, J., & Singer, W. (2023) · Cerebral Cortex · doi:10.1093/cercor/bhad420 · Cited by 8
Chen, H., Guan, L., Lin, B., Chen, L., Huang, J., & Li, Y. (2025) · Electric Power Systems Research · doi:10.1016/j.epsr.2025.111704 · Cited by 7
Relay learning: a physically secure framework for clinical multi-site deep learning
Bo, Z.-H., Guo, Y., Lyu, J., Liang, H., He, J., Deng, S., Xu, F., Lou, X., & Dai, Q. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00934-4 · Cited by 9
“In addition, most of them are assessed in tasks such as image classi fi cation 32,33 , and are still immature in challenging clinical tasks, like pixel-level lesion segmentation (see Supplementary Results section and Supplementary Fig.”
BackgroundHOP to the Next Tasks and Domains for Continual Learning in NLP
Michieli, U., & Özay, M. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i13.29349 · Cited by 7
Xiong, F., Liu, Z., Huang, K., Yang, X., Qiao, H., & Hussain, A. (2020) · Neural Networks · doi:10.1016/j.neunet.2020.06.003 · Cited by 7
Similarity-based context aware continual learning for spiking neural networks
Han, B., Zhao, F., Li, Y., Kong, Q., Li, X., & Zeng, Y. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.107037 · Cited by 6
Training Networks in Null Space of Feature Covariance With Self-Supervision for Incremental Learning
Wang, S., Li, X., Sun, J. Z., & Xu, Z. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3522258 · Cited by 6
“Among these algorithms including CAB [55], OWM [24], GPM [25], and ours, the major difference is the way of constructing the projection matrix.”
InfluentialImitating the oracle: Towards calibrated model for class incremental learning
Zhu, F., Cheng, Z., Zhang, X.-Y., & Liu, C. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.04.010 · Cited by 6
CAT: Continual Adapter Tuning for aspect sentiment classification
Chen, Q., Huang, J., Wen, W., Li, Q., Zhang, R., & Qin, J. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127423 · Cited by 6
CL-BioGAN: Biologically Inspired Cross-Domain Continual Learning for Hyperspectral Anomaly Detection
Wang, J., Zheng, H., Zhang, W., Hao, S., Yao, Y., & Gong, M. (2025) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2025.3561174 · Cited by 6
“These training methods include DL-based joint ( J ) training for OHAD, fine-tune (FT)-based OHAD, and CL-based algorithm MAS [77], EWC [17], OWM [78], and CL-CaGAN [79].”
Learning to Navigate for Mobile Robot with Continual Reinforcement Learning
Wang, N., Zhang, D., & Wang, Y. (2020) · Cybersecurity and Cyberforensics Conference · doi:10.23919/ccc50068.2020.9188558 · Cited by 7
“Besides, there are also many works focucing on continual learning [17]-[19].”
BackgroundState Primitive Learning to Overcome Catastrophic Forgetting in Robotics
Xiong, F., Liu, Z., Huang, K., Yang, X., & Qiao, H. (2020) · Cognitive Computation · doi:10.1007/s12559-020-09784-8 · Cited by 6
“In [28], Zeng et al. present an orthogonal weight modification (OWM) method to modify network weights in the direction orthogonal to the subspace spanned by the previously learned tasks so that they could be protected.”
MethodLifelong Learning Without a Task Oracle
Rios, A., & Itti, L. (2020) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/ictai50040.2020.00049 · Cited by 6
“In general, partitioning methods come at the cost of large storage overheads [4].”
Powerpropagation: A sparsity inducing weight reparameterisationPreprint
Schwarz, J., Jayakumar, S. M., Pascanu, R., Latham, P. E., & Teh, Y. W. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.00296 · Cited by 11
Revisiting Flatness-Aware Optimization in Continual Learning With Orthogonal Gradient Projection
Yang, E., Shen, L., Wang, Z., Liu, S., Guo, G., Wang, X., & Tao, D. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3539019 · Cited by 5
“4) Orthogonal-Projection based methods: OWM [18], GPM [21], FS-DGPM [28], SD [25], DualGPM [29], Adam-NSCL [30], CGP [31], TRGP [20], ROGO [22], SGP [23].”
InfluentialComplementary Learning Subnetworks Towards Parameter-Efficient Class-Incremental Learning
Li, D., Zeng, Z., Dai, W., & Suganthan, P. N. (2025) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2025.3550809 · Cited by 8
“We follow the setting in [20], [40] where a feature extractor pre-trained was first used to analyze the raw images.”
InfluentialInvestigating the benefits of artificial neural networks over linear approaches to BMI decoding
Temmar, H., Willsey, M. S., Costello, J. T., Mender, M. J., Cubillos, L. H., DeMatteo, J. C., Lam, J. L., Wallace, D. M., Kelberman, M. M., Patil, P. G., et al. (2025) · Journal of Neural Engineering · doi:10.1088/1741-2552/ade568 · Cited by 11
Wang, Z., Yang, W., Bai, G., Lu, H., Qin, H., & Sui, Y. (2025) · IEEE Internet of Things Journal · doi:10.1109/jiot.2025.3604223 · Cited by 5
“Optimization-based approaches use gradient projection [35] and meta-learning [36] to achieve CL.”
A neural network for modeling human concept formation, understanding and communication
Guo, L., Chen, H., Chen, Y., Bi, Y., & Yu, S. (2026) · Nature Computational Science · doi:10.1038/s43588-026-00956-4 · Cited by 6
“Here, inspired by a previously proposed network capable of flexible context-dependent processing [24], we propose the CATS Net,a dual-module framework comprising concept-abstraction (CA) and sensorimotor task-solving (TS) modules (Figure 1b).”
Hybrid federated continual graph contrastive learning for evolving money laundering threats
Bashir, Z., Verma, M., & Mohan, C. K. (2026) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-026-01192-0 · Cited by 5
“Memory-free CL methods using orthogonal gradient projec - tion (Guo et al. 2022; Wang et al. 2021a; Zeng et al. 2019) have emerged as a scalable alternative to memory-based approaches.”
Restricted orthogonal gradient projection for continual learning
Yang, Z., Yang, Z., Liu, Y., Li, P., & Liu, Y. (2023) · AI Open · doi:10.1016/j.aiopen.2023.08.010 · Cited by 5
Class-incremental learning with Balanced Embedding Discrimination Maximization
Wei, Q., & Zhang, W. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106487 · Cited by 5
ForestryBERT: A pre-trained language model with continual learning adapted to changing forestry text
Tan, J., Zhang, H., Yang, J., Liu, Y., Zheng, D., & Liu, X. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113706 · Cited by 6
Gradient Projection for Continual Parameter-Efficient Tuning
Qiao, J., Zhang, Z., Tan, X., Qu, Y., Zhang, W., Han, Z., & Xie, Y. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3587032 · Cited by 8
“In seeking to address the forgetting problem, continual learning, or incremental learning [1], [25], [38], [70] is proposed to train a model with continuously expanded datasets by adding novel classes or domains [3], [6], [20], [23], [37], [48], [51], [59], [67], [68], [69].”
Chang, K., Singh, P., Vepakomma, P., Poirot, M. G., Raskar, R., Rubin, D. L., & Kalpathy–Cramer, J. (2020) · Elsevier eBooks · doi:10.1016/b978-0-12-821259-2.00006-5 · Cited by 5
Zhang, D., Zhou, Z., Han, S., Gong, H., Zou, T., & Luo, J. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-022-13143-0 · Cited by 5
“…method, the WofE, etc.), deep learning methods can establish very complex mapping relationships between the output and input, to mine the potential information in data at a deeper level, make full use in the effective information of the data, and then enhance the accuracy of prediction [47, 54].”
MethodPNSP: Overcoming catastrophic forgetting using Primary Null Space Projection in continual learning
Zhou, D., & Song, Y. (2024) · Pattern Recognition Letters · doi:10.1016/j.patrec.2024.02.009 · Cited by 5
Chen, S., Wu, J., Lu, Q., Wang, Y., & Lin, Z. (2021) · International Journal of Advanced Robotic Systems · doi:10.1177/17298814211050560 · Cited by 5
“39 Unlike most methods that only use pretrained models, 13 the pipeline designed in this article includes online and offline phases to make our method have stronger learning capability and adaptability.”
MethodRT-Net: replay-and-transfer network for class incremental object detection
Cui, B., Hu, G., & Yu, S. P. (2022) · Applied Intelligence · doi:10.1007/s10489-022-03509-0 · Cited by 5
“Past studies of incremental learning can be roughly divided into three primary families: regularization-based [21, 49], distillation-based [3, 26, 35, 46] and replay-based meth-ods [20, 40].”
BackgroundLearning with Long-term Remembering: Following the Lead of Mixed Stochastic Gradient.Preprint
Guo, Y., Liu, M., Yang, T., & Rosing, T. (2019) · arXiv (Cornell University) · Cited by 6
Temmar, H., Willsey, M. S., Costello, J. T., Mender, M. J., Cubillos, L. H., Lam, J. L., Wallace, D. M., Kelberman, M. M., Patil, P. G., & Chestek, C. A. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.03.01.583000 · Cited by 5
Yang, Z., Khairuddin, A. S. M., Chuah, J. H., Wong, W. R., Noman, H. M. F., Putri, T. W. O., Xu, X., & Haq, M. M. (2025) · IEEE Access · doi:10.1109/access.2025.3555539 · Cited by 4
Memory Mechanisms Based Few-Shot Continual Learning Railway Obstacle Detection
Li, Z., Shang, D., Ma, Z., Liu, Y., & Lu, Y. (2023) · China Automation Congress (CAC) · doi:10.1109/cac59555.2023.10452127 · Cited by 4
“For outer loop, we introduce the orthogonal weight modification(OWM) [29] meta update strategy, which formed as: where θ k , θ rs , and γ separately denote the parameter of model before inner loop, the parameter of model after inner loop and the meta learning rate.”
MethodSelecting Useful Knowledge from Previous Tasks for Future Learning in a Single Network
Shi, F., Wang, P., Shi, Z., & Rui, Y. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9413105 · Cited by 4
“Regularization-based methods [17] [43] [44] mainly learn new tasks under the constraints of previous knowledge, this type of methods maintain the performance of previous tasks while learning new tasks.”
MethodLee, D. E., Nakamura, K., Tak, J.-H., & Hong, B. (2023) · IEEE Access · doi:10.1109/access.2023.3303869 · Cited by 4
“References [39] and [40] proposed different projection operators to achieve the same”
BackgroundReduce the Difficulty of Incremental Learning With Self-Supervised Learning
Guan, L., & Wu, Y. (2021) · IEEE Access · doi:10.1109/access.2021.3112745 · Cited by 4
“The most related work with ours is [30], this work only discussed the effect of combining self-supervised signals and the OWM algorithm [31].”
BackgroundGradient Regularization with Multivariate Distribution of Previous Knowledge for Continual Learning
Kim, T., Moon, H.-J., & Cho, S. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-21753-1_35 · Cited by 4
Incremental Audio-Visual Fusion for Person Recognition in Earthquake Scene
You, S., Zuo, Y., Yao, H., & Xu, C. (2023) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3614434 · Cited by 4
“For example, Orthogonal Weight Modiication (OWM) [48] ensures the weights can only be modiied in the direction orthogonal to the subspace spanned by all previously learned inputs when training a network for new tasks, which ensures the new learning processes do not interfere with previously learned tasks.”
BackgroundLi, D., Wang, T., Chen, J., Ren, Q., Kawaguchi, K., & Zeng, Z. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i12.29249 · Cited by 6
Generalised Controller Design Using Continual Learning
Benavides‐Prado, D., Wanigasekara, C., & Swain, A. K. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86340-1_32 · Cited by 4
A Cross-Modal Image Fusion Theory Guided by Human Visual Characteristics.Preprint
Fang, A., Zhao, X., & Zhang, Y. (2019) · arXiv (Cornell University) · Cited by 4
If deep learning is the answer, then what is the question?Preprint
Saxe, A., Nelli, S., & Summerfield, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.07580 · Cited by 8
“Hebbian learning might encourage the formation of orthogonal neural codes for different temporally contexts 147 , which in turn allows tasks to be learned in different neural subspaces 146 .”
BackgroundAE-Net: Autonomous Evolution Image Fusion Method Inspired by Human Cognitive MechanismPreprint
Fang, A., Zhao, X., Yang, J., Cao, S., & Zhang, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.08763 · Cited by 4
“rapidly and continuously increase their knowledge and ability without a lot of data driven. However, this process of autonomous evolutionary learning is deficient in the existing deep learning methods [20]. To solve this problem, this ability of human brain has been simulated in the field 5 of computer vision [21], and…”
BackgroundReservoir-based Tracking (TRAKR) For One-shot Classification Of Neural Time-series PatternsPreprint
Afzal, M. F., Márton, C. D., Rich, E. L., Mayberg, H. S., & Rajan, K. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.10.13.464288 · Cited by 4
Human-inspired Perspectives: A Survey on AI Long-term Memory
He, Z., Lin, W., Zheng, H., Zhang, F., Jones, M. W., Aitchison, L., Xu, X., Liu, M., Kristensson, P. O., & Shen, J. (2024) · arXiv · doi:10.48550/arxiv.2411.00489 · Cited by 3
“Zeng et al. [119] and Shen et al. [121] construct a matrix representing all previously trained input vectors when training on a new task.”
The continuous learning algorithm with null space and eigenface-based orthogonal weight modification
Liao, D., Liu, J., Zeng, J., & Xu, S. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.127468 · Cited by 3
Assembly-based computations through contextual dendritic gating of plasticity
Onasch, S., Miehl, C., Miękus, M. M., & Gjorgjieva, J. (2026) · Neuron · doi:10.1016/j.neuron.2026.07.028 · Cited by 4
Hippocampus supports multi-task reinforcement learning under partial observability
Pedamonti, D., Mohinta, S., Dimitrov, M. V., Malagon‐Vina, H., Ciocchi, S., & Costa, R. P. (2025) · Nature Communications · doi:10.1038/s41467-025-64591-9 · Cited by 5
“One possibility is that the brain mitigates interference between tasks through orthogonalisation of neural representations 65 – 67 .”
Trace Back and Go Ahead: Completing partial annotation for continual semantic segmentation
Luo, Y., Chen, J., Cong, R., Ip, H. H. S., & Kwong, S. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.111613 · Cited by 3
Jiang, H., Wei, S., Feng, X., & Yu, H. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.114631 · Cited by 3
A Comprehensive Review of Continual Learning with Machine Learning Models
Liu, S., Pan, T., Wang, C. W., Ma, X., Dong, W., Hu, T., Zhang, S., Zhang, Y., & Yan, Q. (2024) · Lecture notes in electrical engineering · doi:10.1007/978-981-97-0855-0_47 · Cited by 3
Moon, H.-J., & Cho, S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-61140-7_49 · Cited by 3
Digital Twin-Assisted Bearing Fault Diagnosis Using Multi-Fidelity Incremental Learning
Huang, X., Xie, T., & Zhou, Q. (2024) · International Conference on Control, Automation and Robotics (ICCAR) · doi:10.1109/iccar61844.2024.10569708 · Cited by 3
“On this context, the traditional IFD models suffer from catastrophic forgetting, i.e., the IFD models trained with new fault types will drastically forget previously learned information and cannot accurately diagnose previously learned categories [9].”
BackgroundGhosh, T., & Jayanthi, N. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-19592-z · Cited by 3
“2), (3), (4), (5), (6), (7),(8),(9),(10), (11), (12), (13), (14), (15), (16) are GFF [20], RPCNN [25],LPCNN49],NSNT + PAPCNN [26], IFCNN [50], PMGT [51], U2Fusion [52], EWC [53], Adapter [54], EWC + Adapter, OWM [55], TDAN [48],”
InfluentialContinual Learning From a Stream of APIs
Yang, E., Wang, Z., Shen, L., Yin, N., Liu, T., Guo, G., Wang, X., & Tao, D. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3460871 · Cited by 6
“According to the construction method of subspace, it can be further divided into projection in feature (or input) subspace [37], [38], [39], [40], [41] or projection in gradient subspace [42], [43].”
MethodA Two-Stage LoRA Strategy for Expanding Language Capabilities in Multilingual ASR Models
Kwok, C. Y., Liu, H., Yip, J. Q., Li, S., & Chng, E. S. (2025) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2025.3578752 · Cited by 4
Golden, R., Saxena, R., Gonzalez, O. C., Delanois, J. E., Kilianski, S., McNaughton, B. L., & Bazhenov, M. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.06.25.661579 · Cited by 4
“…): rehearsal and pseudorehearsal 1,9-11 , i.e., storing or generating old data samples to mix with new ones during training; and regularization-based methods 53-56 , which modify plasticity rules by adding constraints to gradient descent that preserve weights important for previously learned tasks.”
AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models
Fang, J., Jiang, H., Wang, K., Ma, Y., Jie, S., Wang, X., He, X., & Chua, T.-S. (2024) · arXiv · doi:10.48550/arxiv.2410.02355 · Cited by 170
“Specifically, OWM (Zeng et al., 2019) directs parameter updates orthogonally to the space of the input data from previous tasks.”
CLIP-based autonomous visual prompting for unsupervised domain incremental learning
Yu, J., Yang, M., Yan, J., & Deng, C. D. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.131019 · Cited by 2
SOLO: Spatial online learning at once for spiking neural networks
Lee, S., Moon, H., Yoon, M., Heo, W., & Kim, S. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.131940 · Cited by 2
Zhao, Z., Su, J., Xi, J., Li, Z., & Ersoy, O. K. (2025) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2025.3637249 · Cited by 2
“Methods, such as orthogonal weights modification [18] and Adam-NSCL [19], that do not constrain parameters directly achieve a fine model updating process by controlling the gradient direction of the new task.”
EDCL: An Efficient Dynamic Continual Learning Framework for IoT Systems
Zhang, K., Liu, X., Wang, K., Cai, Q., Xie, X., Zhang, J., Chen, J., Zhang, C., Tong, X., Gong, Z., et al. (2026) · IEEE Transactions on Computers · doi:10.1109/tc.2026.3662718 · Cited by 2
“Some methods retain knowledge of old tasks by updating parameters in a direction orthogonal to the gradients of historical tasks [30].”
A Continuous Learning Approach for Probabilistic Human Motion Prediction
Xu, J., Wang, S., Chen, X., Zhang, J., Lan, X., & Zheng, N. (2022) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra46639.2022.9811906 · Cited by 2
Memory Enhanced Replay for Continual Learning
Xu, G., Guo, W., & Wang, Y. (2022) · International Conference on Signal Processing · doi:10.1109/icsp56322.2022.9965222 · Cited by 2
“Because the proposed method is based on generative replay, we mainly compare it with the other advanced continual learning approaches based on generative replay, e.g. EWC-M [28], DGMw [11], MeRGAN [10], OWM [29] and TMNs [12].”
MethodInfluentialUsing Flexible Memories to Reduce Catastrophic Forgetting
Wong, W., Koh, Y. S., & Dobbie, G. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-33377-4_17 · Cited by 2
Metadata
- Authors
- Zeng, G., Chen, Y., Cui, B., & Yu, S.
- Affiliations
- Chinese Academy of Sciences (CN)
- ORCID
- Yu, Shan
- Venue
- Nature Machine Intelligence
- Published
- 2019-08-09
- Volume
- 1
- Issue
- 8
- Pages
- 364-372
- ISSN
- 2522-5839 (ISSN-L)
- OpenAlex
- W2895699061
- Cited by
- 309
- References
- 41
- Citations by year
- 2026: 51 · 2025: 63 · 2024: 54 · 2023: 52 · 2022: 23 · 2021: 39 · 2020: 21 · 2019: 6
- FWCI
- 10.27
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- Nature Machine Intelligence (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- continual learning, catastrophic forgetting, context-dependent processing, neural network modules, multi-task learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
