The DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
Continual learning of context-dependent processing in neural networks
Open accessOverview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“It has been shown that input created from a generative model can indeed assist with learning, such as preventing catastrophic forgetting.149”
From The overfitted brain: Dreams evolved to assist generalization (2021) · cites this paper as [149]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Golub, G. (2012) · doi:10.56021/9781421407944 · Cited by 34,505
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741
An Integrative Theory of Prefrontal Cortex Function
Miller, E. K., & Cohen, J. D. (2001) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.24.1.167 · Cited by 12,803
Neural Mechanisms of Selective Visual Attention
Desimone, R., & Duncan, J. (1995) · Annual Review of Neuroscience · doi:10.1146/annurev.ne.18.030195.001205 · Cited by 8,388
Overcoming catastrophic forgetting in neural networks
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1611835114 · Cited by 7,899
Deep Learning Face Attributes in the Wild
Liu, Z., Luo, P., Wang, X., & Tang, X. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.425 · Cited by 7,747
International Conference on Learning Representations (ICLR 2013)
松尾, 豊. (2013) · 人工知能学会誌 = Journal of Japanese Society for Artificial Intelligence · Cited by 6,286
Research works printed but not linked: 15
- Newell, A. Unified Theories of Cognition (Harvard Univ. Press, 1994).
- Haykin, S. S. Adaptive Filter theory (Pearson Education India, 2008).
- Singhal, S. & Wu, L. Training feed-forward networks with the extended kalman algorithm. In International Conference on Acoustics, Speech, and Signal Processing 1187–1190 (IEEE, 1989).
- Jaeger, H. Controlling recurrent neural networks by conceptors. Preprint at https://arxiv.org/abs/1403.3369 (2014). · arXiv
- Nair, V. & Hinton, G. E. Rectified linear units improve restricted Boltzmann machines. In International Conference on Machine Learning 807–814 (PMLR, 2010).
- Lee, S.-W., Kim, J.-H., Jun, J., Ha, J.-W. & Zhang, B.-T. Overcoming catastrophic forgetting by incremental moment matching. In Advances in Neural Information Processing Systems 4652–4662 (Curran Associates, 2017).
- Zenke, F., Poole, B. & Ganguli, S. Continual learning through synaptic intelligence. In International Conference on Machine Learning 6072–6082 (PMLR, 2017).
- Řehůřek, R. & Sojka, P. Software framework for topic modelling with large corpora. Proc. LREC 2010 Workshop on New Challenges for NLP Frameworks 45–50 (ELRA, 2010).
- Shin, H., Lee, J. K., Kim, J. & Kim, J. Continual learning with deep generative replay. In Advances in Neural Information Processing Systems 2990–2999 (Curran Associates, 2017).
- Yosinski, J., Clune, J., Bengio, Y. & Lipson, H. How transferable are features in deep neural networks? In Advances in Neural Information Processing Systems 3320–3328 (Curran Associates, 2014).
- Hinton, G., Vinyals, O. & Dean, J. Distilling the knowledge in a neural network. Preprint at https://arxiv.org/abs/1503.02531 (2015). · arXiv
- Glorot, X. & Bengio, Y. Understanding the difficulty of training deep feedforward neural networks. In Proc. Thirteenth International Conference on Artificial Intelligence and Statistics 249–256 (Microtome, 2010).
- Nair, V. & Hinton, G. E. Rectified linear units improve restricted boltzmann machines. In Proc. 27th International Conference on Machine Learning (ICML-10) 807–814 (PMLR, 2010).
- Srivastava, R. K., Masci, J., Kazerounian, S., Gomez, F. & Schmidhuber, J. Compete to compute. In Advances in Neural Information Processing Systems 2310–2318 (Curran Associates, 2013).
- Hu, W. et al. Overcoming catastrophic forgetting via model adaptation. In International Conference on Learning Representations (ICLR, 2019).
Cited by
Sorted by citations
Items 201–300 of 313
A biologically inspired architecture with switching units can learn to generalize across backgrounds
Voina, D., Shea-Brown, E., & Mihalas, S. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.09.014 · Cited by 2
Toward Digits Recognition Using Continual Learning
Kharrat, A., Drira, F., Lebourgeois, F., & García, C. (2023) · IEEE International Workshop on Multimedia Signal Processing (MMSP) · doi:10.1109/mmsp59012.2023.10337747 · Cited by 2
“As noted by the authors of [11], the multi-headed configuration is much easier to deploy and is more likely to be used in similar evaluations[12], [13].”
BackgroundMethodDouble Confidence Calibration Focused Distillation for Task-Incremental Learning
Fu, Z., Wang, Z., Yu, C., Xu, X., & Li, D. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3418811 · Cited by 2
“Finally, the third group encompasses regularization methods, which include EWC [15], MAS [16], MUC-MAS [40], SI [17], OWM [65], LwF [18], Adam-NSCL [42], Adam-NSCL + SD [66], AdNS [67], and ASLC [68].”
Incremental Learning via Robust Parameter Posterior Fusion
Sun, W., Li, Q., Zhang, S., Wang, W., & Geng, Y.-A. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3681164 · Cited by 2
Decoding BatchNorm statistics via anchors pool for data-free models based on continual learning
Li, X., Wang, W., & Xu, G. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-10904-w · Cited by 2
“Compared with other continual learning, like regulariza-tion-based methods [1, 4] and replay-based meth-ods [3, 10], additional stage 1) seems to take extra computation time.”
A brain-inspired sequence learning model based on a logic
Xu, B. (2025) · Scientific Reports · doi:10.1038/s41598-025-97777-8 · Cited by 2
“Some modern research ( e.g. , [21]) over the years still tried to solve this issue.”
Enhancing pre-trained language model by answering natural questions for event extraction
Zhang, Y., & Han, Q. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1520290 · Cited by 2
“Contextual cues play a pivotal role in understanding the nuances and intricacies of language, yet they are often overlooked or underrepresented in current methodologies (Zeng et al., 2019).”
FedSVD: Asynchronous Federated Learning With Stale Weight Vector Decomposition
Sur, G., Kim, H., Yoon, S., & Lim, H. (2025) · IEEE Access · doi:10.1109/access.2025.3573806 · Cited by 2
“To overcome catastrophic forgetting, some works proposed to exploit orthogonal weights or gradients to the optimized model trained by previous tasks [30], [31], [32], [33], [34].”
Continual Diffuser (CoD): Mastering Continual Offline RL With Experience Rehearsal
Hu, J., Shen, L., Huang, S., Yang, Z., Chen, H., Sun, L., Chang, Y., & Tao, D. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3598928 · Cited by 4
“They will show a dilemma between storing historical knowledge (stability) in their brains and adapting to environmental changes (plasticity) [6].”
Flaws (and quality) in research today: can artificial intelligence intervene?
Siristatidis, C. S., & Pouliakis, A. (2020) · Systems Biology in Reproductive Medicine · doi:10.1080/19396368.2020.1749727 · Cited by 2
“…the proposed ANN would have a ‘ cyclic nature and function ’ , i.e., the system should be able to be trained again to accumulate new knowledge and/or adjust existing knowledge from previous training processes, a process so-called ‘ continual learning ’ (Parisi et al. 2019; Zeng et al. 2019).”
BackgroundContinual Correction of Errors Using Smart Memory Replay
Cory, C., Benavides‐Prado, D., & Koh, Y. S. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9534260 · Cited by 2
Prototype Representation Expansion in Incremental Learning
Mao, K., Luo, Y., Ren, Y., & Wang, R. (2023) · Neural Processing Letters · doi:10.1007/s11063-023-11317-x · Cited by 2
“…methods such as elastic weight consolidation (EWC) [13], synaptic intelligence (SI) [39], memory aware synapses (MAS) [1] and orthogonal weights modification (OWM) [38] design different metric to evaluate the importance of parameters and use regularization terms to con-straintparameterupdates.”
Guo, L., Chen, Y., & Yu, S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-78189-6_8 · Cited by 2
Continual Learning Using Multi-view Task Conditional Neural NetworksPreprint
Li, H., Barnaghi, P. M., Enshaeifar, S., & Ganz, F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.05080 · Cited by 2
Zhang, J., Zhou, D., & Chen, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.08579 · Cited by 2
Cognitive Escape Reinforcement Learning for Complex Decision MakingPreprint
Wu, Q., Zhao, S., Zhou, F., Zhang, H., Huang, Y., & Ma, K. (2023) · Research Square · doi:10.21203/rs.3.rs-2661516/v1 · Cited by 2
Unsupervised Replay Strategies for Continual Learning with Limited Data
Bazhenov, A., Dewasurendra, P., Krishnan, G. P., & Delanois, J. E. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650116 · Cited by 1
“Notably, the Elastic Weight Consolidation (EWC), Synaptic Intelligence (SI) and Orthogonal Weight Modification (OWM) techniques penalize weight updates for prior tasks [14], [42], [43].”
BackgroundIntroducing Common Null Space of Gradients for Gradient Projection Methods in Continual Learning
Yang, C., Dong, M., Zhang, X., Qi, J., & Zhou, A. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3680605 · Cited by 1
“Orthogonal Weight Modification (OWM) [40] constructs projection operators through recursive least squares algorithm.”
Prompt-based Continual Learning for Extending Pretrained CLIP Models' Knowledge
Jiao, L., Cao, L., & Wang, T. (2024) · ACM International Conference on Multimedia in Asia (MMAsia) · doi:10.1145/3696409.3700216 · Cited by 1
Uncertainty-guided recurrent prototype distillation for graph few-shot class-incremental learning
Zhu, N., Wang, S., Sun, Y., & Yin, B. (2025) · Multimedia Systems · doi:10.1007/s00530-025-01764-y · Cited by 1
“CIL’s original works include elastic weight consolidation [40], synaptic intelligence [41], orthogonal weight modification [42], orthogonal gradient descent [43], learning without forgetting [44].”
Dual-Stage Clean-Sample Selection for Incremental Noisy Label Learning
Li, J., Ma, X., & Shi, Y. (2025) · Bioengineering · doi:10.3390/bioengineering12070743 · Cited by 1
“OWM (Orthogonal Weight Modification) [25] and AOP (Adaptive Orthogonal Projection) [26] update weights along directions orthogonal to previous tasks.”
A Feedback-Driven Learning Framework for Adaptive Neural Motion Planner
Zheng, H., Liu, S., & Wang, J. (2025) · Chinese Control and Decision Conference (CCDC) · doi:10.1109/ccdc65474.2025.11090812 · Cited by 1
“Similarly, Or-4447 thogonal Weight Modification (OWM) [23] aligns updates to the null space of previous tasks, ensuring task boundaries are respected.”
Research on Understanding and Improving Deep Hashing Retrieval under Incremental Data
Weijian, M., & Xiaoting, L. (2025) · International Conference on IoT, Data Science and Advanced Computing (IDSAC) · doi:10.1109/idsac65763.2025.11170201 · Cited by 1
“…the underlying architecture of deep networks; 3) Algorithm-based updating rules, as seen in GEM [42] and A-GEM [43], among others, including OWM [44], an orthogonal weight correction method, Adam-NSCL [45], which utilizes the null space of all previous data, and the primary challenge for…”
Clustering-Based Weight Orthogonalization for Stabilizing Deep Reinforcement Learning
Ma, G., Zhang, Y., Dai, Y., Hao, G., Chen, Y., & Yu, S. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11227331 · Cited by 1
“Furthermore, considering the constraints of MLPs, the use of a projection matrix can more effectively minimize learning overlap [37].”
Sun, W., Li, Q., Zhang, S., Wang, W., & Geng, Y. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.112948 · Cited by 1
Han, B., Zhao, F., Sun, Y., Pan, W., & Zeng, Y. (2026) · National Science Review · doi:10.1093/nsr/nwag066 · Cited by 1
“Methods based on weight regularization [6 –8 ], experience replay [9 ,10 ] and weight freezing [11 ,12 ] have effectively mitigated catastrophic forgetting in multi-task learning within the same domain.”
Bi 2 CFEL: Continual Feature Evolution Learning for Industrial Online Recognition
Huang, K., Feng, W., Chen, Z., Wu, D., Yang, C., & Gui, W. (2026) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2026.3670028 · Cited by 1
“4) Optimization-based: Constraining the gradient updates to preserve historical performance [17].”
A review on exploring and unlocking the operational mode of the brain
Wang, R., Li, Y., Yan, C., Wu, F., Wang, Y. M., Xu, X., & Pan, X. (2026) · Nonlinear Dynamics · doi:10.1007/s11071-026-12344-8 · Cited by 1
Learning continually with representational drift
van der Veldt, S., van de Ven, G. M., Moorman, S., & Etter, G. (2026) · Trends Open · doi:10.1016/j.treopn.2026.04.013 · Cited by 3
“Maintaining stable performance on past tasks can also be achieved by forcing ANNs to learn new tasks in the null space or orthogonal complement of past tasks [99–101].”
Tran, C. T., Nguyen, T. T., Nguyễn, T. P., & Thirion‐Moreau, N. (2026) · Machine Vision and Applications · doi:10.1007/s00138-026-01883-0 · Cited by 1
“…by maintaining class centroids, aiming to balance representation richness and storage efficiency. iii) Optimization-based approaches modify gradient updates to minimize interference between new and previous tasks, drawing inspiration from meta-learning and multi-task optimization [46–50 ].”
Li, C., Liu, Y., & Zhao, C. (2026) · Concurrency and Computation Practice and Experience · doi:10.1002/cpe.70819 · Cited by 1
“These include FedGen [ 20], a generative data-free distillation network, along with federated variants of classic continual learning methods: Fed-EWC [ 14] (Elastic Weight Consolidation) and OWM [ 38] (Orthogonal Weights Modification).”
Liu, Z., Li, Y., Gong, Y., & Wu, Y. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446458 · Cited by 1
“…(-2.83) 71.66 (-3.72) 52.33 (-6.17) MAS [15] 66.93(-4.03) 63.84(-6.29) 47.96 (-7.04) GEM [7] 49.48 (2.77) 68.89 (-1.2) N/A A-GEM [10] 49.57 (-1 -1.13) 61.91 (-6.88) 53.32 (-7.68) MEGA [16] 54.17 (-2.19) 64.98 (-5.13) 57.12 (-5.90) OWM [17] 68.89 (-1.88) 68.47 (-3.37) 49.98 (-3.64) GPM [8] 73 .”
InfluentialExperimental study of rehearsal-based incremental classification of document streams
Malik, U., Visani, M., Sidère, N., Coustaty, M., & Joseph, A. (2024) · International Journal on Document Analysis and Recognition (IJDAR) · doi:10.1007/s10032-024-00467-w · Cited by 1
“Zeng et al. [20] propose the orthogonal weights modifica-tion (OWM) approach for incremental image classification.”
Online Continual Learning Benefits From Large Number of Task Splits
Zhang, S., & Yi, C. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3405404 · Cited by 1
Incremental Object Detection based on YOLO v5 and EWC Models
Li, Z., Liu, X., Zhang, C., Li, S., Xi, Z., & Deng, W. (2023) · doi:10.1145/3641584.3641651 · Cited by 1
“ZENG G et al. proposed the orthogonal right correction model and scenario-related processing module (OWM-CDP) [13].”
Malviya, A., & Maurya, C. K. (2024) · IEEE Access · doi:10.1109/access.2024.3435555 · Cited by 1
“OGD [48] and OWM [49] address CF by storing gradient directions, but GPM significantly reduces memory requirements.”
Breaking Information Silos: Global Guided Task Prediction for Class-Incremental Learning
Hu, C., Ye, B., Chen, Z., & Lai, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-78189-6_4 · Cited by 1
Neural networks trained by weight permutation are universal approximators
Cai, Y., Chen, G., & Qiao, Z. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107277 · Cited by 2
Geodesic-Aligned Gradient Projection for Continual Task Learning
Qiu, B., Qiu, H., Wen, H., Wang, L., Dai, Y., Meng, F., Wu, Q., & Li, H. (2025) · IEEE Transactions on Image Processing · doi:10.1109/tip.2025.3551139 · Cited by 4
“Gradient projection memory [16], [17], [31] is a novel anti-forgetting paradigm for continual learning.”
InfluentialRemembering Transformer for Continual Learning
Sun, Y., Fujisawa, I., Juliani, A., Sakuma, J., & Kanai, R. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11227313 · Cited by 1
Continual Learning with Laplace Operator Based Node-Importance Dynamic Architecture Neural Network
Li, Z., Meng, M., He, Y., & Liao, Y. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-92270-2_5 · Cited by 1
A Conceptual Framework for Lifelong LearningPreprint
Ling, C. X., & Bohn, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.09704 · Cited by 4
“Orthogonal Weights Modification [25] can be viewed as a type of parameter-based approach.”
BackgroundEfficient and robust multi-task learning in the brain with modular task primitives.Preprint
Márton, C. D., Lajoie, G., & Rajan, K. (2021) · arXiv (Cornell University) · Cited by 1
Knowledge Decoupling via Orthogonal Projection for Lifelong Editing of Large Language Models
Xu, H., Lan, P., Yang, E., Guo, G., Zhao, J., Jiang, L., & Wang, X. (2025) · Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2025.acl-long.646
“Inspired by works in continual learning (Zeng et al., 2019; Saha et al., 2021), we propose a knowledge decoupling method to mitigate the issue of knowledge coupling by adjusting the direction of the gradient updates during editing.”
Modeling of Brain Computer Interface-aided Training in Rehabilitation
Zeng, G., Chen, Y., & Yu, S. P. (2019) · IEEE International Conference on Automation, Electronics and Electrical Engineering (AUTEEE) · doi:10.1109/auteee48671.2019.9033263
A biological plausible audio-visual integration model for continual lifelong learning
Chen, W., Du, F., Wang, Y., & Cao, L. (2020)
Lifelong Learning Without a Task OraclePreprint
Rios, A., & Itti, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.04783
“In general, partitioning methods come at the cost of large storage overheads [4].”
A Biologically Plausible Audio-Visual Integration Model for Continual Learning
Chen, W., Du, F., Wang, Y., & Cao, L. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9533450
“We compare the proposed AVIM with six ANN methods, including ANN(Base), ANN(Offline), iCaRL [16], GEM [15], EWC [5] and OWM [8].”
MethodDefeating Catastrophic Forgetting via Enhanced Orthogonal Weights ModificationPreprint
Li, Y., Liu, B., Yao, K., Kou, X., Lv, P., Xu, Y., & Cui, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.10078
“L G ] 1 9 N ov 2 02 1 (Hsu, Liu, and Kira 2018; van de Ven and Tolias 2019; Zeng et al. 2019).”
BackgroundMethodInfluentialCLUE: Consolidating Learned and Undergoing Experience in Domain-Incremental Classification
Cai, C., Liu, J., Yu, W., & Guo, Y. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-26348-4_17
Characteristic Analysis and Application of the Optoelectronic Neural Network
Chen, Y. (2023) · Highlights in Science Engineering and Technology · doi:10.54097/hset.v57i.9988
Connection-Based Knowledge Transfer for Class Incremental Learning
Zhao, G., & Mu, K. (2023) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10191696 · Cited by 1
“(5) OWM [31] tries to find orthogonal projections of parameter updates in order to preserve outputs of previous tasks.”
BackgroundOnline prediction of photovoltaic power considering concept drift
Zhang, L., Zhu, J., Cheung, K. W., & Jialin, Z. (2023) · IEEE Power Engineering Society General Meeting · doi:10.1109/pesgm52003.2023.10252625
Big data and artificial intelligence in ophthalmology - clinical application and future exploration
(2023) · Frontiers research topics · doi:10.3389/978-2-8325-4171-5
A brief introduction to the brain
Yu, S. (2024) · Elsevier eBooks · doi:10.1016/b978-0-323-98829-2.00012-8
Immunity-Based Orthogonal Weights Modification Algorithm
Liang, X., Yin, S., Zhao, X., & Zhao, X. Y. (2024) · Communications on Applied Mathematics and Computation · doi:10.1007/s42967-024-00377-y
“Kemker et al. [16] summarized five mechanisms for mitigating CI problems, which were rehearsal methods [8, 12, 13, 32], ensemble methods [6], dual memory methods [15, 19], sparse coding methods [5, 9, 35], and regularization methods [16, 18, 29, 37].”
InfluentialCognition of Time and Thinking Beyond
Bi, Z. (2024) · Advances in experimental medicine and biology · doi:10.1007/978-3-031-60183-5_10
Gupta, S., Chang, K., Qu, L., Rana, A., Ahmed, S. R., Aggarwal, M., Arun, N., Vaswani, A., Raghavan, S., Agarwal, V., et al. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-73483-0_6
Huang, L., Zou, B., Zhong, L., & Shan, D. (2024) · International Journal of Data Science and Analytics · doi:10.1007/s41060-024-00660-4
“As model training proceeds in batch mode, an iterative method [34–36] can be used to calculate the projectors, which does not need to store all previous inputs.”
Low-Resource Language Adaptation Using Two-Stage Regularization for Multilingual ASR
Kwok, C. Y., Yip, J. Q., & Chng, E. S. (2025) · International Journal of Asian Language Processing · doi:10.1142/s2717554525500031
Jiao, L., Fu, W., & Chen, X. (2025) · Applied Sciences · doi:10.3390/app15062966
Emotion Recognition System Using Deep Learning Algorithm
Doe, A., Abdulgader, M. M., Nwosu, K. C., & Osunmakinde, I. O. (2024) · International Conference on Computer and Applications (ICCA) · doi:10.1109/icca62237.2024.10927877
“The loss for the backward propagation process was calculated using Categorical Cross Entropy Loss (SoftMax Loss) [22] [23].”
Neural signatures of motor memories emerge in neural network modelsPreprint
Chang, J. C., Clopath, C., & Gallego, J. Á. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2025.04.02.646788
“…levels of synaptic plasticity to protect existing knowledge 50,51 , push ac-426 tivity changes to be in orthogonal subspaces in the population space 52,53 , incorporate context to separate activity 54,55 , and use complementary learning systems that promote replay and con-428 solidation 56 .”
Optimizing English Lexical Databases with BERT and Reinforcement Learning
Saranya, V., Murthy, G. R. K., Alapati, P. R., Mythili, M., Swarnamughi, K., & Bala, B. K. (2025) · International Conference on Advances in Electrical, Computing, Communication and Sustainable Technologies (ICAECT) · doi:10.1109/icaect63952.2025.10958921
“This is why more conventional approaches to IR, and language processing in general, fail to be scalable, context-aware, and fail to incorporate the dynamic nature of practical language use [5].”
A brain-inspired projection contrastive learning network for instantaneous learning
Yang, Y. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.111524
Neural signatures of motor memories emerge in neural network modelsPreprint
Chang, J. C., Clopath, C., & Gallego, J. Á. (2025) · eLife · doi:10.7554/elife.106871
Neural signatures of motor memories emerge in neural network modelsPreprint
Chang, J. C., Clopath, C., & Gallego, J. Á. (2025) · eLife · doi:10.7554/elife.106871.1
Task-Conditioned Ensemble of Expert Models for Continuous Learning
Sharma, R., Pal, D., & Ross, A. A. (2025) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw67362.2025.00645
“…58.85 ± 2.59 Online EWC + [39] 57.33 ± 1.44 SI + [58] 64.76 ± 3.09 KFAC o [37] 67.86 ± 1.33 MAS + [2] 68.57 ± 6.85 LwF + [23] 71.02 ± 1.26 OWM o [57] 87.46 ± 0.74 NCL o [20] 91.48 ± 0.64 BiC ∗ [53] 77.75 ± 1.27 ER − [6] 85.69 GDumb ∗ [34] 88.51 ± 0.52 RM ∗ [3] 92.65 ± 0.33 DGR + [42] 95.74 ±…”
InfluentialKnowledge as Reconstruction: A No-Cloning View of Memory and CommunicationPreprint
Pavone, A. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5576876
SER: A Continual Learning Framework with Spacing Effect-based Review of Forgotten SamplesPreprint
Wang, H., Ye, H., Zhang, H., Liu, X., Wang, Q., & Wang, X. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5690557
Pu, S., Chen, H., Zheng, L., & Sun, Z. (2025) · Communications in computer and information science · doi:10.1007/978-981-95-4091-4_25
“Exemplar-free approaches mainly consist of regularization-based methods [27, 29] and parameter isolation methods [22, 21].”
VPT-NSP 2 ++: Importance-Aware Visual Prompt Tuning in Null Space for Continual Learning
Zhang, S., Lu, Y., Cheng, D., Xing, Y., Wang, N., Wang, P., & Zhang, Y. (2025) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3642298 · Cited by 2
“3 ) The regularization-based methods [8], [9], [10] constrain parameter updates to prevent significant changes during incremental learning.”
InfluentialElectrochemical ohmic memristors for continual learningPreprint
Valov, I., Chen, S., Yang, Z., Hartmann, H., Besmehn, A., & Yang, Y. (2023) · Research Square · doi:10.21203/rs.3.rs-3486022/v1
Mendez, R., Maier, A., & Emmert, J. (2026) · Sensors · doi:10.3390/s26020425
“…gradient vectors used to learn former tasks; this means that even though new gradients can be collected within the process of learning a new task, this matrix remains constant, to be recalculated only after the mentioned task has been learned, and the preparation to learn the next task begins [6].”
Jackson, E. J., & Geangu, E. (2026) · Infant Behavior and Development · doi:10.1016/j.infbeh.2026.102222
Liu, X., Chen, Z., Wu, G., & Cui, S. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6178280
Dual-LoRA and Quality-Enhanced Pseudo Replay for Multimodal Continual Food Learning
Wu, X., Zhu, B., Han, F., Jiao, P., & Chen, J. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-6960-1_13
Zhang, B., Cao, X., Wang, S., & Meng, D. (2026) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2026.3661264
“For example, OWM [28] and AOP [29] update parameters in directions orthogonal to the subspace spanned by previous inputs.”
Learning sculpts orthogonal task manifolds for continual skill learning in recurrent networksPreprint
Liu, Z., Kurth, A. C., Osako, Y., & Asabuki, T. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.02.15.705283 · Cited by 1
“…by allocating task-specific resources, including modular architectures and progressive expansion (Rusu et al., 2016), or by explicitly constraining weight updates, for example using gradient projection or orthogonalization (Lopez-Paz & Ranzato, 2017; Zeng et al., 2019; Farajtabar et al., 2020).”
Swin-NSPT: Null-Space Prompt Tuning for Continual Learning in Swin Transformers
Han, Y., Li, H., Jiang, Z., Zhang, S., Sang, M., & Liu, J. (2025) · International Conference on Computer, Big Data and Artificial Intelligence (ICCBD+AI) · doi:10.1109/iccbdai66607.2025.11388731
“Incontrast, in the era of convolutional neural networks, several orthogonal projection–based CL approaches have been proposed to mitigate forgetting through theoretically grounded formulations [7–10].”
B-Cos Networks as an Architectural Inductive Bias for Mitigating Catastrophic Forgetting
Sorrenti, A., Bellitto, G., Calcagno, S., Hendrix, R., Spampinato, C., & Palazzo, S. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00548
“Approaches like Orthogonal Weights Modification (OWM) [28] project gradients to maintain independence between task solutions, thereby minimizing de-structive interference.”
Ma, G., Chen, Y., Zhang, Y., Guo, L., Wu, Y., & Yu, S. (2026) · Neurocomputing · doi:10.1016/j.neucom.2026.133139
A neural network that bridges sensory experience and symbolic thought
(2026) · Nature Computational Science · doi:10.1038/s43588-026-00968-0
Hu, K., Wen, L., Zhang, T., & Zhang, H. (2026) · Scientific Reports · doi:10.1038/s41598-026-42970-6
Learning Prompt Adapters for Forgetting-Free Continual Image Super-Resolution
Fang, C., Fu, B., Cheng, D., Tang, C., & Li, G. (2026) · IEEE Transactions on Image Processing · doi:10.1109/tip.2026.3671688
“Regularization-based approaches [34], [35], [36] are a classical type of methods proposed for continual learning.”
Training constrains neural routes to knowledge assemblyPreprint
Wang, Q., French, C., Bansiya, P., Rabii, N., & Nelli, S. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.03.13.711547
“Another approach minimizes interference through high dimensional sparse coding strategies including orthogonalization and context-dependent gating 14,21–27 .”
Hu, K., Wen, L., Zhang, T., Zhang, X., & Zhang, H. (2025) · International Conference on Cognitive and Intelligent Technology (ICCIT) · doi:10.1109/iccit68389.2025.11453443
“Prior work has explored orthogonality in continual learning, often by constraining update directions through gradient or subspace projections (e.g., OWM [5] and OGD [6]).”
Continual Learning Through the Lens of Adaptive Filtering: A mathematical tutorial
Peng, L., & Vidal, R. (2026) · IEEE Signal Processing Magazine · doi:10.1109/msp.2026.3657516
“This was recently repurposed for deep continual learning to reduce forgetting [22].”
L, G., & Yu, S. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11464514
Sassioui, A., Elouargui, Y., Kamili, M. E., Benouini, R., Benyoussef, E. M., Chergui, M., & Ouzzif, M. (2026) · Technologies · doi:10.3390/technologies14050250
Wang, J., Hu, M., Li, N., Al-Ali, A., & Suganthan, P. N. (2026) · Neural Networks · doi:10.1016/j.neunet.2026.109115 · Cited by 1
Leveraging Information Flow for Knowledge Transfer in Continual Learning
Andle, J., Payani, A., & Yasaei-Sekeh, S. (2026) · Neural Processing Letters · doi:10.1007/s11063-026-11852-3
Chen, L., Fu, D., Shu, G., Zhang, S., & Liu, B. (2026) · Symmetry · doi:10.3390/sym18050855
SD2-SNN: Self-distillation and structural decomposition framework for SNNs in continual learning
Xie, Z., Xiao, X., Zhang, H., Pang, Y., & Ji, Z. (2026) · Neural Networks · doi:10.1016/j.neunet.2026.109241
Ship hydrodynamic prediction by a novel transformer model with incremental learning
Jin, J., Chen, D., Wang, X. Z., Jiang, S., Zhou, C., & Sun, J. (2026) · Intelligent Data Analysis · doi:10.1177/1088467x261457308
Prompt-Based Hypernetwork in Rehearsal-Free Class Incremental Learning
Liu, Y., & Zeng, Z. (2026) · Communications in computer and information science · doi:10.1007/978-981-92-1599-7_29
Yang, H., Tang, J., & Yang, S. (2026) · The Journal of Supercomputing · doi:10.1007/s11227-026-08667-3
“These include regularization-based methods (e.g., EWC [26]), replay-based meth-ods (e.g., Reservoir Sampling [27]), optimization-based methods (e.g., OWM [28]), representation-based methods [29], and architectural-based methods [30], among others[31–33].”
FreqOrtho-SR: Frequency-Guided Orthogonal Expert Learning for Real-World Image Super-Resolution
Hoang, M. S., Tran, D. P., Duc, Q. N., Phuong, D. H., & Kim, D. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37271-0_8
Learning to Cooperate for Continual Learning with Pre-Training and Parameter-Efficient Fine-TuningPreprint
Zhang, D., Liu, J., Lu, Z. Z., & Song, Z. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7043025
Functionality Separation: Rethinking Dual-Stream Networks for Class-Incremental Learning
Gao, Q., Li, X., Sun, Z., Wang, B., Zhang, Z., Hu, Y., Yin, B., & Gao, W. (2026) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2026.3710882
“Another approach [30], [31], [32] avoids interference with previously learned tasks by constraining gradient updates through subspace projection.”
Metadata
- Authors
- Zeng, G., Chen, Y., Cui, B., & Yu, S.
- Affiliations
- Chinese Academy of Sciences (CN)
- ORCID
- Yu, Shan
- Venue
- Nature Machine Intelligence
- Published
- 2019-08-09
- Volume
- 1
- Issue
- 8
- Pages
- 364-372
- ISSN
- 2522-5839 (ISSN-L)
- OpenAlex
- W2895699061
- Cited by
- 309
- References
- 41
- Citations by year
- 2026: 51 · 2025: 63 · 2024: 54 · 2023: 52 · 2022: 23 · 2021: 39 · 2020: 21 · 2019: 6
- FWCI
- 10.27
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- Nature Machine Intelligence (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- continual learning, catastrophic forgetting, context-dependent processing, neural network modules, multi-task learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
