Continual learning of context-dependent processing in neural networks
Open accessThe claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Input created from a generative model can assist with learning, such as preventing catastrophic forgetting.
“It has been shown that input created from a generative model can indeed assist with learning, such as preventing catastrophic forgetting.149”
From The overfitted brain: Dreams evolved to assist generalization (2021) · cites this paper as [149]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126 · see citing works
Golub, G. (2012) · doi:10.56021/9781421407944 · Cited by 34,505 · see citing works
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741 · see citing works
An Integrative Theory of Prefrontal Cortex Function
Miller, E. K., & Cohen, J. D. (2001) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.24.1.167 · Cited by 12,803 · see citing works
Neural Mechanisms of Selective Visual Attention
Desimone, R., & Duncan, J. (1995) · Annual Review of Neuroscience · doi:10.1146/annurev.ne.18.030195.001205 · Cited by 8,388 · see citing works
Overcoming catastrophic forgetting in neural networks
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1611835114 · Cited by 7,899 · see citing works
Deep Learning Face Attributes in the Wild
Liu, Z., Luo, P., Wang, X., & Tang, X. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.425 · Cited by 7,747 · see citing works
International Conference on Learning Representations (ICLR 2013)
松尾, 豊. (2013) · 人工知能学会誌 = Journal of Japanese Society for Artificial Intelligence · Cited by 6,286 · see citing works
McClelland, J. L., McNaughton, B. L., & O'Reilly, R. C. (1995) · Psychological Review · doi:10.1037/0033-295x.102.3.419 · Cited by 5,256 · see citing works
Li, Z., & Hoiem, D. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2773081 · Cited by 4,202 · see citing works
Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem
McCloskey, M., & Cohen, N. J. (1989) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(08)60536-8 · Cited by 3,770 · see citing works
Continual lifelong learning with neural networks: A review
Parisi, G. I., Kemker, R., Part, J. L., Kanan, C., & Wermter, S. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.01.012 · Cited by 3,117 · see citing works
The prefontral cortex and cognitive control
Miller, E. K. (2000) · Nature Reviews Neuroscience · doi:10.1038/35036228 · Cited by 2,016 · see citing works
Context-dependent computation by recurrent dynamics in prefrontal cortex
Mante, V., Sussillo, D., Shenoy, K. V., & Newsome, W. T. (2013) · Nature · doi:10.1038/nature12742 · Cited by 1,807 · see citing works
The importance of mixed selectivity in complex cognitive tasks
Rigotti, M., Barak, O., Warden, M. R., Wang, X.-J., Daw, N. D., Miller, E. K., & Fusi, S. (2013) · Nature · doi:10.1038/nature12160 · Cited by 1,639 · see citing works
Rusu, A. A., Rabinowitz, N. C., Desjardins, G., Soyer, H., Kirkpatrick, J., Kavukcuoglu, K., Pascanu, R., & Hadsell, R. (2016) · arXiv · doi:10.48550/arxiv.1606.04671 · Cited by 1,307 · cited 1× · see citing works
“In contrast to regularization approaches, other types of continual learning methods involve dynamically introducing extra neurons or layers along the learning process [42], which may help mitigate the tradeoff described above [15].”
BackgroundThe ventral visual pathway: an expanded neural framework for the processing of object quality
Kravitz, D. J., Saleem, K. S., Baker, C. I., Ungerleider, L. G., & Mishkin, M. (2012) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2012.10.011 · Cited by 1,154 · see citing works
Categorical Representation of Visual Stimuli in the Primate Prefrontal Cortex
Freedman, D. J., Riesenhuber, M., Poggio, T., & Miller, E. K. (2001) · Science · doi:10.1126/science.291.5502.312 · Cited by 1,123 · see citing works
Generating Coherent Patterns of Activity from Chaotic Neural Networks
Sussillo, D., & Abbott, L. F. (2009) · Neuron · doi:10.1016/j.neuron.2009.07.018 · Cited by 1,092 · see citing works
An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks
Goodfellow, I. J., Mirza, M., Xiao, D., Courville, A., & Bengio, Y. (2013) · arXiv · doi:10.48550/arxiv.1312.6211 · Cited by 1,089 · cited 1× · see citing works
“Shuffled MNIST Experiment 3 tasks Accuracy (%) 10 tasks Accuracy (%) 100 tasks Accuracy (%) SGD#[14] 71.”
MethodWord learning as Bayesian inference.
Xu, F., & Tenenbaum, J. B. (2007) · Psychological Review · doi:10.1037/0033-295x.114.2.245 · Cited by 1,034 · see citing works
The Frontal Lobes and Voluntary Action
Passingham, R. (1993) · doi:10.1093/oso/9780198521853.001.0001 · Cited by 970 · see citing works
Ratcliff, R. (1990) · Psychological Review · doi:10.1037/0033-295x.97.2.285 · Cited by 892 · see citing works
Fast Readout of Object Identity from Macaque Inferior Temporal Cortex
Hung, C. P., Kreiman, G., Poggio, T., & DiCarlo, J. J. (2005) · Science · doi:10.1126/science.1117593 · Cited by 854 · see citing works
The intelligibility of speech as a function of the context of the test materials.
Miller, G. A., Heise, G. A., & Lichten, W. (1951) · Journal of Experimental Psychology · doi:10.1037/h0062491 · Cited by 770 · see citing works
What Learning Systems do Intelligent Agents Need? Complementary Learning Systems Theory Updated
Kumaran, D., Hassabis, D., & McClelland, J. L. (2016) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2016.05.004 · Cited by 736 · see citing works
Branch-specific dendritic Ca2+ spikes cause persistent synaptic plasticity
Cichon, J., & Gan, W. (2015) · Nature · doi:10.1038/nature14251 · Cited by 509 · see citing works
The Frontal Cortex-Basal Ganglia System in Primates
Wise, S. P., Murray, E. A., & Gerfen, C. R. (1996) · Critical Reviews in Neurobiology · doi:10.1615/critrevneurobiol.v10.i3-4.30 · Cited by 437 · see citing works
Cortical information flow during flexible sensorimotor decisions
Siegel, M., Buschman, T. J., & Miller, E. K. (2015) · Science · doi:10.1126/science.aab0551 · Cited by 434 · see citing works
Progress & Compress: A scalable framework for continual learningPreprint
Schwarz, J., Luketina, J., Czarnecki, W. M., Grabska‐Barwińska, A., Teh, Y. W., Pascanu, R., & Hadsell, R. T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.06370 · Cited by 422 · cited 4× · see citing works
“P If the capacity limit of the entire network is finally approached, two solutions can be considered: 1) introduction of a larger α or the forgetting factor used in RLS [16] and online EWC [50]; and 2) addition of more layer(s), e.g., CDP module (see below for details), to provide more space to preserve previously lear…”
InfluentialICDAR 2013 Chinese Handwriting Recognition Competition
Yin, F., Wang, Q., Zhang, X.-Y., & Liu, C. (2013) · IAPR International Conference on Document Analysis and Recognition (ICDAR) · doi:10.1109/icdar.2013.218 · Cited by 324 · cited 1× · see citing works
“We found that a classifier trained with the OWM could learn to recognize all 3,755 characters sequentially, with a final accuracy ∼ 92% closely approaching the results obtained in human performance when recognizing handwritten Chinese characters (∼ 96%) [28].”
ResultMiller, E. K. (1999) · Neuron · doi:10.1016/s0896-6273(00)80673-x · Cited by 284 · see citing works
What helps where – and why? Semantic relatedness for knowledge transfer
Rohrbach, M., Stark, M. M., Szarvas, G., Gurevych, I., & Schiele, B. (2010) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2010.5540121 · Cited by 273 · see citing works
Anatomy of the Prefrontal Cortex
Fuster, J. M. (2015) · Elsevier eBooks · doi:10.1016/b978-0-12-407815-4.00002-7 · Cited by 272 · see citing works
Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization
Masse, N. Y., Grant, G. D., & Freedman, D. J. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1803839115 · Cited by 260 · see citing works
Microstructural proliferation in human cortex is coupled with the development of face processing
Gomez, J., Barnett, M., Natu, V. S., Mezer, A., Palomero‐Gallagher, N., Weiner, K. S., Amunts, K., Zilles, K., & Kalanit Grill-spector, A. (2017) · Science · doi:10.1126/science.aag0311 · Cited by 173 · see citing works
Optimal filtering algorithms for fast learning in feedforward neural networks
Shah, S., Palmieri, F., & Datum, M. (1992) · Neural Networks · doi:10.1016/s0893-6080(05)80139-x · Cited by 172 · see citing works
Chinese Handwriting Recognition Contest 2010
Liu, C., Yin, F., Wang, D.-H., & Wang, Q. (2010) · Chinese Conference on Pattern Recognition · doi:10.1109/ccpr.2010.5659229 · Cited by 47 · see citing works
Dimensionality of Object Representations in Monkey Inferotemporal Cortex
Lehky, S. R., Kiani, R., Esteky, H., & Tanaka, K. (2014) · Neural Computation · doi:10.1162/neco_a_00648 · Cited by 44 · see citing works
The Role of Prefrontal Mixed Selectivity in Cognitive Control
Cardenas, A. R., & Viswanathan, P. (2016) · Journal of Neuroscience · doi:10.1523/jneurosci.1816-16.2016 · Cited by 8 · see citing works
Beijixiong3510 (2019) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.3346079 · Cited by 1 · see citing works
Flagged first, then by citations
Research works printed but not linked: 14
- [1][1]Newell, A. Unified theories of cognition (Harvard University Press, 1994). -
- [16][16]Haykin, S. S. Adaptive filter theory (Pearson Education India, 2008). -
- [18][18]Singhal, S. & Wu, L. Training feed-forward networks with the extended kalman algorithm. In Acoustics, Speech, and Signal Processing, 1989. ICASSP-89., 1989 International Conference on , 1187-1190 (IEEE, 1989). -
- [22][22]He, X. & Jaeger, H. Overcoming catastrophic interference using conceptor-aided backpropagation. In International Conference on Learning Representations (2018). -
- [23][23]Nair, V. & Hinton, G. E. Rectified linear units improve restricted boltzmann machines. In International Conference on International Conference on Machine Learning (2010). -
- [25][25]Lee, S.-W., Kim, J.-H., Jun, J., Ha, J.-W. & Zhang, B.-T. Overcoming catastrophic forgetting by incremental moment matching. In Advances in Neural Information Processing Systems , 4652-4662 (2017). -
- [30][30]Hu, W. et al. Overcoming catastrophic forgetting via model adaptation. In International Conference on Learning Representations (2019). -
- [31][31]Fuster, J. The prefrontal cortex (Academic Press, 2015). -
- [33][33] ˇ Reh˚ uˇ rek, R. & Sojka, P. Software framework for topic modelling with large corpora. In Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks , 45-50 (ELRA, Valletta, Malta, 2010). -
- [45][45]Shin, H., Lee, J. K., Kim, J. & Kim, J. Continual learning with deep generative replay. In Advances in Neural Information Processing Systems , 2990-2999 (2017). -
- [48][48]Yosinski, J., Clune, J., Bengio, Y . & Lipson, H. How transferable are features in deep neural networks? In Advances in neural information processing systems , 3320-3328 (2014). -
- [51][51]Glorot, X. & Bengio, Y. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the thirteenth international conference on artificial intelligence and statistics , 249256 (2010). -
- [52][52]Nair, V. & Hinton, G. E. Rectified linear units improve restricted boltzmann machines. In Proceedings of the 27th international conference on machine learning (ICML-10) , 807-814 (2010). -
- [53][53]Srivastava, R. K., Masci, J., Kazerounian, S., Gomez, F. & Schmidhuber, J. Compete to compute. In Advances in neural information processing systems , 2310-2318 (2013). -
Cited by
Sorted by citations
A Comprehensive Survey of Continual Learning: Theory, Method and Application
Wang, L., Zhang, X., Su, H., & Zhu, J. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3367329 · Cited by 1,113 · see citing works
“Alternatively, gradient projection can also be performed without storing old training samples [51], [65], [66], [67].”
BackgroundThree types of incremental learning
van de Ven, G. M., Tuytelaars, T., & Tolias, A. S. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00568-3 · Cited by 665 · see citing works
“2 shows how Split MNIST, which is a popular toy problem for continual learning 27,28,31,32 , can be performed according to each of the three scenarios.”
If deep learning is the answer, what is the question?
Saxe, A., Nelli, S., & Summerfield, C. (2021) · Nature Reviews Neuroscience · doi:10.1038/s41583-020-00395-8 · Cited by 397 · see citing works
“Hebbian learning might encourage the formation of orthogonal neural codes for different temporal contexts 133 , which in turn enables tasks to be learned in different neural subspaces 132 .”
BackgroundClass-Incremental Learning: A Survey
Zhou, D.-W., Wang, Q.-W., Qi, Z.-H., Ye, H.-J., Zhan, D.-C., & Liu, Z. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3429383 · Cited by 376 · see citing works
“OWM [56] only allows weight modification in the direction orthogonal to the subspace spanned by all previously learned inputs.”
BackgroundInfluentialOrthogonal Gradient Descent for Continual LearningPreprint
Farajtabar, M., Azizan, N., Mott, A., & Li, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.07104 · Cited by 184 · see citing works
Robust Spike-Based Continual Meta-Learning Improved by Restricted Minimum Error Entropy Criterion
Yang, S., Tan, J., & Chen, B. (2022) · Entropy · doi:10.3390/e24040455 · Cited by 157 · see citing works
“The human brain has implemented an efficient and scalable mechanism for continual learning based on neuronal activity patterns that represent previous experiences [6].”
BackgroundTraining Networks in Null Space of Feature Covariance for Continual Learning
Wang, S., Li, X., Sun, J., & Xu, Z. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00025 · Cited by 126 · see citing works
“Different from GEM and A-GEM, OWM [54] projects the parameter up-date into the orthogonal space of the space spanned by input features of each linear layer.”
BackgroundMethodInfluentialMgSvF: Multi-Grained Slow versus Fast Framework for Few-Shot Class-Incremental Learning
Zhao, H., Fu, Y., Kang, M., Tian, Q., Wu, F., & Li, X. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3133897 · Cited by 115 · see citing works
“Recently, there has been a large body of research in incremental learning [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38], [39], [40], [41], [42], [43].”
A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning
Wang, Z., Yang, E., Shen, L., & Huang, H. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3498346 · Cited by 101 · see citing works
“(2) Orthogonal Feature Subspace: These require that the parameter update direction of the new task is orthogonal to the subspace spanned by the input(feature) of the old tasks [62], [63], [64], [65], [66].”
The Computational and Neural Bases of Context-Dependent Learning
Heald, J. B., Wolpert, D. M., & Lengyel, M. (2023) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-092322-100402 · Cited by 84 · see citing works
A framework for the general design and computation of hybrid neural networks
Zhao, R. J., Yang, Z., Zheng, H. R., Wu, Y., Liu, F., Wu, Z., Li, L. T., Chen, F., Song, S., Zhu, J., et al. (2022) · Nature Communications · doi:10.1038/s41467-022-30964-7 · Cited by 77 · see citing works
GAN Memory with No ForgettingPreprint
Cong, Y., Zhao, M., Li, J., Wang, S., & Carin, L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.07543 · Cited by 77 · see citing works
“To alleviate catastrophic forgetting, many methods have been proposed, with most focusing on discriminative/classification tasks [36, 65, 95, 55, 94].”
MethodZhang, T., Cheng, X., Jia, S., Poo, M., Zeng, Y., & Xu, B. (2021) · Science Advances · doi:10.1126/sciadv.abh0146 · Cited by 75 · see citing works
“E j SNN = a V j (t) 2 + b ∑ i N ( V i (t ) W i,j V j (t ) ) + c V j (t) (6)”
The overfitted brain: Dreams evolved to assist generalization
Hoel, E. (2021) · Patterns · doi:10.1016/j.patter.2021.100244 · Cited by 70 · see citing works
“It has been shown that input created from a generative model can indeed assist with learning, such as preventing catastrophic forgetting.(149) It should be noted that in most these cases the generative model exists outside the network itself, which is not biologically realistic in the case of the brain, although there…”
BackgroundContinual Learning: A Review of Techniques, Challenges, and Future Directions
Wickramasinghe, B., Saha, G., & Roy, K. (2023) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2023.3339091 · Cited by 73 · see citing works
“Orthogonal weight modulation (OWM) reduces interference to old information by modifying the weights of the network in a direction orthogonal to the subspace spanned by the inputs of old tasks [142].”
BackgroundA Continual Deepfake Detection Benchmark: Dataset, Methods, and Essentials
Li, C., Huang, Z., Paudel, D. P., Wang, Y., Shahbazi, M., Hong, X., & Gool, L. V. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00139 · Cited by 69 · see citing works
“1 Gradient-based methods (e.g., [62, 85, 23, 65, 70, 75]) overcome catastrophic forgetting by minimizing the interference among task-wise gradients when updating network weights.”
MethodRecent Advances of Continual Learning in Computer Vision: An Overview
Qu, H., Rahmani, H., Xu, L., Williams, B. M., & Liu, J. (2025) · IET Computer Vision · doi:10.1049/cvi2.70013 · Cited by 80 · see citing works
Lifelong Incremental Reinforcement Learning With Online Bayesian Inference
Wang, Z., Chen, C., & Dong, D. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3055499 · Cited by 62 · see citing works
Towards Better Plasticity-Stability Trade-off in Incremental Learning: A Simple Linear Connector
Lin, G., Chu, H., & Lai, H. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00019 · Cited by 67 · see citing works
“OWM [43] is orthogonal weight modification method to overcome catastrophic forgetting.”
An Intelligent Transient Stability Assessment Framework with Continual Learning Ability
Li, X., Yang, Z., Guo, P., & Cheng, J. (2021) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2021.3064052 · Cited by 62 · see citing works
Continual All-in-One Adverse Weather Removal With Knowledge Replay on a Unified Network Structure
Cheng, D., Ji, Y., Gong, D., Li, Y., Wang, N., Han, J., & Zhang, D. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3377136 · Cited by 60 · see citing works
“Some others, like Adam-NSC [33] and OWM [34], regulate the direction of parameter change.”
BackgroundTuning Convolutional Spiking Neural Network With Biologically Plausible Reward Propagation
Zhang, T., Jia, S., Cheng, X., & Xu, B. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3085966 · Cited by 58 · see citing works
Sleep-like unsupervised replay reduces catastrophic forgetting in artificial neural networks
Tadros, T., Krishnan, G. P., Ramyaa, R., & Bazhenov, M. (2022) · Nature Communications · doi:10.1038/s41467-022-34938-7 · Cited by 59 · see citing works
Adapting BERT for Continual Learning of a Sequence of Aspect Sentiment Classification Tasks
Ke, Z., Xu, H., & Liu, B. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.378 · Cited by 64 · see citing works
“HAT, UCL, EWC and OWM were originally designed for image classification.”
MethodInfluentialContinual learning and catastrophic forgetting
van de Ven, G. M., Soures, N., & Kudithipudi, D. (2024) · Elsevier eBooks · doi:10.1016/b978-0-443-15754-7.00073-0 · Cited by 81 · see citing works
“A first popular approach is ‘orthogonal gradient projection’ (Zeng et al., 2019; Farajtabar et al., 2020; Saha et al., 2021).”
BackgroundCatastrophic Forgetting in Deep Learning: A Comprehensive Taxonomy
Aleixo, E. L., Colonna, J. G., Cristo, M., & Fernandes, E. (2023) · arXiv · doi:10.48550/arxiv.2312.10549 · Cited by 54
“It is supported by finding orthogonal filters for each task and, in inference time, using the task identifier to retrieve the right filters [Zeng et al ., 2019].”
MethodChen, J., & Qiao, H. (2020) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2020.3045574 · Cited by 52 · see citing works
“To realize continuous multitask learning with RNN, the OWM method [60] is introduced and improved in this section.”
MethodHuang, X., Xie, T., Luo, S., Wu, J., Luo, R., & Zhou, Q. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.108212 · Cited by 51 · see citing works
Catastrophic Forgetting in Deep Learning: A Comprehensive Taxonomy
Aleixo, E. L., Colonna, J. G., Cristo, M., & Fernandes, E. (2024) · Journal of the Brazilian Computer Society · doi:10.5753/jbcs.2024.3966 · Cited by 54 · see citing works
Flesch, T., Nagy, D., Saxe, A., & Summerfield, C. (2023) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1010808 · Cited by 49 · see citing works
“Orthogonalisation approaches in particular have gained attention in neuroscience [31, 33], as more recent investigations of neural geometry have shown that during multi-task performance, mutual interference among tasks is minimised by projecting relevant dimensions into orthogonal, low-dimensional subspaces [15, 17, 4…”
BackgroundCLASSIC: Continual and Contrastive Learning of Aspect Sentiment Classification Tasks
Ke, Z., Liu, B., Xu, H., & Shu, L. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.550 · Cited by 49 · see citing works
Organizing recurrent network dynamics by task-computation to enable continual learning
Duncker, L., Driscoll, L. N., Shenoy, K. V., Sahani, M., & Sussillo, D. (2020) · UCL Discovery (University College London) · Cited by 45 · see citing works
Brain works principle followed by neural information processing: a review of novel brain theory
Wang, R., Wang, Y., Xu, X., Li, Y., & Pan, X. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10520-5 · Cited by 43 · see citing works
Electrochemical ohmic memristors for continual learning
Chen, S., Yang, Z., Hartmann, H., Besmehn, A., Yang, Y., & Valov, I. (2025) · Nature Communications · doi:10.1038/s41467-025-57543-w · Cited by 43 · see citing works
“5a) 45 .”
Linear Mode Connectivity in Multitask and Continual LearningPreprint
Mirzadeh, S. I., Farajtabar, M., Görür, D., Pascanu, R., & Ghasemzadeh, H. G. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.04495 · Cited by 55 · see citing works
Sun, W., Li, Q., Zhang, J., Wang, D., Wang, W., & Geng, Y. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2023.109561 · Cited by 39 · see citing works
Memory Recall: A Simple Neural Network Training Framework Against Catastrophic Forgetting
Zhang, B., Guo, Y., Li, Y., He, Y., Wang, H., & Dai, Q. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3099700 · Cited by 41 · see citing works
A Survey of Geometric Optimization for Deep Learning: From Euclidean Space to Riemannian Manifold
Fei, Y., Liu, Y., Jia, C., Zheng-yu, L., Wei, X. H., & Chen, M. (2024) · ACM Computing Surveys · doi:10.1145/3708498 · Cited by 39 · see citing works
“of gradient orthogonality was first proposed in [90].”
Hardware Acceleration of EEG-Based Emotion Classification Systems: A Comprehensive Survey
Gonzalez, H. A., George, R., Muzaffar, S., Acevedo, J., Höppner, S., Mayr, C., Yoo, J., Fitzek, F. H. P., & Elfadel, I. M. (2021) · IEEE Transactions on Biomedical Circuits and Systems · doi:10.1109/tbcas.2021.3089132 · Cited by 37 · see citing works
“Another learning paradigm that is also geared toward improving model performance as old data is updated or new data becomes available is task-progressive [212], continual [213], or lifelong learning [214].”
BackgroundCKDF: Cascaded Knowledge Distillation Framework for Robust Incremental Learning
Li, K., Wan, J., & Yu, S. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3176130 · Cited by 36 · see citing works
“1) Regularization Approaches: The first is regularization approaches [14]–[20].”
BackgroundZhang, T., Cheng, X., Jia, S., Li, C., Poo, M., & Xu, B. (2023) · Science Advances · doi:10.1126/sciadv.adi2947 · Cited by 36 · see citing works
CRNet: A Fast Continual Learning Framework With Random Theory
Li, D., & Zeng, Z. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3262853 · Cited by 36 · see citing works
“We follow the setting in OWM [56] where a feature extractor pretrained by the complete training sets is first used to analyze the”
MethodInfluentialContinual task learning in natural and artificial agents
Flesch, T., Saxe, A., & Summerfield, C. (2023) · Trends in Neurosciences · doi:10.1016/j.tins.2022.12.006 · Cited by 37 · see citing works
“As a proof of concept, methods that project learning updates (gradients) for new tasks into independent neural subspaces have been shown to help with continual learning in machine learning models, including recurrent networks [67,68].”
Variational Data-Free Knowledge Distillation for Continual Learning
Li, X., Wang, S., Sun, J. Z., & Xu, Z. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3271626 · Cited by 33 · see citing works
One-shot Federated Learning without server-side training
Su, S., Li, B., & Xue, X. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.04.035 · Cited by 34 · see citing works
“The projection matrix has been applied into Continual Learning [40, 41, 42].”
MethodRegularizing Second-Order Influences for Continual Learning
Sun, Z., Mu, Y., & Hua, G. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01931 · Cited by 33 · see citing works
“To overcome this effect, three main streams of methods have been developed: weight regularization [28, 33, 57], parameter isolation [2, 43, 45, 56] and memory replay [34, 41, 46].”
Domain-incremental learning for fire detection in space-air-ground integrated observation network
Wang, M., Yu, D., He, W., Yue, P., & Liang, Z. (2023) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2023.103279 · Cited by 33 · see citing works
Balancing Stability and Plasticity Through Advanced Null Space in Continual Learning
Kong, Y., Liu, L., Wang, Z., & Tao, D. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19809-0_13 · Cited by 32 · see citing works
Embodied tactile perception and learning
Liu, H., Guo, D., Sun, F., Yang, W., Furber, S. B., & Sun, T. (2020) · Brain Science Advances · doi:10.26599/bsa.2020.9050012 · Cited by 31 · see citing works
“The team demonstrated a good lifelong learning ability in applications such as handwriting and face recognitions [134].”
BackgroundWhat to remember: Self-adaptive continual learning for audio deepfake detection
Zhang, X., Yi, J., Wang, C., Zhang, C. Y., Zeng, S., & Tao, J. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29929 · Cited by 32 · see citing works
Neural Manifold Modulated Continual Reinforcement Learning for Musculoskeletal Robots
Chen, J., Chen, Z., Yao, C., & Qiao, H. (2022) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2022.3231055 · Cited by 28 · see citing works
“For projection constraints-based methods, the input space of each layer in network is constructed by different ways and the update of weights is projected to the direction orthogonal to the input space [26], [27].”
BackgroundMethodHua, J., Li, Y., Mou, W., & Liu, C. (2021) · Proceedings of the Institution of Mechanical Engineers Part B Journal of Engineering Manufacture · doi:10.1177/0954405421993694 · Cited by 28 · see citing works
“24,25 To avoid catastrophic forgetting, the orthogonal weights modification method 26 is used for continual learning.”
MethodSpiking neural predictive coding for continually learning from data streams
Ororbia, A. G. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126292 · Cited by 38 · see citing works
Continual Learning with Scaled Gradient Projection
Saha, G., & Roy, K. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i8.26157 · Cited by 28 · see citing works
Kumari, P., Chauhan, J., Bozorgpour, A., Huang, B., Azad, R., & Merhof, D. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103730 · Cited by 33 · see citing works
Rethinking Gradient Projection Continual Learning: Stability/Plasticity Feature Space Decoupling
Zhao, Z. W., Zhang, Z., Tan, X., Liu, J., Qu, Y., Xie, Y., & Ma, L. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00362 · Cited by 25 · see citing works
A Dual-Channel Collaborative Transformer for continual learning
Cai, H., Wang, Y., Luo, Y., & Mao, K. (2025) · Applied Soft Computing · doi:10.1016/j.asoc.2025.112792 · Cited by 25 · see citing works
Continual Learning with Knowledge Transfer for Sentiment Classification
Ke, Z., Liu, B., Wang, H., & Shu, L. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-67664-3_41 · Cited by 25 · see citing works
“Orthogonal Weights Modification (OWM) [32] is a state-of-the-art class continual learning (CCL) method.”
BackgroundMethodInfluentialDu, J., Liu, P., Vong, C., Chen, C., Wang, T., & Chen, C. L. P. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3259016 · Cited by 24 · see citing works
“…and it is also compared with the existing DCIL methods, including EWC (2017) [33], LwF (2017) [34], incremental moment matching (IMM) (2017) [47], OWM (2019) [35], PGMA (2019) [36], random patch selection network (RPSnet) (2019) [48], and PCL (2021) [18] over five real-world image datasets.”
MethodLifelong Learning With Cycle Memory Networks
Peng, J., Ye, D., Tang, B., Lei, Y., Liu, Y., & Li, H. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3294495 · Cited by 25 · see citing works
“Zeng et al. [46] and Tang et al. [47] kept the parameter space of the preceding and following tasks orthogonal to avoid interference.”
BackgroundNikolić, D. (2023) · Computational Biology and Chemistry · doi:10.1016/j.compbiolchem.2023.107820 · Cited by 25 · see citing works
Understanding neural flexibility from a multifaceted definition
Yin, D., & Kaiser, M. (2021) · NeuroImage · doi:10.1016/j.neuroimage.2021.118027 · Cited by 24 · see citing works
“recently proposed an approach that includes an orthogonal weight modification algorithm and a context-dependent processing module, enabling a description of a neural network to progressively learn various mapping rules in a context-dependent manner (Zeng et al., 2019).”
Disturbance-immune weight sharing for neural architecture search
Niu, S., Wu, J., Zhang, Y., Guo, Y. F., Zhao, P., Huang, J., & Tan, M. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.09.002 · Cited by 23 · see citing works
Verma, T., Jin, L., Zhou, J. W., Huang, J., Tan, M., Choong, B. C. M., Tan, T. F., Gao, F., Xu, X., Ting, D. S. W., et al. (2023) · Frontiers in Medicine · doi:10.3389/fmed.2023.1227515 · Cited by 25 · see citing works
Ma, L., Zhou, Y., Ma, Y., Yu, G., Li, Q., He, Q., & Pei, Y. (2025) · IEEE Transactions on Computers · doi:10.1109/tc.2025.3540650 · Cited by 22 · see citing works
“Unfortunately, the projector attenuation will ultimately and severely degrade the efficiency [24] of the weight learning as well as the prediction ability of the supernet.”
Kang, M., Zhang, J., Zhang, J., Wang, X., Yang, C., Ma, Z., & Huang, X. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01732 · Cited by 22 · see citing works
Generative Feature Replay with Orthogonal Weight Modification for Continual Learning
Shen, G., Zhang, S., Chen, X., & Deng, Z. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9534437 · Cited by 22 · see citing works
Remaining Useful Life Prediction Based on Incremental Learning
Que, Z., Jin, X., Xu, Z., & Hu, C. (2023) · IEEE Transactions on Reliability · doi:10.1109/tr.2023.3294939 · Cited by 23 · see citing works
“The OWM algorithm [27], as shown in Fig.”
MethodFedSOL: Stabilized Orthogonal Learning with Proximal Restrictions in Federated Learning
Lee, G., Jeong, M., Kim, S., Oh, J., & Yun, S.-Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01189 · Cited by 21 · see citing works
“Second, unlike in CL where task distributions are often disjoint [13, 72], FL clients may have overlapping data distributions, including instances from the same class.”
BackgroundDealing with Cross-Task Class Discrimination in Online Continual Learning
Guo, Y., Liu, B., & Zhao, D. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01143 · Cited by 21 · see citing works
Brain-Inspired Collaborative Power Dynamic Dispatch for the Digital Twin Smart Generation System
Liu, Z., Xi, L., Quan, Y., & Cheng, C. (2024) · IEEE Transactions on Industry Applications · doi:10.1109/tia.2024.3373371 · Cited by 21 · see citing works
Rapid context inference in a thalamocortical model using recurrent neural networks
Zheng, W., Wu, Z., Hummos, A., Yang, G. R., & Halassa, M. M. (2024) · Nature Communications · doi:10.1038/s41467-024-52289-3 · Cited by 20 · see citing works
Recursive Least-Squares Estimator-Aided Online Learning for Visual Tracking
Gao, J. G., Hu, W., & Lu, Y. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00741 · Cited by 21 · see citing works
“It has recently proved valuable in numerous supervised learning and reinforcement learning-based applications along with deep learning success, leading to an emerging field [18, 15, 39, 27].”
BackgroundContinual learning in medical image analysis: A survey
Wu, X., Xu, Z., & Tong, R. K. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.109206 · Cited by 22 · see citing works
Learning Without Forgetting: A New Framework for Network Cyber Security Threat Detection
Karn, R. R., Kudva, P. N., & Elfadel, I. M. (2021) · IEEE Access · doi:10.1109/access.2021.3115946 · Cited by 20 · see citing works
“The recommended method to compute O µ + 1 is through an iterative process as in the Recursive Least Squares algo-rithm [6], with O µ + 1 computed for each layer separately using only the training dataset D µ + 1 and the projection for the most recent task O µ .”
BackgroundMethodInfluentialOpen-world continual learning: Unifying novelty detection and continual learning
Kim, G., Xiao, C., Konishi, T., Ke, Z., & Liu, B. (2025) · Artificial Intelligence · doi:10.1016/j.artint.2024.104237 · Cited by 19 · see citing works
Continual Learning Based on OOD Detection and Task Masking
Kim, G., Esmaeilpour, S., Xiao, C., & Liu, B. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00431 · Cited by 19 · see citing works
Multi-view class incremental learning
Li, D., Wang, T., Chen, J., Kawaguchi, K., Lian, C., & Zeng, Z. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.102021 · Cited by 19 · see citing works
“To make the comparison more complete, we also compare with CIL algorithms: (1) EWC [45]; (2) PCL [80]; (3) OWM [62]; (4) BiC [81]; (5) FS-DGPM [82]; (6) GEM [83]; (7) LOGD [84]; (8) IL2M [41].”
BackgroundMethodInfluentialExamining Adaptive E-Learning Approaches to Enhance Learning and Individual Experiences
Benkhalfallah, F., Laouar, M. R., & Benkhalfallah, M. S. (2024) · Acta Informatica Pragensia · doi:10.18267/j.aip.240 · Cited by 19 · see citing works
Energy-Based Models for Continual LearningPreprint
Li, S., Du, Y., van de Ven, G. M., & Mordatch, I. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.12216 · Cited by 19 · see citing works
“, 2018) or by using a different set of parameters for each task (Zeng et al., 2019; Hu et al., 2019).”
BackgroundNatural continual learning: success is a journey, not (just) a destination
Kao, T.-C., Jensen, K. T., van de Ven, G. M., Bernacchia, A., & Hennequin, G. (2021) · Lirias · doi:10.48550/arxiv.2106.08085 · Cited by 18 · see citing works
“…projection matrix P L designed to approximate the posterior covariance of previous tasks Λ − 1 k − 1 ≈ P R ⊗ P L (i.e., the prior covariance on task k ; Appendix D), while Zeng et al. (2019) use the identity matrix I and Duncker et al. (2020) use the covariance of recurrent inputs (Appendix F).”
BackgroundMethodInfluentialMemory efficient data-free distillation for continual learning
Li, X., Wang, S., Sun, J., & Xu, Z. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2023.109875 · Cited by 18 · see citing works
Data Augmented Flatness-aware Gradient Projection for Continual Learning
Yang, E., Shen, L., Wang, Z., Liu, S., Guo, G., & Wang, X. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00518 · Cited by 18 · see citing works
“To alleviate catastrophic forgetting of old tasks, several works [57, 15, 8, 27, 43] propose to constrain the gradient update direction of the new tasks.”
BackgroundInfluentialLi, K., Chen, H., Wan, J., & Yu, S. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3376197 · Cited by 19 · see citing works
“Regularization-based approaches introduce different regularization terms in loss functions to penalize the changes of network outputs to alleviate forgetting [9], [10].”
BackgroundSPACE: Structured Compression and Sharing of Representational Space for Continual Learning
Saha, G., Garg, I., Ankit, A., & Roy, K. (2021) · IEEE Access · doi:10.1109/access.2021.3126027 · Cited by 20 · see citing works
“However, in the context of continual learning (CL) [3, 4, 5, 6, 7], whereas humans can continuously adapt and learn new tasks throughout their lifetimes without forgetting the old ones, current deep learning methods struggle to perform well since the data distribution changes over the course of learning.”
BackgroundBeneficial Perturbation Network for Designing General Adaptive Artificial Intelligence Systems
Wen, S., Rios, A., Ge, Y., & Itti, L. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3054423 · Cited by 18 · see citing works
Hierarchically structured task-agnostic continual learning
Hihn, H., & Braun, D. A. (2022) · Machine Learning · doi:10.1007/s10994-022-06283-9 · Cited by 19 · see citing works
“, by mapping the gradient updates into a different space [26, 27].”
BackgroundHan, S., Liu, B., Fan, X., Feng, T., Yang, J., Zhou, Z., Gong, H., & Luo, J. (2023) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2023.105759 · Cited by 17 · see citing works
Signatures of task learning in neural representations
Gurnani, H., & Cayco-Gajic, N. A. (2023) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2023.102759 · Cited by 18 · see citing works
GopGAN: Gradients Orthogonal Projection Generative Adversarial Network With Continual Learning
Li, X., & Wang, W. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3093319 · Cited by 16 · see citing works
“The pioneering works, CAB [10] and OWM [39], are the variants of the backpropagation algorithm and have been successfully applied in classification tasks, but we find that there is no work to apply these two methods to GANs.”
MethodInfluentialContinual learning: A systematic literature review
Yang, Q., Wang, L., Wicker, J., & Dobbie, G. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.108226 · Cited by 17 · see citing works
Class Gradient Projection For Continual Learning
Chen, C., Zhang, J., Song, J., & Gao, L. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548054 · Cited by 15 · see citing works
“Recently, the gradient projection based paradigms [14, 35, 44] have achieved remarkable performance in tackling the catastrophic forgetting problem in CL.”
BackgroundMethodZheng, H., Liu, B., Han, S., Fan, X., Zou, T., Zhou, Z., & Gong, H. (2022) · Environmental Earth Sciences · doi:10.1007/s12665-022-10369-x · Cited by 15 · see citing works
A Two-Level Dynamic Adaptive Network for Medical Image Fusion
Huang, W., Zhang, H., Quan, X., & Wang, J. (2022) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2022.3169546 · Cited by 15 · see citing works
LoRA Subtraction for Drift-Resistant Space in Exemplar-Free Continual Learning
Liu, X., & Chang, X. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.01426 · Cited by 14 · see citing works
“Existing works [31, 37, 45] have shown that the gradient update in a linear layer lies within the span of the input data.”
Research on landslide susceptibility prediction model based on LSTM-RF-MDBN
Yang, X., Fan, X., Wang, K., & Zhou, Z. (2023) · Environmental Science and Pollution Research · doi:10.1007/s11356-023-31232-x · Cited by 14 · see citing works
Holton, E., Braun, L., Thompson, J. A. F., Grohn, J., & Summerfield, C. (2025) · Nature Human Behaviour · doi:10.1038/s41562-025-02318-y · Cited by 17 · see citing works
“This aligns with a growing literature in machine learning showing that catastrophic forgetting can be mitigated by encouraging networks to learn in orthogonal subspaces in more complex settings 45–49 .”
Adaptive Domain-Aware Representation Learning for Speech Emotion Recognition
Fan, W., Xu, X., Xing, X., & Huang, D. (2020) · Interspeech 2020 · doi:10.21437/interspeech.2020-2572 · Cited by 14 · see citing works
“Inspired by [20], we propose an adaptive domain-aware representation Learning (ADARL) in this paper, which consists of a multi-task representation learning by combining domain clas-sification and emotion recognition, and an extracting domain aware representation through the domain knowledge…”
BackgroundMethodCognitively Inspired Learning of Incremental Drifting Concepts
Rostami, M. G., & Galstyan, A. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/341 · Cited by 14 · see citing works
incDFM: Incremental Deep Feature Modeling for Continual Novelty Detection
Rios, A., Ahuja, N., Ndiour, I. J., Genc, U., Itti, L., & Tickoo, O. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19806-9_34 · Cited by 13 · see citing works
“…of task labels which are usually provided by a task oracle and utilize this information by employing task-specific classification heads and other task-dependent parameters to share the rest of the network for different tasks, e.g. partitioning with context [48, 47, 5] or mask matrices [27, 9].”
Method
- Authors
- Zeng, G., Chen, Y., Cui, B., & Yu, S.
- Affiliations
- Chinese Academy of Sciences (CN)
- ORCID
- Yu, Shan
- Venue
- Nature Machine Intelligence
- Published
- 2019-08-09
- Volume
- 1
- Issue
- 8
- Pages
- 364-372
- ISSN
- 2522-5839 (ISSN-L)
- OpenAlex
- W2895699061
- Cited by
- 309
- References
- 41
- Citations by year
- 2026: 51 · 2025: 63 · 2024: 54 · 2023: 52 · 2022: 23 · 2021: 39 · 2020: 21 · 2019: 6
- FWCI
- 10.27
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- Nature Machine Intelligence (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- continual learning, catastrophic forgetting, context-dependent processing, neural network modules, multi-task learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
