The DOI matches its Crossref record; 36 of the 37 references it cites were found in a registry or an index; none of the 28 sources its claims cite is checked for support yet.
Memory Aware Synapses: Learning What (not) to Forget
Open accessAbstract
Humans can learn in a continuous manner. Old rarely utilized knowledge can be overwritten by new incoming information while important, frequently used knowledge is prevented from being erased. In artificial learning systems, lifelong learning so far has focused mainly on accumulating knowledge over tasks and overcoming catastrophic forgetting. In this paper, we argue that, given the limited model capacity and the unlimited new information to be learned, knowledge has to be preserved or erased selectively. Inspired by neuroplasticity, we propose a novel approach for lifelong learning, coined Memory Aware Synapses (MAS). It computes the importance of the parameters of a neural network in an unsupervised and online manner. Given a new sample which is fed to the network, MAS accumulates an importance measure for each parameter of the network, based on how sensitive the predicted output function is to a change in this parameter. When learning a new task, changes to important parameters can then be penalized, effectively preventing important knowledge related to previous tasks from being overwritten. Further, we show an interesting connection between a local version of our method and Hebb's rule,which is a model for the learning process in the brain. We test our method on a sequence of object recognition tasks and on the challenging problem of learning an embedding for predicting $ $ triplets. We show state-of-the-art performance and, for the first time, the ability to adapt the importance of the parameters based on unlabeled data towards what the network needs (not) to forget, which may vary depending on test conditions.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
Memory Aware Synapses (MAS) estimates each network parameter’s importance from the sensitivity of the learned function’s output, not the loss, so importance can be computed online and unsupervised on labeled or unlabeled data. It penalizes changes to important parameters when learning new tasks and reports state-of-the-art lifelong learning results on object recognition and fact-learning embedding tasks.
Key findings
- MAS reduces catastrophic forgetting in object recognition: in two-task sequences it achieves average forgetting around 1%, while FineTune drops 8–13%; in an 8-task sequence MAS has average forgetting of 0.49% and average accuracy 52.695, compared with SI at 50.49 and Finetune at 32.67.
- In fact learning on the 6DS dataset, MAS reaches overall MAP 0.29 on a 4-task random split versus SI 0.25 and Finetune 0.18; using both training and test data for importance further improves MAS to 0.30.
- MAS can adapt importance to unlabeled test conditions: for the Krishna Kamal flower subset, forgetting was 2% versus 8% on the whole Flower dataset, and last-task accuracy was 84% instead of 69% without adaptation; on a sports-fact subset, MAS reached 0.50 MAP while Finetune and SI were close to 0.20.
- A local variant, l-MAS, is linked to Hebbian learning and computes importance from neuron activation correlations; it is faster but less accurate, with average forgetting of 3% versus 1% for global MAS.
Methods
- MAS is a model-based lifelong learning method: after convergence, it accumulates an importance weight Ω_ij for each parameter by measuring how sensitive the squared ℓ2 norm of the learned function output is to parameter changes over available data points.
- When learning a new task, it adds a regularizer that penalizes changes to parameters with high Ω_ij, allowing low-importance parameters to adapt; Ω can be updated online and computed on training, validation, or unlabeled test data.
- Experiments use AlexNet pretrained on ImageNet for sequences of object recognition datasets (MIT Scenes, Caltech-UCSD Birds, Oxford Flowers, Stanford Cars, FGVCAircraft, VOC Actions, Letters, SVHN) and a VGG-16-based embedding model on the 6DS fact-learning dataset, reporting classification accuracy and mean average precision (MAP).
Limitations
- The paper states that l-MAS is faster but less accurate than global MAS, with average forgetting of 3% versus 1%.
- The paper states that λ was set to 1 in object recognition experiments without tuning, because it assumes no access to previous task data.
- Not explicitly stated as a limitation, but evident from the text: the object recognition setup assumes an oracle at test time to select the task-specific classification head, and separate unshared heads per task.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
34 Own interpretation · 28 Own result · 15 Not checked yet · 12 No basis given · 1 Own method
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, here, our method differentiates itself clearly, showing 6 % better MAP on the first two tasks compared to SI .”
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“SI [39] followed by EWC [12] has the least forgetting among our methods competitors.”
What the sources say
- Not checked yetcites [39] Zenke, F., Poole, B., Ganguli, S.: Improved multitask learning through synaptic intelligence. In: Proceedings of the International Conference on Machine Learning (ICML) (2017)
- Not checked yetcites [12] Kirkpatrick, J., et al.: Overcoming catastrophic forgetting in neural networks. arXiv preprint arXiv:1612.00796 (2016)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We observe no significant difference on forgetting over 3 random trials where we get a mean, over 6 numbers, of 0 . 51% ± 0 . 18 for the drop on the first task in the vector output case compared to 0 . 50% ± 0 . 19 for the ℓ 2 2 norm case.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Table 3 shows, independent from the set used for computing the importance of the weights, for both l-MAS and MAS the preservation of the previous task and the performance on the current task are quite similar.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“At the end of the sequence, we observe a minimal forgetting on that subset of 2% compared to 8% forgetting on the Flower dataset as a whole.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“for the first time, the ability to adapt the importance of the parameters based on unlabeled data towards what the network needs (not) to forget, which may vary depending on test conditions.”
- Also stated in: § 1 Introduction · p. 2 · plain
“our LLL method can learn what parts of the model are important using unlabelled data . This allows for adaptation to specific test conditions and continuous updating of importance weights.”
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § 1 Introduction · p. 2 · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“MAS , our method, shows a minimal or no forgetting on the different tasks in the sequence with an average forgetting of 0 . 49% .”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“After Finetune that doesn't treat forgetting, our method has the least amount of memory consumption.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“However, looking at existing LLL solutions, we observe that none of them satisfies all the characteristics one would expect or desire from a lifelong learning approach (see Table 1).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As shown in Table 2, FineTune clearly suffers from catastrophic forgetting with a drop in performance from 8% to 13% .”
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- It may state two separate claims as one. 39
- Probably not a substantial claim. 19
None of the sources cited for this claim have been checked yet.
In the paper
“LwF [17] suffers from a buildup of errors when facing a long sequence while EBLL [28] reduces slightly this effect.”
What the sources say
- Not checked yetcites [17] Li, Z., Hoiem, D.: Learning without forgetting. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 614–629. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46493-0_37 source
- Not checked yetcites [28] Rannen, A., Aljundi, R., Blaschko, M.B., Tuytelaars, T.: Encoder based lifelong learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1320–1328 (2017) source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Note that IMM grows linearly in storage, but at inference time it only uses the obtained model.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Overall, MAS achieves a MAP of 0 . 29 compared to 0 . 25 by SI and only 0 . 18 by Finetune .”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We test our method on a sequence of object recognition tasks and on the challenging problem of learning an embedding for predicting < subject, predicate, object > triplets. We show state-of-the-art performance”
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“On average, our method method achieves the lowest forgetting rates (around 1% ) while performance on the new task is almost similar ( 0 -3% lower).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Our method requires a constant amount of memory and enjoys the main desired characteristics of lifelong learning we listed above while achieving stateof-the-art performance.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Further, computing the gradients locally at each layer for l-MAS allows for faster computations but less accurate estimations. As such, l-MAS shows an average forgetting of 3% compared to 1% by MAS .”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“IMM [16] merges the models at the end of the sequence and the drop in performance differs between tasks. More importantly, the method performance on the last task is highly affected by the moment matching.”
What the sources say
- Not checked yetcites [16] Lee, S.W., Kim, J.H., Jun, J., Ha, J.W., Zhang, B.T.: Overcoming catastrophic forgetting by incremental moment matching. In: Advances in Neural Information Processing Systems, pp. 4652–4662 (2017)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Finetuning suffers from a severe forgetting on the previous tasks while being advantageous for the last task, as expected.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our MAS (pink) succeeds to learn that this set is important to preserve and achieves a performance of 0.50 at the end of the sequence, while the performance of finetuning and SI on this set was close to 0.20.”
plainn = 7- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As such, using ℓ 2 2 is n times faster (where n is the length of the output vector) without loss in performance.”
plaindirectn = 2- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“This makes our method not only more versatile, but also simpler, more memory-efficient, and, as it turns out, more effective in learning what not to forget, compared to other model-based LLL approaches.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It is worth noting that our method's absolute performance on average including the last task is 2% better than SI which indicates our method ability to accurately estimate the importance weights and the new tasks to adjust accordingly.”
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We see the sensitivity around λ = 1 is very low with low forgetting, although further improvements could be achieved.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Similar to previous experiments, Finetune is only advantageous on the last task while drastically suffering on the previous tasks.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“All the considered methods manage to reduce the forgetting over fine-tuning significantly while having performance close to fine-tuning on the new task.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“SI [39] and MAS (ours) need the same offline storage requirements (importance weights and model parameters). However, at training phase, SI needs additional memory to accumulate the contributions of each parameter to the change in the loss.”
What the sources say
- Not checked yetcites [39] Zenke, F., Poole, B., Ganguli, S.: Improved multitask learning through synaptic intelligence. In: Proceedings of the International Conference on Machine Learning (ICML) (2017)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“After the third task, as shown in figure F2, the Adaptive projections are closer to the original ones than the Non Adaptive that considers this subset as part of the full task being preserved and tries to prevent forgetting them as well. Finetuning started destroying the learned topology of this subset and lies further apart.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It can be seen that the Adaptive and Non Adaptive variants of our method try to preserve the projections from this subset. The adaptive projections are closer to the original one, if we look closely, while Finetuning projections starts drifting away from where they were.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Finetuning needs no additional memory requirements other than the model and the tasks' heads.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“In spite of the fact that IMM [16] consumes the same amount of memory as Finetuning while training, it requires an offline storage that increases linearly with the number of tasks. That's due to the storage of a Fisher information matrix for each task and previous tasks models.”
What the sources say
- Not checked yetcites [16] Lee, S.W., Kim, J.H., Jun, J., Ha, J.W., Zhang, B.T.: Overcoming catastrophic forgetting by incremental moment matching. In: Advances in Neural Information Processing Systems, pp. 4652–4662 (2017)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“LwF [17], while training a new task, needs to load the last task model and all the previous heads to get the target predictions. However, LwF has the same offline storage requirements between tasks as Finetuning.”
What the sources say
- Not checked yetcites [17] Li, Z., Hoiem, D.: Learning without forgetting. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 614–629. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46493-0_37 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Nevertheless, the Adaptive MAS and Non Adaptive MAS still preserve the topology of the learned projections. The Adaptive projections lie closer and look more similar to the originals than the Non Adaptive MAS. Finetune, on the other hand, forgets completely about this subset and all the samples get projected in one point where it becomes quite hard to recognize their corresponding facts.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Model-based approaches [6,12,16,39], like our method, focus on the parameters of the network instead of depending on the task data.”
What the sources say
- Not checked yetcites [6] Fernando, C., et al.: PathNet: evolution channels gradient descent in super neural networks. arXiv preprint arXiv:1701.08734 (2017)
- Not checked yetcites [12] Kirkpatrick, J., et al.: Overcoming catastrophic forgetting in neural networks. arXiv preprint arXiv:1612.00796 (2016)
- Not checked yetcites [16] Lee, S.W., Kim, J.H., Jun, J., Ha, J.W., Zhang, B.T.: Overcoming catastrophic forgetting by incremental moment matching. In: Advances in Neural Information Processing Systems, pp. 4652–4662 (2017)
- Not checked yetcites [39] Zenke, F., Poole, B., Ganguli, S.: Improved multitask learning through synaptic intelligence. In: Proceedings of the International Conference on Machine Learning (ICML) (2017)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Yet in practice, the biggest challenge is to avoid catastrophic forgetting of the old tasks' knowledge (i.e., forgetting how to perform the old tasks well).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This calls for a LLL method that can learn what (not) to forget using unlabeled test data.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In contrast to the above two methods, we propose to look at the sensitivity of the learned function, rather than the loss. This simplifies the setup considerably since, unlike the loss, the learned function is not in a local minimum, so complications with gradients being close to zero are avoided.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Hence, we only need to compute one backward pass and can use the resulting gradients for estimating the parameters importance.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This is achieved by estimating importance weights for the network parameters without relying on the loss, but by looking at the sensitivity of the output function instead. This way, our method not only avoids the need for labeled data, but importantly it also avoids complications due to the loss being in a local minimum, resulting in gradients being close to zero.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Using our method, for regions in the input space that are sampled densely, the function will be preserved and catastrophic forgetting is avoided. However, parameters not affecting those regions will be given low importance weights, and can be used to optimize the function for other tasks, affecting the function over other regions of the input space.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Yet in our case, these importance weights approximate the sensitivity of the learned function to a parameter change rather than a measure of the (inverse of) parameter uncertainty, as in [12], or the sensitivity of the loss to a parameter change, as in [39]”
What the sources say
- Not checked yetcites [12] Kirkpatrick, J., et al.: Overcoming catastrophic forgetting in neural networks. arXiv preprint arXiv:1612.00796 (2016)
- Not checked yetcites [39] Zenke, F., Poole, B., Ganguli, S.: Improved multitask learning through synaptic intelligence. In: Proceedings of the International Conference on Machine Learning (ICML) (2017)
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Data-based approaches [1,17,28,34] use data from the new task to approximate the performance of the previous tasks. This works best if the data distribution mismatch between tasks is limited.”
What the sources say
- Not checked yetcites [1] Aljundi, R., Chakravarty, P., Tuytelaars, T.: Expert gate: lifelong learning with a network of experts. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017) source
- Not checked yetcites [17] Li, Z., Hoiem, D.: Learning without forgetting. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 614–629. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46493-0_37 source
- Not checked yetcites [28] Rannen, A., Aljundi, R., Blaschko, M.B., Tuytelaars, T.: Encoder based lifelong learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1320–1328 (2017) source
- Not checked yetcites [34] Shmelkov, K., Schmid, C., Alahari, K.: Incremental learning of object detectors without catastrophic forgetting. In: The IEEE International Conference on Computer Vision (ICCV) (2017) source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We can conclude that applying Hebb's rule to measure the importance of the parameters in a neural network can be seen as a local variant of our method that considers only one layer at a time instead of the global function learned by the network.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Further, the method can be added after the network is trained. It can be applied on top of any pretrained network and compute the importance on any set of data without the need to have the labels.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The global version needs to compute the gradients of the output function while the local variant (Hebbian based) can be computed locally by multiplying the input with the output of the connecting neurons.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“When the training procedure converges to a local minimum, the model has learned an approximation F of the true function ¯ F .”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As it does not depend on the ground truth labels, our approach allows computing the importance using any available data (unlabeled) which in turn allows for an adaptation to user-specific settings.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“It focuses on image classification and consists of a sequence of disjoint tasks , which are learned one after the other. Tasks may correspond to different datasets, or different splits of a dataset, without overlap in category labels.”
What the sources say
- Not checked yetcites [1] Aljundi, R., Chakravarty, P., Tuytelaars, T.: Expert gate: lifelong learning with a network of experts. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017) source
- Not checked yetcites [16] Lee, S.W., Kim, J.H., Jun, J., Ha, J.W., Zhang, B.T.: Overcoming catastrophic forgetting by incremental moment matching. In: Advances in Neural Information Processing Systems, pp. 4652–4662 (2017)
- Not checked yetcites [17] Li, Z., Hoiem, D.: Learning without forgetting. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 614–629. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46493-0_37 source
- Not checked yetcites [28] Rannen, A., Aljundi, R., Blaschko, M.B., Tuytelaars, T.: Encoder based lifelong learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1320–1328 (2017) source
- Not checked yetcites [39] Zenke, F., Poole, B., Ganguli, S.: Improved multitask learning through synaptic intelligence. In: Proceedings of the International Conference on Machine Learning (ICML) (2017)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- The authors state this as an assumption: In the standard LLL setup, when training a task, only the data related to that task is accessible.
In the paper
“In the Synaptic Intelligence work [39], importance weights are computed during training in an online manner. To this end, they record how much the loss would change due to a change in a specific parameter and accumulate this information over the training trajectory.”
What the sources say
- Not checked yetcites [39] Zenke, F., Poole, B., Ganguli, S.: Improved multitask learning through synaptic intelligence. In: Proceedings of the International Conference on Machine Learning (ICML) (2017)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Note that SI can only learn importance weights during training, and therefore cannot adapt to a particular subset.”
What the sources say
- Not checked yetcites [39] Zenke, F., Poole, B., Ganguli, S.: Improved multitask learning through synaptic intelligence. In: Proceedings of the International Conference on Machine Learning (ICML) (2017)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Data based approaches are mainly designed for a classification scenario and overall, the need of these approaches to have a preprocessing step before each new task, to record the targets for the previous tasks is an additional limitation.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“According to Hebbian theory [10], the process at the basis of this phenomenon is the strengthening of synapses connecting neurons that fire synchronously, compared to those connecting neurons with unrelated firing behavior.”
What the sources say
- Not checked yetcites [10] Hebb, D.: The organization of behavior 1949. New York Wiely 2, 8 (2002)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our global and local methods both have the advantage of computing the importance of the parameters on any given data point without the need to access the labels or the condition of being computed while training the model.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our proposed method (both the local and global version) resembles an implicit memory included for each parameter of the network. We, therefore, refer to it as Memory Aware Synapses. It keeps updating its value based on the activations of the network when applied to new data points. It can adapt and specialize to a given subset of data points rather than preserving every functionality in the”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Parameters with small importance weights do not affect the output much, and can, therefore, be changed to minimize the loss for subsequent tasks, while parameters with large weights should ideally be left unchanged.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Instead of aiming for no forgetting at all, figuring out what can possibly be forgotten becomes at least as important.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We use a regularization parameter λ of 1; note that no tuning of λ was performed as we assume no access to previous task data.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
- The authors state this as an assumption: No access to previous task data is available.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“This is a far more challenging setup than joint learning, as typically used in the multitask learning literature, where all tasks are trained simultaneously.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“In the Elastic Weight Consolidation work [12], this is done based on an approximation of the diagonal of the Fisher information matrix.”
What the sources say
- Not checked yetcites [12] Kirkpatrick, J., et al.: Overcoming catastrophic forgetting in neural networks. arXiv preprint arXiv:1612.00796 (2016)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, in a true LLL setting with a never-ending list of tasks, the capacity of the model sooner or later reaches its limits and compromises need to be made.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“which is remarkably similar to the Hebbian learning rule as explained in the main paper.”
plaincorrelationaln = 2- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This indicates that our method ( MAS ) will allow changes on most of the parameters that were unused by the first task while penalizing changes to those few crucial parameters that carry meaningful information for the learned task.”
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- The authors state this as an assumption: A good importance measure would give very low importance values to unused parameters and high values for those that are crucial for the task at hand.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This suggests that the method identifies the important parameters needed for each subset and when those parameters are shared the parameters importance is correlated between the two subsets while when those are different, different parameters receive different importance values based on the used subset.”
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The authors state this as an assumption.
No access to previous task data is available.
“note that no tuning of λ was performed as we assume no access to previous task data.”
The authors state this as an assumption.
In the standard LLL setup, when training a task, only the data related to that task is accessible.
“The assumption of this setup is that, when training a task, only the data related to that task is accessible.”
The authors state this as an assumption.
In the object recognition experiments, different classification layers for each task remain unshared.
“Note that this assumes having different classification layers for each task (different 'heads') that remain unshared.”
The paper relies on this without support.
In the absence of error signals, synapses connecting biological neurons strengthen or weaken based on the concurrence of the connected neurons activations.
“In the absence of error signals, synapses connecting biological neurons strengthen or weaken based on the concurrence of the connected neurons activations.”
The authors state this as an assumption.
A good importance measure would give very low importance values to unused parameters and high values for those that are crucial for the task at hand.
“Ideally, a good importance measure would give very low importance values to the unused parameters and high values for those that are crucial for the task at hand.”
The authors state this as an assumption.
A plot where points are closely lying around a straight line indicates that the parameters from the two Ωs have similar importance values.
“A plot where the points are closely lying around a straight line indicates that the parameters from the two Ω s have similar importance values.”
The authors state this as an assumption.
A plot where points are spread further from such a line and scattered among the plotted area indicates a lower correlation between the Ωs.
“A plot where the points are spread further from such a line and scattered among the plotted area indicates a lower correlation between the Ω s.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“However, fixed-representations work well only if the task does not change over time, as it is the case in the evaluated scenarios presented in [2].”
From A Comprehensive Study of Class Incremental Learning Algorithms for Visual Tasks (2020) · cites this paper as [2]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredIn the citing paper
“The MAS method was proposed by Aljundi et al. [49], which was partly similar with SI but it supports the use of unlabeled data to obtain weight sensitivity.”
From An Appraisal of Incremental Learning Methods (2020) · cites this paper as [49]
In the citing paper
“Aljundi et al. [49] compared the classification performance between a multi-head classifier and single-head classifier. It can be seen that a multi-head classifier can achieve higher classification accuracy than a single-head classifier.”
From An Appraisal of Incremental Learning Methods (2020) · cites this paper as [49]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“| Finetune | 33.98 | 22.57 | 21.48 | 21.48 | 13.32 | 26.64 | 36.18 | 90.33 | 32.67 |”
Reported 32.67 · recomputed 33.2475
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ C2 Details on methods' memory requirements estimation · p. 18
“In the main paper, section 5 . 1 Figure 6, we report an approximation to the memory requirements of each method in the 8 tasks object recognition sequence.”
§ Longer Sequence · p. 12
“Fig. 6: avg. performance, left, and avg. forgetting, right, on permuted mnist sequence.”
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Hebb, D. O. (2005) · doi:10.4324/9781410612403 · Cited by 9,825
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
Finn, C., Abbeel, P., & Levine, S. (2017) · arXiv · doi:10.48550/arxiv.1703.03400 · Cited by 8,512
Overcoming catastrophic forgetting in neural networks
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1611835114 · Cited by 7,899
McClelland, J. L., McNaughton, B. L., & O'Reilly, R. C. (1995) · Psychological Review · doi:10.1037/0033-295x.102.3.419 · Cited by 5,256
Reading digits in natural images with unsupervised feature learning
Netzer, Y. (2024) · doi:10.57702/e7wwo0zk · Cited by 4,261
iCaRL: Incremental Classifier and Representation Learning
Rebuffi, S.-A., Kolesnikov, A., Sperl, G., & Lampert, C. H. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.587 · Cited by 3,988
Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem
McCloskey, M., & Cohen, N. J. (1989) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(08)60536-8 · Cited by 3,770
Research works printed but not linked: 5
- Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The PASCAL Visual Object Classes Challenge 2012 (VOC2012) Results (2012). http://www.pascal-network.org/challenges/VOC/voc2012/workshop/index.html
- Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Pereira, F., Burges, C.J.C., Bottou, L., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 25, pp. 1097–1105. Curran Associates, Inc. (2012)
- Pentina, A., Lampert, C.H.: Lifelong learning with non-iid tasks. In: Cortes, C., Lawrence, N.D., Lee, D.D., Sugiyama, M., Garnett, R. (eds.) Advances in Neural Information Processing Systems 28, pp. 1540–1548 (2015)
- Pentina, A., Lampert, C.H.: Lifelong learning with non-iid tasks. In: Advances in Neural Information Processing Systems, pp. 1540–1548 (2015)
- Quadrianto, N., Petterson, J., Smola, A.J.: Distribution matching for transduction. In: Advances in Neural Information Processing Systems, pp. 1500–1508 (2009)
Cited by
Sorted by citations
A Comprehensive Survey of Continual Learning: Theory, Method and Application
Wang, L., Zhang, X., Su, H., & Zhu, J. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3367329 · Cited by 1,113
Learning a Unified Classifier Incrementally via Rebalancing
Hou, S., Pan, X., Loy, C. C., Wang, Z., & Lin, D. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00092 · Cited by 1,195
A Continual Learning Survey: Defying Forgetting in Classification Tasks
Delange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., & Tuytelaars, T. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3057446 · Cited by 1,647
Self-citationLearning to Prompt for Continual Learning
Wang, Z., Zhang, Z., Lee, C., Zhang, H., Sun, R., Ren, X., Su, G., Perot, V., Dy, J., & Pfister, T. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00024 · Cited by 936
Class-Incremental Learning: Survey and Performance Evaluation on Image Classification
Masana, M., Liu, X., Twardowski, B., Menta, M., Bagdanov, A. D., & van de Weijer, J. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3213473 · Cited by 740
PODNet: Pooled Outputs Distillation for Small-Tasks Incremental Learning
Douillard, A., Cord, M., Ollion, C., Robert, T., & Junior, E. A. D. V. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58565-5_6 · Cited by 748
DER: Dynamically Expandable Representation for Class Incremental Learning
Yan, S., Xie, J., & He, X. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00303 · Cited by 803
Towards Open World Object Detection
Joseph, K. J., Khan, S. H., Khan, F. S., & Balasubramanian, V. N. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00577 · Cited by 606
Dhar, P., Singh, R., Peng, K., Wu, Z., & Chellappa, R. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00528 · Cited by 548
Maintaining Discrimination and Fairness in Class Incremental Learning
Zhao, B., Xiao, X., Gan, G., Zhang, B., & Xia, S. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01322 · Cited by 525
DualPrompt: Complementary Prompting for Rehearsal-Free Continual Learning
Wang, Z., Zhang, Z., Ebrahimi, S., Sun, R., Zhang, H., Lee, C.-Y., Ren, X., Su, G., Perot, V., Dy, J., et al. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19809-0_36 · Cited by 597
GDumb: A Simple Approach that Questions Our Progress in Continual Learning
Prabhu, A., Torr, P. H. S., & Dokania, P. K. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58536-5_31 · Cited by 524
Few-Shot Class-Incremental Learning
Tao, X., Hong, X., Chang, X., Dong, S., Wei, X., & Gong, Y. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01220 · Cited by 499
Embracing Change: Continual Learning in Deep Neural Networks
Hadsell, R., Rao, D., Rusu, A. A., & Pascanu, R. (2020) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2020.09.004 · Cited by 467
Online continual learning in image classification: An empirical survey
Mai, Z., Li, R., Jeong, J., Quispe, D., Kim, H., & Sanner, S. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.10.021 · Cited by 468
Prototype Augmentation and Self-Supervision for Incremental Learning
Zhu, F., Zhang, X.-Y., Wang, C., Yin, F., & Liu, C.-L. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00581 · Cited by 427
Class-Incremental Learning: A Survey
Zhou, D.-W., Wang, Q.-W., Qi, Z.-H., Ye, H.-J., Zhan, D.-C., & Liu, Z. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3429383 · Cited by 376
DyTox: Transformers for Continual Learning with DYnamic TOken eXpansion
Douillard, A., Ramé, A., Couairon, G., & Cord, M. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00907 · Cited by 417
Class-incremental Learning via Deep Model Consolidation
Zhang, J., Zhang, J., Ghosh, S., Li, D., Taşcı, Ş., Heck, L. P., Zhang, H., & Kuo, C. ‐. J. (2020) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv45572.2020.9093365 · Cited by 361
Modeling the Background for Incremental Learning in Semantic Segmentation
Cermelli, F., Mancini, M., Bulò, S. R., Ricci, E., & Barbara Caputo, B. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00925 · Cited by 364
CODA-Prompt: COntinual Decomposed Attention-Based Prompting for Rehearsal-Free Continual Learning
Smith, J. S., Karlinsky, L., Gutta, V., Cascante-Bonilla, P., Kim, D., Arbelle, A., Panda, R., Feris, R., & Kira, Z. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01146 · Cited by 409
On Tiny Episodic Memories in Continual LearningPreprint
Chaudhry, A., Rohrbach, M., Elhoseiny, M., Ajanthan, T., Dokania, P. K., Torr, P. H. S., & Ranzato, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.10486 · Cited by 506
Self-citationAljundi, R., Kelchtermans, K., & Tuytelaars, T. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01151 · Cited by 335
Self-citationForward Compatible Few-Shot Class-Incremental Learning
Zhou, D.-W., Wang, F., Ye, H.-J., Ma, L., Pu, S., & Zhan, D. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00884 · Cited by 318
Learning to Remember: A Synaptic Plasticity Driven Framework for Continual Learning
Ostapenko, O., Puscas, M., Klein, T., Jähnichen, P., & Nabi, M. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01158 · Cited by 294
Semantic Drift Compensation for Class-Incremental Learning
Yu, L., Twardowski, B., Liu, X., Herranz, L., Wang, K., Cheng, Y., Jui, S., & van de Weijer, J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00701 · Cited by 340
PLOP: Learning without Forgetting for Continual Semantic Segmentation
Douillard, A., Chen, Y., Dapogny, A., & Cord, M. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00403 · Cited by 290
REMIND Your Neural Network to Prevent Catastrophic Forgetting
Hayes, T. L., Kafle, K., Shrestha, R., Acharya, M., & Kanan, C. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58598-3_28 · Cited by 286
Incremental Learning Techniques for Semantic Segmentation
Michieli, U., & Zanuttigh, P. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00400 · Cited by 274
Alleviating catastrophic forgetting using context-dependent gating and synaptic stabilization
Masse, N. Y., Grant, G. D., & Freedman, D. J. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1803839115 · Cited by 278
Biological underpinnings for lifelong learning machines
Kudithipudi, D., Aguilar-Simon, M., Babb, J., Bazhenov, M., Blackiston, D., Bongard, J., Brna, A. P., Chakravarthi Raja, S., Cheney, N., Clune, J., et al. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00452-0 · Cited by 236
Continuous learning in single-incremental-task scenarios
Maltoni, D., & Lomonaco, V. (2019) · Neural Networks · doi:10.1016/j.neunet.2019.03.010 · Cited by 309
Semi-Supervised and Unsupervised Deep Visual Learning: A Survey
Chen, Y., Mancini, M., Zhu, X., & Akata, Z. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3201576 · Cited by 257
Class-Incremental Learning by Knowledge Distillation with Adaptive Feature Consolidation
Kang, M., Park, J., & Han, B. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01560 · Cited by 220
(2019) · arXiv (Cornell University) · Cited by 214
Self-citationOvercoming Catastrophic Forgetting With Unlabeled Data in the Wild
Lee, K., Lee, K., Shin, J., & Lee, H. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00040 · Cited by 218
Re-evaluating Continual Learning Scenarios: A Categorization and Case for Strong Baselines
Hsu, Y.-C., Liu, Y.-C., Ramasamy, A., & Kira, Z. (2018) · arXiv · doi:10.48550/arxiv.1810.12488 · Cited by 211
Using Hindsight to Anchor Past Knowledge in Continual Learning
Chaudhry, A., Gordo, A., Dokania, P. K., Torr, P. H. S., & López-Paz, D. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i8.16861 · Cited by 208
Online Class-Incremental Continual Learning with Adversarial Shapley Value
Shim, D., Mai, Z., Jeong, J., Sanner, S., Kim, H., & Jang, J. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i11.17159 · Cited by 203
Continual Prototype Evolution: Learning Online from Non-Stationary Data Streams
Lange, M. D., & Tuytelaars, T. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00814 · Cited by 210
Self-citationLifelong Learning via Progressive Distillation and Retrospection
Hou, S., Pan, X., Loy, C. C., Wang, Z., & Lin, D. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01219-9_27 · Cited by 190
Micro-Doppler Based Target Recognition With Radars: A Review
Hanif, A., Muaz, M., Hasan, A., & Adeel, M. (2022) · IEEE Sensors Journal · doi:10.1109/jsen.2022.3141213 · Cited by 201
Mai, Z., Li, R., Kim, H., & Sanner, S. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00398 · Cited by 197
Conditional Channel Gated Networks for Task-Aware Continual Learning
Abati, D., Tomczak, J. M., Blankevoort, T., Calderara, S., Cucchiara, R., & Bejnordi, B. E. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00399 · Cited by 182
Lifelong GAN: Continual Learning for Conditional Image Generation
Zhai, M., Chen, L., Tung, F., He, J., Nawhal, M., & Mori, G. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00285 · Cited by 188
Adversarial Continual Learning
Ebrahimi, S., Meier, F., Calandra, R., Darrell, T., & Rohrbach, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58621-8_23 · Cited by 188
Self-citationOrthogonal Gradient Descent for Continual LearningPreprint
Farajtabar, M., Azizan, N., Mott, A., & Li, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.07104 · Cited by 184
Memory-Efficient Incremental Learning Through Feature Adaptation
İşcen, A., Zhang, J., Lazebnik, S., & Schmid, C. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58517-4_41 · Cited by 176
Always Be Dreaming: A New Approach for Data-Free Class-Incremental Learning
Smith, J., Hsu, Y.-C., Balloch, J., Shen, Y., Jin, H., & Kira, Z. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00924 · Cited by 169
Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgetting
Chen, S., Hou, Y., Cui, Y., Che, W., Liu, T., & Yu, X. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.634 · Cited by 226
Few-Shot Class-Incremental Learning by Sampling Multi-Phase Tasks
Zhou, D.-W., Ye, H.-J., Ma, L., Xie, D., Pu, S., & Zhan, D.-C. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3200865 · Cited by 174
Topology-Preserving Class-Incremental Learning
Tao, X., Chang, X., Hong, X., Wei, X., & Gong, Y. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58529-7_16 · Cited by 162
AutoNovel: Automatically Discovering and Learning Novel Visual Categories
Han, K., Rebuffi, S.-A., Ehrhardt, S., Vedaldi, A., & Zisserman, A. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3091944 · Cited by 257
Self-Supervised Models are Continual Learners
Fini, E., da Costa, V. G. T., Alameda-Pineda, X., Ricci, E., Alahari, K., & Mairal, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00940 · Cited by 155
Lifelong Machine Learning with Deep Streaming Linear Discriminant Analysis
Hayes, T. L., & Kanan, C. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00118 · Cited by 148
iTAML: An Incremental Task-Agnostic Meta-learning Approach
Rajasegaran, J., Khan, S. H., Hayat, M., Khan, F. S., & Shah, M. A. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01360 · Cited by 147
Incremental Learning for Semantic Segmentation of Large-Scale Remote Sensing Data
Taşar, O., Tarabalka, Y., & Alliez, P. (2019) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2019.2925416 · Cited by 144
Interactive Few-Shot Learning: Limited Supervision, Better Medical Image Segmentation
Feng, R., Zheng, X., Gao, T., Chen, J., Wang, W., Chen, D. Z., & Wu, J. (2021) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2021.3060551 · Cited by 141
A Neural Dirichlet Process Mixture Model for Task-Free Continual LearningPreprint
Lee, S., Ha, J., Zhang, D., & Kim, G. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.00689 · Cited by 189
Mundt, M., Hong, Y., Pliushch, I., & Ramesh, V. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.01.014 · Cited by 167
Generative Feature Replay For Class-Incremental Learning
Liu, X., Wu, C., Menta, M., Herranz, L., Raducanu, B., Bagdanov, A. D., Jui, S., & van de Weijer, J. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00121 · Cited by 142
Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapters
Yu, J., Zhuge, Y., Zhang, L., Hu, P., Wang, D., Lu, H., & He, Y. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02191 · Cited by 173
Expandable Subspace Ensemble for Pre-Trained Model-Based Class-Incremental Learning
Zhou, D.-W., Sun, H., Ye, H.-J., & Zhan, D. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02223 · Cited by 148
Mazumder, P., Singh, P. P., & Rai, P. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i3.16334 · Cited by 13
Continual learning for recurrent neural networks: An empirical evaluation
Cossu, A., Carta, A., Lomonaco, V., & Bacciu, D. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.07.021 · Cited by 129
SLCA: Slow Learner with Classifier Alignment for Continual Learning on a Pre-trained Model
Zhang, G., Wang, L., Kang, G., Chen, L., & Wei, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01754 · Cited by 156
Meta-Learning Representations for Continual LearningPreprint
Javed, K., & White, M. (2019) · Neural Information Processing Systems · doi:10.48550/arxiv.1905.12588 · Cited by 397
LAMOL: LAnguage MOdeling for Lifelong Language LearningPreprint
Sun, F.-K., Ho, C.-H., & Lee, H.-Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.03329 · Cited by 201
Overcoming Catastrophic Forgetting in Incremental Object Detection via Elastic Response Distillation
Feng, T., Wang, M., & Yuan, H. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00921 · Cited by 132
An Appraisal of Incremental Learning Methods
Luo, Y., Yin, L., Bai, W., & Mao, K. (2020) · Entropy · doi:10.3390/e22111190 · Cited by 120
Training Networks in Null Space of Feature Covariance for Continual Learning
Wang, S., Li, X., Sun, J., & Xu, Z. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00025 · Cited by 135
Incremental Object Detection via Meta-Learning
Joseph, K. J., Rajasegaran, J., Khan, S. H., Khan, F. S., & Balasubramanian, V. N. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3124133 · Cited by 139
Uncertainty-guided Continual Learning with Bayesian Neural NetworksPreprint
Ebrahimi, S., Elhoseiny, M., Darrell, T., & Rohrbach, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.02425 · Cited by 150
Self-citationMgSvF: Multi-Grained Slow versus Fast Framework for Few-Shot Class-Incremental Learning
Zhao, H., Fu, Y., Kang, M., Tian, Q., Wu, F., & Li, X. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3133897 · Cited by 115
Overcoming Catastrophic Forgetting in Graph Neural Networks
Liu, H., Yang, Y., & Wang, X. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i10.17049 · Cited by 127
Streaming Graph Neural Networks via Continual Learning
Wang, J., Song, G., Wu, Y., & Wang, L. (2020) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3340531.3411963 · Cited by 118
Representation-Enhanced Status Replay Network for Multisource Remote-Sensing Image Classification
Wang, J., Li, W., Wang, Y., Tao, R., & Du, Q. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3286422 · Cited by 111
Dynamic Memory to Alleviate Catastrophic Forgetting in Continual Learning with Medical Imaging
Perkonigg, M., Hofmanninger, J., Herold, C. J., Brink, J. A., Pianykh, O., Prosch, H., & Langs, G. (2021) · Nature Communications · doi:10.1038/s41467-021-25858-z · Cited by 106
Replay in Deep Learning: Current Approaches and Missing Biological Elements
Hayes, T. L., Krishnan, G. P., Bazhenov, M., Siegelmann, H. T., Sejnowski, T. J., & Kanan, C. (2021) · Neural Computation · doi:10.1162/neco_a_01433 · Cited by 106
Continual Learning for Blind Image Quality Assessment
Zhang, W., Li, D., Ma, C., Zhai, G., Yang, X., & Ma, K. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3178874 · Cited by 112
Continual Relation Learning via Episodic Memory Activation and Reconsolidation
Han, X., Dai, Y., Gao, T., Lin, Y., Liu, Z., Li, P., Sun, M., & Zhou, J. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.573 · Cited by 104
A Unified Continual Learning Framework with General Parameter-Efficient Tuning
Gao, Q., Zhao, C., Sun, Y., Xi, T., Zhang, G., Ghanem, B., & Zhang, J. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01055 · Cited by 104
Automatically Discovering and Learning New Visual Categories with Ranking StatisticsPreprint
Han, K., Rebuffi, S.-A., Ehrhardt, S., Vedaldi, A., & Zisserman, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.05714 · Cited by 103
Preventing Zero-Shot Transfer Degradation in Continual Learning of Vision-Language Models
Zheng, Z., Ma, M., Wang, K., Qin, Z., Yue, X., & You, Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01752 · Cited by 122
Li, D., Tasci, S., Ghosh, S., Zhu, J., Zhang, J., & Heck, L. (2019) · Proceedings of the 4th ACM/IEEE Symposium on Edge Computing · doi:10.1145/3318216.3363317 · Cited by 94
Few-Shot Class-Incremental Learning for Medical Time Series Classification
Sun, L., Zhang, M., Wang, B., & Tiwari, P. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2023.3247861 · Cited by 98
Online Continual Learning with Maximally Interfered Retrieval
Aljundi, R., Caccia, L., Belilovsky, E., Caccia, M., Lin, M., Charlin, L., & Tuytelaars, T. (2019) · arXiv · doi:10.48550/arxiv.1908.04742 · Cited by 140
Self-citationKnowledge distillation for incremental learning in semantic segmentation
Michieli, U., & Zanuttigh, P. (2021) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2021.103167 · Cited by 100
More Classifiers, Less Forgetting: A Generic Multi-classifier Paradigm for Incremental Learning
Liu, Y., Parisot, S., Slabaugh, G., Jia, X., Leonardis, A., & Tuytelaars, T. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58574-7_42 · Cited by 96
Self-citationIncorporating neuro-inspired adaptability for continual learning in artificial intelligence
Wang, L., Zhang, X., Li, Q., Zhang, M., Su, H., Zhu, J., & Zhong, Y. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00747-w · Cited by 91
Grid-Based Crime Prediction Using Geographical Features
Lin, Y.-L., Yen, M.-F., & Yu, L.-C. (2018) · ISPRS International Journal of Geo-Information · doi:10.3390/ijgi7080298 · Cited by 90
A Comprehensive Survey of Forgetting in Deep Learning Beyond Continual Learning
Wang, Z., Yang, E., Shen, L., & Huang, H. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3498346 · Cited by 132
IncDet: In Defense of Elastic Weight Consolidation for Incremental Object Detection
Liu, L., Kuang, Z., Chen, Y., Xue, J.-H., Yang, W., & Zhang, W. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3002583 · Cited by 90
Continual Local Training For Better Initialization Of Federated Models
Yao, X., & Sun, L. (2020) · Proceedings - International Conference on Image Processing · doi:10.1109/icip40778.2020.9190968 · Cited by 91
Continual Learning of Large Language Models: A Comprehensive Survey
Shi, H., Xu, Z., Wang, H., Qin, W., Wang, W., Wang, Y., Wang, Z., Ebrahimi, S., & Wang, H. (2025) · ACM Computing Surveys · doi:10.1145/3735633 · Cited by 291
SHINE-Mapping: Large-Scale 3D Mapping Using Sparse Hierarchical Implicit Neural Representations
Zhong, X., Pan, Y., Behley, J., & Stachniss, C. (2023) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra48891.2023.10160907 · Cited by 86
Synaptic metaplasticity in binarized neural networks
Laborieux, A., Ernoult, M., Hirtzlin, T., & Querlioz, D. (2021) · Nature Communications · doi:10.1038/s41467-021-22768-y · Cited by 103
Selfless Sequential LearningPreprint
Aljundi, R., Rohrbach, M., & Tuytelaars, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.05421 · Cited by 114
Self-citationCo-Transport for Class-Incremental Learning
Zhou, D.-W., Ye, H.-J., & Zhan, D. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475306 · Cited by 83
A Few-Shot Class-Incremental Learning Method for Network Intrusion Detection
Du, L., Gu, Z., Wang, Y., Wang, L., & Jia, Y. (2023) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2023.3332284 · Cited by 83
Metadata
- Authors
- Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., & Tuytelaars, T.
- Affiliations
- IMEC (BE)
- Meta (United States) (US)
- Corresponding author
- Aljundi, Rahaf
- Venue
- Lecture Notes in Computer Science
- Publisher
- Springer International Publishing
- Published
- 2018
- Pages
- 144-161
- ISSN
- 0302-9743 (ISSN-L)
- 1611-3349
- OpenAlex
- W2768412495
- Semantic Scholar
- 713b0d9005944f80af00addc81b162ca74ea4b14
- Cited by
- 1,590
- References
- 37
- Citations by year
- 2026: 166 · 2025: 281 · 2024: 288 · 2023: 295 · 2022: 184 · 2021: 203 · 2020: 105 · 2019: 52 · 2018: 15
- FWCI
- 38.29
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Lecture notes in computer science (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Lirias (submittedVersion, other-oa)
- arXiv (Cornell University)
- Funders
- Fonds Wetenschappelijk Onderzoek
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- catastrophic forgetting, lifelong learning, parameter importance, neuroplasticity, Hebbian learning, online learning, object recognition, knowledge preservation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-08, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-08 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
