Memory Aware Synapses: Learning What (not) to Forget

May combine two claims.

LwF suffers from a buildup of errors when facing a long sequence while EBLL reduces slightly this effect.

“LwF [17] suffers from a buildup of errors when facing a long sequence while EBLL [28] reduces slightly this effect.”

• cites [17] Li, Z., Hoiem, D.: Learning without forgetting. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 614–629. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46493-0_37 sourceNot checked yet

  Not checked in the free run.

• cites [28] Rannen, A., Aljundi, R., Blaschko, M.B., Tuytelaars, T.: Encoder based lifelong learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1320–1328 (2017) sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

IMM grows linearly in storage, but at inference time it only uses the obtained model.

“Note that IMM grows linearly in storage, but at inference time it only uses the obtained model.”

May combine two claims.

In the 4-task fact learning experiments, overall MAS achieves a MAP of 0.29 compared to 0.25 by SI and 0.18 by Finetune.

“Overall, MAS achieves a MAP of 0 . 29 compared to 0 . 25 by SI and only 0 . 18 by Finetune .”

May combine two claims.

MAS achieves state-of-the-art performance on a sequence of object recognition tasks and on learning an embedding for predicting <subject, predicate, object> triplets.

“We test our method on a sequence of object recognition tasks and on the challenging problem of learning an embedding for predicting < subject, predicate, object > triplets. We show state-of-the-art performance”

May combine two claims.

On average, MAS achieves the lowest forgetting rates (around 1%) while performance on the new task is almost similar (0-3% lower) in two-task experiments.

“On average, our method method achieves the lowest forgetting rates (around 1% ) while performance on the new task is almost similar ( 0 -3% lower).”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Our method requires a constant amount of memory and enjoys the main desired characteristics of lifelong learning while achieving state-of-the-art performance.

“Our method requires a constant amount of memory and enjoys the main desired characteristics of lifelong learning we listed above while achieving stateof-the-art performance.”

May combine two claims.

Computing the gradients locally at each layer for l-MAS allows for faster computations but less accurate estimations. As such, l-MAS shows an average forgetting of 3% compared to 1% by MAS.

“Further, computing the gradients locally at each layer for l-MAS allows for faster computations but less accurate estimations. As such, l-MAS shows an average forgetting of 3% compared to 1% by MAS .”

May combine two claims.

IMM merges the models at the end of the sequence and the drop in performance differs between tasks; the method performance on the last task is highly affected by the moment matching.

“IMM [16] merges the models at the end of the sequence and the drop in performance differs between tasks. More importantly, the method performance on the last task is highly affected by the moment matching.”

• cites [16] Lee, S.W., Kim, J.H., Jun, J., Ha, J.W., Zhang, B.T.: Overcoming catastrophic forgetting by incremental moment matching. In: Advances in Neural Information Processing Systems, pp. 4652–4662 (2017)Not checked yet

  Not checked in the free run.

May combine two claims.

In the 8-task sequence, Finetuning suffers from severe forgetting on previous tasks while being advantageous for the last task.

“Finetuning suffers from a severe forgetting on the previous tasks while being advantageous for the last task, as expected.”

May combine two claims.

In the sports subset adaptation test, MAS achieves a performance of 0.50 at the end of the 4-task sequence, while finetuning and SI performance on this set was close to 0.20.

“Our MAS (pink) succeeds to learn that this set is important to preserve and achieves a performance of 0.50 at the end of the sequence, while the performance of finetuning and SI on this set was close to 0.20.”

May combine two claims.

Using l2^2 is n times faster (where n is the length of the output vector) without loss in performance.

“As such, using ℓ 2 2 is n times faster (where n is the length of the output vector) without loss in performance.”

May combine two claims.

MAS is more versatile, simpler, more memory-efficient, and more effective in learning what not to forget, compared to other model-based LLL approaches.

“This makes our method not only more versatile, but also simpler, more memory-efficient, and, as it turns out, more effective in learning what not to forget, compared to other model-based LLL approaches.”

May combine two claims.

MAS's absolute performance on average including the last task is 2% better than SI, which indicates MAS's ability to accurately estimate the importance weights and the new tasks to adjust accordingly.

“It is worth noting that our method's absolute performance on average including the last task is 2% better than SI which indicates our method ability to accurately estimate the importance weights and the new tasks to adjust accordingly.”

May combine two claims.

For MAS on a sequence of 5 permuted MNIST tasks with a 2-layer perceptron (512 units), the sensitivity of performance and forgetting to λ around λ=1 is very low, with low forgetting.

“We see the sensitivity around λ = 1 is very low with low forgetting, although further improvements could be achieved.”

May combine two claims.

In the 4-task fact learning experiments, Finetune is only advantageous on the last task while drastically suffering on previous tasks.

“Similar to previous experiments, Finetune is only advantageous on the last task while drastically suffering on the previous tasks.”

May combine two claims.

All considered methods manage to reduce the forgetting over fine-tuning significantly while having performance close to fine-tuning on the new task.

“All the considered methods manage to reduce the forgetting over fine-tuning significantly while having performance close to fine-tuning on the new task.”

May combine two claims.

SI and MAS need the same offline storage requirements (importance weights and model parameters), but at training phase SI needs additional memory to accumulate the contributions of each parameter to the change in the loss.

“SI [39] and MAS (ours) need the same offline storage requirements (importance weights and model parameters). However, at training phase, SI needs additional memory to accumulate the contributions of each parameter to the change in the loss.”

• cites [39] Zenke, F., Poole, B., Ganguli, S.: Improved multitask learning through synaptic intelligence. In: Proceedings of the International Conference on Machine Learning (ICML) (2017)Not checked yet

  Not checked in the free run.

May combine two claims.

After the third task, Adaptive MAS projections are closer to the original than Non Adaptive MAS, and Finetuning started destroying the learned topology of the subset.

“After the third task, as shown in figure F2, the Adaptive projections are closer to the original ones than the Non Adaptive that considers this subset as part of the full task being preserved and tries to prevent forgetting them as well. Finetuning started destroying the learned topology of this subset and lies further apart.”

May combine two claims.

In the embedding visualization, Adaptive and Non Adaptive MAS variants try to preserve the projections from the sport subset, with Adaptive projections closer to the original than Non Adaptive, while Finetuning projections drift away.

“It can be seen that the Adaptive and Non Adaptive variants of our method try to preserve the projections from this subset. The adaptive projections are closer to the original one, if we look closely, while Finetuning projections starts drifting away from where they were.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Finetuning needs no additional memory requirements other than the model and the tasks' heads.

“Finetuning needs no additional memory requirements other than the model and the tasks' heads.”

May combine two claims.

IMM consumes the same amount of memory as Finetuning while training, but requires offline storage that increases linearly with the number of tasks due to storing a Fisher information matrix for each task and previous task models.

“In spite of the fact that IMM [16] consumes the same amount of memory as Finetuning while training, it requires an offline storage that increases linearly with the number of tasks. That's due to the storage of a Fisher information matrix for each task and previous tasks models.”

• cites [16] Lee, S.W., Kim, J.H., Jun, J., Ha, J.W., Zhang, B.T.: Overcoming catastrophic forgetting by incremental moment matching. In: Advances in Neural Information Processing Systems, pp. 4652–4662 (2017)Not checked yet

  Not checked in the free run.

May combine two claims.

LwF, while training a new task, needs to load the last task model and all previous heads to get target predictions, but has the same offline storage requirements between tasks as Finetuning.

“LwF [17], while training a new task, needs to load the last task model and all the previous heads to get the target predictions. However, LwF has the same offline storage requirements between tasks as Finetuning.”

• cites [17] Li, Z., Hoiem, D.: Learning without forgetting. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 614–629. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46493-0_37 sourceNot checked yet

  Not checked in the free run.

May combine two claims.

After the fourth task, Adaptive MAS and Non Adaptive MAS still preserve the topology of the learned projections, with Adaptive closer and more similar to originals than Non Adaptive, while Finetune forgets completely and projects all samples to one point.

“Nevertheless, the Adaptive MAS and Non Adaptive MAS still preserve the topology of the learned projections. The Adaptive projections lie closer and look more similar to the originals than the Non Adaptive MAS. Finetune, on the other hand, forgets completely about this subset and all the samples get projected in one point where it becomes quite hard to recognize their corresponding facts.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Model-based approaches focus on the parameters of the network instead of depending on the task data.

“Model-based approaches [6,12,16,39], like our method, focus on the parameters of the network instead of depending on the task data.”

• cites [6] Fernando, C., et al.: PathNet: evolution channels gradient descent in super neural networks. arXiv preprint arXiv:1701.08734 (2017)Not checked yet

  Not checked in the free run.

• cites [12] Kirkpatrick, J., et al.: Overcoming catastrophic forgetting in neural networks. arXiv preprint arXiv:1612.00796 (2016)Not checked yet

  Not checked in the free run.

• cites [16] Lee, S.W., Kim, J.H., Jun, J., Ha, J.W., Zhang, B.T.: Overcoming catastrophic forgetting by incremental moment matching. In: Advances in Neural Information Processing Systems, pp. 4652–4662 (2017)Not checked yet

  Not checked in the free run.

• cites [39] Zenke, F., Poole, B., Ganguli, S.: Improved multitask learning through synaptic intelligence. In: Proceedings of the International Conference on Machine Learning (ICML) (2017)Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S4 (based).

In practice, the biggest challenge in LLL is to avoid catastrophic forgetting of the old tasks' knowledge.

“Yet in practice, the biggest challenge is to avoid catastrophic forgetting of the old tasks' knowledge (i.e., forgetting how to perform the old tasks well).”

Probably not a substantial claim: may not pass S2 (checkable).

This calls for a LLL method that can learn what (not) to forget using unlabeled test data.

“This calls for a LLL method that can learn what (not) to forget using unlabeled test data.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

We propose to look at the sensitivity of the learned function, rather than the loss, which simplifies the setup since the learned function is not in a local minimum, so complications with gradients being close to zero are avoided.

“In contrast to the above two methods, we propose to look at the sensitivity of the learned function, rather than the loss. This simplifies the setup considerably since, unlike the loss, the learned function is not in a local minimum, so complications with gradients being close to zero are avoided.”

May combine two claims.

Using the gradients of the squared l2 norm of the learned function output instead of the vector output requires only one backward pass and can be used for estimating parameter importance.

“Hence, we only need to compute one backward pass and can use the resulting gradients for estimating the parameters importance.”

May combine two claims.

Estimating importance weights without relying on the loss, but by looking at the sensitivity of the output function, avoids the need for labeled data and avoids complications due to the loss being in a local minimum, resulting in gradients close to zero.

“This is achieved by estimating importance weights for the network parameters without relying on the loss, but by looking at the sensitivity of the output function instead. This way, our method not only avoids the need for labeled data, but importantly it also avoids complications due to the loss being in a local minimum, resulting in gradients being close to zero.”

May combine two claims.

Using our method, for regions in the input space that are sampled densely, the function will be preserved and catastrophic forgetting is avoided. However, parameters not affecting those regions will be given low importance weights, and can be used to optimize the function for other tasks, affecting the function over other regions of the input space.

“Using our method, for regions in the input space that are sampled densely, the function will be preserved and catastrophic forgetting is avoided. However, parameters not affecting those regions will be given low importance weights, and can be used to optimize the function for other tasks, affecting the function over other regions of the input space.”

May combine two claims.

Our importance weights approximate the sensitivity of the learned function to a parameter change rather than a measure of the (inverse of) parameter uncertainty, as in EWC, or the sensitivity of the loss to a parameter change, as in SI.

“Yet in our case, these importance weights approximate the sensitivity of the learned function to a parameter change rather than a measure of the (inverse of) parameter uncertainty, as in [12], or the sensitivity of the loss to a parameter change, as in [39]”

• cites [12] Kirkpatrick, J., et al.: Overcoming catastrophic forgetting in neural networks. arXiv preprint arXiv:1612.00796 (2016)Not checked yet

  Not checked in the free run.

• cites [39] Zenke, F., Poole, B., Ganguli, S.: Improved multitask learning through synaptic intelligence. In: Proceedings of the International Conference on Machine Learning (ICML) (2017)Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Data-based approaches use data from the new task to approximate the performance of the previous tasks, and this works best if the data distribution mismatch between tasks is limited.

“Data-based approaches [1,17,28,34] use data from the new task to approximate the performance of the previous tasks. This works best if the data distribution mismatch between tasks is limited.”

• cites [1] Aljundi, R., Chakravarty, P., Tuytelaars, T.: Expert gate: lifelong learning with a network of experts. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017) sourceNot checked yet

  Not checked in the free run.

• cites [17] Li, Z., Hoiem, D.: Learning without forgetting. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 614–629. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46493-0_37 sourceNot checked yet

  Not checked in the free run.

• cites [28] Rannen, A., Aljundi, R., Blaschko, M.B., Tuytelaars, T.: Encoder based lifelong learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1320–1328 (2017) sourceNot checked yet

  Not checked in the free run.

• cites [34] Shmelkov, K., Schmid, C., Alahari, K.: Incremental learning of object detectors without catastrophic forgetting. In: The IEEE International Conference on Computer Vision (ICCV) (2017) sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Applying Hebb's rule to measure the importance of the parameters in a neural network can be seen as a local variant of our method that considers only one layer at a time instead of the global function learned by the network.

“We can conclude that applying Hebb's rule to measure the importance of the parameters in a neural network can be seen as a local variant of our method that considers only one layer at a time instead of the global function learned by the network.”

May combine two claims.

MAS can be added after the network is trained, applied on top of any pretrained network, and compute the importance on any set of data without the need to have the labels.

“Further, the method can be added after the network is trained. It can be applied on top of any pretrained network and compute the importance on any set of data without the need to have the labels.”

May combine two claims.

The global version needs to compute the gradients of the output function while the local variant (Hebbian based) can be computed locally by multiplying the input with the output of the connecting neurons.

“The global version needs to compute the gradients of the output function while the local variant (Hebbian based) can be computed locally by multiplying the input with the output of the connecting neurons.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

When the training procedure converges to a local minimum, the model has learned an approximation F of the true function.

“When the training procedure converges to a local minimum, the model has learned an approximation F of the true function ¯ F .”

May combine two claims.

As it does not depend on the ground truth labels, our approach allows computing the importance using any available data (unlabeled) which in turn allows for an adaptation to user-specific settings.

“As it does not depend on the ground truth labels, our approach allows computing the importance using any available data (unlabeled) which in turn allows for an adaptation to user-specific settings.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The standard LLL setup consists of a sequence of disjoint tasks learned one after the other, where tasks may correspond to different datasets or different splits of a dataset without overlap in category labels.

“It focuses on image classification and consists of a sequence of disjoint tasks , which are learned one after the other. Tasks may correspond to different datasets, or different splits of a dataset, without overlap in category labels.”

• cites [1] Aljundi, R., Chakravarty, P., Tuytelaars, T.: Expert gate: lifelong learning with a network of experts. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017) sourceNot checked yet

  Not checked in the free run.

• cites [16] Lee, S.W., Kim, J.H., Jun, J., Ha, J.W., Zhang, B.T.: Overcoming catastrophic forgetting by incremental moment matching. In: Advances in Neural Information Processing Systems, pp. 4652–4662 (2017)Not checked yet

  Not checked in the free run.

• cites [17] Li, Z., Hoiem, D.: Learning without forgetting. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 614–629. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46493-0_37 sourceNot checked yet

  Not checked in the free run.

• cites [28] Rannen, A., Aljundi, R., Blaschko, M.B., Tuytelaars, T.: Encoder based lifelong learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1320–1328 (2017) sourceNot checked yet

  Not checked in the free run.

• cites [39] Zenke, F., Poole, B., Ganguli, S.: Improved multitask learning through synaptic intelligence. In: Proceedings of the International Conference on Machine Learning (ICML) (2017)Not checked yet

  Not checked in the free run.

• AssumptionThe authors state this as an assumption: In the standard LLL setup, when training a task, only the data related to that task is accessible.

Probably not a substantial claim: may not pass S3 (load-bearing).

In SI, importance weights are computed during training in an online manner by recording how much the loss would change due to a change in a specific parameter and accumulating this information over the training trajectory.

“In the Synaptic Intelligence work [39], importance weights are computed during training in an online manner. To this end, they record how much the loss would change due to a change in a specific parameter and accumulate this information over the training trajectory.”

• cites [39] Zenke, F., Poole, B., Ganguli, S.: Improved multitask learning through synaptic intelligence. In: Proceedings of the International Conference on Machine Learning (ICML) (2017)Not checked yet

  Not checked in the free run.

May combine two claims.

SI can only learn importance weights during training, and therefore cannot adapt to a particular subset.

“Note that SI can only learn importance weights during training, and therefore cannot adapt to a particular subset.”

• cites [39] Zenke, F., Poole, B., Ganguli, S.: Improved multitask learning through synaptic intelligence. In: Proceedings of the International Conference on Machine Learning (ICML) (2017)Not checked yet

  Not checked in the free run.

May combine two claims.

Data-based approaches are mainly designed for a classification scenario and require a preprocessing step before each new task to record targets for previous tasks, which is an additional limitation.

“Data based approaches are mainly designed for a classification scenario and overall, the need of these approaches to have a preprocessing step before each new task, to record the targets for the previous tasks is an additional limitation.”

Probably not a substantial claim: may not pass S3 (load-bearing).

According to Hebbian theory, the process at the basis of synaptic plasticity is the strengthening of synapses connecting neurons that fire synchronously, compared to those connecting neurons with unrelated firing behavior.

“According to Hebbian theory [10], the process at the basis of this phenomenon is the strengthening of synapses connecting neurons that fire synchronously, compared to those connecting neurons with unrelated firing behavior.”

• cites [10] Hebb, D.: The organization of behavior 1949. New York Wiely 2, 8 (2002)Not checked yet

  Not checked in the free run.

May combine two claims.

Our global and local methods both have the advantage of computing the importance of the parameters on any given data point without the need to access the labels or the condition of being computed while training the model.

“Our global and local methods both have the advantage of computing the importance of the parameters on any given data point without the need to access the labels or the condition of being computed while training the model.”

May combine two claims.

MAS resembles an implicit memory included for each parameter of the network, keeps updating its value based on the activations of the network when applied to new data points, and can adapt and specialize to a given subset of data points rather than preserving every functionality in the network.

“Our proposed method (both the local and global version) resembles an implicit memory included for each parameter of the network. We, therefore, refer to it as Memory Aware Synapses. It keeps updating its value based on the activations of the network when applied to new data points. It can adapt and specialize to a given subset of data points rather than preserving every functionality in the”

May combine two claims.

Parameters with small importance weights do not affect the output much, and can therefore be changed to minimize the loss for subsequent tasks, while parameters with large weights should ideally be left unchanged.

“Parameters with small importance weights do not affect the output much, and can, therefore, be changed to minimize the loss for subsequent tasks, while parameters with large weights should ideally be left unchanged.”

Probably not a substantial claim: may not pass S2 (checkable).

Instead of aiming for no forgetting at all, figuring out what can possibly be forgotten becomes at least as important.

“Instead of aiming for no forgetting at all, figuring out what can possibly be forgotten becomes at least as important.”

Probably not a substantial claim: may not pass S3 (load-bearing).

No tuning of λ was performed as we assume no access to previous task data.

“We use a regularization parameter λ of 1; note that no tuning of λ was performed as we assume no access to previous task data.”

• AssumptionThe authors state this as an assumption: No access to previous task data is available.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

This is a far more challenging setup than joint learning, as typically used in the multitask learning literature, where all tasks are trained simultaneously.

“This is a far more challenging setup than joint learning, as typically used in the multitask learning literature, where all tasks are trained simultaneously.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In EWC, importance weights are computed based on an approximation of the diagonal of the Fisher information matrix.

“In the Elastic Weight Consolidation work [12], this is done based on an approximation of the diagonal of the Fisher information matrix.”

• cites [12] Kirkpatrick, J., et al.: Overcoming catastrophic forgetting in neural networks. arXiv preprint arXiv:1612.00796 (2016)Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

In a true LLL setting with a never-ending list of tasks, the capacity of the model sooner or later reaches its limits and compromises need to be made.

“However, in a true LLL setting with a never-ending list of tasks, the capacity of the model sooner or later reaches its limits and compromises need to be made.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The derivation for l-MAS yields an expression remarkably similar to the Hebbian learning rule.

“which is remarkably similar to the Hebbian learning rule as explained in the main paper.”

May combine two claims.

MAS will allow changes on most parameters unused by the first task while penalizing changes to few crucial parameters that carry meaningful information for the learned task.

“This indicates that our method ( MAS ) will allow changes on most of the parameters that were unused by the first task while penalizing changes to those few crucial parameters that carry meaningful information for the learned task.”

• AssumptionThe authors state this as an assumption: A good importance measure would give very low importance values to unused parameters and high values for those that are crucial for the task at hand.

May combine two claims.

The method identifies important parameters needed for each subset; when those parameters are shared, importance is correlated between subsets, and when different, different parameters receive different importance values based on the used subset.

“This suggests that the method identifies the important parameters needed for each subset and when those parameters are shared the parameters importance is correlated between the two subsets while when those are different, different parameters receive different importance values based on the used subset.”