Connectionist models of recognition memory: Constraints imposed by learning and forgetting functions.

Abstract

Multilayer connectionist models of memory based on the encoder model using the backpropagation learning rule are evaluated. The models are applied to standard recognition memory procedures in which items are studied sequentially and then tested for retention. Sequential learning in these models leads to 2 major problems. First, well-learned information is forgotten rapidly as new information is learned. Second, discrimination between studied items and new items either decreases or is nonmonotonic as a function of learning. To address these problems, manipulations of the network within the multilayer model and several variants of the multilayer model were examined, including a model with prelearned memory and a context model, but none solved the problems. The problems discussed provide limitations on connectionist models applied to human memory and in tasks where information to be learned is not all available during learning.

Cites (6)

1. Human Memory: A Proposed System and its Control Processes
   Atkinson, R.C., Shiffrin, R.M. · 1968 · Psychology of Learning and Motivation
2. Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem
   McCloskey, Michael, Cohen, Neal J. · 1989 · Psychology of Learning and Motivation
3. A learning algorithm for Boltzmann machines
   Ackley, David H., Hinton, Geoffrey E., Sejnowski, Terrence J. · 1985 · Cognitive Science
4. A massively parallel architecture for a self-organizing neural pattern recognition machine
   Carpenter, Gail A., Grossberg, Stephen · 1987 · Computer Vision, Graphics, and Image Processing
5. Recognition and retrieval processes in free recall.
   Anderson, John R., Bower, Gordon H. · 1972 · Psychological Review
6. A retrieval theory of priming in memory.
   Ratcliff, Roger, McKoon, Gail · 1988 · Psychological Review

Show all 6 references

Cited by (53)

1. Overcoming catastrophic forgetting in neural networks
   Kirkpatrick, James, Pascanu, Razvan, Rabinowitz, Neil, Veness, Joel, Desjardins, Guillaume, Rusu, Andrei A., Milan, Kieran, Quan, John, Ramalho, Tiago, Grabska-Barwinska, Agnieszka et al. · 2017 · Proceedings of the National Academy of Sciences
2. Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory
   McClelland, James L., McNaughton, Bruce L., O'Reilly, Randall C. · 1995 · Psychological Review
3. Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem
   McCloskey, Michael, Cohen, Neal J. · 1989 · Psychology of Learning and Motivation
4. Continual lifelong learning with neural networks: A review
   Parisi, German I., Kemker, Ronald, Part, Jose L., Kanan, Christopher, Wermter, Stefan · 2019 · Neural Networks
5. On the Opportunities and Risks of Foundation Models
   Bommasani, Rishi, Hudson, Drew A., Adeli, Ehsan, Altman, Russ, Arora, Simran, von Arx, Sydney, Bernstein, Michael S., Bohg, Jeannette, Bosselut, Antoine, Brunskill, Emma et al. · 2021 · arXiv

See all 885