Parallel Distributed Processing Theory in the Age of Deep Networks
Open accessThe DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
Parallel distributed processing (PDP) models in psychology are the precursors of deep networks used in computer science. However, only PDP models are associated with two core psychological claims, namely that all knowledge is coded in a distributed format and cognition is mediated by non-symbolic computations. These claims have long been debated in cognitive science, and recent work with deep networks speaks to this debate. Specifically, single-unit recordings show that deep networks learn units that respond selectively to meaningful categories, and researchers are finding that deep networks need to be supplemented with symbolic systems to perform some tasks. Given the close links between PDP and deep networks, it is surprising that research with deep networks is challenging PDP theory.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
DRC: A dual route cascaded model of visual word recognition and reading aloud.
Coltheart, M., Rastle, K., Perry, C., Langdon, R., & Ziegler, J. (2001) · Psychological Review · doi:10.1037/0033-295x.108.1.204 · Cited by 4,500 · cited 1×
“This was taken to undermine the dual-route model of naming that includes a symbolic grapheme–phoneme conversion system for the sake of generalizing to novel words (as well as localist codes for the sake of naming irregular words) [43].”
BackgroundConnectionism and cognitive architecture: A critical analysis
Fodor, J. A., & Pylyshyn, Z. W. (1988) · Cognition · doi:10.1016/0010-0277(88)90031-5 · Cited by 4,022
Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem
McCloskey, M., & Cohen, N. J. (1989) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(08)60536-8 · Cited by 3,770
McClelland, J. L., Rumelhart, D. E., & AU (1987) · doi:10.7551/mitpress/5237.001.0001 · Cited by 3,315
Human-level concept learning through probabilistic program induction
Lake, B. M., Salakhutdinov, R., & Tenenbaum, J. B. (2015) · Science · doi:10.1126/science.aab3050 · Cited by 2,817
Understanding normal and impaired word reading: Computational principles in quasi-regular domains.
Plaut, D. C., McClelland, J. L., Seidenberg, M. S., & Patterson, K. (1996) · Psychological Review · doi:10.1037/0033-295x.103.1.56 · Cited by 2,698 · cited 1×
“Perhaps the best-known example of PDP models generalizing comes from models of single-word reading that not only successfully name trained words, but also many novel words (e.g., [39, 40, 41]; but see [42]).”
BackgroundDeep Networks with Stochastic Depth
Huang, G., Sun, Y., Liu, Z., Sedra, D., & Weinberger, K. Q. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_39 · Cited by 2,104
Hybrid computing using a neural network with dynamic external memory
Graves, A., Wayne, G., Reynolds, M., Harley, T., Danihelka, I., Grabska-Barwińska, A., Colmenarejo, S. G., Grefenstette, E., Ramalho, T., Agapiou, J., et al. (2016) · Nature · doi:10.1038/nature20101 · Cited by 1,412
Cited by
Theories of Error Back-Propagation in the Brain
Whittington, J. C. R., & Bogacz, R. (2019) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2018.12.005 · Cited by 420
“Furthermore, thanks to current computational advances, arti fi cial neural networks can now provide useful insights on how complex cognitive functions are achieved in the brain [8].”
BackgroundIf deep learning is the answer, what is the question?
Saxe, A., Nelli, S., & Summerfield, C. (2021) · Nature Reviews Neuroscience · doi:10.1038/s41583-020-00395-8 · Cited by 397
“Contemporary deep networks resemble scaled-up connectionist models, and recent advances in machine learning are also heavily indebted to the ubiquity of digital data and the relatively low cost of computation in the twenty-first century 26 .”
BackgroundDeep problems with neural network models of human vision
Bowers, J. S., Malhotra, G., Dujmović, M., Llera Montero, M., Tsvetkov, C., Biscione, V., Puebla, G., Adolfi, F., Hummel, J. E., Heaton, R. F., et al. (2023) · Behavioral and Brain Sciences · doi:10.1017/s0140525x22002813 · Cited by 204
Self-citation
Metadata
- Authors
- Bowers, J. S.
- Venue
- Trends in Cognitive Sciences
- Publisher
- Elsevier BV
- Published
- 2017-12
- Volume
- 21
- Issue
- 12
- Pages
- 950-961
- ISSN
- 1364-6613 (ISSN-L)
- OpenAlex
- W2766235997
- Cited by
- 40
- References
- 15
- FWCI
- 1.46
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Parallel Distributed Processing Theory in the Age of Deep Networks. https://app.cochat.ai/papers/10.1016/j.tics.2017.09.013
- The paper
- Bowers, J. S. (2017). Parallel Distributed Processing Theory in the Age of Deep Networks. Trends in Cognitive Sciences. https://doi.org/10.1016/j.tics.2017.09.013
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.