Sparse coding with an overcomplete basis set: A strategy employed by V1?
Open accessAbstract
The spatial receptive fields of simple cells in mammalian striate cortex have been reasonably well described physiologically and can be characterized as being localized, oriented, and bandpass, comparable with the basis functions of wavelet transforms. Previously, we have shown that these receptive field properties may be accounted for in terms of a strategy for producing a sparse distribution of output activity in response to natural images. Here, in addition to describing this work in a more expansive fashion, we examine the neurobiological implications of sparse coding. Of particular interest is the case when the code is overcomplete--i.e., when the number of code elements is greater than the effective dimensionality of the input space. Because the basis functions are non-orthogonal and not linearly independent of each other, sparsifying the code will recruit only those basis functions necessary for representing a given input, and so the input-output function will deviate from being purely linear. These deviations from linearity provide a potential explanation for the weak forms of non-linearity observed in the response properties of cortical simple cells, and they further make predictions about the expected interactions among units in response to naturalistic stimuli.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The fundamental problem faced by brains and machines is the same: efficiently extracting and understanding the underlying structure in images, text, sounds, etc.
“Even though the mediums may differ -silicon transistors versus biological neurons -the fundamental problem faced by brains and machines is the same: efficiently extracting and understanding the underlying structure in images, text, sounds, etc. (Barlow et al., 1961;Olshausen & Field, 1997).”
From The Platonic Representation Hypothesis (2024) · cites this paper as Olshausen & Field, 1997
Any deviation from typical behaviour is considered an anomaly and will be detected as an attack.
“Any deviation from this typical behaviour is considered an anomaly and will be detected as an attack [180] [153].”
From Comparative Analysis of Intrusion Detection Systems and Machine Learning Based Model Analysis through Decision Tree (2023) · cites this paper as [180]
Cites
A theory for multiresolution signal decomposition: the wavelet representation
Mallat, S. (1989) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.192463 · Cited by 21,463 · see citing works
An Information-Maximization Approach to Blind Separation and Blind Deconvolution
Bell, A. J., & Sejnowski, T. J. (1995) · Neural Computation · doi:10.1162/neco.1995.7.6.1129 · Cited by 8,986 · see citing works
Independent component analysis, A new concept?
Comon, P. (1994) · Signal Processing · doi:10.1016/0165-1684(94)90029-9 · Cited by 8,150 · see citing works
Information theory and statistics
(1959) · Journal of the Franklin Institute · doi:10.1016/0016-0032(59)90368-0 · Cited by 7,318 · see citing works
Emergence of simple-cell receptive field properties by learning a sparse code for natural images
Olshausen, B. A., & Field, D. J. (1996) · Nature · doi:10.1038/381607a0 · Cited by 5,694 · see citing works
Self-citationThe Handbook of brain theory and neural networks
(1996) · Choice Reviews Online · doi:10.5860/choice.33-4484 · Cited by 3,913 · see citing works
Relations between the statistics of natural images and the response properties of cortical cells
Field, D. J. (1987) · Journal of the Optical Society of America A · doi:10.1364/josaa.4.002379 · Cited by 3,241 · see citing works
Self-citation(1961) · Information and Control · doi:10.1016/s0019-9958(61)80064-8 · Cited by 2,904 · see citing works
A New Learning Algorithm for Blind Signal Separation
Амари, Ш.-И., Cichocki, A. S., & Yang, H. H. (1995) · Cited by 1,904 · see citing works
Barlow, H. B. (1989) · Neural Computation · doi:10.1162/neco.1989.1.3.295 · Cited by 1,677 · see citing works
Spatial frequency selectivity of cells in macaque visual cortex
De Valois, R. L., Albrecht, D. G., & Thorell, L. G. (1982) · Vision Research · doi:10.1016/0042-6989(82)90113-4 · Cited by 1,489 · see citing works
Shiftable multiscale transforms
Simoncelli, E. P., Freeman, W. T., Adelson, E. H., & Heeger, D. J. (1992) · IEEE Transactions on Information Theory · doi:10.1109/18.119725 · Cited by 1,331 · see citing works
Dayan, P., Hinton, G. E., Neal, R. M., & Zemel, R. S. (1995) · Neural Computation · doi:10.1162/neco.1995.7.5.889 · Cited by 1,207 · see citing works
What Is the Goal of Sensory Coding?
Field, D. J. (1994) · Neural Computation · doi:10.1162/neco.1994.6.4.559 · Cited by 1,183 · see citing works
Self-citationExploratory Projection Pursuit
Friedman, J. H. (1987) · Journal of the American Statistical Association · doi:10.1080/01621459.1987.10478427 · Cited by 801 · see citing works
What Does the Retina Know about Natural Scenes?
Atick, J. J., & Redlich, A. (1992) · Neural Computation · doi:10.1162/neco.1992.4.2.196 · Cited by 706 · see citing works
Could information theory provide an ecological theory of sensory processing?
Atick, J. J. (2011) · Network: Computation in Neural Systems · doi:10.3109/0954898x.2011.638888 · Cited by 679 · see citing works
Large-scale neuronal theories of the brain
(1995) · Choice Reviews Online · doi:10.5860/choice.32-4182 · Cited by 624 · see citing works
Natural image statistics and efficient coding
Olshausen, B. A., & Field, D. (1996) · Network Computation in Neural Systems · doi:10.1088/0954-898x/7/2/014 · Cited by 555 · see citing works
Self-citationTowards a Theory of Early Visual Processing
Atick, J. J., & Redlich, A. (1990) · Neural Computation · doi:10.1162/neco.1990.2.3.308 · Cited by 552 · see citing works
Forming sparse representations by local anti-Hebbian learning
Földiák, P. (1990) · Biological Cybernetics · doi:10.1007/bf02331346 · Cited by 546 · see citing works
Dan, Y., Atick, J. J., & Reid, R. C. (1996) · Journal of Neuroscience · doi:10.1523/jneurosci.16-10-03351.1996 · Cited by 476 · see citing works
Could information theory provide an ecological theory of sensory processing?
Atick, J. J. (1992) · Network Computation in Neural Systems · doi:10.1088/0954-898x_3_2_009 · Cited by 448 · see citing works
Large-scale neuronal theories of the brain
Schutter, E. D. (1995) · Trends in Neurosciences · doi:10.1016/0166-2236(95)90055-1 · Cited by 384 · see citing works
Dynamic Model of Visual Recognition Predicts Neural Response Properties in the Visual Cortex
Rao, R. P. N., & Ballard, D. H. (1997) · Neural Computation · doi:10.1162/neco.1997.9.4.721 · Cited by 290 · see citing works
Natural image statistics and efficient coding
Olshausen, B. A., & Field, D. J. (1996) · Network Computation in Neural Systems · doi:10.1088/0954-898x_7_2_014 · Cited by 273 · see citing works
Self-citationGenerative models for discovering sparse distributed representations
Hinton, G. E., & Ghahramani, Z. (1997) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.1997.0101 · Cited by 225 · see citing works
Sparse coding in the primate cortex
Földiák, P., & Young, M. P. (1998) · St Andrews Research Repository (St Andrews Research Repository) · Cited by 184 · see citing works
Temporal decorrelation: a theory of lagged and nonlagged responses in the lateral geniculate nucleus
Dong, D. W., & Atick, J. J. (1995) · Network Computation in Neural Systems · doi:10.1088/0954-898x_6_2_003 · Cited by 183 · see citing works
Entropy reduction and decorrelation in visual coding by oriented neural receptive fields
Daugman, J. G. (1989) · IEEE Transactions on Biomedical Engineering · doi:10.1109/10.16456 · Cited by 152 · see citing works
A Context-Sensitive Generalization of ICA
Pearlmutter, B. A., & Parra, L. C. (1996) · Maynooth University ePrints and eTheses Archive (Maynooth University) · Cited by 148 · see citing works
Neuronal architectures for pattern-theoretic problems.
Mumford, D. B. (1994) · Cited by 146 · see citing works
Internal representations for associative memory
Baum, E. B., Moody, J. E., & Wilczek, F. A. (1988) · Biological Cybernetics · doi:10.1007/bf00332910 · Cited by 129 · see citing works
A Multiple Cause Mixture Model for Unsupervised Learning
Saund, E. (1995) · Neural Computation · doi:10.1162/neco.1995.7.1.51 · Cited by 111 · see citing works
A minimum description length framework for unsupervised learning
Zemel, R. S. (1994) · Cited by 96 · see citing works
Development of low entropy coding in a recurrent network
Harpur, G. F., & Prager, R. W. (1996) · Network Computation in Neural Systems · doi:10.1088/0954-898x/7/2/007 · Cited by 92 · see citing works
Feature Extraction Using an Unsupervised Neural Network
Intrator, N. (1992) · Neural Computation · doi:10.1162/neco.1992.4.1.98 · Cited by 90 · see citing works
Two-dimensional spatial structure of receptive fields in monkey striate cortex
Parker, A. J., & Hawken, M. J. (1988) · Journal of the Optical Society of America A · doi:10.1364/josaa.5.000598 · Cited by 86 · see citing works
Law, C.-T., & Cooper, L. N. (1994) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.91.16.7797 · Cited by 84 · see citing works
Baddeley, R. J. (1996) · Nature · doi:10.1038/381560a0 · Cited by 66 · see citing works
Bayesian Unsupervised Learning of Higher Order Structure
Lewicki, M. S., & Sejnowski, T. J. (1996) · Cited by 55 · see citing works
The Neurobiology of Computation
Bower, J. M. (1995) · Springer eBooks · doi:10.1007/978-1-4615-2235-5 · Cited by 44 · see citing works
Finding compact and sparse-distributed representations of visual images
Fyfe, C., & Baddeley, R. J. (1995) · Network Computation in Neural Systems · doi:10.1088/0954-898x_6_3_002 · Cited by 41 · see citing works
Learning linear, sparse, factorial codes
Olshausen, B. A. (1996) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 40 · see citing works
Self-citationDevelopment of low entropy coding in a recurrent network
Harpur, G., & Prager, R. W. (1996) · Network Computation in Neural Systems · doi:10.1088/0954-898x_7_2_007 · Cited by 36 · see citing works
Tadmor, Y., & Tolhurst, D. J. (1989) · Visual Neuroscience · doi:10.1017/s0952523800005940 · Cited by 26 · see citing works
A Model of the Spatial-Frequency Organization in Primate Striate Cortex
Olshausen, B. A., & Anderson, C. H. (1995) · Springer eBooks · doi:10.1007/978-1-4615-2235-5_45 · Cited by 6 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Rao, R. P. N., & Ballard, D. H. (1999) · Nature Neuroscience · doi:10.1038/4580 · Cited by 5,483 · see citing works
Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries
Elad, M., & Aharon, M. (2006) · IEEE Transactions on Image Processing · doi:10.1109/tip.2006.881969 · Cited by 5,321 · see citing works
Deo, R. C. (2015) · Circulation · doi:10.1161/circulationaha.115.001593 · Cited by 3,466 · see citing works
Deep learning for visual understanding: A review
Guo, Y., Liu, Y., Oerlemans, A., Lao, S., Wu, S., & Lew, M. S. (2016) · Neurocomputing · doi:10.1016/j.neucom.2015.09.116 · Cited by 2,504 · see citing works
Natural Image Statistics and Neural Representation
Simoncelli, E. P., & Olshausen, B. A. (2001) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.24.1.1193 · Cited by 2,460 · see citing works
Self-citationDefinitions, methods, and applications in interpretable machine learning
Murdoch, W. J., Singh, C., Kumbier, K., Abbasi-Asl, R., & Yu, B. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1900654116 · Cited by 2,167 · see citing works
“Methods for obtaining sparsity often utilize a penalty on a loss function, such as LASSO (31) and sparse coding (32), or on a model selection criteria such as AIC or BIC (33, 34).”
MethodConvolutional networks and applications in vision
LeCun, Y., Kavukcuoglu, K., & Farabet, C. (2010) · Proceedings of 2010 IEEE International Symposium on Circuits and Systems · doi:10.1109/iscas.2010.5537907 · Cited by 2,213 · see citing works
What is the best multi-stage architecture for object recognition?
Jarrett, K., Kavukcuoglu, K., Ranzato, M. A., & LeCun, Y. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459469 · Cited by 2,153 · see citing works
Practical Recommendations for Gradient-Based Training of Deep Architectures
Bengio, Y. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-35289-8_26 · Cited by 2,259 · see citing works
How does the brain solve visual object recognition?
DiCarlo, J. J., Zoccolan, D., & Rust, N. C. (2012) · Neuron · doi:10.1016/j.neuron.2012.01.010 · Cited by 1,790 · see citing works
Chen, C. L. P., & Liu, Z. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2716952 · Cited by 1,957 · see citing works
Sparse Representation for Color Image Restoration
Mairal, J., Elad, M., & Sapiro, G. (2008) · IEEE Transactions on Image Processing · doi:10.1109/tip.2007.911828 · Cited by 1,716 · see citing works
Sparse coding of sensory inputs
Olshausen, B. A., & Field, D. J. (2004) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2004.07.007 · Cited by 1,404 · see citing works
Self-citationKriegeskorte, N. (2015) · Annual Review of Vision Science · doi:10.1146/annurev-vision-082114-035447 · Cited by 1,093 · see citing works
Independent component filters of natural images compared with simple cells in primary visual cortex
van Hateren, J. H., & van der Schaaf, A. (1998) · Proceedings of the Royal Society of London. Series B: Biological Sciences · doi:10.1098/rspb.1998.0303 · Cited by 1,106 · see citing works
Efficient Learning of Sparse Representations with an Energy-Based Model
Ranzato, M., Poultney, C., Chopra, S., & LeCun, Y. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0147 · Cited by 1,106 · see citing works
Multi-column deep neural network for traffic sign classification
Cireşan, D., Meier, U., Masci, J., & Schmidhuber, J. (2012) · Neural Networks · doi:10.1016/j.neunet.2012.02.023 · Cited by 1,057 · see citing works
Unsupervised Learning of Visual Representations Using Videos
Wang, X., & Gupta, A. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.320 · Cited by 1,005 · see citing works
Nweke, H. F., Teh, Y. W., Al-garadi, M. A., & Alo, U. R. (2018) · Expert Systems with Applications · doi:10.1016/j.eswa.2018.03.056 · Cited by 930 · see citing works
Lieder, F., & Griffiths, T. L. (2020) · Behavioral and Brain Sciences · doi:10.1017/s0140525x1900061x · Cited by 892 · see citing works
A fully integrated reprogrammable memristor–CMOS system for efficient multiply–accumulate operations
Cai, F., Correll, J. M., Lee, S. H., Lim, Y., Bothra, V., Zhang, Z., Flynn, M. P., & Lu, W. D. (2019) · Nature Electronics · doi:10.1038/s41928-019-0270-x · Cited by 728 · see citing works
Medical Image Denoising Using Convolutional Denoising Autoencoders
Gondara, L. (2016) · 2016 IEEE 16th International Conference on Data Mining Workshops (ICDMW) · doi:10.1109/icdmw.2016.0041 · Cited by 737 · see citing works
Sparse coding with memristor networks
Sheridan, P. M., Cai, F., Du, C., Ma, W., Zhang, Z., & Lu, W. D. (2017) · Nature Nanotechnology · doi:10.1038/nnano.2017.83 · Cited by 619 · see citing works
Shin, H.-C., Orton, M. R., Collins, D. J., Doran, S. J., & Leach, M. O. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.277 · Cited by 488 · see citing works
Simplicity: a unifying principle in cognitive science?
Chater, N., & Vitányi, P. (2003) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(02)00005-0 · Cited by 494 · see citing works
A review of predictive coding algorithms
Spratling, M. W. (2017) · Brain and Cognition · doi:10.1016/j.bandc.2015.11.003 · Cited by 437 · see citing works
Restoring an Image Taken through a Window Covered with Dirt or Rain
Eigen, D., Krishnan, D., & Fergus, R. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.84 · Cited by 454 · see citing works
Huang, Y., & Rao, R. P. N. (2011) · WIREs Cognitive Science · doi:10.1002/wcs.142 · Cited by 470 · see citing works
Big Data: Deep Learning for financial sentiment analysis
Sohangir, S., Wang, D., Pomeranets, A., & Khoshgoftaar, T. M. (2018) · Journal of Big Data · doi:10.1186/s40537-017-0111-6 · Cited by 390 · see citing works
Computational approaches to sensorimotor transformations
Pouget, A., & Snyder, L. H. (2000) · Nature Neuroscience · doi:10.1038/81469 · Cited by 357 · see citing works
Ecological statistics of Gestalt laws for the perceptual organization of contours
Elder, J. H., & Goldberg, R. M. (2002) · Journal of Vision · doi:10.1167/2.4.5 · Cited by 333 · see citing works
Learning to Generate Reviews and Discovering Sentiment
Radford, A., Jozefowicz, R., & Sutskever, I. (2017) · arXiv · doi:10.48550/arxiv.1704.01444 · Cited by 330 · see citing works
“There is also a long history of unsupervised representation learning (Olshausen & Field, 1997).”
BackgroundDynamic Model of Visual Recognition Predicts Neural Response Properties in the Visual Cortex
Rao, R. P. N., & Ballard, D. H. (1997) · Neural Computation · doi:10.1162/neco.1997.9.4.721 · Cited by 290 · see citing works
Sparse coding based visual tracking: Review and experimental comparison
Zhang, S., Yao, H., Sun, X., & Lu, X. (2013) · Pattern Recognition · doi:10.1016/j.patcog.2012.10.006 · Cited by 284 · see citing works
Azam, Z., Islam, M. M., & Huda, M. N. (2023) · IEEE Access · doi:10.1109/access.2023.3296444 · Cited by 265 · see citing works
The neuroconnectionist research programme
Doerig, A., Sommers, R. P., Seeliger, K., Richards, B., Ismael, J., Lindsay, G. W., Kording, K. P., Konkle, T., van Gerven, M. A. J., Kriegeskorte, N., et al. (2023) · Nature Reviews Neuroscience · doi:10.1038/s41583-023-00705-w · Cited by 233 · see citing works
Bhattacharya, S., & Lane, N. D. (2016) · Proceedings of the 14th ACM Conference on Embedded Networked Sensor Systems CD-ROM · doi:10.1145/2994551.2994564 · Cited by 236 · see citing works
Perceptual Decision-Making as Probabilistic Inference by Neural Sampling
Haefner, R. M., Berkes, P., & Fiser, J. (2016) · Neuron · doi:10.1016/j.neuron.2016.03.020 · Cited by 207 · see citing works
Linear Algebraic Structure of Word Senses, with Applications to Polysemy
Arora, S., Li, Y., Liang, Y., Ma, T., & Risteski, A. (2018) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00034 · Cited by 186 · see citing works
A Systematic Review of Detecting Sleep Apnea Using Deep Learning
Mostafa, S. S., Mendonça, F., G. Ravelo-García, A., & Morgado-Dias, F. (2019) · Sensors · doi:10.3390/s19224934 · Cited by 182 · see citing works
Learning optimized features for hierarchical models of invariant object recognition.
Wersing, H., & Körner, E. (2003) · Neural Computation · doi:10.1162/089976603321891800 · Cited by 169 · see citing works
Forms of prediction in the nervous system
Teufel, C., & Fletcher, P. C. (2020) · Nature Reviews Neuroscience · doi:10.1038/s41583-020-0275-5 · Cited by 155 · see citing works
The Rhetoric and Reality of Anthropomorphism in Artificial Intelligence
Watson, D. (2019) · Minds and Machines · doi:10.1007/s11023-019-09506-6 · Cited by 148 · see citing works
Zhou, H., Chen, J., Dong, G., & Wang, R. (2016) · Mechanical Systems and Signal Processing · doi:10.1016/j.ymssp.2015.11.022 · Cited by 131 · see citing works
Toward an integration of deep learning and neuroscience
Marblestone, A. H., Wayne, G., & Kording, K. P. (2016) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2016.00094 · Cited by 119 · see citing works
Bayesian natural selection and the evolution of perceptual systems
Geisler, W. S., & Diehl, R. L. (2002) · Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences · doi:10.1098/rstb.2001.1055 · Cited by 101 · see citing works
Zhang, Y., Moorthy, A. K., Chandler, D. M., & Bovik, A. C. (2014) · Signal Processing: Image Communication · doi:10.1016/j.image.2014.05.004 · Cited by 102 · see citing works
Learning the invariance properties of complex cells from their responses to natural stimuli
Einhäuser, W., Kayser, C., König, P., & Körding, K. P. (2002) · European Journal of Neuroscience · doi:10.1046/j.0953-816x.2001.01885.x · Cited by 91 · see citing works
Deep Neural Networks for Modeling Visual Perceptual Learning
Wenliang, L. K., & Seitz, A. R. (2018) · The Journal of Neuroscience · doi:10.1523/jneurosci.1620-17.2018 · Cited by 83 · see citing works
Enhanced functional connectivity properties of human brains during in-situ nature experience
Chen, Z., He, Y., & Yu, Y. (2016) · PeerJ · doi:10.7717/peerj.2210 · Cited by 74 · see citing works
Learning Multisensory Integration and Coordinate Transformation via Density Estimation
Makin, J. G., Fellows, M. R., & Sabes, P. N. (2013) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1003035 · Cited by 53 · see citing works
Wasserstein Introspective Neural Networks
Lee, K., Xu, W., Fan, F., & Tu, Z. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00390 · Cited by 56 · see citing works
Montijn, J. S., Goltstein, P. M., & Pennartz, C. M. (2015) · eLife · doi:10.7554/elife.10163 · Cited by 46 · see citing works
Coding of the contrasts in natural images by populations of neurons in primary visual cortex (V1)
Clatworthy, P. L., Chirimuuta, M., Lauritzen, J. S., & Tolhurst, D. J. (2003) · Vision Research · doi:10.1016/s0042-6989(03)00277-3 · Cited by 54 · see citing works
Contrastive Learning Models for Sentence Representations
Xu, L., Xie, H., Li, Z., Wang, F. L., Wang, W., & Li, Q. (2023) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3593590 · Cited by 50 · see citing works
Ezzati Khatab, Z., Hajihoseini Gazestani, A., Ghorashi, S. A., & Ghavami, M. (2021) · Signal Processing · doi:10.1016/j.sigpro.2020.107915 · Cited by 44 · see citing works
Learning probabilistic neural representations with randomly connected circuits
Maoz, O., Tkačik, G., Esteki, M. S., Kiani, R., & Schneidman, E. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1912804117 · Cited by 44 · see citing works
Deep learning of smartphone sensor data for personal health assistance
Li, H., & Trocan, M. (2019) · Microelectronics Journal · doi:10.1016/j.mejo.2018.01.015 · Cited by 30 · see citing works
The Platonic Representation Hypothesis
Huh, M., Cheung, B., Wang, T., & Isola, P. (2024) · arXiv · doi:10.48550/arxiv.2405.07987 · Cited by 35 · see citing works
“See Appendix F for additional results on GSM8K (math) (Cobbe et al., 2021). by brains and machines is the same: efficiently extracting and understanding the underlying structure in images, text, sounds, etc. (Barlow et al., 1961; Olshausen & Field, 1997).”
Parziale, M., Lomazzi, L., Giglio, M., & Cadini, F. (2022) · Structural Control and Health Monitoring · doi:10.1002/stc.3076 · Cited by 22 · see citing works
Willmore, B. D. B., Bulstrode, H., & Tolhurst, D. J. (2012) · Vision Research · doi:10.1016/j.visres.2011.12.008 · Cited by 10 · see citing works
Transient localist representations in critical branching networks
Rodny, J. J., Shea, T. M., & Kello, C. T. (2017) · Language, Cognition and Neuroscience · doi:10.1080/23273798.2016.1242760 · Cited by 4 · see citing works
Improving Dictionary Learning with Gated Sparse Autoencoders
Rajamanoharan, S., Conmy, A., Smith, L., Lieberum, T., Varma, V., Kramár, J., Shah, R., & Nanda, N. (2024) · arXiv · doi:10.48550/arxiv.2404.16014 · Cited by 3 · see citing works
“For example, sparse coding (Elad, 2010) studies how discrete and continuous representations can involve more representations than basis vectors, like our setup in Section 1, and sparse representations are also studied in neuroscience (Olshausen and Field, 1997; Thorpe, 1989).”
BackgroundA survey on neuro-mimetic deep learning via predictive coding
Salvatori, T., Mali, A., Buckley, C. L., Lukasiewicz, T., Rao, R. P. N., Friston, K., & Ororbia, A. (2026) · Neural Networks · doi:10.1016/j.neunet.2025.108161 · Cited by 2 · see citing works
- Authors
- Olshausen, B. A., & Field, D. J.
- Affiliations
- Cornell University (US)
- Corresponding author
- Olshausen, Bruno A.
- ORCID
- Olshausen, Bruno A.
- Venue
- Vision Research
- Publisher
- Elsevier BV
- Published
- 1997-12
- Volume
- 37
- Issue
- 23
- Pages
- 3311-3325
- ISSN
- 0042-6989 (ISSN-L)
- OpenAlex
- W2105464873
- Cited by
- 3,642
- References
- 47
- Citations by year
- 2026: 49 · 2025: 95 · 2024: 90 · 2023: 114 · 2022: 121 · 2021: 179 · 2020: 233 · 2019: 224 · 2018: 244 · 2017: 251 · 2016: 239 · 2015: 228 · 2014: 264 · 2013: 208 · 2012: 169
- FWCI
- 13.17
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Vision Research (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Funders
- National Institutes of Health (5f32mh011062-02, 5r29mh050588-05)
- National Institute of Mental Health (R29-MH50588, F32-MH11062)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Physical Sciences › Engineering › Computational Mechanics › Sparse and Compressive Sensing Techniques
- Keywords
- sparse coding, primary visual cortex, simple cells, receptive fields, non-orthogonal basis functions, natural image statistics, neural coding
- MeSH terms
- Algorithms, Animals, Mammals, Models, Psychological, Visual Cortex, Visual Perception
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
