Learning Multisensory Integration and Coordinate Transformation via Density Estimation
Open accessThe DOI matches its Crossref record; 72 of the 72 references it cites were found in a registry or an index.
Abstract
Sensory processing in the brain includes three key operations: multisensory integration-the task of combining cues into a single estimate of a common underlying stimulus; coordinate transformations-the change of reference frame for a stimulus (e.g., retinotopic to body-centered) effected through knowledge about an intervening variable (e.g., gaze position); and the incorporation of prior information. Statistically optimal sensory processing requires that each of these operations maintains the correct posterior distribution over the stimulus. Elements of this optimality have been demonstrated in many behavioral contexts in humans and other animals, suggesting that the neural computations are indeed optimal. That the relationships between sensory modalities are complex and plastic further suggests that these computations are learned-but how? We provide a principled answer, by treating the acquisition of these mappings as a case of density estimation, a well-studied problem in machine learning and statistics, in which the distribution of observed data is modeled in terms of a set of fixed parameters and a set of latent variables. In our case, the observed data are unisensory-population activities, the fixed parameters are synaptic connections, and the latent variables are multisensory-population activities. In particular, we train a restricted Boltzmann machine with the biologically plausible contrastive-divergence rule to learn a range of neural computations not previously demonstrated under a single approach: optimal integration; encoding of priors; hierarchical integration of cues; learning when not to integrate; and coordinate transformation. The model makes testable predictions about the nature of multisensory representations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures







![Figure 8 Tuning curves in the hidden layer. (A) Tuning curves for the multisensory-integration model/data (Figs. 1 and 2). The left column shows tuning curves in the space of joint angles for sixteen randomly chosen hidden units; the right column shows those same units for the arc of reach endpoints from [29]. The top row shows tuning curves for the trained model; the second row shows the same curves for the untrained model. The location of the arc in joint space is shown by the black slash through the tuning curves in the left column. Whereas the left-column tuning curves were collected for a single gain (), the right-column curves were collected for (blue), (green), and (red) (the same gain was used for both populations, vis and prop). (B) Example hidden-unit tuning curves from the coordinate transformation model for body-centered hand position (; see text for details), for two different gaze positions (red and green curves). The dashed blue curves show where the red tuning curves would lie for the second gaze position if they shifted completely with the eyes, as illustrated by the red arrows, i.e. if they were retinotopic. Some cells (second column) are; some are body-centered (first column); some partially shift (third column); and some even shift in the opposite direction of the gaze angle. (C) Coordinate-transforming cells can be tuned for any of the variables on the continuum from gaze angle (E), to retinotopic hand position (), to body-centered hand position (), to body-centered hand position plus gaze angle (). The histogram shows the distributions of such tunings in the hidden layer, using the analysis of [31].](https://figures.cochat.ai/paper-figures/95ce420e5a7b4290b2090f72d11fd0b3/learning-multisensory-integration-and-coordinate-transformation-via-density-figure-8.jpg)
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Normal processing in the brain is highly tolerant of abnormal patterns and classifies patterns according to best matching prior sensory experiences (19).”
From A neural interface provides long-term stable natural touch perception (2014) · cites this paper as [19]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“The complex perceptions described in response to patterned stimulation intensity paradigms suggest that sensory perception is a pattern recognition activity (19).”
From A neural interface provides long-term stable natural touch perception (2014) · cites this paper as [19]
Experimental — accuracy not yet measuredIn the citing paper
“and related models (Makin et al., 2013, 2016) have been used to account for optimal multi-sensory integration in the brain.”
From Toward an integration of deep learning and neuroscience (2016) · cites this paper as Makin et al., 2013
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
(2005) · Statistics for Biology and Health · doi:10.1007/0-387-27255-0_9 · Cited by 20,133
McCullagh, P., & Nelder, J. A. (1989) · doi:10.1007/978-1-4899-3242-6 · Cited by 16,836
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336 · cited 2×
“In particular, we model the cortical association area with a restricted Boltzmann machine (RBM), an undirected generative model trained with a fast, effective Hebbian learning rule, contrastive divergence [16,17].”
MethodTraining Products of Experts by Minimizing Contrastive Divergence
Hinton, G. E. (2002) · Neural Computation · doi:10.1162/089976602760128018 · Cited by 4,879 · cited 4×
“This rule approximates gradient descent on an objective function for density estimation (Hinton’s ‘‘contrastive divergence’’ [17], or alternatively ‘‘probability flow’’ [64]).”
BackgroundMethodInfluentialHumans integrate visual and haptic information in a statistically optimal fashion
Ernst, M. O., & Banks, M. S. (2002) · Nature · doi:10.1038/415429a · Cited by 4,844 · cited 2×
“based on the peak of the optimal posterior over the stimulus, given a variety of multisensory inputs [1,3–7].”
BackgroundSparse coding with an overcomplete basis set: A strategy employed by V1?
Olshausen, B. A., & Field, D. J. (1997) · Vision Research · doi:10.1016/s0042-6989(97)00169-7 · Cited by 3,642 · cited 1×
“One promising theoretical strategy would be to take ‘‘passing on all the information’’ as a given, and then to seek the set of constraints—fewest spikes [49], topography [50], fewest neurons, least processing time, computational efficiency [51], etc.”
BackgroundSome informational aspects of visual perception.
Attneave, F. (1954) · Psychological Review · doi:10.1037/h0054663 · Cited by 2,890 · cited 1×
“Finally, many authors have either anticipated [51,56,57] or”
BackgroundThe Bayesian brain: The role of uncertainty in neural coding and computation
Knill, D. C., & Pouget, A. (2004) · Trends in Neurosciences · doi:10.1016/j.tins.2004.10.007 · Cited by 2,588 · cited 1×
“based on the peak of the optimal posterior over the stimulus, given a variety of multisensory inputs [1,3–7].”
BackgroundBayesian integration in sensorimotor learning
Körding, K. P., & Wolpert, D. M. (2004) · Nature · doi:10.1038/nature02169 · Cited by 2,004
The Ventriloquist Effect Results from Near-Optimal Bimodal Integration
Alais, D., & Burr, D. (2004) · Current Biology · doi:10.1016/j.cub.2004.01.029 · Cited by 1,936 · cited 2×
“, prism goggles or virtual feedback [1,3,4,18–20].”
BackgroundMerging the senses into a robust percept
Ernst, M. O., & Bülthoff, H. H. (2004) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2004.02.002 · Cited by 1,906 · cited 1×
“We consider now another, seemingly different, computational problem studied in the sensorimotor literature, coordinate transformation (sometimes called ‘‘sensory combination’’ [22]).”
BackgroundBayesian inference with probabilistic population codes
Ma, W. J., Beck, J. M., Latham, P. E., & Pouget, A. (2006) · Nature Neuroscience · doi:10.1038/nn1790 · Cited by 1,457
Possible Principles Underlying the Transformations of Sensory Messages
Barlow, H. B. (2012) · Sensory Communication · doi:10.7551/mitpress/9780262518420.003.0013 · Cited by 1,269
Movement-produced stimulation in the development of visually guided behavior.
Held, R., & Hein, A. (1963) · Journal of Comparative and Physiological Psychology · doi:10.1037/h0040546 · Cited by 1,161 · cited 1×
“Prism and virtualfeedback adapation experiments [8–12] have demonstrated the plasticity of these multisensory mappings, and it is not likely limited to recalibration: Deprivation studies [13]; afferent rerouting experiments [14,15]; the ability to learn novel, cross-modal mappings; and genetic-information constraints…”
BackgroundA mechanism for the Hebb and the anti-Hebb processes underlying learning and memory.
Lisman, J. (1989) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.86.23.9574 · Cited by 1,127 · cited 1×
“Anti-Hebbian learning has been observed in a neural circuit [65], albeit not in mammalian cortex, and plausible cellular mechanisms for it have been described [66].”
BackgroundCausal Inference in Multisensory Perception
Körding, K. P., Beierholm, U., Ma, W. J., Quartz, S., Tenenbaum, J. B., & Shams, L. (2007) · PLoS ONE · doi:10.1371/journal.pone.0000943 · Cited by 1,069 · cited 1×
“[21]).”
Chklovskii, D. (2002) · Neuron · doi:10.1016/s0896-6273(02)00703-1 · Cited by 1,064
PREMOTOR AND PARIETAL CORTEX: Corticocortical Connectivity and Combinatorial Computations
Wise, S. P., Boussaoud, D., Johnson, P. B., & Caminiti, R. (1997) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.20.1.25 · Cited by 953 · cited 1×
“The fact that our network can be hierarchically composed is central to this hypothesis: these brain areas receive overlapping but distinct sets of inputs and with a rough hierarchical organization within them [39–43].”
BackgroundNoise characteristics and prior expectations in human visual speed perception
Stocker, A. A., & Simoncelli, E. P. (2006) · Nature Neuroscience · doi:10.1038/nn1669 · Cited by 797 · cited 1×
“based on the peak of the optimal posterior over the stimulus, given a variety of multisensory inputs [1,3–7].”
BackgroundVentral intraparietal area of the macaque: congruent visual and somatic response properties
Duhamel, J.-R., Colby, C. L., & Goldberg, M. E. (1998) · Journal of Neurophysiology · doi:10.1152/jn.1998.79.1.126 · Cited by 789
Integration of Proprioceptive and Visual Position-Information: An Experimentally Supported Model
van Beers, R. J., Sittig, A. C., & Gon, J. J. D. V. D. (1999) · Journal of Neurophysiology · doi:10.1152/jn.1999.81.3.1355 · Cited by 742
Johnson, P. B., Ferraina, S., Bianchi, L., & Caminiti, R. (1996) · Cerebral Cortex · doi:10.1093/cercor/6.2.102 · Cited by 693
Efficient coding of natural sounds
Lewicki, M. S. (2002) · Nature Neuroscience · doi:10.1038/nn831 · Cited by 674 · cited 1×
“explicitly proposed [58–60] that learning to process early sensory”
BackgroundSpatial invariance of visual receptive fields in parietal cortex neurons
Duhamel, J.-R., Bremmer, F., Ben Hamed, S., & Graf, W. (1997) · Nature · doi:10.1038/39865 · Cited by 642
Redundancy reduction revisited
Barlow, H. (2001) · Network: Computation in Neural Systems · doi:10.1080/net.12.3.241.253 · Cited by 620 · cited 1×
“Finally, many authors have either anticipated [51,56,57] or”
BackgroundBuneo, C. A., & Andersen, R. A. (2006) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2005.10.011 · Cited by 598 · cited 1×
“There is considerable evidence for multimodal, reaching-related signals across several brain areas in the posterior parietal cortex, including Area 5d, MIP, VIP, V6, and Area 7 [33–38].”
BackgroundGraziano, M. S. A. (1999) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.96.18.10418 · Cited by 500 · cited 1×
“There is considerable evidence for multimodal, reaching-related signals across several brain areas in the posterior parietal cortex, including Area 5d, MIP, VIP, V6, and Area 7 [33–38].”
BackgroundSelf-organizing neural network that discovers surfaces in random-dot stereograms
Becker, S., & Hinton, G. E. (1992) · Nature · doi:10.1038/355161a0 · Cited by 495 · cited 2×
“Multisensory integration was first considered from the standpoint of information theory and unsupervised learning in [52], and in a related work [50], and our approach is similar in spirit, but with important differences.”
BackgroundExponential Family Harmoniums with an Application to Information Retrieval
Welling, M., Rosen‐Zvi, M., & Hinton, G. E. (2004) · neural information processing systems · Cited by 486
Flexible strategies for sensory integration during motor planning
Sober, S. J., & Sabes, P. N. (2005) · Nature Neuroscience · doi:10.1038/nn1427 · Cited by 480 · cited 1×
“, prism goggles or virtual feedback [1,3,4,18–20].”
Self-citationReference frames for representing visual and tactile locations in parietal cortex
Avillac, M., Denève, S., Olivier, E., Pouget, A., & Duhamel, J.-R. (2005) · Nature Neuroscience · doi:10.1038/nn1480 · Cited by 464 · cited 1×
“dynamics [27,30,54] ignore prior distributions but more signifi-”
BackgroundNeural correlates of reliability-based cue weighting during multisensory integration
Fetsch, C. R., Pouget, A., DeAngelis, G. C., & Angelaki, D. E. (2012) · Nature Neuroscience · doi:10.1038/nn.2983 · Cited by 459 · cited 1×
“, prism goggles or virtual feedback [1,3,4,18–20].”
Multisensory Integration during Motor Planning
Sober, S. J., & Sabes, P. N. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-18-06982.2003 · Cited by 430
Self-citationA computational perspective on the neural basis of multisensory spatial representations
Pouget, A., Deneve, S., & Duhamel, J.-R. (2002) · Nature Reviews Neuroscience · doi:10.1038/nrn914 · Cited by 413 · cited 1×
“dynamics [27,30,54] ignore prior distributions but more signifi-”
BackgroundApplications of prism adaptation: a tutorial in theory and method
Redding, G. M., Rossetti, Y., & Wallace, B. (2005) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2004.12.004 · Cited by 412
Pesaran, B., Nelson, M. J., & Andersen, R. A. (2006) · Neuron · doi:10.1016/j.neuron.2006.05.025 · Cited by 370 · cited 1×
“More recently, Andersen and colleagues [31,32] have proposed to analyze these qualitative descriptions in terms of (1) the ‘‘separability’’ of the tuning curve—whether it can be written f (s,e)~fs(s)fe(e); and (2) the reference frame in which the neuron is tuned—body, eye, or something intermediate—as measured by the…”
BackgroundEfficient computation and cue integration with noisy population codes
Deneve, S., Latham, P. E., & Pouget, A. (2001) · Nature Neuroscience · doi:10.1038/90541 · Cited by 351 · cited 2×
“dynamics [27,30,54] ignore prior distributions but more signifi-”
BackgroundHeld, R., & Freedman, S. J. (1963) · Science · doi:10.1126/science.142.3591.455 · Cited by 338 · cited 1×
“Prism and virtualfeedback adapation experiments [8–12] have demonstrated the plasticity of these multisensory mappings, and it is not likely limited to recalibration: Deprivation studies [13]; afferent rerouting experiments [14,15]; the ability to learn novel, cross-modal mappings; and genetic-information constraints…”
BackgroundSpatial maps for the control of movement.
Graziano, M. S., & Gross, C. G. (1998) · Current Opinion in Neurobiology · doi:10.1016/s0959-4388(98)80140-2 · Cited by 323
Probabilistic framework for the adaptation and comparison of image codes
Lewicki, M. S., & Olshausen, B. A. (1999) · Journal of the Optical Society of America A · doi:10.1364/josaa.16.001587 · Cited by 308 · cited 1×
“explicitly proposed [58–60] that learning to process early sensory”
BackgroundTransfer of coded information from sensory to motor networks
Salinas, E., & Abbott, L. (1995) · The Journal of Neuroscience · doi:10.1523/jneurosci.15-10-06461.1995 · Cited by 295 · cited 2×
“Other models of learning [24–26,55]”
BackgroundMultiple Levels of Representation of Reaching in the Parieto-frontal Network
Battaglia-Mayer, A. (2003) · Cerebral Cortex · doi:10.1093/cercor/13.10.1009 · Cited by 251 · cited 1×
“The fact that our network can be hierarchically composed is central to this hypothesis: these brain areas receive overlapping but distinct sets of inputs and with a rough hierarchical organization within them [39–43].”
BackgroundThe cortical connections of area V6: an occipito‐parietal network processing visual information
Galletti, C., Gamberini, M., Kutz, D. F., Fattori, P., Luppino, G., & Matelli, M. (2001) · European Journal of Neuroscience · doi:10.1046/j.0953-816x.2001.01538.x · Cited by 239 · cited 1×
“There is considerable evidence for multimodal, reaching-related signals across several brain areas in the posterior parietal cortex, including Area 5d, MIP, VIP, V6, and Area 7 [33–38].”
BackgroundSur, M., Pallas, S. L., & Roe, A. W. (1990) · Trends in Neurosciences · doi:10.1016/0166-2236(90)90165-7 · Cited by 237 · cited 1×
“Prism and virtualfeedback adapation experiments [8–12] have demonstrated the plasticity of these multisensory mappings, and it is not likely limited to recalibration: Deprivation studies [13]; afferent rerouting experiments [14,15]; the ability to learn novel, cross-modal mappings; and genetic-information constraints…”
BackgroundVerstynen, T., & Sabes, P. N. (2011) · Journal of Neuroscience · doi:10.1523/jneurosci.6525-10.2011 · Cited by 230 · cited 3×
“Whether prior distributions in the brain are encoded in synaptic weights [46,47], as a separate neural population [2], or something else again, remains an area of active research (see also Text S1).”
BackgroundSelf-citationDayan, P., & Abbott, L. (2001) · UCL Discovery (University College London) · Cited by 216
Shipp, S., Blanton, M., & Zeki, S. (1998) · European Journal of Neuroscience · doi:10.1046/j.1460-9568.1998.00327.x · Cited by 207 · cited 1×
“There is considerable evidence for multimodal, reaching-related signals across several brain areas in the posterior parietal cortex, including Area 5d, MIP, VIP, V6, and Area 7 [33–38].”
BackgroundGeneralization to Local Remappings of the Visuomotor Coordinate Transformation
Ghahramani, Z., Wolpert, D. M., & Jordan, M. I. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-21-07085.1996 · Cited by 204 · cited 1×
“Prism and virtualfeedback adapation experiments [8–12] have demonstrated the plasticity of these multisensory mappings, and it is not likely limited to recalibration: Deprivation studies [13]; afferent rerouting experiments [14,15]; the ability to learn novel, cross-modal mappings; and genetic-information constraints…”
BackgroundSensory transformations and the use of multiple reference frames for reach planning
McGuire, L. M. M., & Sabes, P. N. (2009) · Nature Neuroscience · doi:10.1038/nn.2357 · Cited by 199 · cited 2×
“We have previously argued that these three operations are exactly those required for planning multisensoryguided reaching movements [23].”
BackgroundSelf-citationXing, J., & Andersen, R. A. (2000) · Journal of Cognitive Neuroscience · doi:10.1162/089892900562363 · Cited by 189
Burnod, Y., Grandguillaume, P., Otto, I., Ferraina, S., Johnson, P., & Caminiti, R. (1992) · The Journal of Neuroscience · doi:10.1523/jneurosci.12-04-01435.1992 · Cited by 133 · cited 1×
“Other models of learning [24–26,55]”
Idiosyncratic and systematic aspects of spatial representations in the macaque parietal cortex
Chang, S. W. C., & Snyder, L. H. (2010) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0913209107 · Cited by 121 · cited 1×
“Density estimation on these inputs, then, is expected to yield activity patterns that are also highly overlapping but distinct, as observed, for example, in [29,44].”
BackgroundStorage of a sensory pattern by anti-Hebbian synaptic plasticity in an electric fish.
Bell, C. C., Caputi, A., Grant, K., & Serrier, J. (1993) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.90.10.4650 · Cited by 118 · cited 1×
“Anti-Hebbian learning has been observed in a neural circuit [65], albeit not in mammalian cortex, and plausible cellular mechanisms for it have been described [66].”
BackgroundCombination of Hand and Gaze Signals During Reaching: Activity in Parietal Area 7m of the Monkey
Ferraina, S., Johnson, P. B., Garasto, M. R., Battaglia‐Mayer, A., Ercolani, L., Bianchi, L., Lacquaniti, F., & Caminiti, R. (1997) · Journal of Neurophysiology · doi:10.1152/jn.1997.77.2.1034 · Cited by 117 · cited 1×
“There is considerable evidence for multimodal, reaching-related signals across several brain areas in the posterior parietal cortex, including Area 5d, MIP, VIP, V6, and Area 7 [33–38].”
BackgroundMcGuire, L. M., & Sabes, P. N. (2011) · Journal of Neuroscience · doi:10.1523/jneurosci.2921-10.2011 · Cited by 113 · cited 4×
“Furthermore, although none of these 16 cells exhibited bimodal tuning for this arc, from the distribution of planar tuning we expect that some cells would; and indeed a subset of cells in [29] exhibit bimodal tuning (see Fig.”
BackgroundMethodInfluentialSelf-citationVisual-Shift Adaptation Is Composed of Separable Sensory and Task-Dependent Effects
Simani, M. C., McGuire, L. M. M., & Sabes, P. N. (2007) · Journal of Neurophysiology · doi:10.1152/jn.00290.2007 · Cited by 113 · cited 1×
“Prism and virtualfeedback adapation experiments [8–12] have demonstrated the plasticity of these multisensory mappings, and it is not likely limited to recalibration: Deprivation studies [13]; afferent rerouting experiments [14,15]; the ability to learn novel, cross-modal mappings; and genetic-information constraints…”
BackgroundSelf-citationNew Method for Parameter Estimation in Probabilistic Models: Minimum Probability Flow
Sohl-Dickstein, J., Battaglino, P. B., & DeWeese, M. R. (2011) · Physical Review Letters · doi:10.1103/physrevlett.107.220601 · Cited by 112
Vision calibrates sound localization in developing barn owls
Knudsen, E., & Knudsen, P. (1989) · The Journal of Neuroscience · doi:10.1523/jneurosci.09-09-03306.1989 · Cited by 112 · cited 1×
“Prism and virtualfeedback adapation experiments [8–12] have demonstrated the plasticity of these multisensory mappings, and it is not likely limited to recalibration: Deprivation studies [13]; afferent rerouting experiments [14,15]; the ability to learn novel, cross-modal mappings; and genetic-information constraints…”
BackgroundCoding of the Reach Vector in Parietal Area 5d
Bremner, L. R., & Andersen, R. A. (2012) · Neuron · doi:10.1016/j.neuron.2012.03.041 · Cited by 88
The ‘Ideal Homunculus’: Statistical Inference from Neural Population Responses
Földiák, P. (1993) · Springer eBooks · doi:10.1007/978-1-4615-3254-5_9 · Cited by 83 · cited 1×
“Of course, in theory one can always compute a posterior over the stimulus given some population activities [48]; but whether the posterior conditioned on activities deep in the hierarchy matches that conditioned on the activity in early sensory cortices, as in our model, is unknown.”
ResultSensory integration for reaching
Sabes, P. N. (2011) · Progress in brain research · doi:10.1016/b978-0-444-53752-2.00004-7 · Cited by 71 · cited 1×
“We have previously argued that having a collection of such representations allows for the flexible and (nearly) optimal use of a wide range of sensory inputs [45].”
BackgroundSelf-citationComputing with Continuous Attractors: Stability and Online Aspects
Wu, S., & Амари, Ш.-И. (2005) · Neural Computation · doi:10.1162/0899766054615626 · Cited by 69 · cited 1×
“Whether prior distributions in the brain are encoded in synaptic weights [46,47], as a separate neural population [2], or something else again, remains an area of active research (see also Text S1).”
BackgroundFactorial Learning and the EM Algorithm
Ghahramani, Z. (1994) · Cited by 67
Natural Image Coding in V1: How Much Use Is Orientation Selectivity?
Eichhorn, J., Sinz, F., & Bethge, M. (2009) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000336 · Cited by 62
Bayesian inference with probabilistic population codes.
Ma, W., Beck, J., Latham, P., & Pouget, A. (2006) · UCL Discovery (University College London) · Cited by 47
Learning Cross-Modal Spatial Transformations through Spike Timing-Dependent Plasticity
Davison, A. P., & Frégnac, Y. (2006) · Journal of Neuroscience · doi:10.1523/jneurosci.5263-05.2006 · Cited by 45 · cited 2×
“Other models of learning [24–26,55]”
BackgroundMinimum Probability Flow Learning
Sohl‐Dickstein, J., Battaglino, P., & DeWeese, M. R. (2009) · arXiv (Cornell University) · doi:10.48550/arxiv.0906.4779 · Cited by 45 · cited 1×
“This rule approximates gradient descent on an objective function for density estimation (Hinton’s ‘‘contrastive divergence’’ [17], or alternatively ‘‘probability flow’’ [64]).”
A Rational Analysis of the Acquisition of Multisensory Representations
Yildirim, I., & Jacobs, R. A. (2011) · Cognitive Science · doi:10.1111/j.1551-6709.2011.01216.x · Cited by 32 · cited 1×
“More recently, the idea of treating multisensory integration as a density estimation problem has been proposed independently by [53], a complementary report that explores both cognitive and neural implications of this view, without proposing an explicit neural implementation.”
BackgroundHamed, S. B., Duhamel, J., Bremmer, F., & Graf, W. M. (1997) · Springer eBooks · doi:10.1007/978-3-642-60661-8_21 · Cited by 15
Role of Afferent Activity in the Development of Cortical Specification
Lyckman, A. W., & Sur, M. (2002) · Results and problems in cell differentiation · doi:10.1007/978-3-540-46006-0_7 · Cited by 12
(2000) · doi:10.1002/1096-9861(20001204)428:1<112::aid-cne8>3.0.co;2-9 · cited 1×
“The fact that our network can be hierarchically composed is central to this hypothesis: these brain areas receive overlapping but distinct sets of inputs and with a rough hierarchical organization within them [39–43].”
Background
Cited by
A neural interface provides long-term stable natural touch perception
Tan, D. W., Schiefer, M. A., Keith, M. W., Anderson, J. R., Tyler, J., & Tyler, D. J. (2014) · Science Translational Medicine · doi:10.1126/scitranslmed.3008669 · Cited by 751
“Normal processing in the brain is highly tolerant of abnormal patterns and classifies patterns according to best matching prior sensory experiences (19).”
BackgroundBehavioral, Neural, and Computational Principles of Bodily Self-Consciousness
Blanke, O., Slater, M., & Serino, A. (2015) · Neuron · doi:10.1016/j.neuron.2015.09.029 · Cited by 698
Serino, A. (2019) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2019.01.016 · Cited by 304
“The neural mechanism allowing reference frame transformation system and the PPS multisensory integration system might interact, for instance, via intermediate neuronal layers between the unimodal and the multisensory layers, which re-code unisensory inputs into a common, body-part centered reference frame, necessary f…”
Grivaz, P., Blanke, O., & Serino, A. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2016.12.052 · Cited by 203
“…during the rubber hand illusion is the so-called proprioceptive drift, whereby participants feel their hand to be closer to the location where they see the rubber hand during synchronous visuo-tactile stimulation (see also Makin et al., 2013; Samad et al., 2015 for computational approaches).”
A learning-based approach to artificial sensory feedback leads to optimal integration
Dadarlat, M. C., O'Doherty, J. E., & Sabes, P. N. (2015) · Nature Neuroscience · doi:10.1038/nn.3883 · Cited by 210
Self-citation
Metadata
- Authors
- Makin, J. G., Fellows, M. R., & Sabes, P. N.
- Affiliations
- University of California, San Francisco (US)
- Corresponding author
- Makin, Joseph G.
- Venue
- PLoS Computational Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2013-04-18
- Volume
- 9
- Issue
- 4
- Pages
- e1003035
- ISSN
- 1553-734X (ISSN-L)
- 1553-7358
- PMC
- PMC3630212
- OpenAlex
- W2119669612
- Cited by
- 53
- References
- 71
- Citations by year
- 2026: 1 · 2025: 4 · 2024: 2 · 2023: 2 · 2022: 2 · 2021: 4 · 2020: 2 · 2019: 3 · 2017: 5 · 2016: 8 · 2015: 7 · 2014: 12 · 2013: 1
- FWCI
- 12.62
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS Computational Biology (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- eScholarship (California Digital Library) (submittedVersion)
- Figshare (submittedVersion, cc-by)
- Funders
- National Institutes of Health
- National Eye Institute (EY015679, R01EY015679)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Multisensory perception and integration
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural dynamics and brain function
- Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
- Keywords
- multisensory integration, coordinate transformation, density estimation, restricted Boltzmann machine, contrastive divergence, prior information, hierarchical integration, population activity, gaze position
- MeSH terms
- Algorithms, Animals, Brain, Cues, Humans, Learning, Models, Neurological, Photic Stimulation, Probability, Sensation, Visual Perception, Normal Distribution, Poisson Distribution
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.