Spatiotemporal energy models for the perception of motion
Abstract
A motion sequence may be represented as a single pattern in x-y-t space; a velocity of motion corresponds to a three-dimensional orientation in this space. Motion sinformation can be extracted by a system that responds to the oriented spatiotemporal energy. We discuss a class of models for human motion mechanisms in which the first stage consists of linear filters that are oriented in space-time and tuned in spatial frequency. The outputs of quadrature pairs of such filters are squared and summed to give a measure of motion energy. These responses are then fed into an opponent stage. Energy models can be built from elements that are consistent with known physiology and psychophysics, and they permit a qualitative understanding of a variety of motion phenomena.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Spatiotemporal Gabor filters have been used in several models of the response of motion selective neurons in the visual cortex.
“Spatiotemporal Gabor filters have been used in several models of the response of motion selective neurons in the visual cortex [57], [59], [39].”
From Motion Tuned Spatio-Temporal Quality Assessment of Natural Videos (2010) · cites this paper as [59]
STS images have been effectively used in a model of human visual-motion sensing, in energy models of motion perception, and in video motion analysis.
“The STS images have been effectively used in a model of human visual-motion sensing,22 in energy models of motion perception,23 and in video motion analysis.20,21”
From ViS3: an algorithm for video quality assessment via analysis of spatial and spatiotemporal slices (2014) · cites this paper as [23]
Cites
The contrast sensitivity of retinal ganglion cells of the cat
Enroth-Cugell, C., & Robson, J. G. (1966) · The Journal of Physiology · doi:10.1113/jphysiol.1966.sp008107 · Cited by 2,625 · see citing works
Spatial and temporal contrast sensitivity of striate cortical neurones
TOLHURST, D. J., & MOVSHON, J. A. (1975) · Nature · doi:10.1038/257674a0 · Cited by 167 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Performance of optical flow techniques
Barron, J. L., Fleet, D. J., & Beauchemin, S. S. (1994) · International Journal of Computer Vision · doi:10.1007/bf01420984 · Cited by 4,305 · see citing works
SlowFast Networks for Video Recognition
Feichtenhofer, C., Fan, H., Malik, J., & He, K. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00630 · Cited by 4,007 · see citing works
A Neural Algorithm of Artistic Style
Gatys, L., Ecker, A., & Bethge, M. (2016) · Journal of Vision · doi:10.1167/16.12.326 · Cited by 2,612 · see citing works
How does the brain solve visual object recognition?
DiCarlo, J. J., Zoccolan, D., & Rust, N. C. (2012) · Neuron · doi:10.1016/j.neuron.2012.01.010 · Cited by 1,790 · see citing works
Merging the senses into a robust percept
Ernst, M. O., & Bülthoff, H. H. (2004) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2004.02.002 · Cited by 1,906 · see citing works
The Normalization Model of Attention
Reynolds, J. H., & Heeger, D. J. (2009) · Neuron · doi:10.1016/j.neuron.2009.01.002 · Cited by 1,457 · see citing works
Multichannel texture analysis using localized spatial filters
Bovik, A. C., Clark, M., & Geisler, W. S. (1990) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.41384 · Cited by 1,382 · see citing works
Computation of component image velocity from local phase information
Fleet, D. J., & Jepson, A. D. (1990) · International Journal of Computer Vision · doi:10.1007/bf00056772 · Cited by 1,085 · see citing works
A computational framework and an algorithm for the measurement of visual motion
Anandan, P. (1989) · International Journal of Computer Vision · doi:10.1007/bf00158167 · Cited by 1,073 · see citing works
STRUCTURE AND FUNCTION OF VISUAL AREA MT
Born, R. T., & Bradley, D. C. (2005) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.26.041002.131052 · Cited by 1,023 · see citing works
Motion Tuned Spatio-Temporal Quality Assessment of Natural Videos
Seshadrinathan, K., & Bovik, A. C. (2010) · IEEE Transactions on Image Processing · doi:10.1109/tip.2009.2034992 · Cited by 708 · see citing works
Metamers of the ventral stream
Freeman, J., & Simoncelli, E. P. (2011) · Nature Neuroscience · doi:10.1038/nn.2889 · Cited by 667 · see citing works
Changes of mind in decision-making
Resulaj, A., Kiani, R., Wolpert, D. M., & Shadlen, M. N. (2009) · Nature · doi:10.1038/nature08275 · Cited by 678 · see citing works
Visual Response Properties of Striate Cortical Neurons Projecting to Area MT in Macaque Monkeys
Movshon, J. A., & Newsome, W. T. (1996) · The Journal of Neuroscience · doi:10.1523/jneurosci.16-23-07733.1996 · Cited by 651 · see citing works
Reconstructing Speech from Human Auditory Cortex
Pasley, B. N., David, S. V., Mesgarani, N., Flinker, A., Shamma, S. A., Crone, N. E., Knight, R. T., & Chang, E. F. (2012) · PLoS Biology · doi:10.1371/journal.pbio.1001251 · Cited by 604 · see citing works
Responses of neurons in macaque MT to stochastic motion signals
Britten, K. H., Shadlen, M. N., Newsome, W. T., & Movshon, J. A. (1993) · Visual Neuroscience · doi:10.1017/s0952523800010269 · Cited by 594 · see citing works
Huk, A. C., & Shadlen, M. N. (2005) · The Journal of Neuroscience · doi:10.1523/jneurosci.4684-04.2005 · Cited by 538 · see citing works
Optical flow using spatiotemporal filters
Heeger, D. J. (1988) · International Journal of Computer Vision · doi:10.1007/bf00133568 · Cited by 534 · see citing works
Kiani, R., Hanks, T. D., & Shadlen, M. N. (2008) · The Journal of Neuroscience · doi:10.1523/jneurosci.4761-07.2008 · Cited by 500 · see citing works
Choice Certainty Is Informed by Both Evidence and Decision Time
Kiani, R., Corthell, L., & Shadlen, M. N. (2014) · Neuron · doi:10.1016/j.neuron.2014.12.015 · Cited by 470 · see citing works
Motion selectivity and the contrast-response function of simple cells in the visual cortex
Albrecht, D. G., & Geisler, W. S. (1991) · Visual Neuroscience · doi:10.1017/s0952523800010336 · Cited by 433 · see citing works
Elapsed Decision Time Affects the Weighting of Prior Probability in a Perceptual Decision Task
Hanks, T. D., Mazurek, M. E., Kiani, R., Hopp, E., & Shadlen, M. N. (2011) · The Journal of Neuroscience · doi:10.1523/jneurosci.5613-10.2011 · Cited by 378 · see citing works
Perceiver: General Perception with Iterative Attention
Jaegle, A., Gimeno, F., Brock, A., Zisserman, A., Vinyals, O., & Carreira, J. (2021) · arXiv · doi:10.48550/arxiv.2103.03206 · Cited by 414 · see citing works
A Comparison of Visual and Auditory Motion Processing in Human Cerebral Cortex
Lewis, J. W. (2000) · Cerebral Cortex · doi:10.1093/cercor/10.9.873 · Cited by 381 · see citing works
Neuronal Adaptation to Visual Motion in Area MT of the Macaque
Kohn, A., & Movshon, J. A. (2003) · Neuron · doi:10.1016/s0896-6273(03)00438-0 · Cited by 353 · see citing works
Functional connectivity between simple cells and complex cells in cat striate cortex
Alonso, J.-M., & Martinez, L. M. (1998) · Nature Neuroscience · doi:10.1038/1609 · Cited by 278 · see citing works
A common mechanism underlies changes of mind about decisions and confidence
van den Berg, R., Anandalingam, K., Zylberberg, A., Kiani, R., Shadlen, M. N., & Wolpert, D. M. (2016) · eLife · doi:10.7554/elife.12192 · Cited by 270 · see citing works
McCane, B., Novins, K., Crannitch, D., & Galvin, B. (2001) · Computer Vision and Image Understanding · doi:10.1006/cviu.2001.0930 · Cited by 250 · see citing works
Deniz, F., Nunez-Elizalde, A. O., Huth, A. G., & Gallant, J. L. (2019) · The Journal of Neuroscience · doi:10.1523/jneurosci.0675-19.2019 · Cited by 214 · see citing works
“Motion-energy model construction A spatiotemporal Gabor pyramid was used to extract low-level visual features from the sequence of word frames used in the reading experiment (Adelson and Bergen, 1985; Watson and Ahumada, 1985).”
MethodMicrostimulation of visual cortex affects the speed of perceptual decisions
Ditterich, J., Mazurek, M. E., & Shadlen, M. N. (2003) · Nature Neuroscience · doi:10.1038/nn1094 · Cited by 234 · see citing works
Motion perception in schizophrenia
Chen, Y., Palafox, G. P., Nakayama, K., Levy, D. L., Matthysse, S., & Holzman, P. S. (1999) · Archives of General Psychiatry · doi:10.1001/archpsyc.56.2.149 · Cited by 225 · see citing works
Visuomotor behaviors in larval zebrafish after GFP-guided laser ablation of the optic tectum
Roeser, T., & Baier, H. (2003) · The Journal of Neuroscience · doi:10.1523/jneurosci.23-09-03726.2003 · Cited by 222 · see citing works
ViS3: an algorithm for video quality assessment via analysis of spatial and spatiotemporal slices
Vu, P. V., & Chandler, D. M. (2014) · Journal of Electronic Imaging · doi:10.1117/1.jei.23.1.013016 · Cited by 185 · see citing works
The timing of response onset and offset in macaque visual neurons
Bair, W., Cavanaugh, J. R., Smith, M. A., & Movshon, J. A. (2002) · The Journal of Neuroscience · doi:10.1523/jneurosci.22-08-03189.2002 · Cited by 170 · see citing works
Representation of accumulating evidence for a decision in two parietal areas
de Lafuente, V., Jazayeri, M., & Shadlen, M. N. (2015) · The Journal of Neuroscience · doi:10.1523/jneurosci.2451-14.2015 · Cited by 162 · see citing works
“We estimated the rate of this rise or decline by fitting lines to the response averages in the 150 ms epoch from the earliest point that these trends appear to emerge (see Materials and Methods).”
BackgroundInfluentialA computational perspective on autism
Rosenberg, A., Patterson, J. S., & Angelaki, D. E. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1510583112 · Cited by 156 · see citing works
A model of the ventral visual system based on temporal stability and local memory
Wyss, R., König, P., & Verschure, P. F. M. J. (2006) · PLoS Biology · doi:10.1371/journal.pbio.0040120 · Cited by 145 · see citing works
Lampl, I., Ferster, D., Poggio, T., & Riesenhuber, M. (2004) · Journal of Neurophysiology · doi:10.1152/jn.00060.2004 · Cited by 139 · see citing works
Orientation Selectivity Sharpens Motion Detection in Drosophila
Fisher, Y. E., Silies, M., & Clandinin, T. R. (2015) · Neuron · doi:10.1016/j.neuron.2015.09.033 · Cited by 117 · see citing works
Facial expression recognition using Gabor motion energy filters
Wu, T., Bartlett, M. S., & Movellan, J. R. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops · doi:10.1109/cvprw.2010.5543267 · Cited by 126 · see citing works
How are complex cell properties adapted to the statistics of natural stimuli?
Körding, K. P., Kayser, C., Einhäuser, W., & König, P. (2004) · Journal of Neurophysiology · doi:10.1152/jn.00149.2003 · Cited by 121 · see citing works
Gruntman, E., Romani, S., & Reiser, M. B. (2018) · Nature Neuroscience · doi:10.1038/s41593-017-0046-4 · Cited by 92 · see citing works
Unsupervised learning of invariant representations
Anselmi, F., Leibo, J. Z., Rosasco, L., Mutch, J., Tacchetti, A., & Poggio, T. (2016) · Theoretical Computer Science · doi:10.1016/j.tcs.2015.06.048 · Cited by 88 · see citing works
A visual motion sensor based on the properties of V1 and MT neurons
Perrone, J. A. (2004) · Vision Research · doi:10.1016/j.visres.2004.03.003 · Cited by 60 · see citing works
Estimating average single-neuron visual receptive field sizes by fMRI
Keliris, G. A., Li, Q., Papanikolaou, A., Logothetis, N. K., & Smirnakis, S. M. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1809612116 · Cited by 48 · see citing works
Nonlinear circuits for naturalistic visual motion estimation
Fitzgerald, J. E., & Clark, D. A. (2015) · eLife · doi:10.7554/elife.09123 · Cited by 52 · see citing works
A Neural Representation of Naturalistic Motion-Guided Behavior in the Zebrafish Brain
Yildizoglu, T., Riegler, C., Fitzgerald, J. E., & Portugues, R. (2020) · Current Biology · doi:10.1016/j.cub.2020.04.043 · Cited by 34 · see citing works
Predicting individual neuron responses with anatomically constrained task optimization
Mano, O., Creamer, M. S., Badwan, B. A., & Clark, D. A. (2021) · Current Biology · doi:10.1016/j.cub.2021.06.090 · Cited by 24 · see citing works
Multilevel visual motion opponency in Drosophila
Ammer, G., Serbe-Kamp, E., Mauss, A. S., Richter, F. G., Fendl, S., & Borst, A. (2023) · Nature Neuroscience · doi:10.1038/s41593-023-01443-z · Cited by 17 · see citing works
Neural mechanisms underlying sensitivity to reverse-phi motion in the fly
Leonhardt, A., Meier, M., Serbe, E., Eichner, H., & Borst, A. (2017) · PLOS ONE · doi:10.1371/journal.pone.0189019 · Cited by 16 · see citing works
- Authors
- Adelson, E. H., & Bergen, J. R.
- Affiliations
- RCA (United States) (US)
- ORCID
- Bergen, James R.
- Venue
- Journal of the Optical Society of America A
- Publisher
- Optica Publishing Group
- Published
- 1985-02-01
- Volume
- 2
- Issue
- 2
- Pages
- 284
- ISSN
- 1084-7529 (ISSN-L)
- 1520-8532
- OpenAlex
- W2108992228
- Cited by
- 3,442
- References
- 2
- Citations by year
- 2026: 59 · 2025: 67 · 2024: 50 · 2023: 66 · 2022: 68 · 2021: 67 · 2020: 91 · 2019: 102 · 2018: 110 · 2017: 99 · 2016: 101 · 2015: 98 · 2014: 123 · 2013: 89 · 2012: 118
- FWCI
- 19.72
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of the Optical Society of America A (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Visual perception and processing mechanisms
- Life Sciences › Neuroscience › Cognitive Neuroscience › Motor Control and Adaptation
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Keywords
- motion perception, motion energy, linear filters, spatial frequency tuning, quadrature filters, opponent processes
- MeSH terms
- Humans, Models, Psychological, Motion Perception, Orientation, Space Perception, Time Perception
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
