The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.
Automatic Foveation for Video Compression Using a Neurobiological Model of Visual Attention
Abstract
We evaluate the applicability of a biologically-motivated algorithm to select visually-salient regions of interest in video streams for multiply-foveated video compression. Regions are selected based on a nonlinear integration of low-level visual cues, mimicking processing in primate occipital, and posterior parietal cortex. A dynamic foveation filter then blurs every frame, increasingly with distance from salient locations. Sixty-three variants of the algorithm (varying number and shape of virtual foveas, maximum blur, and saliency competition) are evaluated against an outdoor video scene, using MPEG-1 and constant-quality MPEG-4 (DivX) encoding. Additional compression radios of 1.1 to 8.5 are achieved by foveation. Two variants of the algorithm are validated against eye fixations recorded from four to six human observers on a heterogeneous collection of 50 video clips (over 45 000 frames in total). Significantly higher overlap than expected by chance is found between human and algorithmic foveations. With both variants, foveated clips are, on average, approximately half the size of unfoveated clips, for both MPEG-1 and MPEG-4. These results suggest a general-purpose usefulness of the algorithm in improving compression ratios of unconstrained video.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex
Hubel, D. H., & Wiesel, T. N. (1962) · The Journal of Physiology · doi:10.1113/jphysiol.1962.sp006837 · Cited by 13,680
A feature-integration theory of attention.
Treisman, A. M., & Gelade, G. (1980) · Cognitive Psychology · doi:10.1016/0010-0285(80)90005-5 · Cited by 12,366
A model of saliency-based visual attention for rapid scene analysis
Itti, L., Koch, C., & Niebur, E. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.730558 · Cited by 11,271
Self-citationPrincipal warps: thin-plate splines and the decomposition of deformations
Bookstein, F. L. (1989) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.24792 · Cited by 4,913
Embedded image coding using zerotrees of wavelet coefficients
Shapiro, J. M. (1993) · IEEE Transactions on Signal Processing · doi:10.1109/78.258085 · Cited by 4,815
Computational modelling of visual attention
Itti, L., & Koch, C. (2001) · Nature Reviews Neuroscience · doi:10.1038/35058500 · Cited by 4,733
Self-citationShifts in Selective Visual Attention: Towards the Underlying Neural Circuitry
Koch, C., & Ullman, S. (1987) · Matters of Intelligence · doi:10.1007/978-94-009-3833-5_5 · Cited by 3,722
IARBUS, A. L. (1967) · Medical Entomology and Zoology · Cited by 3,319
Yarbus, A. L. (1967) · doi:10.1007/978-1-4899-5379-7 · Cited by 3,304
A saliency-based search mechanism for overt and covert shifts of visual attention
Itti, L., & Koch, C. (2000) · Vision Research · doi:10.1016/s0042-6989(99)00163-7 · Cited by 3,156
Self-citationIntegration of Visual and Linguistic Information in Spoken Language Comprehension
Tanenhaus, M. K., Spivey-Knowlton, M. J., Eberhard, K. M., & Sedivy, J. C. (1995) · Science · doi:10.1126/science.7777863 · Cited by 2,929
DISCHARGE PATTERNS AND FUNCTIONAL ORGANIZATION OF MAMMALIAN RETINA
Kuffler, S. W. (1953) · Journal of Neurophysiology · doi:10.1152/jn.1953.16.1.37 · Cited by 2,609
Selective Attention Gates Visual Processing in the Extrastriate Cortex
Moran, J., & Desimone, R. (1985) · Science · doi:10.1126/science.4023713 · Cited by 2,433
Guided search: An alternative to the feature integration model for visual search.
Wolfe, J. M., Cave, K. R., & Franzel, S. L. (1989) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037//0096-1523.15.3.419 · Cited by 2,147
Selective Attention Gates Visual Processing in the Extrastriate Cortex
Moran, J., & Desimone, R. (2003) · Essential Sources in the Scientific Study of Consciousness · doi:10.7551/mitpress/2834.003.0024 · Cited by 2,144
Distance transformations in digital images
Borgefors, G. (1986) · Computer Vision, Graphics, and Image Processing · doi:10.1016/s0734-189x(86)80047-0 · Cited by 1,898
Feature-based attention influences motion processing gain in macaque visual cortex
Treue, S., & Trujillo, J. C. M. (1999) · Nature · doi:10.1038/21176 · Cited by 1,465
Modeling the role of salience in the allocation of overt visual attention
Parkhurst, D., Law, K., & Niebur, E. (2002) · Vision Research · doi:10.1016/s0042-6989(01)00250-4 · Cited by 1,388
(1995) · Medical Entomology and Zoology · Cited by 1,212
The Dynamic Representation of Scenes
Rensink, R. A. (2000) · Visual Cognition · doi:10.1080/135062800394667 · Cited by 1,078
Attention Increases Sensitivity of V4 Neurons
Reynolds, J. H., Pasternak, T., & Desimone, R. (2000) · Neuron · doi:10.1016/s0896-6273(00)81206-4 · Cited by 1,014
Henderson, J. M., & Hollingworth, A. (1999) · Annual Review of Psychology · doi:10.1146/annurev.psych.50.1.243 · Cited by 967
Attention improves or impairs visual performance by enhancing spatial resolution
Yeshurun, Y., & Carrasco, M. (1998) · Nature · doi:10.1038/23936 · Cited by 767
Framing pictures: The role of knowledge in automatized encoding and memory for gist.
Friedman, A. (1979) · Journal of Experimental Psychology: General · doi:10.1037//0096-3445.108.3.316 · Cited by 741
The brainstem control of saccadic eye movements
Sparks, D. L. (2002) · Nature Reviews Neuroscience · doi:10.1038/nrn986 · Cited by 717
Scanpaths in Eye Movements during Pattern Perception
Noton, D., & Stark, L. (1971) · Science · doi:10.1126/science.171.3968.308 · Cited by 645
Visual search, anticipation and expertise in soccer goalkeepers
Savelsbergh, G. J. P., Williams, A. M., Kamp, J. V. D., & Ward, P. (2002) · Journal of Sports Sciences · doi:10.1080/026404102317284826 · Cited by 616
Algorithms for defining visual regions-of-interest: comparison with eye fixations
Privitera, C. M., & Stark, L. W. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.877520 · Cited by 535
Recognition memory for a rapid sequence of pictures.
Potter, M. C., & Levy, E. I. (1969) · Journal of Experimental Psychology · doi:10.1037/h0027470 · Cited by 527
Neural correlates of attentive selection for color or luminance in extrastriate area V4
Motter, B. (1994) · The Journal of Neuroscience · doi:10.1523/jneurosci.14-04-02178.1994 · Cited by 482
Attention Modulates Contextual Influences in the Primary Visual Cortex of Alert Monkeys
Ito, M., & Gilbert, C. D. (1999) · Neuron · doi:10.1016/s0896-6273(00)80713-8 · Cited by 456
Feature combination strategies for saliency-based visual attention systems
Itti, L. (2001) · Journal of Electronic Imaging · doi:10.1117/1.1333677 · Cited by 441
Self-citationOliva, A., & Schyns, P. G. (1997) · Cognitive Psychology · doi:10.1006/cogp.1997.0667 · Cited by 423
Marcellin, M. W., Gormish, M. J., Bilgin, A., & Boliek, M. P. (2002) · Proceedings DCC 2000. Data Compression Conference · doi:10.1109/dcc.2000.838192 · Cited by 405
Real-time foveated multiresolution system for low-bandwidth video communication:
Geisler, W. S., & Perry, J. S. (1998) · SPIE Proceedings · doi:10.1117/12.320120 · Cited by 369
Realistic avatar eye and head animation using a neurobiological model of visual attention
Itti, L., Dhavale, N., & Pighin, F. (2004) · SPIE Proceedings · doi:10.1117/12.512618 · Cited by 289
Self-citationHeuristic filtering and reliable calibration methods for video-based pupil-tracking systems
Stampe, D. M. (1993) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03204486 · Cited by 276
Does consistent scene context facilitate object perception?
Hollingworth, A. (1998) · Journal of Experimental Psychology: General · doi:10.1037//0096-3445.127.4.398 · Cited by 267
Hollingworth, A., Williams, C. C., & Henderson, J. M. (2001) · Psychonomic Bulletin & Review · doi:10.3758/bf03196215 · Cited by 256
Modeling global scene factors in attention
Torralba, A. (2003) · Journal of the Optical Society of America A · doi:10.1364/josaa.20.001407 · Cited by 247
Attentional Selection for Object Recognition — A Gentle Way
Walther, D., Itti, L., Riesenhuber, M., Poggio, T., & Koch, C. (2002) · Lecture Notes in Computer Science · doi:10.1007/3-540-36181-2_47 · Cited by 218
Self-citationFoveation scalable video coding with automatic fixation selection
Wang, Z., Lu, L., & Bovik, A. C. (2003) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.809015 · Cited by 196
Automatic identification of perceptually important regions in an image
Osberger, W. M., & Maeder, A. J. (2002) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.1998.711240 · Cited by 146
Mr. Chips 2002: new insights from an ideal-observer model of reading
Legge, G. E., Hooven, T. A., Klitz, T. S., Mansfield, J. S., & Tjan, B. S. (2002) · Vision Research · doi:10.1016/s0042-6989(02)00131-1 · Cited by 135
Modeling the Effect of Task and Graphical Representation on Response Latency in a Graph Reading Task
Peebles, D., & Cheng, P. (2003) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.45.1.28.27225 · Cited by 125
Perceptual skill, radiology expertise, and visual test performance with NINA and WALDO
Nodine, C. F., & Krupinski, E. A. (1998) · Academic Radiology · doi:10.1016/s1076-6332(98)80295-x · Cited by 112
Global Transsaccadic Change Blindness During Scene Perception
Henderson, J. M., & Hollingworth, A. (2003) · Psychological Science · doi:10.1111/1467-9280.02459 · Cited by 102
Visual memory: What do you know about what you saw?
Wolfe, J. M. (1998) · Current Biology · doi:10.1016/s0960-9822(98)70192-7 · Cited by 101
A Goal Oriented Attention Guidance Model
Navalpakkam, V., & Itti, L. (2002) · Lecture notes in computer science · doi:10.1007/3-540-36181-2_45 · Cited by 92
Self-citationAn estimator for visual attention through competitive novelty with application to image compression
Stentiford, F. W. M., & Heath, M. (2001) · Cited by 84
Automatic detection of regions of interest in complex video sequences
Osberger, W. M., & Rohaly, A. M. (2001) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.429506 · Cited by 61
Scene perception: A failure to find a benefit from prior expectancy or familiarity.
Biederman, I., Teitelbaum, R. C., & Mezzanotte, R. J. (1983) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037//0278-7393.9.3.411 · Cited by 53
VISUAL SEARCH STRATEGIES IN EXPERIENCED AND INEXPERIENCED GYMNASTIC COACHES
Moreno, F. J., Reina, R., del Campo, V. L., & Sabido, R. (2002) · Perceptual and Motor Skills · doi:10.2466/pms.2002.95.3.901 · Cited by 51
Real-time data collection in Linux: A case study
Finney, S. A. (2001) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03195362 · Cited by 47
Saliency of peripheral targets in gaze-contingent multiresolutional displays
Reingold, E. M., & Loschky, L. C. (2002) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03195478 · Cited by 36
Maskell, S. R., Rollason, M. P., Gordon, N. J., et al. (2002) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.478509 · Cited by 28
Diepen, P. M. J. V., Graef, P. D., & Rensbergen, J. V. (1994) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03204665 · Cited by 22
Cited by
Saliency Detection via Graph-Based Manifold Ranking
Yang, C., Zhang, L., Lu, H., Ruan, X., & Yang, M.-H. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.407 · Cited by 2,554
Bayesian Saliency via Low and Mid Level Cues
Xie, Y., Lu, H., & Yang, M.-H. (2013) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2216276 · Cited by 367
Visual Importance Pooling for Image Quality Assessment
Moorthy, A. K., & Bovik, A. C. (2009) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2009.2015374 · Cited by 340
State-of-the-Art in 360° Video/Image Processing: Perception, Assessment and Compression
Xu, M., Li, C., Zhang, S., & Callet, P. L. (2020) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2020.2966864 · Cited by 281
Revisiting Video Saliency: A Large-Scale Benchmark and a New Model
Wang, W., Shen, J., Guo, F., Cheng, M.-M., & Borji, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00514 · Cited by 268
Metadata
- Authors
- Itti, L.
- Venue
- IEEE Transactions on Image Processing
- Published
- 2004-10
- Volume
- 13
- Issue
- 10
- Pages
- 1304-1318
- ISSN
- 1057-7149 (ISSN-L)
- OpenAlex
- W2130184867
- Cited by
- 759
- References
- 57
- FWCI
- 21.25
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.