The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
A Novel Multiresolution Spatiotemporal Saliency Detection Model and Its Applications in Image and Video Compression
Abstract
Salient areas in natural scenes are generally regarded as areas which the human eye will typically focus on, and finding these areas is the key step in object detection. In computer vision, many models have been proposed to simulate the behavior of eyes such as SaliencyToolBox (STB), Neuromorphic Vision Toolkit (NVT), and others, but they demand high computational cost and computing useful results mostly relies on their choice of parameters. Although some region-based approaches were proposed to reduce the computational complexity of feature maps, these approaches still were not able to work in real time. Recently, a simple and fast approach called spectral residual (SR) was proposed, which uses the SR of the amplitude spectrum to calculate the image's saliency map. However, in our previous work, we pointed out that it is the phase spectrum, not the amplitude spectrum, of an image's Fourier transform that is key to calculating the location of salient areas, and proposed the phase spectrum of Fourier transform (PFT) model. In this paper, we present a quaternion representation of an image which is composed of intensity, color, and motion features. Based on the principle of PFT, a novel multiresolution spatiotemporal saliency detection model called phase spectrum of quaternion Fourier transform (PQFT) is proposed in this paper to calculate the spatiotemporal saliency map of an image by its quaternion representation. Distinct from other models, the added motion dimension allows the phase spectrum to represent spatiotemporal saliency in order to perform attention selection not only for images but also for videos. In addition, the PQFT model can compute the saliency map of an image under various resolutions from coarse to fine. Therefore, the hierarchical selectivity (HS) framework based on the PQFT model is introduced here to construct the tree structure representation of an image. With the help of HS, a model called multiresolution wavelet domain foveation (MWDF) is proposed in this paper to improve coding efficiency in image and video compression. Extensive tests of videos, natural images, and psychological patterns show that the proposed PQFT model is more effective in saliency detection and can predict eye fixations better than other state-of-the-art models in previous literature. Moreover, our model requires low computational cost and, therefore, can work in real time. Additional experiments on image and video compression show that the HS-MWDF model can achieve higher compression rate than the traditional model.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
James, W. (1913) · Henry Holt and Company eBooks · doi:10.1037/11059-000 · Cited by 25,226
A feature-integration theory of attention.
Treisman, A. M., & Gelade, G. (1980) · Cognitive Psychology · doi:10.1016/0010-0285(80)90005-5 · Cited by 12,366
A model of saliency-based visual attention for rapid scene analysis
Itti, L., Koch, C., & Niebur, E. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.730558 · Cited by 11,271
Horn, B. K. P., & Schunck, B. G. (1981) · Artificial Intelligence · doi:10.1016/0004-3702(81)90024-2 · Cited by 10,312
Horn, B. K., & Schunck, B. G. (1981) · SPIE Proceedings · doi:10.1117/12.965761 · Cited by 7,394
The principles of psychology, Vol I.
James, W. (1890) · doi:10.1037/10538-000 · Cited by 4,514
Shifts in Selective Visual Attention: Towards the Underlying Neural Circuitry
Koch, C., & Ullman, S. (1987) · Matters of Intelligence · doi:10.1007/978-94-009-3833-5_5 · Cited by 3,722
Guided Search 2.0 A revised model of visual search
Wolfe, J. M. (1994) · Psychonomic Bulletin & Review · doi:10.3758/bf03200774 · Cited by 3,504
Harel, J., Koch, C., & Perona, P. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0073 · Cited by 3,439
Saliency Detection: A Spectral Residual Approach
Hou, X., & Zhang, L. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383267 · Cited by 3,422
Self-citationA saliency-based search mechanism for overt and covert shifts of visual attention
Itti, L., & Koch, C. (2000) · Vision Research · doi:10.1016/s0042-6989(99)00163-7 · Cited by 3,156
Components of visual orienting
Posner, M. I. (1984) · Attention and performance · Cited by 2,980
High Accuracy Optical Flow Estimation Based on a Theory for Warping
Brox, T., Bruhn, A., Papenberg, N., & Weickert, J. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-24673-2_3 · Cited by 2,570
Feature analysis in early vision: Evidence from search asymmetries.
Treisman, A., & Gormican, S. (1988) · Psychological Review · doi:10.1037//0033-295x.95.1.15 · Cited by 2,257
Object class recognition by unsupervised scale-invariant learning
Fergus, R., Perona, P., & Zisserman, A. (2003) · 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings. · doi:10.1109/cvpr.2003.1211479 · Cited by 2,023
Saliency Based on Information Maximization
Bruce, N. D. B., & Tsotsos, J. K. (2005) · neural information processing systems · Cited by 1,307
Modeling attention to salient proto-objects
Walther, D., & Koch, C. (2006) · Neural Networks · doi:10.1016/j.neunet.2006.10.001 · Cited by 1,249
Bayesian surprise attracts human attention
Itti, L., & Baldi, P. (2009) · Vision Research · doi:10.1016/j.visres.2008.09.007 · Cited by 1,224
(1995) · Medical Entomology and Zoology · Cited by 1,212
Modeling visual attention via selective tuning
Tsotsos, J. K., Culhane, S. M., Kei Wai, W. Y., Lai, Y., Davis, N., & Nuflo, F. (1995) · Artificial Intelligence · doi:10.1016/0004-3702(95)00025-9 · Cited by 1,143
Objects and attention: the state of the art
Scholl, B. J. (2001) · Cognition · doi:10.1016/s0010-0277(00)00152-9 · Cited by 971
Learning to Detect A Salient Object
Liu, T., Sun, J., Zheng, N.-N., Tang, X., & Shum, H.-Y. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383047 · Cited by 946
Automatic Foveation for Video Compression Using a Neurobiological Model of Visual Attention
Itti, L. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2004.834657 · Cited by 759
Spatio-temporal Saliency detection using phase spectrum of quaternion fourier transform
Guo, C., Ma, Q., & Zhang, L. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587715 · Cited by 754
Self-citationVisual correlates of fixation selection: effects of scale and time
Tatler, B. W., Baddeley, R. J., & Gilchrist, I. D. (2005) · Vision Research · doi:10.1016/j.visres.2004.09.017 · Cited by 700
A coherent computational approach to model bottom-up visual attention
Le Meur, O., Le Callet, P., Barba, D., & Thoreau, D. (2006) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2006.86 · Cited by 642
Is bottom-up attention useful for object recognition?
Rutishauser, U., Walther, D., Koch, C., & Perona, P. (2004) · Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004. · doi:10.1109/cvpr.2004.1315142 · Cited by 592
Algorithms for defining visual regions-of-interest: comparison with eye fixations
Privitera, C. M., & Stark, L. W. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.877520 · Cited by 535
Seeing, sensing, and scrutinizing
Rensink, R. A. (2000) · Vision Research · doi:10.1016/s0042-6989(00)00003-1 · Cited by 484
Jähne, B. (1995) · Springer eBooks · doi:10.1007/978-3-662-03174-2 · Cited by 457
Gale, A., & Edwards, J. A. (1983) · Academic Press eBooks · Cited by 394
A Principled Approach to Detecting Surprising Events in Video
Itti, L., & Baldi, P. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.40 · Cited by 376
Colour tuning in human visual cortex measured with functional magnetic resonance imaging
Engel, S., Zhang, X., & Wandell, B. (1997) · Nature · doi:10.1038/40398 · Cited by 369
An Integrated Model of Top-Down and Bottom-Up Attention for Optimizing Detection Speed
Navalpakkam, V., & Itti, L. (2006) · 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Volume 2 (CVPR'06) · doi:10.1109/cvpr.2006.54 · Cited by 369
Fourier transforms of colour images usingquaternion or hypercomplex, numbers
Sangwine, S. J. (1996) · Electronics Letters · doi:10.1049/el:19961331 · Cited by 353
VOCUS: A Visual Attention System for Object Detection and Goal-Directed Search
Frintrop, S. (2006) · Lecture Notes in Computer Science · doi:10.1007/11682110 · Cited by 345
Object-based visual attention for computer vision
Sun, Y., & Fisher, R. (2003) · Artificial Intelligence · doi:10.1016/s0004-3702(02)00399-5 · Cited by 295
Models of bottom-up and top-down visual attention
Koch, C., & Itti, L. (2000) · doi:10.7907/md7v-ne41 · Cited by 261
Attentional Selection for Object Recognition — A Gentle Way
Walther, D., Itti, L., Riesenhuber, M., Poggio, T., & Koch, C. (2002) · Lecture Notes in Computer Science · doi:10.1007/3-540-36181-2_47 · Cited by 218
Foveation scalable video coding with automatic fixation selection
Wang, Z., Lu, L., & Bovik, A. C. (2003) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.809015 · Cited by 196
The discriminant center-surround hypothesis for bottom-up saliency
Gao, D., Mahadevan, V., & Vasconcelos, N. M. (2007) · neural information processing systems · Cited by 195
Automatic identification of perceptually important regions in an image
Osberger, W. M., & Maeder, A. (2002) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.1998.711240 · Cited by 149
Data- and model-driven gaze control for an active-vision system
Backer, G., Mertsching, B., & Bollmann, M. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.977565 · Cited by 119
Fast and Robust Generation of Feature Maps for Region-Based Visual Attention
Aziz, M. Z., & Mertsching, B. (2008) · IEEE Transactions on Image Processing · doi:10.1109/tip.2008.919365 · Cited by 115
An estimator for visual attention through competitive novelty with application to image compression
Stentiford, F. W. M., & Heath, M. (2001) · UCL Discovery (University College London) · Cited by 90
Saliency of peripheral targets in gaze-contingent multiresolutional displays
Reingold, E. M., & Loschky, L. C. (2002) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03195478 · Cited by 36
Cited by
State-of-the-Art in Visual Attention Modeling
Borji, A., & Itti, L. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.89 · Cited by 1,817
Salient Object Detection: A Benchmark
Borji, A., Cheng, M.-M., Jiang, H., & Li, J. (2015) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2487833 · Cited by 1,364
Deeply Supervised Salient Object Detection with Short Connections
Hou, Q., Cheng, M.-M., Hu, X., Borji, A., Tu, Z., & Torr, P. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.563 · Cited by 1,143
Salient Object Detection in the Deep Learning Era: An In-Depth Survey
Wang, W., Lai, Q., Fu, H., Shen, J., Ling, H., & Yang, R. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3051099 · Cited by 811
Deeply Supervised Salient Object Detection with Short Connections
Hou, Q., Cheng, M.-M., Hu, X., Borji, A., Tu, Z., & Torr, P. H. S. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2815688 · Cited by 676
Metadata
- Authors
- Guo, C., & Zhang, L.
- Venue
- IEEE Transactions on Image Processing
- Published
- 2010-01
- Volume
- 19
- Issue
- 1
- Pages
- 185-198
- ISSN
- 1057-7149 (ISSN-L)
- 1941-0042
- OpenAlex
- W2098702446
- Cited by
- 934
- References
- 46
- FWCI
- 27.48
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.