The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
A Novel Multiresolution Spatiotemporal Saliency Detection Model and Its Applications in Image and Video Compression
Abstract
Salient areas in natural scenes are generally regarded as areas which the human eye will typically focus on, and finding these areas is the key step in object detection. In computer vision, many models have been proposed to simulate the behavior of eyes such as SaliencyToolBox (STB), Neuromorphic Vision Toolkit (NVT), and others, but they demand high computational cost and computing useful results mostly relies on their choice of parameters. Although some region-based approaches were proposed to reduce the computational complexity of feature maps, these approaches still were not able to work in real time. Recently, a simple and fast approach called spectral residual (SR) was proposed, which uses the SR of the amplitude spectrum to calculate the image's saliency map. However, in our previous work, we pointed out that it is the phase spectrum, not the amplitude spectrum, of an image's Fourier transform that is key to calculating the location of salient areas, and proposed the phase spectrum of Fourier transform (PFT) model. In this paper, we present a quaternion representation of an image which is composed of intensity, color, and motion features. Based on the principle of PFT, a novel multiresolution spatiotemporal saliency detection model called phase spectrum of quaternion Fourier transform (PQFT) is proposed in this paper to calculate the spatiotemporal saliency map of an image by its quaternion representation. Distinct from other models, the added motion dimension allows the phase spectrum to represent spatiotemporal saliency in order to perform attention selection not only for images but also for videos. In addition, the PQFT model can compute the saliency map of an image under various resolutions from coarse to fine. Therefore, the hierarchical selectivity (HS) framework based on the PQFT model is introduced here to construct the tree structure representation of an image. With the help of HS, a model called multiresolution wavelet domain foveation (MWDF) is proposed in this paper to improve coding efficiency in image and video compression. Extensive tests of videos, natural images, and psychological patterns show that the proposed PQFT model is more effective in saliency detection and can predict eye fixations better than other state-of-the-art models in previous literature. Moreover, our model requires low computational cost and, therefore, can work in real time. Additional experiments on image and video compression show that the HS-MWDF model can achieve higher compression rate than the traditional model.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
James, W. (1913) · Henry Holt and Company eBooks · doi:10.1037/11059-000 · Cited by 25,226
A feature-integration theory of attention.
Treisman, A. M., & Gelade, G. (1980) · Cognitive Psychology · doi:10.1016/0010-0285(80)90005-5 · Cited by 12,366
A model of saliency-based visual attention for rapid scene analysis
Itti, L., Koch, C., & Niebur, E. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.730558 · Cited by 11,271
Horn, B. K. P., & Schunck, B. G. (1981) · Artificial Intelligence · doi:10.1016/0004-3702(81)90024-2 · Cited by 10,312
Horn, B. K., & Schunck, B. G. (1981) · SPIE Proceedings · doi:10.1117/12.965761 · Cited by 7,394
The principles of psychology, Vol I.
James, W. (1890) · doi:10.1037/10538-000 · Cited by 4,514
Shifts in Selective Visual Attention: Towards the Underlying Neural Circuitry
Koch, C., & Ullman, S. (1987) · Matters of Intelligence · doi:10.1007/978-94-009-3833-5_5 · Cited by 3,722
Guided Search 2.0 A revised model of visual search
Wolfe, J. M. (1994) · Psychonomic Bulletin & Review · doi:10.3758/bf03200774 · Cited by 3,504
Harel, J., Koch, C., & Perona, P. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0073 · Cited by 3,439
Saliency Detection: A Spectral Residual Approach
Hou, X., & Zhang, L. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383267 · Cited by 3,422
Self-citation
Cited by
State-of-the-Art in Visual Attention Modeling
Borji, A., & Itti, L. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.89 · Cited by 1,817
Salient Object Detection: A Benchmark
Borji, A., Cheng, M.-M., Jiang, H., & Li, J. (2015) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2487833 · Cited by 1,364
Deeply Supervised Salient Object Detection with Short Connections
Hou, Q., Cheng, M.-M., Hu, X., Borji, A., Tu, Z., & Torr, P. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.563 · Cited by 1,143
Salient Object Detection in the Deep Learning Era: An In-Depth Survey
Wang, W., Lai, Q., Fu, H., Shen, J., Ling, H., & Yang, R. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3051099 · Cited by 811
Deeply Supervised Salient Object Detection with Short Connections
Hou, Q., Cheng, M.-M., Hu, X., Borji, A., Tu, Z., & Torr, P. H. S. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2815688 · Cited by 676
Non-local Deep Features for Salient Object Detection
Luo, Z., Mishra, A., Achkar, A., Eichel, J., Li, S., & Jodoin, P.-M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.698 · Cited by 621
Quantitative Analysis of Human-Model Agreement in Visual Saliency Modeling: A Comparative Study
Borji, A., Sihite, D. N., & Itti, L. (2013) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2210727 · Cited by 613
Generative Adversarial Networks for Extreme Learned Image Compression
Agustsson, E., Tschannen, M., Mentzer, F., Timofte, R., & Van Gool, L. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00031 · Cited by 575
Visual Saliency Based on Scale-Space Analysis in the Frequency Domain
Li, J., Levine, M. D., An, X., Xu, X., & He, H. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.147 · Cited by 574
Salient object detection: A survey
Borji, A., Cheng, M.-M., Hou, Q., Jiang, H., & Li, J. (2019) · Computational Visual Media · doi:10.1007/s41095-019-0149-9 · Cited by 524
Shifting More Attention to Video Salient Object Detection
Fan, D.-P., Wang, W., Cheng, M.-M., & Shen, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00875 · Cited by 510
The Seventh Visual Object Tracking VOT2019 Challenge Results
Kristan, M., Matas, J., Leonardis, A., Felsberg, M., Pflugfelder, R., Kamarainen, J.-K., Zajc, L. C., Drbohlav, O., Lukezic, A., Berg, A., et al. (2019) · 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW) · doi:10.1109/iccvw.2019.00276 · Cited by 461
Self-citationSalient Object Detection: A Benchmark
Borji, A., Sihite, D. N., & Itti, L. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33709-3_30 · Cited by 416
Exploiting local and global patch rarities for saliency detection
Borji, A., & Itti, L. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247711 · Cited by 376
Bayesian Saliency via Low and Mid Level Cues
Xie, Y., Lu, H., & Yang, M.-H. (2013) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2216276 · Cited by 367
JL-DCF: Joint Learning and Densely-Cooperative Fusion Framework for RGB-D Salient Object Detection
Fu, K., Fan, D.-P., Ji, G.-P., & Zhao, Q. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00312 · Cited by 365
ECFFNet: Effective and Consistent Feature Fusion Network for RGB-T Salient Object Detection
Zhou, W., Guo, Q., Lei, J., Yu, L., & Hwang, J.-N. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3077058 · Cited by 340
Revisiting Video Saliency Prediction in the Deep Learning Era
Wang, W., Shen, J., Xie, J., Cheng, M.-M., Ling, H., & Borji, A. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2924417 · Cited by 330
LSNet: Lightweight Spatial Boosting Network for Detecting Salient Objects in RGB-Thermal Images
Zhou, W., Zhu, Y., Lei, J., Yang, R., & Yu, L. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3242775 · Cited by 298
Gaze Prediction in Dynamic 360° Immersive Videos
Xu, Y., Dong, Y., Wu, J., Sun, Z., Shi, Z., Yu, J., & Gao, S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00559 · Cited by 273
Revisiting Video Saliency: A Large-Scale Benchmark and a New Model
Wang, W., Shen, J., Guo, F., Cheng, M.-M., & Borji, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00514 · Cited by 268
Boosting bottom-up and top-down visual features for saliency estimation
Borji, A. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247706 · Cited by 266
Multimodal Saliency and Fusion for Movie Summarization Based on Aural, Visual, and Textual Attention
Evangelopoulos, G., Zlatintsi, A., Potamianos, A., Maragos, P., Rapantzikos, K., Skoumas, G., & Avrithis, Y. (2013) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2013.2267205 · Cited by 263
The Eighth Visual Object Tracking VOT2020 Challenge Results
Kristan, M., Leonardis, A., Matas, J., Felsberg, M., Pflugfelder, R., Kämäräinen, J.-K., Danelljan, M., Zajc, L. Č., Lukežič, A., Drbohlav, O., et al. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-68238-5_39 · Cited by 259
Self-citationPredicting Head Movement in Panoramic Video: A Deep Reinforcement Learning Approach
Xu, M., Song, Y., Wang, J., Qiao, M., Huo, L., & Wang, Z. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2858783 · Cited by 259
RGB-T Salient Object Detection via Fusing Multi-Level CNN Features
Zhang, Q., Huang, N., Yao, L., Zhang, D., Shan, C., & Han, J. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2959253 · Cited by 247
Distributed and efficient object detection in edge computing: Challenges and solutions
Ren, J., Guo, Y., Zhang, D., Liu, Q., & Zhang, Y. (2018) · IEEE Network · doi:10.1109/mnet.2018.1700415 · Cited by 148
“[11] C.”
An effective computer aided diagnosis model for pancreas cancer on PET/CT images
Li, S., Jiang, H., Wang, Z., Zhang, G., & Yao, Y.-D. (2018) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2018.09.001 · Cited by 75
Metadata
- Authors
- Guo, C., & Zhang, L.
- Venue
- IEEE Transactions on Image Processing
- Published
- 2010-01
- Volume
- 19
- Issue
- 1
- Pages
- 185-198
- ISSN
- 1057-7149 (ISSN-L)
- 1941-0042
- OpenAlex
- W2098702446
- Cited by
- 934
- References
- 46
- FWCI
- 27.48
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.