Perceptual Hashing With Visual Content Understanding for Reduced-Reference Screen Content Image Quality Assessment
Abstract
Numerous screen content images (SCIs) have been produced to meet the needs of virtual desktop and remote display, which put forward a very urgent requirement for security and management of SCIs. Perceptual hashing is an effective way to deal with this issue. However, since SCIs are generally composed of pictures, graphics and texts, their intrinsic characteristics are different from those of natural images. Thus the previous hashing methods for natural images are not suitable for SCIs. In this article, we propose a perceptual hashing method for SCIs from the perspective of visual content understanding. Specifically, considering that the visual content understanding of SCIs mainly comes from textual regions, while the contours of text always have thinner width and higher contrast, it is decided to generate hash in the gradient field. An input screen image is first performed by some joint preprocessing operations. Then the maximum gradient magnitude and corresponding orientation information are extracted from three color channels R, G and B. Normalized histogram and local frequency coefficient features are further obtained from the maximum gradient magnitude. Finally, a hash sequence is constructed by statistics that are derived from extracted features. Experiments validated on three SCIs databases were conducted to evaluate classification between robustness and discrimination. Receiver operating characteristics (ROC) results demonstrate that the proposed method is superior to the state-of-the-art algorithms. Besides, SIQAD and SCID databases were leveraged to present the application in reduced-reference screen content image quality assessment, and comparisons show that our hashing could provide accurate predictions than other metrics.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584 · see citing works
Making a “Completely Blind” Image Quality Analyzer
Mittal, A., Soundararajan, R., & Bovik, A. C. (2013) · IEEE Signal Processing Letters · doi:10.1109/lsp.2012.2227726 · Cited by 6,766 · see citing works
Human gaze control during real-world scene perception
Henderson, J. (2003) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2003.09.006 · Cited by 1,409 · see citing works
Bayesian surprise attracts human attention
Itti, L., & Baldi, P. (2009) · Vision Research · doi:10.1016/j.visres.2008.09.007 · Cited by 1,224 · see citing works
Image statistics and the perception of surface qualities
Motoyoshi, I., Nishida, S., Sharan, L., & Adelson, E. H. (2007) · Nature · doi:10.1038/nature05724 · Cited by 590 · see citing works
A robust image authentication method distinguishing JPEG compression from malicious manipulation
Lin, C., & Chang, S.-F. (2001) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/76.905982 · Cited by 556 · see citing works
Venkatesan, R., Koon, S.-M., Jakubowski, M. H., & Moulin, P. (2002) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2000.899541 · Cited by 527 · see citing works
A robust content based digital signature for image authentication
Schneider, M., & Chang, S.-F. (2002) · Proceedings of 3rd IEEE International Conference on Image Processing · doi:10.1109/icip.1996.560425 · Cited by 421 · see citing works
Triplet-Based Deep Hashing Network for Cross-Modal Retrieval
Deng, C. D., Chen, Z., Liu, X., Gao, X., & Tao, D. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2821921 · Cited by 402 · see citing works
Perceptual Image Hashing Via Feature Points: Performance Evaluation and Tradeoffs
Monga, V., & Evans, B. L. (2006) · IEEE Transactions on Image Processing · doi:10.1109/tip.2006.881948 · Cited by 363 · see citing works
Saliency-Guided Quality Assessment of Screen Content Images
Gu, K., Wang, S., Yang, H., Lin, W., Zhai, G., Yang, X., & Zhang, W. (2016) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2016.2547343 · Cited by 303 · see citing works
Deep Sketch Hashing: Fast Free-Hand Sketch-Based Image Retrieval
Liu, L., Shen, F., Shen, Y., Liu, X., & Shao, L. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.247 · Cited by 289 · see citing works
Robust perceptual image hashing via matrix invariants
Kozat, S. S., Venkatesan, R., & Mihcak, M. K. (2005) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2004.1421855 · Cited by 256 · see citing works
Robust Image Hashing With Ring Partition and Invariant Vector Distance
Tang, Z., Zhang, X., Li, X., & Zhang, S. (2016) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2015.2485163 · Cited by 247 · see citing works
Perceptual Quality Assessment of Screen Content Images
Yang, H., Fang, Y., & Lin, W. (2015) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2465145 · Cited by 238 · see citing works
Robust Hashing for Image Authentication Using Zernike Moments and Local Features
Zhao, Y. Q., Wang, S., Zhang, X., & Yao, H. (2012) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2012.2223680 · Cited by 227 · see citing works
Robust Perceptual Image Hashing Based on Ring Partition and NMF
Tang, Z., Zhang, X., & Zhang, S. (2013) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2013.45 · Cited by 199 · see citing works
ESIM: Edge Similarity for Screen Content Image Quality Assessment
Ni, Z., Ma, L., Zeng, H., et al. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2718185 · Cited by 184 · see citing works
Quantization-based hashing: a general framework for scalable image and video retrieval
Song, J., Gao, L., Liu, L., Zhu, X., & Sebe, N. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2017.03.021 · Cited by 183 · see citing works
Virtualized Screen: A Third Element for Cloud–Mobile Convergence
Lu, Y., Li, S., & Shen, H. (2011) · IEEE Multimedia · doi:10.1109/mmul.2011.33 · Cited by 159 · see citing works
Zhou, W., Yu, L., Zhou, Y., Qiu, W., Wu, M., & Luo, T. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2794207 · Cited by 159 · see citing works
SSDH: Semi-Supervised Deep Hashing for Large Scale Image Retrieval
Zhang, J., & Peng, Y. (2017) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2017.2771332 · Cited by 155 · see citing works
Robust Image Hashing with Tensor Decomposition
Tang, Z., Chen, L., Zhang, X., & Zhang, S. (2019) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2018.2837745 · Cited by 134 · see citing works
A Visual Model-Based Perceptual Image Hash for Content Authentication
Wang, X., Pang, K., Zhou, X., Zhou, Y., Li, L., & Xue, J. (2015) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2015.2407698 · Cited by 128 · see citing works
Deep Asymmetric Pairwise Hashing
Shen, F., Gao, X., Liu, L., Yang, Y., & Shen, H. T. (2017) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3123266.3123345 · Cited by 123 · see citing works
A Gabor Feature-Based Quality Assessment Model for the Screen Content Images
Ni, Z., Zeng, H., Ma, L., Hou, J., Chen, J., & Ma, K.-K. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2839890 · Cited by 118 · see citing works
Reduced-Reference Quality Assessment of Screen Content Images
Wang, S., Gu, K., Zhang, X., Lin, W., Ma, S., & Gao, W. (2016) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2016.2602764 · Cited by 110 · see citing works
Large-Scale Video Retrieval Using Image Queries
Araújo, A. D. M., & Girod, B. (2017) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2017.2667710 · Cited by 94 · see citing works
Perceptual image hashing using center-symmetric local binary patterns
Davarzani, R., Mozaffari, S., & Yaghmaie, K. (2015) · Multimedia Tools and Applications · doi:10.1007/s11042-015-2496-6 · Cited by 91 · see citing works
Evaluating Quality of Screen Content Images Via Structural Variation Analysis
Gu, K., Qiao, J.-F., Min, X.-K., Yue, G., Lin, W.-S., & Thalmann, D. (2018) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2017.2771284 · Cited by 91 · see citing works
Robust Image Hashing Based on Random Gabor Filtering and Dithered Lattice Vector Quantization
Li, Y., Lu, Z., Zhu, C., & Niu, X. (2011) · IEEE Transactions on Image Processing · doi:10.1109/tip.2011.2171698 · Cited by 90 · see citing works
Quaternion-Based Image Hashing for Adaptive Tampering Localization
Yan, C.-P., Pun, C., & Yuan, X. (2016) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2016.2594136 · Cited by 88 · see citing works
Gradient Direction for Screen Content Image Quality Assessment
Ni, Z., Ma, L., Zeng, H., Cai, C., & Ma, K. (2016) · IEEE Signal Processing Letters · doi:10.1109/lsp.2016.2599294 · Cited by 87 · see citing works
Robust image hashing using non-uniform sampling in discrete Fourier domain
Qin, C., Chang, C.-C., & Tsou, P.-L. (2013) · Digital Signal Processing · doi:10.1016/j.dsp.2012.11.002 · Cited by 81 · see citing works
Robust image hashing with multidimensional scaling
Tang, Z., Huang, Z., Zhang, X., & Lao, H. (2017) · Signal Processing · doi:10.1016/j.sigpro.2017.02.008 · Cited by 80 · see citing works
Self-citationBlind Quality Metric of DIBR-Synthesized Images in the Discrete Wavelet Transform Domain
Wang, G., Wang, Z., Gu, K., Li, L., Xia, Z., & Wu, L. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2945675 · Cited by 80 · see citing works
Subjective and Objective Quality Assessment of Compressed Screen Content Images
Wang, S., Gu, K., Zhang, X., Lin, W., Zhang, L., Ma, S., & Gao, W. (2016) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2016.2598756 · Cited by 79 · see citing works
Saliency-induced reduced-reference quality index for natural scene and screen content images
Min, X., Gu, K., Zhai, G., Hu, M., & Yang, X. (2017) · Signal Processing · doi:10.1016/j.sigpro.2017.10.025 · Cited by 77 · see citing works
Distributed Adaptive Binary Quantization for Fast Nearest Neighbor Search
Liu, X., Li, Z., Deng, C. D., & Tao, D. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2729896 · Cited by 76 · see citing works
Chang, T.-H., & Li, Y. (2011) · ACM SIGCHI Conference on Human Factors in Computing Systems (CHI) · doi:10.1145/1978942.1979257 · Cited by 71 · see citing works
Robust image hashing via DCT and LLE
Tang, Z., Lao, H., Zhang, X., & Liu, K. (2016) · Computers & Security · doi:10.1016/j.cose.2016.07.006 · Cited by 56 · see citing works
Screen image quality assessment incorporating structural degradation measurement
Gu, K., Wang, S., Zhai, G., et al. (2015) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.2015.7168586 · Cited by 53 · see citing works
Perceptual Image Hashing with Weighted DWT Features for Reduced-Reference Image Quality Assessment
Tang, Z., Huang, Z., Yao, H., Zhang, X., Lv, C., & Yu, C. (2018) · The Computer Journal · doi:10.1093/comjnl/bxy047 · Cited by 52 · see citing works
Self-citationChen, W., Gu, K., Lin, W., Yuan, F., & Cheng, E. (2019) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2019.2890878 · Cited by 50 · see citing works
A high-performanance remote computing platform
Shen, H., Lu, Y., Wu, F. H., et al. (2009) · IEEE International Conference on Pervasive Computing and Communications · doi:10.1109/percom.2009.4912855 · Cited by 49 · see citing works
A Prediction Backed Model for Quality Assessment of Screen Content and 3-D Synthesized Images
Jakhetiya, V., Gu, K., Lin, W., Li, Q., & Jaiswal, S. P. (2017) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2017.2756666 · Cited by 43 · see citing works
SCID: A database for screen content images quality assessment
Ni, Z., Ma, L., Zeng, H., et al. (2017) · International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS) · doi:10.1109/ispacs.2017.8266580 · Cited by 41 · see citing works
Efficient Image Hashing with Geometric Invariant Vector Distance for Copy Detection
Liu, S., & Huang, Z. (2019) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3355394 · Cited by 41 · see citing works
Self-citationRobust hashing for image authentication using SIFT feature and quaternion Zernike moments
Ouyang, J., Liu, Y., & Shu, H. (2016) · Multimedia Tools and Applications · doi:10.1007/s11042-015-3225-x · Cited by 40 · see citing works
Image hashing with color vector angle
Tang, Z., Li, X., Zhang, X., et al. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.04.057 · Cited by 38 · see citing works
Kasaei, S., Deriche, M. A., & Boashash, B. (2002) · IEEE Transactions on Image Processing · doi:10.1109/tip.2002.802534 · Cited by 34 · see citing works
Robustness and Discrimination Oriented Hashing Combining Texture and Invariant Vector Distance
Huang, Z., & Liu, S. (2018) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3240508.3240690 · Cited by 24 · see citing works
Self-citationFQI: feature‐based reduced‐reference image quality assessment method for screen content images
Rahul, K., & Tiwari, A. K. (2019) · IET Image Processing · doi:10.1049/iet-ipr.2018.5496 · Cited by 20 · see citing works
Toward Accurate Quality Estimation of Screen Content Pictures With Very Sparse Reference Information
Xia, Z., Gu, K., Wang, S., Liu, H., & Kwong, S. (2019) · IEEE Transactions on Industrial Electronics · doi:10.1109/tie.2019.2905831 · Cited by 19 · see citing works
A High-Performance Remote Computing Platform.
Shen, H., Yan, L., Wu, F., et al. (2009) · Cited by 16 · see citing works
Digital image hashing using local histogram of Oriented Gradients
Setyawan, I., & Timotius, I. K. (2014) · doi:10.1109/iciteed.2014.7007903 · Cited by 8 · see citing works
Perceptual image hashing using block truncation coding and local binary pattern
Chen, X., Qin, C., & Ji, P. (2015) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.1109/apsipa.2015.7415392 · Cited by 8 · see citing works
Robust perceptual color image hashing using randomized hypercomplex matrix factorizations
Ghouti, L. (2017) · Multimedia Tools and Applications · doi:10.1007/s11042-017-5355-9 · Cited by 6 · see citing works
Histogram-Based Perceptual Image Hashing in the DWT Domain
Xiang, S. (2010) · International Conference on Multimedia Information Networking and Security · doi:10.1109/mines.2010.142 · Cited by 5 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Qin, C., Liu, E., Feng, G., & Zhang, X. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2020.3047142 · Cited by 78 · see citing works
Robust Image Hashing With Isomap and Saliency Map for Copy Detection
Liang, X., Tang, Z., Wu, J., Li, Z., & Zhang, X. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3139217 · Cited by 32 · see citing works
Deep Perceptual Hash Based on Hash Center for Image Copyright Protection
Sun, X., & Zhou, J. (2022) · IEEE Access · doi:10.1109/access.2022.3221980 · Cited by 20 · see citing works
“They further obtained Normalized histogram and local frequency coefficient features from the maximum gradient magnitude, which improved the processing ability of screen content images (SCIs) [25].”
BackgroundDocument-Image Perceptual Hashing for Content Authentication
Situ, X., Liu, T., Yao, H., Qin, C., & Zhang, X. (2025) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2025.3596854 · Cited by 2 · see citing works
- Authors
- Huang, Z., & Liu, S.
- Affiliations
- Tianjin University (CN)
- Venue
- IEEE Transactions on Circuits and Systems for Video Technology
- Published
- 2021-07
- Volume
- 31
- Issue
- 7
- Pages
- 2808-2823
- ISSN
- 1051-8215 (ISSN-L)
- 1558-2205
- OpenAlex
- W3089867232
- Cited by
- 56
- References
- 59
- Citations by year
- 2026: 2 · 2025: 10 · 2024: 11 · 2023: 11 · 2022: 12 · 2021: 9 · 2020: 1
- FWCI
- 3.21
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Circuits and Systems for Video Technology (publishedVersion)
- Funders
- Natural Science Foundation of China (61672375, 61170118)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Video Quality Assessment
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Steganography and Watermarking Techniques
- Keywords
- perceptual hashing, screen content images, reduced-reference image quality assessment, gradient field, normalized histogram, remote display
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
