CoChat
Sign Up
Cited by 11Image and Video Quality AssessmentImage Retrieval and Classification TechniquesDigital Media Forensic Detection

The DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.

A Combined Full-Reference Image Quality Assessment Method Based on Convolutional Activation Maps

Open access
Varga, Domonkos·2020·Algorithms
DOI 10.3390/a13120313
Open this paper in CoChat — free accountOpen PDF

Abstract

The goal of full-reference image quality assessment (FR-IQA) is to predict the perceptual quality of an image as perceived by human observers using its pristine (distortion free) reference counterpart. In this study, we explore a novel, combined approach which predicts the perceptual quality of a distorted image by compiling a feature vector from convolutional activation maps. More specifically, a reference-distorted image pair is run through a pretrained convolutional neural network and the activation maps are compared with a traditional image similarity metric. Subsequently, the resulting feature vector is mapped onto perceptual quality scores with the help of a trained support vector regressor. A detailed parameter study is also presented in which the design choices of the proposed method is explained. Furthermore, we study the relationship between the amount of training images and the prediction performance. Specifically, it is demonstrated that the proposed method can be trained with a small amount of data to reach high prediction performance. Our best proposal—called ActMapFeat—is compared to the state-of-the-art on six publicly available benchmark IQA databases, such as KADID-10k, TID2013, TID2008, MDID, CSIQ, and VCL-FER. Specifically, our method is able to significantly outperform the state-of-the-art on these benchmark databases.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3390/a13120313

Cites

7 of 7 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Show all 7 references

Research works printed but not linked: 67

  1. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7–12). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA. · DOI · Paper page
  2. Oord, A.v.d., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A., and Kavukcuoglu, K. (2016). Wavenet: A generative model for raw audio. arXiv.
  3. Russakovsky, 2015, Imagenet large scale visual recognition challenge, Int. J. Comput. Vis., 115, 211, 10.1007/s11263-015-0816-y · DOI · Paper page
  4. Penatti, O.A., Nogueira, K., and Dos Santos, J.A. (2015, January 7–12). Do deep features generalize from everyday objects to remote sensing and aerial scenes domains?. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Boston, MA, USA. · DOI
  5. Bousetouane, F., and Morris, B. (2015). Off-the-shelf CNN features for fine-grained classification of vessels in a maritime environment. International Symposium on Visual Computing, Springer. · DOI
  6. Chou, 1995, A perceptually tuned subband image coder based on the measure of just-noticeable-distortion profile, IEEE Trans. Circuits Syst. Video Technol., 5, 467, 10.1109/76.475889 · DOI
  7. Daly, 1992, Visible differences predictor: An algorithm for the assessment of image fidelity, Human Vision, Visual Processing, and Digital Display III. International Society for Optics and Photonics, Volume 1666, 2
  8. Watson, 1997, Image quality and entropy masking, Human Vision and Electronic Imaging II. International Society for Optics and Photonics, Volume 3016, 2
  9. Chen, G.H., Yang, C.L., Po, L.M., and Xie, S.L. (2006, January 14–19). Edge-based structural similarity for image quality assessment. Proceedings of the 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings, Toulouse, France.
  10. Li, C., and Bovik, A.C. (2009, January 19–21). Three-component weighted structural similarity index. Proceedings of the Image Quality and System Performance VI, International Society for Optics and Photonics, San Jose, CA, USA. · DOI
  11. Liu, 2011, Visual attention in objective image quality assessment: Based on eye-tracking data, IEEE Trans. Circuits Syst. Video Technol., 21, 971, 10.1109/TCSVT.2011.2133770 · DOI
  12. Wang, 2010, Information content weighting for perceptual image quality assessment, IEEE Trans. Image Process., 20, 1185, 10.1109/TIP.2010.2092435 · DOI · Paper page
  13. Zhang, 2011, FSIM: A feature similarity index for image quality assessment, IEEE Trans. Image Process., 20, 2378, 10.1109/TIP.2011.2109730 · DOI
  14. Xue, 2013, Gradient magnitude similarity deviation: A highly efficient perceptual image quality index, IEEE Trans. Image Process., 23, 684, 10.1109/TIP.2013.2293423 · DOI · Paper page
  15. Reisenhofer, 2018, A Haar wavelet-based perceptual similarity index for image quality assessment, Signal Process. Image Commun., 61, 33, 10.1016/j.image.2017.11.001 · DOI
  16. Wang, Y., Liu, W., and Wang, Y. (2008, January 27–30). Color image quality assessment based on quaternion singular value decomposition. Proceedings of the 2008 Congress on Image and Signal Processing, Sanya, China. · DOI
  17. Kolaman, 2011, Quaternion structural similarity: A new quality index for color images, IEEE Trans. Image Process., 21, 1526, 10.1109/TIP.2011.2181522 · DOI
  18. Grega, 2010, Automated qualitative assessment of multi-modal distortions in digital images based on GLZ, Ann. Telecommun., 65, 3, 10.1007/s12243-009-0146-6 · DOI
  19. Liang, Y., Wang, J., Wan, X., Gong, Y., and Zheng, N. (2016). Image quality assessment using similar scene as reference. European Conference on Computer Vision, Springer. · DOI
  20. Kim, J., and Lee, S. (2017, January 21–26). Deep learning of human visual sensitivity in image quality assessment framework. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA. · DOI
  21. Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., and LeCun, Y. (2013). Overfeat: Integrated recognition, localization and detection using convolutional networks. arXiv.
  22. Zhang, R., Isola, P., Efros, A.A., Shechtman, E., and Wang, O. (2018, January 18–23). The unreasonable effectiveness of deep features as a perceptual metric. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA. · DOI · Paper page
  23. Pedersen, 2017, Image quality assessment by comparing cnn features between images, Electron. Imaging, 2017, 42
  24. Bosse, 2017, Deep neural networks for no-reference and full-reference image quality assessment, IEEE Trans. Image Process., 27, 206, 10.1109/TIP.2017.2760518 · DOI
  25. Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv.
  26. Varga, 2020, Composition-preserving deep approach to full-reference image quality assessment, Signal Image Video Process., 14, 1265, 10.1007/s11760-020-01664-w · DOI
  27. Okarma, K. (2010). Combined full-reference image quality metric linearly correlated with subjective assessment. International Conference on Artificial Intelligence and Soft Computing, Springer. · DOI
  28. Sheikh, 2006, Image information and visual quality, IEEE Trans. Image Process., 15, 430, 10.1109/TIP.2005.859378 · DOI · Paper page
  29. Mansouri, 2009, Image quality assessment using the singular value decomposition theorem, Opt. Rev., 16, 49, 10.1007/s10043-009-0010-y · DOI
  30. Okarma, 2012, Combined image similarity index, Opt. Rev., 19, 349, 10.1007/s10043-012-0055-1 · DOI
  31. Okarma, 2013, Extended hybrid image similarity–combined full-reference image quality metric linearly correlated with subjective scores, Elektron. Elektrotechnika, 19, 129
  32. Oszust, 2017, Image quality assessment with lasso regression and pairwise score differences, Multimed. Tools Appl., 76, 13255, 10.1007/s11042-016-3755-x · DOI
  33. Yuan, 2015, Image quality assessment: A sparse learning way, Neurocomputing, 159, 227, 10.1016/j.neucom.2015.01.066 · DOI
  34. Lukin, 2015, Combining full-reference image visual quality metrics by neural network, Human Vision and Electronic Imaging XX, Volume 9394, 93940K
  35. Oszust, M. (2016). Full-reference image quality assessment with linear combination of genetically selected quality measures. PLoS ONE, 11. · DOI · Paper page
  36. Amirshahi, S.A., Pedersen, M., and Beghdadi, A. (2018, January 12–16). Reviving Traditional Image Quality Metrics Using CNNs. Proceedings of the Color and Imaging Conference, Society for Imaging Science and Technology, Vancouver, BC, Canada. · DOI
  37. Lin, H., Hosu, V., and Saupe, D. (2020). DeepFL-IQA: Weak Supervision for Deep IQA Feature Learning. arXiv.
  38. Gao, 2018, Blind image quality prediction by exploiting multi-level deep representations, Pattern Recognit., 81, 432, 10.1016/j.patcog.2018.04.016 · DOI
  39. Lin, H., Hosu, V., and Saupe, D. (2018). KonIQ-10K: Towards an ecologically valid and large-scale IQA database. arXiv.
  40. Ponomarenko, 2009, TID2008-a database for evaluation of full-reference visual quality assessment metrics, Adv. Mod. Radioelectron., 10, 30
  41. Hlevnjak, 2012, VCL@ FER image quality assessment database, AUTOMATIKA Časopis Autom. Mjer. Elektron. Računarstvo Komun., 53, 344
  42. Sun, 2017, MDID: A multiply distorted image database for image quality assessment, Pattern Recognit., 61, 153, 10.1016/j.patcog.2016.07.033 · DOI
  43. Larson, 2010, Most apparent distortion: Full-reference image quality assessment and the role of strategy, J. Electron. Imaging, 19, 011006, 10.1117/1.3267105 · DOI · Paper page
  44. Hii, Y.L., See, J., Kairanbay, M., and Wong, L.K. (2017, January 17–20). Multigap: Multi-pooled inception network with text augmentation for aesthetic prediction of photographs. Proceedings of the 2017 IEEE International Conference on Image Processing (ICIP), Beijing, China. · DOI
  45. Drucker, H., Burges, C.J., Kaufman, L., Smola, A.J., and Vapnik, V. (1997). Support vector regression machines. Advances in Neural Information Processing Systems, MIT Press.
  46. Williams, C.K., and Rasmussen, C.E. (2006). Gaussian Processes for Machine Learning, MIT Press. · DOI · Paper page
  47. Ponomarenko, 2015, Image database TID2013: Peculiarities, results and perspectives, Signal Process. Image Commun., 30, 57, 10.1016/j.image.2014.10.009 · DOI · Paper page
  48. Sheikh, 2006, A statistical evaluation of recent full reference image quality assessment algorithms, IEEE Trans. Image Process., 15, 3440, 10.1109/TIP.2006.881959 · DOI
  49. Cho, J., Lee, K., Shin, E., Choy, G., and Do, S. (2015). How much data is needed to train a medical image deep learning system to achieve necessary high accuracy?. arXiv.
  50. Liu, 2011, Image quality assessment based on gradient similarity, IEEE Trans. Image Process., 21, 1500
  51. Nafchi, 2016, Mean deviation similarity index: Efficient and reliable full-reference image quality evaluator, IEEE Access, 4, 5579, 10.1109/ACCESS.2016.2604042 · DOI
  52. Temel, 2016, CSV: Image quality assessment based on color, structure, and visual system, Signal Process. Image Commun., 48, 92, 10.1016/j.image.2016.08.008 · DOI
  53. Balanov, A., Schwartz, A., Moshe, Y., and Peleg, N. (2015, January 27–30). Image quality assessment based on DCT subband similarity. Proceedings of the 2015 IEEE International Conference on Image Processing (ICIP), Quebec City, QC, Canada. · DOI
  54. Zhang, 2014, VSI: A visual saliency-induced index for perceptual image quality assessment, IEEE Trans. Image Process., 23, 4270, 10.1109/TIP.2014.2346028 · DOI
  55. Temel, D., and AlRegib, G. (2015, January 27–30). PerSIM: Multi-resolution image quality assessment in the perceptually uniform color domain. Proceedings of the 2015 IEEE International Conference on Image Processing (ICIP), Quebec City, QC, Canada. · DOI
  56. Temel, D., and AlRegib, G. (2016, January 11–15). BLeSS: Bio-inspired low-level spatiochromatic similarity assisted image quality assessment. Proceedings of the 2016 IEEE International Conference on Multimedia and Expo (ICME), Seattle, WA, USA. · DOI
  57. Temel, D., and AlRegib, G. (2016, January 25–28). ReSIFT: Reliability-weighted sift-based image quality assessment. Proceedings of the 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA. · DOI
  58. Prabhushankar, 2017, Ms-unique: Multi-model and sharpness-weighted unsupervised image quality estimation, Electron. Imaging, 2017, 30, 10.2352/ISSN.2470-1173.2017.12.IQSP-223 · DOI
  59. Yang, 2018, RVSIM: A feature similarity method for full-reference image quality assessment, EURASIP J. Image Video Process., 2018, 6, 10.1186/s13640-018-0246-1 · DOI
  60. Yu, 2019, Predicting the quality of images compressed after distortion in two steps, IEEE Trans. Image Process., 28, 5757, 10.1109/TIP.2019.2922850 · DOI
  61. Temel, 2019, Perceptual image quality assessment through spectral analysis of error representations, Signal Process. Image Commun., 70, 37, 10.1016/j.image.2018.09.005 · DOI
  62. Layek, M., Uddin, A., Le, T.P., Chung, T., and Huh, E.-N. (2019). Center-emphasized visual saliency and a contrast-based full reference image quality index. Symmetry, 11. · DOI
  63. Shi, C., and Lin, Y. (2020). Full Reference Image Quality Assessment Based on Visual Salience with Color Appearance and Gradient Similarity. IEEE Access. · DOI
  64. Ding, K., Ma, K., Wang, S., and Simoncelli, E.P. (2020). Image quality assessment: Unifying structure and texture similarity. arXiv. · DOI
  65. Liu, X., van de Weijer, J., and Bagdanov, A.D. (2017, January 22–29). Rankiqa: Learning from rankings for no-reference image quality assessment. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy. · DOI
  66. Pan, D., Shi, P., Hou, M., Ying, Z., Fu, S., and Zhang, Y. (2018, January 18–23). Blind predicting similar quality map for image quality assessment. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA. · DOI
  67. Yan, 2019, Naturalness-aware deep no-reference image quality assessment, IEEE Trans. Multimed., 21, 2603, 10.1109/TMM.2019.2904879 · DOI

Cited by

Sorted by citations

See all 11

Metadata

Authors
Varga, D.
Affiliations
Budapest University of Technology and Economics (HU)
Corresponding author
Varga, Domonkos
Venue
Algorithms
Publisher
MDPI AG
Published
2020-11-28
Volume
13
Issue
12
Pages
313
ISSN
1999-4893 (ISSN-L)
DOI
10.3390/a13120313
OpenAlex
W3092866836
Cited by
11
References
7
Citations by year
2025: 3 · 2024: 2 · 2023: 3 · 2022: 2 · 2021: 1
FWCI
0.58
Type
journal article
Language
en
Open access
Licence
Locations
  • Algorithms (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Video Quality Assessment
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
Keywords
full-reference image quality assessment, perceptual image quality, feature fusion, support vector regression, pre-trained convolutional neural networks, image similarity measures, training data size

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review