CoChat
Sign Up
Cited by 322Image and Video Quality AssessmentImage Retrieval and Classification TechniquesDigital Media Forensic Detection

Perceptual quality prediction on authentically distorted images using a bag of features approach

Open access
Ghadiyaram, Deepti, Bovik, Alan C.·2017·Journal of Vision
DOI 10.1167/17.1.32PMC PMC5283082
Open this paper in CoChat — free account

Abstract

Current top-performing blind perceptual image quality prediction models are generally trained on legacy databases of human quality opinion scores on synthetically distorted images. Therefore, they learn image features that effectively predict human visual quality judgments of inauthentic and usually isolated (single) distortions. However, real-world images usually contain complex composite mixtures of multiple distortions. We study the perceptually relevant natural scene statistics of such authentically distorted images in different color spaces and transform domains. We propose a "bag of feature maps" approach that avoids assumptions about the type of distortion(s) contained in an image and instead focuses on capturing consistencies-or departures therefrom-of the statistics of real-world images. Using a large database of authentically distorted images, human opinions of them, and bags of features computed on them, we train a regressor to conduct image quality prediction. We demonstrate the competence of the features toward improving automatic perceptual quality prediction by testing a learned algorithm using them on a benchmark legacy database as well as on a newly introduced distortion-realistic resource called the LIVE In the Wild Image Quality Challenge Database. We extensively evaluate the perceptual quality prediction model and algorithm and show that it is able to achieve good-quality prediction power that is better than other leading models.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1167/17.1.32

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The FRIQUEE model has been observed to achieve SOTA performance on UGC/consumer video/picture databases like LIVEChallenge, CVD2014, and KoNViD-1k.

    “The FRIQUEE model [41] has been observed to achieve SOTA performance both on UGC/consumer video/picture databases like LIVEChallenge [14], CVD2014 [12], and KoNViD-1k [10] by leveraging a bag of NSS features drawn from diverse color spaces and perceptually motivated transform domains.”

    From UGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Content (2021) · cites this paper as [41]

    The Experiments section describes FRIQUEE as using perceptually relevant statistical features across multiple domains, and ‘Comparing different IQA techniques’ reports performance significantly better than other top-performing models on LIVE Challenge (page not provided). The paper does not report FRIQUEE results on the named video databases, CVD2014 or KoNViD-1k, so it does not support the claim of state-of-the-art performance across all three databases. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 55

  1. Barland, R., & Saadane, A. (2006) Reference free quality metric using a region-based attention model for jpeg-2000 compressed images. Proc. SPIE, 6059, 605905– 1 –605905-10.
  2. Ben-Hur, A., & Weston, J. (2010) A user's guide to support vector machines. In XX (Ed.), Data mining techniques for the life sciences (pp. 223–239). Berlin: Springer. 20221922
  3. Bovik, A. C. (2013) Automatic prediction of perceptual image and video quality. Proc. IEEE, 101, 2008– 2024.
  4. Campbell, F. W., & Robson, J. G. (1968) Application of Fourier analysis to the visibility of gratings. The Journal of Physiology, 197 2, 551. 5666169 PMC1351748
  5. Chen, J., Zhang, Y., Liang, L., Ma, S., Wang, R., & Gao, W. (2008) A no-reference blocking artifacts metric using selective gradient and plainness measures. In XX (Ed.), Proc. Pacific Rim Conf. Multimedia, Adv. Multimedia Inf. Process (pp. 894–897). Berlin: Springer.
  6. Csurka, G., Dance, C. R., Fan, L., Willamowski, J., & Bray, C. (2004) Visual categorization with bags of keypoints. In XX (Ed.), Workshop on Statistical Learning in Computer Vision, ECCV (pp. 1–22).
  7. Ferzli, R., & Karam, L. J. (2009) A no-reference objective image sharpness metric based on the notion of just noticeable blur (jnb). IEEE Trans. Image Process, 18 4, 717– 728. 19278916
  8. Field, D. J. (1987) Relations between the statistics of natural images and the response properties of cortical cells. J. Opt. Soc. Am. A, 4, 2379– 2394 3430225
  9. Ghadiyaram, D., & Bovik, A. C. (2014) Crowdsourced study of subjective image quality. Asilomar Conf. Signals, Syst. Comput, 101, 2008– 2024.
  10. Ghadiyaram, D., & Bovik, A. C. (2015) Feature maps driven no-reference image quality prediction of authentically distorted images. In Proceedings of the SPIE conference on human vision and electronic imaging (pp. 93940J). Bellingham, WA: SPIE. PMC5283082 28129417
  11. Ghadiyaram, D., & Bovik, A. C. (2016) Massive online crowdsourced study of subjective and objective picture quality. IEEE Trans. on Image Proc, 25 1, 372– 387. 26571530
  12. Golestaneh, S. A., & Chandler, D. M. (2014) No-reference quality assessment of jpeg images via a quality relevance map. IEEE Sig. Proc. Letters, 21 2, 155– 158.
  13. Goodall, T. R., Bovik, A. C., & Paulter, N. G. (2016) Tasking on natural statistics of infrared images. IEEE Transactions on Image Processing, 25 1, 65– 79. 26540687
  14. Grauman, K., & Darrell, T. (2005) The pyramid match kernel: Discriminative classification with sets of image features. Int. Conf. Comp. Vision, 2, 1458– 1465.
  15. Heeger, D. J. (1992) Normalization of cell responses in cat striate cortex. Visual Neuroscience, 9 2, 181– 197. 1504027
  16. Hurvich, L. M., & Jameson, D. (1957) An opponent-process theory of color vision. Psychological Review, 64 6, 384– 404. 13505974
  17. Itti, L., Koch, C., & Niebur, E. (1998) A model of saliency-based visual attention for rapid scene analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20 11, 1254– 1259.
  18. Jayaraman, D., Mittal, A., Moorthy, A. K., & Bovik, A. C. (2012) Objective quality assessment of multiply distorted images. In XX (Ed.), Proc. Asilomar Conf. Signals, Syst. Comput (pp. 1693–1697). New York: IEEE.
  19. Kang, L., Ye, P., Li, Y., & Doermann, D. (2014) Convolutional neural networks for no-reference image quality assessment. In Proceedings of the international conference on computer vision and pattern recognition. New York: IEEE.
  20. Kang, L., Ye, P., Li, Y., & Doermann, D. (2015) Simultaneous estimation of image quality and distortion via multi-task convolutional neural networks. In 2015 IEEE international conference on image processing (ICIP) (pp. 2791–2795. New York: IEEE.
  21. Kuffler, S. W. (1953) Discharge patterns and functional organization of mammalian retina. Journal of Neurophysiology, 16 1, 37– 68. 13035466
  22. Larson, E. C., & Chandler, D. M. (2010) Most apparent distortion: Full-reference image quality assessment and the role of strategy. Journal of Electronic Imaging, 19 1, 011006.
  23. Lasmar, N. E., Stitou, Y., & Berthoumieu, Y. (2009) Multiscale skewed heavy tailed model for texture analysis. In IEEE international conference on image processing (pp. 2281–2284). New York: IEEE.
  24. Mittal, A., Soundararajan, R., & Bovik, A. C. (2012) Making a “completely blind” image quality analyzer. IEEE Sig. Proc. Letters, 20 3, 209– 212.
  25. Moorthy, A. K., & Bovik, A. C. (2010) A two-step framework for constructing blind image quality indices. IEEE Signal Processing Letters, 17 5, 513– 516.
  26. Moorthy, A. K., & Bovik, A. C. (2011) Blind image quality assessment: From natural scene statistics to perceptual quality. IEEE Trans. Image Process, 20 12, 3350– 3364. 21521667
  27. Narvekar, N. D., & Karam, L. J. (2009) A no-reference perceptual image sharpness metric based on a cumulative probability of blur detection. In IEEE international workshop on qualitative multimedia experience (pp. 87–91). New York: IEEE. 21447451
  28. Ponomarenko, N., Ieremeiev, O., Lukin, V., Egiazarian, K., Jin, L.,... Kuo, C.-C. J. (2013) Color image database tid2013: Peculiarities and preliminary results. In XX (Ed.), Proc. of 4th European Workshop on Visual Info (pp. 106–111). Location: Publisher.
  29. Rodieck, R. W. (1965) Quantitative analysis of cat retinal ganglion cell response to visual stimuli. Vision Research, 5 12, 583– 601. 5862581
  30. Saad, M., Bovik, A. C., & Charrier, C. (2012) Blind image quality assessment: A natural scene statistics approach in the DCT domain. IEEE Trans. Image Process, 21 8, 3339– 3352. 22453635
  31. Saad, M., Bovik, A. C., & Charrier, C. (2014) Blind prediction of natural video quality. IEEE Trans. on Image Proc, 23 3, 1352– 1365. 24723532
  32. Saha, A., & Wu, Q. M. J. (2015) Utilizing image scales towards totally training free blind image quality assessment. IEEE Trans. Image Process, 24 6, 1879– 1892. 25775489
  33. Sekuler, R., & Blake, R. (2002) Perception. New York, NY: McGraw Hill.
  34. Seshadrinathan, K., & Bovik, A. C. (2007) A structural similarity metric for video based on motion models. In IEEE international conference on acoustics, speech, and signal processing (pp. 1–869). New York: IEEE.
  35. Sharifi, K., & Leon-Garcia, A. (1995) Estimation of shape parameter for generalized gaussian distributions in subband decompositions of video. IEEE Trans. Circuits Syst. Video Technol, 5 1, 52– 56.
  36. Sheikh, H. R., Bovik, A. C., & Cormack, L. K. (2005) No-reference quality assessment using natural scene statistics: Jpeg2000. IEEE Trans. Image Process, 14 11, 1918– 1927. 16279189
  37. Sheikh, H. R., Sabir, M. F., & Bovik, A. C. (2006) A statistical evaluation of recent full reference image quality assessment algorithms. IEEE Trans. Image Process, 15 11, 3440– 3451. 17076403
  38. Sheskin, D. (2004) Handbook of parametric and nonparametric statistical procedures. In Proceedings of Asilomar conference on signals, systems, and computers. Boca Raton, FL: CRC Press.
  39. Soundararajan, R., & Bovik, A. C. (2013) Video quality assessment by reduced reference spatio-temporal entropic differencing. IEEE Trans. Circ. and Sys. Video Tech, 23 4, 684– 694.
  40. Srivastava, A., Lee, A. B., Simoncelli, E. P., & Zhu, S.-C. (2003) On advances in statistical modeling of natural images. Journal of Mathematical Imaging and Vision, 18 1, 17– 33.
  41. Tang, H., Joshi, N., & Kapoor, A. (2011) Learning a blind measure of perceptual image quality. In IEEE international conference on computational visual pattern recognition (pp. 305–312). New York: IEEE.
  42. Tang, H., Joshi, N., & Kapoor, A. (2014) Blind image quality assessment using semi-supervised rectifier networks. In IEEE international conference on computational visual pattern recognition (pp. 2877–2884). New York: IEEE.
  43. Varadarajan, S., & Karam, L. J. (2008a) An improved perception-based no-reference objective image sharpness metric using iterative edge refinement. In IEEE international conference on image processing (pp. 401–404). New York: IEEE.
  44. Varadarajan, S., & Karam, L. J. (2008b) A no-reference perceptual image sharpness metric based on saliency-weighted foveal pooling. In Proceedings of the IEEE international conference on image processing (pp. 369–372). New York: IEEE.
  45. Wainwright, M. J., Schwartz, O., & Simoncelli, E. P. (2002) Natural image statistics and divisive normalization: Modeling nonlinearities and adaptation in cortical neurons. In Statistical theories of the brain (pp. 203–222). Cambridge, MA: MIT Press.
  46. Wang, Z., Bovik, A., Sheikh, H., & Simoncelli, E. (2004) Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13 4, 600– 612. 15376593
  47. Wang, Z., Lu, L., & Bovik, A. C. (2004) Video quality assessment based on structural distortion measurement. Sig. Proc. Img. Comm, 19 2, 121– 132.
  48. Wang, Z., Simoncelli, E. P., & Bovik, A. (2003) Multiscale structural similarity for image quality assessment. Proc. Asilomar Conf. Signals, Syst. Comput, 2, 1398– 1402.
  49. Wilson, H. R., & Bergen, J. R. (1979) A four mechanism model for threshold spatial vision. Vision Research, 19 1, 19– 32. 419698
  50. Xu, L., Lin, W., & Kuo, C.-C. J. (2015) Visual quality assessment by machine learning. Berlin, Germany: Springer.
  51. Ye, P., & Doermann, D. (2011) No-reference image quality assessment using visual codebooks. In IEEE international conference on image processing (pp. 3129–3138). New York: IEEE. 22410336
  52. Zhang, L., Zhang, L., & Bovik, A. C. (2015) A feature-enriched completely blind image quality evaluator. IEEE Trans. on Image Proc. 25915960
  53. Zhang, Y., & Chandler, D. (2013) No-reference image quality assessment based on log-derivative statistics of natural scenes. Journal of Electronic Imaging, 22 4, 43025.
  54. Zhang, Y., Moorthy, A. K., Chandler, D., & Bovik, A. (2014) C-DIIVINE: No-reference image quality assessment based on local magnitude and phase statistics of natural scenes. Sig. Proc. Image Commun, 29 7, 725– 747.
  55. Zhu, T., & Karam, L. J. (2014) A no-reference objective image quality metric based on perceptually weighted local noise. EURASIP Journal of Image and Video Processing, 4 1, 1– 8.

Cited by

Sorted by citations

Authors
Ghadiyaram, D., & Bovik, A. C.
Affiliations
  • The University of Texas at Austin (US)
  • bovik@ece.utexas.eduhttp://live.ece.utexas.edu/bovik.php
ORCID
Venue
Journal of Vision
Publisher
Association for Research in Vision and Ophthalmology (ARVO)
Published
2017-01-27
Volume
17
Issue
1
Pages
32
ISSN
1534-7362 (ISSN-L)
DOI
10.1167/17.1.32
PMC
PMC5283082
OpenAlex
W2521359813
Cited by
322
References
1
Citations by year
2026: 11 · 2025: 29 · 2024: 37 · 2023: 45 · 2022: 46 · 2021: 44 · 2020: 40 · 2019: 36 · 2018: 21 · 2017: 13
FWCI
12.49
Type
journal article
Language
en
Open access
Licence
Locations
  • Journal of Vision (publishedVersion, cc-by-nc-nd)
  • PubMed Central (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Video Quality Assessment
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
Keywords
no-reference image quality assessment, perceptual quality prediction, natural scene statistics, color space, transform domain
MeSH terms
Algorithms, Humans, Image Interpretation, Computer-Assisted, Models, Theoretical, Pattern Recognition, Automated, Perceptual Distortion, Photography, Reproducibility of Results, Databases, Factual

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review