Blind Image Quality Assessment via Deep Learning
Open accessThe DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
Abstract
This paper investigates how to blindly evaluate the visual quality of an image by learning rules from linguistic descriptions. Extensive psychological evidence shows that humans prefer to conduct evaluations qualitatively rather than numerically. The qualitative evaluations are then converted into the numerical scores to fairly benchmark objective image quality assessment (IQA) metrics. Recently, lots of learning-based IQA models are proposed by analyzing the mapping from the images to numerical ratings. However, the learnt mapping can hardly be accurate enough because some information has been lost in such an irreversible conversion from the linguistic descriptions to numerical scores. In this paper, we propose a blind IQA model, which learns qualitative evaluations directly and outputs numerical scores for general utilization and fair comparison. Images are represented by natural scene statistics features. A discriminative deep model is trained to classify the features into five grades, corresponding to five explicit mental concepts, i.e., excellent, good, fair, poor, and bad. A newly designed quality pooling is then applied to convert the qualitative labels into scores. The classification framework is not only much more natural than the regression-based models, but also robust to the small sample size problem. Thorough experiments are conducted on popular databases to verify the model's effectiveness, efficiency, and robustness.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336 · cited 10×
“The discriminative deep model is pretrained by DBN [21] and fine-tuned by backpropagation.”
BackgroundMethodInfluentialHutchins, E. (1995) · doi:10.7551/mitpress/1881.001.0001 · Cited by 10,801
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 9,239
No-Reference Image Quality Assessment in the Spatial Domain
Mittal, A., Moorthy, A. K., & Bovik, A. C. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2214050 · Cited by 6,064 · cited 8×
“Mittal et al. [26] presented BRISQUE to predict image quality in the spatial domain, Ye et al. [12] proposed CORNIA to extract features using K-means clustering and applied SVM to map the feature to quality.”
BackgroundInfluentialA unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058 · cited 1×
“Deep learning models have thrived over the years [30], not only in the field of computer vision, but also in many others, such as audio [31], natural language processing [32], and so on [21], [33]–[35].”
BackgroundLearning Deep Architectures for AI
Bengio, Y. (2009) · now publishers, Inc. eBooks · doi:10.1561/9781601982957 · Cited by 4,941
Image information and visual quality
Sheikh, H. R., & Bovik, A. C. (2006) · IEEE Transactions on Image Processing · doi:10.1109/tip.2005.859378 · Cited by 4,104
A Statistical Evaluation of Recent Full Reference Image Quality Assessment Algorithms
Sheikh, H. R., Sabir, M. F., & Bovik, A. C. (2006) · IEEE Transactions on Image Processing · doi:10.1109/tip.2006.881959 · Cited by 2,966
Cited by
Chen, C. L. P., & Liu, Z. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2716952 · Cited by 1,957
“DEEP structure neural networks and learnings have been applied in many fields and have achieved breakthrough successes in a great number of applications [1], [2], particularly in large scale data processing [3], [4].”
BackgroundPerceptual image quality assessment: a survey
Zhai, G., & Min, X. (2020) · Science China Information Sciences · doi:10.1007/s11432-019-2757-1 · Cited by 655
Liu, J., Gong, M., Qin, K., & Zhang, P. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2636227 · Cited by 522
Deep learning in microscopy image analysis: a survey
Xing, F., Xie, Y., Su, H., Liu, F., & Yang, L. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2766168 · Cited by 460
Fully Deep Blind Image Quality Predictor
Kim, J., & Lee, S. (2017) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2016.2639328 · Cited by 459
Learning a No-Reference Quality Assessment Model of Enhanced Images With Big Data
Gu, K., Tao, D., Qiao, J.-F., & Lin, W. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2017.2649101 · Cited by 443
Self-citationLightweight Pyramid Networks for Image Deraining
Fu, X., Liang, B., Huang, Y., Ding, X., & Paisley, J. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2926481 · Cited by 421
dipIQ: Blind Image Quality Assessment by Learning-to-Rank Discriminable Image Pairs
Ma, K., Liu, W., Liu, T., Wang, Z., & Tao, D. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2708503 · Cited by 353
Self-citationDeep CNN-Based Blind Image Quality Predictor
Kim, J., Nguyen, A.-D., & Lee, S. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2829819 · Cited by 331
Kim, J., Zeng, H., Ghadiyaram, D., Lee, S., Zhang, L., & Bovik, A. C. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2736018 · Cited by 283
“trained a deep belief network on wavelet domain NSS features to classify distorted images into five discrete score categories [17], and Li et al.”
BackgroundBlind image quality assessment by relative gradient statistics and adaboosting neural network
Liu, L., Hua, Y., Zhao, Q., Huang, H., & Bovik, A. C. (2016) · Signal Processing: Image Communication · doi:10.1016/j.image.2015.10.005 · Cited by 282
A survey of infrared and visual image fusion methods
Jin, X., Jiang, Q., Yao, S., Zhou, D., Nie, R., Hai, J., & He, K. (2017) · Infrared Physics & Technology · doi:10.1016/j.infrared.2017.07.010 · Cited by 280
On the use of deep learning for blind image quality assessment
Bianco, S., Celona, L., Napoletano, P., & Schettini, R. (2018) · Signal, Image and Video Processing · doi:10.1007/s11760-017-1166-8 · Cited by 277
TOPIQ: A Top-Down Approach From Semantics to Distortions for Image Quality Assessment
Chen, C., Mo, J., Hou, J., Wu, H., Liao, L., Sun, W., Yan, Q., & Lin, W. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2024.3378466 · Cited by 276
Deep Learning of Human Visual Sensitivity in Image Quality Assessment Framework
Kim, J., & Lee, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.213 · Cited by 265
Gong, M., Yang, H., & Zhang, P. (2017) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2017.05.001 · Cited by 226
A Multiobjective Sparse Feature Learning Model for Deep Neural Networks
Gong, M., Liu, J., Li, H., Cai, Q., & Su, L. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2469673 · Cited by 198
Two-Stream Convolutional Networks for Blind Image Quality Assessment
Yan, Q., Gong, D., & Zhang, Y. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2883741 · Cited by 181
“Following the settings in [37], we use the scatter plots to evaluate the proposed method visually.”
BackgroundMethodDeepSim: Deep similarity for image quality assessment
Gao, F., Wang, Y., Li, P., Tan, M., Yu, J., & Zhu, Y. (2017) · Neurocomputing · doi:10.1016/j.neucom.2017.01.054 · Cited by 172
Metadata
- Authors
- Hou, W., Gao, X., Tao, D., & Li, X.
- Venue
- IEEE Transactions on Neural Networks and Learning Systems
- Published
- 2015-06
- Volume
- 26
- Issue
- 6
- Pages
- 1275-1286
- ISSN
- 2162-237X (ISSN-L)
- 2162-2388
- OpenAlex
- W2170947705
- Cited by
- 368
- References
- 39
- FWCI
- 26.71
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.