Two-Level Approach for No-Reference Consumer Video Quality Assessment
The DOI matches its Crossref record; 43 of the 48 references it cites were found in a registry or an index.
Abstract
Smartphones and other consumer devices capable of capturing video content and sharing it on social media in nearly real time are widely available at a reasonable cost. Thus, there is a growing need for no-reference video quality assessment (NR-VQA) of consumer produced video content, typically characterized by capture impairments that are qualitatively different from those observed in professionally produced video content. To date, most of the NR-VQA models in prior art have been developed for assessing coding and transmission distortions, rather than capture impairments. In addition, the most accurate NR-VQA methods known in prior art are often computationally complex, and therefore impractical for many real life applications. In this paper, we propose a new approach for learning-based video quality assessment, based on the idea of computing features in two levels so that low complexity features are computed for the full sequence first, and then high complexity features are extracted from a subset of representative video frames, selected by using the low complexity features. We have compared the proposed method against several relevant benchmark methods using three recently published annotated public video quality databases, and our results show that the proposed method can predict subjective video quality more accurately than the benchmark methods. The best performing prior method achieves nearly similar accuracy, but at substantially higher computational cost.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Temporal pooling is one of the most important, unresolved problems for video quality prediction [42], [60], [76], [82], [83].”
From UGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Content (2021) · cites this paper as [42]
In the citing paper
“The model currently achieves SOTA performance on three UGC video quality databases, CVD2014 [12], KoNViD-1k [10], and LIVE-Qualcomm [13], at a reasonably low complexity, as reported by the authors.”
From UGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Content (2021) · cites this paper as [42]
In the citing paper
“We observed in Section II-B that LIVE-VQC videos generally contain more (camera) motions than KoNViD-1k and YouTube-UGC, and TLVQM computes multiple motion relevant features.”
From UGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Content (2021) · cites this paper as [42]
In the citing paper
“TLVQM does require that a large set of parameters (around 30) be specified, which may affect performance on datasets or application scenarios it has not been exposed to.”
From UGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Content (2021) · cites this paper as [42]
Citations
Cites
Making a “Completely Blind” Image Quality Analyzer
Mittal, A., Soundararajan, R., & Bovik, A. C. (2013) · IEEE Signal Processing Letters · doi:10.1109/lsp.2012.2227726 · Cited by 6,766 · cited 4×
“In the field of NR-IQA and NR-VQA, Support Vector Regression (SVR) is the most commonly used regression model [5], [9], [38], [39], [43], but also neural networks and Random Forest Regression (RFR) have been employed successfully [7], [8], [12], [46].”
BackgroundMethodInfluentialNo-Reference Image Quality Assessment in the Spatial Domain
Mittal, A., Moorthy, A. K., & Bovik, A. C. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2214050 · Cited by 6,064 · cited 3×
“In the field of NR-IQA and NR-VQA, Support Vector Regression (SVR) is the most commonly used regression model [5], [9], [38], [39], [43], but also neural networks and Random Forest Regression (RFR) have been employed successfully [7], [8], [12], [46].”
MethodNatural Image Statistics and Neural Representation
Simoncelli, E. P., & Olshausen, B. A. (2001) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.24.1.1193 · Cited by 2,460 · cited 1×
“between neighboring locations, as well as spatial and temporal frequencies, follow certain typical statistical distributions [37].”
BackgroundDesign of an image edge detection filter using the Sobel operator
Kanopoulos, N., Vasanthavada, N., & Baker, R. L. (1988) · IEEE Journal of Solid-State Circuits · doi:10.1109/4.996 · Cited by 1,212
No-reference perceptual quality assessment of JPEG compressed images
Wang, Z., Sheikh, H. R., & Bovik, A. C. (2003) · Proceedings. International Conference on Image Processing · doi:10.1109/icip.2002.1038064 · Cited by 834
Unsupervised feature learning framework for no-reference image quality assessment
Ye, P., Kumar, J., Kang, L., & Doermann, D. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247789 · Cited by 785
A no-reference objective image sharpness metric based on the notion of just noticeable blur (JNB)
Ferzli, R., & Karam, L. J. (2009) · IEEE Transactions on Image Processing · doi:10.1109/tip.2008.2011760 · Cited by 758
The Evolution of Video Quality Measurement: From PSNR to Hybrid Metrics
Winkler, S., & Mohandas, P. (2008) · IEEE Transactions on Broadcasting · doi:10.1109/tbc.2008.2000733 · Cited by 598
Blind Prediction of Natural Video Quality
Saad, M. A., Bovik, A. C., & Charrier, C. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2014.2299154 · Cited by 471 · cited 4×
“In the field of NR-IQA and NR-VQA, Support Vector Regression (SVR) is the most commonly used regression model [5], [9], [38], [39], [43], but also neural networks and Random Forest Regression (RFR) have been employed successfully [7], [8], [12], [46].”
MethodInfluentialA Completely Blind Video Integrity Oracle
Mittal, A., Saad, M. A., & Bovik, A. C. (2016) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2502725 · Cited by 357 · cited 2×
“Pixel-based models use a raw video signal solely as an input [4]–[5], [7]–[10], [12], and hybrid models combine features from the bitstream with features extracted from the”
BackgroundMethodThe Konstanz natural video database (KoNViD-1k)
Hosu, V., Hahn, F., Jenadeleh, M., Lin, H., Men, H., Sziranyi, T., Li, S., & Saupe, D. (2017) · 2017 Ninth International Conference on Quality of Multimedia Experience (QoMEX) · doi:10.1109/qomex.2017.7965673 · Cited by 343
Perceptual quality prediction on authentically distorted images using a bag of features approach
Ghadiyaram, D., & Bovik, A. C. (2017) · Journal of Vision · doi:10.1167/17.1.32 · Cited by 322
Large-Scale Study of Perceptual Video Quality
Sinno, Z., & Bovik, A. C. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2869673 · Cited by 310
Analysis of Public Image and Video Databases for Quality Assessment
Winkler, S. (2012) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2012.2215007 · Cited by 299
Image Sharpness Assessment Based on Local Phase Coherence
Hassen, R., Wang, Z., & Salama, M. M. A. (2013) · IEEE Transactions on Image Processing · doi:10.1109/tip.2013.2251643 · Cited by 290
Best Practices for QoE Crowdtesting: QoE Assessment With Crowdsourcing
Hossfeld, T., Keimel, C., Hirth, M., Gardlo, B., Habigt, J., Diepold, K., & Tran-Gia, P. (2014) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2013.2291663 · Cited by 283
VMAF reproducibility: Validating a perceptual practical video quality metric
Rassool, R. (2017) · 2017 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB) · doi:10.1109/bmsb.2017.7986143 · Cited by 235
CVD2014—A Database for Evaluating No-Reference Video Quality Assessment Algorithms
Nuutinen, M., Virtanen, T., Vaahteranoksa, M., Vuori, T., Oittinen, P., & Hakkinen, J. (2016) · IEEE Transactions on Image Processing · doi:10.1109/tip.2016.2562513 · Cited by 227
No-Reference Quality Assessment of Tone-Mapped HDR Pictures
Kundu, D., Ghadiyaram, D., Bovik, A. C., & Evans, B. L. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2685941 · Cited by 182
VideoSet: A large-scale compressed video quality dataset based on JND measurement
Wang, H., Katsavounidis, I., Zhou, J., Park, J., Lei, S., Zhou, X., Pun, M.-O., Jin, X., Wang, R., Wang, X., et al. (2017) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2017.04.009 · Cited by 176
In-Capture Mobile Video Distortions: A Study of Subjective Behavior and Objective Algorithms
Ghadiyaram, D., Pan, J., Bovik, A. C., Moorthy, A. K., Panda, P., & Yang, K.-C. (2018) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2017.2707479 · Cited by 154
Image Quality Assessment Using Human Visual DOG Model Fused With Random Forest
Pei, S.-C., & Chen, L.-H. (2015) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2440172 · Cited by 132
Spatiotemporal Feature Integration and Model Fusion for Full Reference Video Quality Assessment
Bampis, C. G., Li, Z., & Bovik, A. C. (2019) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2018.2868262 · Cited by 128
Wireless Video Quality Assessment: A Study of Subjective Scores and Objective Algorithms
Moorthy, A. K., Seshadrinathan, K., Soundararajan, R., & Bovik, A. C. (2010) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2010.2041829 · Cited by 124
No-Reference Video Quality Assessment With 3D Shearlet Transform and Convolutional Neural Networks
Li, Y., Po, L.-M., Cheung, C.-H., Xu, X., Feng, L., Yuan, F., & Cheung, K.-W. (2016) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2015.2430711 · Cited by 122
Optimizing Support Vector regression hyperparameters based on cross-validation
Ito, K., & Nakano, R. (2004) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2003.1223728 · Cited by 97
No-reference video quality assessment via feature learning
Xu, J., Ye, P., Liu, Y., & Doermann, D. (2014) · 2014 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip.2014.7025098 · Cited by 85
No-Reference Video Quality Assessment Based on Artifact Measurement and Statistical Analysis
Zhu, K., Li, C., Asari, V., & Saupe, D. (2015) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2014.2363737 · Cited by 77
A hyperparameters selection technique for support vector regression models
Tsirikoglou, P., Abraham, S., Contino, F., et al. (2017) · Applied Soft Computing · doi:10.1016/j.asoc.2017.07.017 · Cited by 75
Motion blur and motion sharpening in the human visual system
Hammett, S. T. (1997) · Vision Research · doi:10.1016/s0042-6989(97)00059-x · Cited by 49
SAMVIQ—A New EBU Methodology for Video Quality Evaluations in Multimedia
Kozamernik, F., Steinmann, V., Sunna, P., et al. (2005) · SMPTE Motion Imaging Journal · doi:10.5594/j11535 · Cited by 49
No-Reference Video Quality Assessment Using Codec Analysis
Søgaard, J., Forchhammer, S. O., & Korhonen, J. (2015) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2015.2397207 · Cited by 48
Self-citationSubjective Quality Assessment of H.264/AVC Video Streaming with Packet Losses
Francesca De Simone, A., Naccari, M., Tagliasacchi, M., Frederic Dufaux, A., Tubaro, S., & Ebrahimi, T. (2011) · EURASIP Journal on Image and Video Processing · doi:10.1155/2011/190431 · Cited by 45
No-Reference Pixel Video Quality Monitoring of Channel-Induced Distortion
Valenzise, G., Magni, S., Tagliasacchi, M., & Tubaro, S. (2011) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2011.2171211 · Cited by 40
Prediction of Transmission Distortion for Wireless Video Communication: Analysis
Chen, Z., & Wu, D. O. (2011) · IEEE Transactions on Image Processing · doi:10.1109/tip.2011.2168411 · Cited by 38
Argyropoulos, S., Raake, A., Garcia, M.-N., & List, P. (2011) · International Conference on Quality of Multimedia Experience (QoMEX) · doi:10.1109/qomex.2011.6065708 · Cited by 37
Predictive no-reference assessment of video quality
Vega, M. T., Mocanu, D. C., Stavrou, S., & Liotta, A. (2016) · Signal Processing Image Communication · doi:10.1016/j.image.2016.12.001 · Cited by 36
Spatiotemporal Feature Combination Model for No-Reference Video Quality Assessment
Men, H., Lin, H., & Saupe, D. (2018) · International Conference on Quality of Multimedia Experience (QoMEX) · doi:10.1109/qomex.2018.8463426 · Cited by 29
Pandremmenou, K., Shahid, M. U., Kondi, L. P., & Lövström, B. (2015) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.2077709 · Cited by 22
Design of no-reference video quality metrics with multiway partial least squares regression
Keimel, C., Habigt, J., Klimpke, M., et al. (2011) · International Workshop on Quality of Multimedia Experience (QoMEX) · doi:10.1109/qomex.2011.6065711 · Cited by 20
Expertise screening in crowdsourcing image quality
Hosu, V., Lin, H., & Saupe, D. (2018) · International Conference on Quality of Multimedia Experience (QoMEX) · doi:10.1109/qomex.2018.8463427 · Cited by 20
Learning-based Prediction of Packet Loss Artifact Visibility in Networked Video
Korhonen, J. (2018) · International Conference on Quality of Multimedia Experience (QoMEX) · doi:10.1109/qomex.2018.8463394 · Cited by 7
Self-citationTechniques for efficient MATLAB-to-C compilation
Bispo, J., Reis, L. P., & Cardoso, J. M. P. (2015) · doi:10.1145/2774959.2774961 · Cited by 4
Research works printed but not linked: 13
- 2011, Objective Multimedia Video Quality Measurement of HDTV for Digital Cable Television in the Presence of a Reduced Reference Signal
- 2016, Objective Perceptual Multimedia Video Quality Measurement of HDTV for Digital Cable Television in the Presence of a Full Reference
- 2012, Methodology for the Subjective Assessment of the Quality of Television Pictures
- 2010, Report on the validation of video quality models for high definition video content
- ghadiyaram, 2017, LIVE-Qualcomm Mobile in-Capture Video Quality Database
- hosu, 2017, KoNViD-1k Video Database
- korhonen, 2019, Two Level Video Quality Model (TlvQM)
- zheng, 2018, Source Code for Classical Biqa Methods
- 2018, Image & Video Quality Assessment Algorithms Laboratory for Image & Video Engineering the University of Texas in Austin
- 2018, Scikit-Learn Toolbox
- alpert, 1997, Subjective quality evaluation–the SSCQE and DSCQE methodologies, EBU Tech Rev, 271, 12
- 1999, Subjective video quality assessment methods for multimedia applications
- nuutinen, 2018, The CVD Video Database
Cited by
UGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Content
Tu, Z., Wang, Y., Birkbeck, N., Adsumilli, B., & Bovik, A. C. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3072221 · Cited by 363
“Some motion-related features (as in TLVQM) may not apply as well in this scenario.”
BackgroundMethodInfluentialA Deep Learning based No-reference Quality Assessment Model for UGC Videos
Sun, W., Min, X., Lu, W., & Zhai, G. (2022) · Proceedings of the 30th ACM International Conference on Multimedia · doi:10.1145/3503161.3548329 · Cited by 240
“For the NR VQA methods, the hand-crafted feature based NR VQA methods such as TLVQM and VIDEVAL achieve pretty well performance by incorporating the rich spatial and temporal quality features, such as NSS features, motion features, etc., but they are inferior to the deep learning based NR VQA methods due to the strong…”
BackgroundMethodInfluentialFAST-VQA: Efficient End-to-End Video Quality Assessment with Fragment Sampling
Wu, H., Chen, C., Hou, J., Liao, L., Wang, A., Sun, W., Yan, Q., & Lin, W. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-20068-7_31 · Cited by 244
Perceptual video quality assessment: a survey
Min, X., Duan, H., Sun, W., Zhu, Y., & Zhai, G. (2024) · Science China Information Sciences · doi:10.1007/s11432-024-4133-3 · Cited by 218
RAPIQUE: Rapid and Accurate Video Quality Prediction of User Generated Content
Tu, Z., Yu, X., Wang, Y., Birkbeck, N., Adsumilli, B., & Bovik, A. C. (2021) · IEEE Open Journal of Signal Processing · doi:10.1109/ojsp.2021.3090333 · Cited by 215
“Evaluation Method: We used a support vector regressor (SVR) as the back-end regression model to learn the feature-to-score mappings [8], [9], [12], [13], [61].”
BackgroundMethodInfluential
Metadata
- Authors
- Korhonen, J.
- Affiliations
- Shenzhen University (CN)
- Corresponding author
- Korhonen, Jari
- ORCID
- Korhonen, Jari
- Venue
- IEEE Transactions on Image Processing
- Published
- 2019-12
- Volume
- 28
- Issue
- 12
- Pages
- 5923-5938
- ISSN
- 1057-7149 (ISSN-L)
- 1941-0042
- OpenAlex
- W2950154603
- Semantic Scholar
- f441c421b710c164872298679de173621dc980dc
- Cited by
- 354
- References
- 48
- Citations by year
- 2026: 38 · 2025: 67 · 2024: 65 · 2023: 72 · 2022: 48 · 2021: 53 · 2020: 10 · 2019: 1
- FWCI
- 12.53
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Image Processing (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Natural Science Foundation of China (61772348)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Video Quality Assessment
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Keywords
- no-reference video quality assessment, consumer video, representative frame selection, computational efficiency
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.