The DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
RAPIQUE: Rapid and Accurate Video Quality Prediction of User Generated Content
Open accessAbstract
Blind or no-reference video quality assessment of user-generated content (UGC) has become a trending, challenging, heretofore unsolved problem. Accurate and efficient video quality predictors suitable for this content are thus in great demand to achieve more intelligent analysis and processing of UGC videos. Previous studies have shown that natural scene statistics and deep learning features are both sufficient to capture spatial distortions, which contribute to a significant aspect of UGC video quality issues. However, these models are either incapable or inefficient for predicting the quality of complex and diverse UGC videos in practical applications. Here we introduce an effective and efficient video quality model for UGC content, which we dub the Rapid and Accurate Video Quality Evaluator (RAPIQUE), which we show performs comparably to state-of-the-art (SOTA) models but with orders-of-magnitude faster runtime. RAPIQUE combines and leverages the advantages of both quality-aware scene statistics features and semantics-aware deep convolutional features, allowing us to design the first general and efficient spatial and temporal (space-time) bandpass statistics model for video quality modeling. Our experimental results on recent large-scale UGC video quality databases show that RAPIQUE delivers top performances on all the datasets at a considerably lower computational expense. We hope this work promotes and inspires further efforts towards practical modeling of video quality problems for potential real-time and low-latency applications.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
Self-citationSpatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2389824 · Cited by 11,971
Making a “Completely Blind” Image Quality Analyzer
Mittal, A., Soundararajan, R., & Bovik, A. C. (2013) · IEEE Signal Processing Letters · doi:10.1109/lsp.2012.2227726 · Cited by 6,766
Self-citationMultiscale structural similarity for image quality assessment
Wang, Z., Simoncelli, E. P., & Bovik, A. C. (2004) · The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003 · doi:10.1109/acssc.2003.1292216 · Cited by 6,253 · cited 1×
“Incorporating multiscale information in quality models provides significant performance improvements [8], [77].”
BackgroundSelf-citationNo-Reference Image Quality Assessment in the Spatial Domain
Mittal, A., Moorthy, A. K., & Bovik, A. C. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2214050 · Cited by 6,064 · cited 10×
“Among these, BRISQUE [8], GM-LOG [11], FRIQUEE [12], V-BLIINDS [9], and VIDEVAL [6] have leveraged different sets of natu-ral scene statistics (NSS)-based quality-aware features, using them to train shallow regressors to predict subjective quality scores.”
BackgroundMethodInfluentialSelf-citationA Two-Step Framework for Constructing Blind Image Quality Indices
Moorthy, A. K., & Bovik, A. C. (2010) · IEEE Signal Processing Letters · doi:10.1109/lsp.2010.2043888 · Cited by 1,196
Self-citationStudy of Subjective and Objective Quality Assessment of Video
Seshadrinathan, K., Soundararajan, R., Bovik, A. C., & Cormack, L. K. (2010) · IEEE Transactions on Image Processing · doi:10.1109/tip.2010.2042111 · Cited by 1,160
Self-citationSTRUCTURE AND FUNCTION OF VISUAL AREA MT
Born, R. T., & Bradley, D. C. (2005) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.26.041002.131052 · Cited by 1,023 · cited 1×
“The perception of video correlates highly with motion and temporal change [52].”
BackgroundKonIQ-10k: An Ecologically Valid Database for Deep Learning of Blind Image Quality Assessment
Hosu, V., Lin, H., Sziranyi, T., & Saupe, D. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.2967829 · Cited by 759
Blind Image Quality Assessment Using Joint Statistics of Gradient Magnitude and Laplacian Features
Xue, W., Mou, X., Zhang, L., Bovik, A. C., & Feng, X. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2014.2355716 · Cited by 641 · cited 4×
“The baseline models used for comparison are BRISQUE [8], GM-LOG [11], HIGRADE [10], FRIQUEE [12], the codebook-based models CORNIA [14] and HOSA [81], and the deep learning models, KonCept512 [51], PaQ-2-PiQ [17] which are all spatial-only models.”
MethodInfluentialSelf-citationBlind Prediction of Natural Video Quality
Saad, M. A., Bovik, A. C., & Charrier, C. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2014.2299154 · Cited by 471 · cited 7×
“Among these, BRISQUE [8], GM-LOG [11], FRIQUEE [12], V-BLIINDS [9], and VIDEVAL [6] have leveraged different sets of natu-ral scene statistics (NSS)-based quality-aware features, using them to train shallow regressors to predict subjective quality scores.”
BackgroundMethodInfluentialSelf-citationBlind measurement of blocking artifacts in images
Wang, Z., Bovik, A. C., & Evan, B. L. (2002) · Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101) · doi:10.1109/icip.2000.899622 · Cited by 413
Self-citationFrom Patches to Pictures (PaQ-2-PiQ): Mapping the Perceptual Space of Picture Quality
Ying, Z., Niu, H., Gupta, P., Mahajan, D., Ghadiyaram, D., & Bovik, A. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00363 · Cited by 406
Self-citationQuality Assessment of In-the-Wild Videos
Li, D., Jiang, T., & Jiang, M. (2019) · Proceedings of the 27th ACM International Conference on Multimedia · doi:10.1145/3343031.3351028 · Cited by 387
Video Quality Assessment by Reduced Reference Spatio-Temporal Entropic Differencing
Soundararajan, R., & Bovik, A. C. (2013) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2012.2214933 · Cited by 368 · cited 1×
“To the best of our knowledge, we propose the first general, effective and efficient temporal statistics model based on bandpass regularities of natural videos along the time dimension, going beyond simpler frame-difference models [78].”
Self-citationUGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Content
Tu, Z., Wang, Y., Birkbeck, N., Adsumilli, B., & Bovik, A. C. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3072221 · Cited by 363 · cited 10×
“Among these, BRISQUE [8], GM-LOG [11], FRIQUEE [12], V-BLIINDS [9], and VIDEVAL [6] have leveraged different sets of natu-ral scene statistics (NSS)-based quality-aware features, using them to train shallow regressors to predict subjective quality scores.”
BackgroundMethodInfluentialSelf-citationA Completely Blind Video Integrity Oracle
Mittal, A., Saad, M. A., & Bovik, A. C. (2016) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2502725 · Cited by 357 · cited 4×
“Since ‘completely blind’ models such as NIQE [33] and VIIDEO [47] were not observed to perform reasonably well on these natural video datasets [6], we did not include them.”
BackgroundMethodInfluentialSelf-citationTwo-Level Approach for No-Reference Consumer Video Quality Assessment
Korhonen, J. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2923051 · Cited by 354 · cited 10×
“Evaluation Method: We used a support vector regressor (SVR) as the back-end regression model to learn the feature-to-score mappings [8], [9], [12], [13], [61].”
BackgroundMethodInfluentialPerceptual quality prediction on authentically distorted images using a bag of features approach
Ghadiyaram, D., & Bovik, A. C. (2017) · Journal of Vision · doi:10.1167/17.1.32 · Cited by 322
Self-citationLarge-Scale Study of Perceptual Video Quality
Sinno, Z., & Bovik, A. C. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2869673 · Cited by 310
Self-citationKim, J., Zeng, H., Ghadiyaram, D., Lee, S., Zhang, L., & Bovik, A. C. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2736018 · Cited by 283 · cited 3×
“The authors of [17], [61] suggested applying a CNN on spatially sampled small patches, then aggregating the locally predicted scores to obtain global quality scores.”
MethodSelf-citationYouTube UGC Dataset for Video Compression Research
Wang, Y., Inguva, S., & Adsumilli, B. (2019) · 2019 IEEE 21st International Workshop on Multimedia Signal Processing (MMSP) · doi:10.1109/mmsp.2019.8901772 · Cited by 282
Self-citationCVD2014—A Database for Evaluating No-Reference Video Quality Assessment Algorithms
Nuutinen, M., Virtanen, T., Vaahteranoksa, M., Vuori, T., Oittinen, P., & Hakkinen, J. (2016) · IEEE Transactions on Image Processing · doi:10.1109/tip.2016.2562513 · Cited by 227
ViS3: an algorithm for video quality assessment via analysis of spatial and spatiotemporal slices
Vu, P. V., & Chandler, D. M. (2014) · Journal of Electronic Imaging · doi:10.1117/1.jei.23.1.013016 · Cited by 185 · cited 1×
“Several other popular CNN-based BVQA models [19], [26], [27], [65], [67] produce accurate quality predictions on legacy (single synthetic distortion) video datasets [1], [68], but struggle on recent in-the-wild UGC databases [31], [50], [66].”
MethodA Comparative Evaluation Of Temporal Pooling Methods For Blind Video Quality Assessment
Tu, Z., Chen, C.-J., Chen, L.-H., Birkbeck, N., Adsumilli, B., & Bovik, A. C. (2020) · 2020 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip40778.2020.9191169 · Cited by 71
Self-citationBBAND INDEX: A NO-REFERENCE BANDING ARTIFACT PREDICTOR
Tu, Z., Lin, J., Wang, Y., Adsumilli, B., & Bovik, A. C. (2020) · ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp40776.2020.9053634 · Cited by 46
Self-citationTu, Z., Lin, J., Wang, Y., Adsumilli, B., & Bovik, A. C. (2020) · IEEE Signal Processing Letters · doi:10.1109/lsp.2020.3024985 · Cited by 31
Self-citation
Cited by
UGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Content
Tu, Z., Wang, Y., Birkbeck, N., Adsumilli, B., & Bovik, A. C. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3072221 · Cited by 363
Self-citationImage Quality Assessment Using Contrastive Learning
Madhusudana, P. C., Birkbeck, N., Wang, Y., Adsumilli, B., & Bovik, A. C. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3181496 · Cited by 285
Self-citationPerceptual video quality assessment: a survey
Min, X., Duan, H., Sun, W., Zhu, Y., & Zhai, G. (2024) · Science China Information Sciences · doi:10.1007/s11432-024-4133-3 · Cited by 218
Metadata
- Authors
- Tu, Z., Yu, X., Wang, Y., Birkbeck, N., Adsumilli, B., & Bovik, A. C.
- Venue
- IEEE Open Journal of Signal Processing
- Published
- 2021
- Volume
- 2
- Pages
- 425-440
- ISSN
- 2644-1322 (ISSN-L)
- OpenAlex
- W3125631097
- Cited by
- 215
- References
- 29
- FWCI
- 11.99
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.