The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Blindly Assess Quality of In-the-Wild Videos via Quality-Aware Pre-Training and Motion Perception
Abstract
Perceptual quality assessment of the videos acquired in the wilds is of vital importance for quality assurance of video services. The inaccessibility of reference videos with pristine quality and the complexity of authentic distortions pose great challenges for this kind of blind video quality assessment (BVQA) task. Although model-based transfer learning is an effective and efficient paradigm for the BVQA task, it remains to be a challenge to explorewhatandhowto bridge the domain shifts for better video representation. In this work, we propose to transfer knowledge from image quality assessment (IQA) databases with authentic distortions and large-scale action recognition with rich motion patterns. We rely on both groups of data to learn the feature extractor and use a mixed list-wise ranking loss function to train the entire model on the target VQA databases. Extensive experiments on six benchmarking databases demonstrate that our method performs very competitively under both individual database and mixed databases training settings. We also verify the rationality of each component of the proposed method and explore a simple ensemble trick for further improvement.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Making a “Completely Blind” Image Quality Analyzer
Mittal, A., Soundararajan, R., & Bovik, A. C. (2013) · IEEE Signal Processing Letters · doi:10.1109/lsp.2012.2227726 · Cited by 6,766
Multiscale structural similarity for image quality assessment
Wang, Z., Simoncelli, E. P., & Bovik, A. C. (2004) · The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003 · doi:10.1109/acssc.2003.1292216 · Cited by 6,253 · cited 1×
“Specifically, similar to [12], we generate pseudo-labels for KADIS-700k [78] using MS-SSIM [79], and use them to pre-train a ResNet50 in a regression manner (dubbed as SFR).”
MethodNo-Reference Image Quality Assessment in the Spatial Domain
Mittal, A., Moorthy, A. K., & Bovik, A. C. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2214050 · Cited by 6,064 · cited 4×
“Although a plethora of general-purpose BVQA models was developed subsequently, they were still designed for handling synthetic distortions using hand-crafted features [8]–[10].”
BackgroundMethodInfluentialSlowFast Networks for Video Recognition
Feichtenhofer, C., Fan, H., Malik, J., & He, K. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00630 · Cited by 4,007
Deep CORAL: Correlation Alignment for Deep Domain Adaptation
Sun, B., & Saenko, K. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-49409-8_35 · Cited by 3,757
KonIQ-10k: An Ecologically Valid Database for Deep Learning of Blind Image Quality Assessment
Hosu, V., Lin, H., Sziranyi, T., & Saupe, D. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.2967829 · Cited by 759
Learning Spatio-Temporal Features with 3D Residual Networks for Action Recognition
Hara, K., Kataoka, H., & Satoh, Y. (2017) · 2017 IEEE International Conference on Computer Vision Workshops (ICCVW) · doi:10.1109/iccvw.2017.373 · Cited by 726
Motion Tuned Spatio-Temporal Quality Assessment of Natural Videos
Seshadrinathan, K., & Bovik, A. C. (2010) · IEEE Transactions on Image Processing · doi:10.1109/tip.2009.2034992 · Cited by 708 · cited 1×
“In addition, motion information has also shown its promises in the perceived quality of videos [19]–[21].”
BackgroundBlind Image Quality Assessment Using Joint Statistics of Gradient Magnitude and Laplacian Features
Xue, W., Mou, X., Zhang, L., Bovik, A. C., & Feng, X. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2014.2355716 · Cited by 641 · cited 1×
“The representative BIQA models are NIQE [25], IL-NIQE [67], BRISQUE [8], M3 [68], HIGRADE [69], FRIQUEE [26], CORNIA [30], HOSA [70], and pre-trained DNN models, VGG-19 [71], ResNet-50 [50].”
MethodKADID-10k: A Large-scale Artificially Distorted IQA Database
Lin, H., Hosu, V., & Saupe, D. (2019) · 2019 Eleventh International Conference on Quality of Multimedia Experience (QoMEX) · doi:10.1109/qomex.2019.8743252 · Cited by 523 · cited 3×
“We also follow the practice of [37] to train the frame-level feature extractor on KADIS-700k using a cross-entropy loss function and a pairwise hinge loss function, accounting for distortion types classification and degradation levels ranking.”
MethodBlind Prediction of Natural Video Quality
Saad, M. A., Bovik, A. C., & Charrier, C. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2014.2299154 · Cited by 471 · cited 3×
“[9] further introduced motion information to enhance the representa-”
BackgroundMethodFrom Patches to Pictures (PaQ-2-PiQ): Mapping the Perceptual Space of Picture Quality
Ying, Z., Niu, H., Gupta, P., Mahajan, D., Ghadiyaram, D., & Bovik, A. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00363 · Cited by 406
Quality Assessment of In-the-Wild Videos
Li, D., Jiang, T., & Jiang, M. (2019) · Proceedings of the 27th ACM International Conference on Multimedia · doi:10.1145/3343031.3351028 · Cited by 387
UGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Content
Tu, Z., Wang, Y., Birkbeck, N., Adsumilli, B., & Bovik, A. C. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3072221 · Cited by 363 · cited 4×
“5537 [73] to partition CombinedU into 1,469 low quality and 1,458 high quality materials, and tabulate the comparisons in the “Quality” column of Table VI.”
BackgroundMethodInfluentialA Completely Blind Video Integrity Oracle
Mittal, A., Saad, M. A., & Bovik, A. C. (2016) · IEEE Transactions on Image Processing · doi:10.1109/tip.2015.2502725 · Cited by 357 · cited 6×
“One exception is VIIDEO, which is calibrated to handle synthetic distortions.”
BackgroundMethodInfluentialTwo-Level Approach for No-Reference Consumer Video Quality Assessment
Korhonen, J. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2923051 · Cited by 354 · cited 4×
“For example, with the resolution growing from 360p to 2160p, our algorithm (Proposed_GPU_PAL) can achieve an increase in speed from 3 times to 16 times compared against TLVQM.”
MethodInfluentialPerceptual quality prediction on authentically distorted images using a bag of features approach
Ghadiyaram, D., & Bovik, A. C. (2017) · Journal of Vision · doi:10.1167/17.1.32 · Cited by 322
YouTube UGC Dataset for Video Compression Research
Wang, Y., Inguva, S., & Adsumilli, B. (2019) · 2019 IEEE 21st International Workshop on Multimedia Signal Processing (MMSP) · doi:10.1109/mmsp.2019.8901772 · Cited by 282
CVD2014—A Database for Evaluating No-Reference Video Quality Assessment Algorithms
Nuutinen, M., Virtanen, T., Vaahteranoksa, M., Vuori, T., Oittinen, P., & Hakkinen, J. (2016) · IEEE Transactions on Image Processing · doi:10.1109/tip.2016.2562513 · Cited by 227
A Comparative Evaluation Of Temporal Pooling Methods For Blind Video Quality Assessment
Tu, Z., Chen, C.-J., Chen, L.-H., Birkbeck, N., Adsumilli, B., & Bovik, A. C. (2020) · 2020 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip40778.2020.9191169 · Cited by 71 · cited 1×
“In addition, motion information has also shown its promises in the perceived quality of videos [19]–[21].”
Background
Cited by
Perceptual video quality assessment: a survey
Min, X., Duan, H., Sun, W., Zhu, Y., & Zhai, G. (2024) · Science China Information Sciences · doi:10.1007/s11432-024-4133-3 · Cited by 218
Self-citationQu, B., Liang, X., Sun, S., & Gao, W. (2024) · arXiv · doi:10.48550/arxiv.2404.13573 · Cited by 44
Metadata
- Authors
- Li, B., Zhang, W., Tian, M., Zhai, G., & Wang, X.
- Venue
- IEEE Transactions on Circuits and Systems for Video Technology
- Published
- 2022-09
- Volume
- 32
- Issue
- 9
- Pages
- 5944-5958
- ISSN
- 1051-8215 (ISSN-L)
- 1558-2205
- OpenAlex
- W3193919962
- Cited by
- 171
- References
- 23
- FWCI
- 11.98
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.