Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 94 of the 94 references it cites were found in a registry or an index; 10 of the 27 sources its claims cite were checked for support.

UGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Content

Open access
Tu, Zhengzhong, Wang, Yilin, Birkbeck, Neil, Adsumilli, Balu, Bovik, Alan C. · 2021 · IEEE Transactions on Image Processing

Abstract

Recent years have witnessed an explosion of user-generated content (UGC) videos shared and streamed over the Internet, thanks to the evolution of affordable and reliable consumer capture devices, and the tremendous popularity of social media platforms. Accordingly, there is a great need for accurate video quality assessment (VQA) models for UGC/consumer videos to monitor, control, and optimize this vast content. Blind quality prediction of in-the-wild videos is quite challenging, since the quality degradations of UGC videos are unpredictable, complicated, and often commingled. Here we contribute to advancing the UGC-VQA problem by conducting a comprehensive evaluation of leading no-reference/blind VQA (BVQA) features and models on a fixed evaluation architecture, yielding new empirical insights on both subjective video quality studies and objective VQA model design. By employing a feature selection strategy on top of efficient BVQA models, we are able to extract 60 out of 763 statistical features used in existing methods to create a new fusion-based model, which we dub the VIDeo quality EVALuator (VIDEVAL), that effectively balances the trade-off between VQA performance and efficiency. Our experimental results show that VIDEVAL achieves state-of-the-art performance at considerably lower computational cost than other leading models. Our study protocol also defines a reliable benchmark for the UGC-VQA problem, which we believe will facilitate further research on deep learning-based VQA modeling, as well as perceptually-optimized efficient UGC video processing, transcoding, and streaming. To promote reproducible research and public evaluation, an implementation of VIDEVAL has been made available online: https://github.com/vztu/VIDEVAL.

Overview

CoChat Summary

TL;DR

This paper benchmarks blind video quality assessment (BVQA) models on large-scale user-generated content (UGC) video databases and proposes VIDEVAL, a feature-fusion model selected from 60 of 763 statistical features. VIDEVAL achieves state-of-the-art or near-state-of-the-art performance at lower computational cost than leading models, and the study provides a reproducible benchmark for UGC-VQA.

Key findings

  • VIDEVAL selected 60 of 763 features and achieved SOTA/near-SOTA on all test sets: All-Combinedc SRCC 0.7960, PLCC 0.7939; KoNViD-1k SRCC 0.7832, PLCC 0.7803; LIVE-VQC SRCC 0.7522, PLCC 0.7514; YouTube-UGC SRCC 0.7787, PLCC 0.7733.
  • TLVQM outperformed other BVQA models on LIVE-VQC by a notable margin (SRCC 0.7988, PLCC 0.8025) but underperformed on YouTube-UGC (SRCC 0.6693, PLCC 0.6590); models containing temporal features excelled on LIVE-VQC.
  • Completely blind NIQE, ILNIQE, and VIIDEO performed poorly; on All-Combinedc, NIQE had SRCC 0.4622/PLCC 0.4773 and VIIDEO had SRCC 0.1039/PLCC 0.1621.
  • Deep CNN baselines transferred well: ResNet-50 on KoNViD-1k had SRCC 0.8018/PLCC 0.8104; ensembling VIDEVAL with deep features improved performance by up to ∼4%, e.g., VIDEVAL+ResNet-50 on All-Combinedc had SRCC 0.8115/PLCC 0.8286.

Methods

  • Evaluated leading BVQA/BIQA models on KoNViD-1k (1,200 videos), LIVE-VQC (585), YouTube-UGC (1,380; 1,323 color videos as YouTube-UGCc), and All-Combinedc (3,108) after INLSA MOS calibration; used SVR regression, 80%/20% train-test splits repeated 100 times, and reported SRCC, KRCC, PLCC, and RMSE.
  • VIDEVAL was built from an initial 763-dimensional feature set drawn from BRISQUE, GM-LOG, HIGRADE-GRAD, FRIQUEE variants, and TLVQM; feature selection used random forest, linear SVM, and SFFS, choosing SVM importance with k=60 over 100 random splits; spatial features were computed every second frame within 1-second chunks with average and standard deviation pooling, then average pooled across chunks.
  • Additional analyses included categorical subsets by resolution, content, and quality; cross-dataset generalization; eleven temporal pooling methods; complexity/runtime comparisons; and ensembling VIDEVAL with deep features.

Limitations

  • Paper-stated: MOS standard deviation analysis is not presented because LIVE-VQC does not provide it; YouTube-UGC contains 57 grayscale videos that cause numerical errors for FRIQUEE, so color models were evaluated on a 1,323-video subset; temporal pooling results are only self-consistent and not comparable to prior experiments; not enough samples (8–20) per video were available for statistically meaningful pooling; non-natural/computer-generated content remains hard and lacks sufficiently large UGC-VQA datasets.
  • Evident from the text: the benchmark covers three databases and a calibrated composite; categorical subsets for animation (81), screen content (163), and gaming (209) are small; cross-dataset generalization is modest (best SRCC values in Tables IX/X range roughly 0.33–0.70); and the deep models evaluated are mostly image-pretrained or picture-quality models rather than video-trained temporal CNNs.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1109/tip.2021.3072221

Cites

94 of 94 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

7 Valid · 1 Not valid · 9 Not checked · 69 Own results and interpretations

43 more claims

Create a free account to see all 86 claims and to verify the 6 references still open.

See all claims — free account
  • Probably not a substantial claim. 24
  • It may state two separate claims as one. 20

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

4 of 4

Citations

Cites

Show all 81 references

Research works printed but not linked: 4

  1. li, 2016, Toward A Practical Perceptual Video Quality Metric
  2. van der maaten, 2008, Visualizing data using t-SNE, J Mach Learn Res, 9, 2579
  3. zhang, 2011, IVP subjective quality video database
  4. 2010, VQEG HDTV Phase I Database

Cited by

See all 363

Metadata

Authors
Tu, Z., Wang, Y., Birkbeck, N., Adsumilli, B., & Bovik, A. C.
Affiliations
  • The University of Texas at Austin (US)
  • Google (United States) (US)
Venue
IEEE Transactions on Image Processing
Published
2021
Volume
30
Pages
4449-4464
ISSN
  • 1057-7149 (ISSN-L)
  • 1941-0042
OpenAlex
W3030701471
Cited by
363
References
81
Citations by year
2026: 40 · 2025: 75 · 2024: 84 · 2023: 74 · 2022: 46 · 2021: 36
FWCI
21.56
Type
journal article
Language
en
Open access
Locations
  • IEEE Transactions on Image Processing (publishedVersion)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
Google
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Video Quality Assessment
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image Processing Techniques
Keywords
blind video quality assessment, user-generated content, benchmark, feature selection, feature fusion, computational efficiency, perceptual video quality, in-the-wild videos, video streaming

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: UGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Content. https://app.cochat.ai/papers/10.1109/tip.2021.3072221
The paper
Tu, Z., Wang, Y., Birkbeck, N., Adsumilli, B., & Bovik, A. C. (2021). UGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Content. IEEE Transactions on Image Processing. https://doi.org/10.1109/tip.2021.3072221

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.