An Optical Flow-Based Full Reference Video Quality Assessment Algorithm
The DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Abstract
We present a simple yet effective optical flow-based full-reference video quality assessment (FR-VQA) algorithm for assessing the perceptual quality of natural videos. Our algorithm is based on the premise that local optical flow statistics are affected by distortions and the deviation from pristine flow statistics is proportional to the amount of distortion. We characterize the local flow statistics using the mean, the standard deviation, the coefficient of variation (CV), and the minimum eigenvalue ( λ min ) of the local flow patches. Temporal distortion is estimated as the change in the CV of the distorted flow with respect to the reference flow, and the correlation between λ min of the reference and of the distorted patches. We rely on the robust multi-scale structural similarity index for spatial quality estimation. The computed temporal and spatial distortions, thus, are then pooled using a perceptually motivated heuristic to generate a spatio-temporal quality score. The proposed method is shown to be competitive with the state-of-the-art when evaluated on the LIVE SD database, the EPFL Polimi SD database, and the LIVE Mobile HD database. The distortions considered in these databases include those due to compression, packet-loss, wireless channel errors, and rate-adaptation. Our algorithm is flexible enough to allow for any robust FR spatial distortion metric for spatial distortion estimation. In addition, the proposed method is not only parameter-free but also independent of the choice of the optical flow algorithm. Finally, we show that the replacement of the optical flow vectors in our proposed method with the much coarser block motion vectors also results in an acceptable FR-VQA algorithm. Our algorithm is called the flow similarity index.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
Multiscale structural similarity for image quality assessment
Wang, Z., Simoncelli, E. P., & Bovik, A. C. (2004) · The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003 · doi:10.1109/acssc.2003.1292216 · Cited by 6,253
Structural and Functional Brain Networks: From Connections to Cognition
Park, H.-J., & Friston, K. (2013) · Science · doi:10.1126/science.1238411 · Cited by 2,106
Most apparent distortion: full-reference image quality assessment and the role of strategy
Larson, E. C., & Chandler, D. M. (2010) · Journal of Electronic Imaging · doi:10.1117/1.3267105 · Cited by 1,953 · cited 1×
“A video quality assessment model based on Most Apparent Distortion (MAD) [44] called Spatiotemporal MAD (ST-MAD) [45], is designed on the assumption that motion artifacts will manifest as spatial artifacts and the appearance-based model of MAD can measure these changes to agree with human perception.”
MethodA new standardized method for objectively measuring video quality
Pinson, M. H., & Wolf, S. (2004) · IEEE Transactions on Broadcasting · doi:10.1109/tbc.2004.834028 · Cited by 1,182 · cited 2×
“This is an extension to their previous work called video quality metric (VQM) [37].”
BackgroundMethodStudy of Subjective and Objective Quality Assessment of Video
Seshadrinathan, K., Soundararajan, R., Bovik, A. C., & Cormack, L. K. (2010) · IEEE Transactions on Image Processing · doi:10.1109/tip.2010.2042111 · Cited by 1,160 · cited 1×
“We have evaluated the performance of our algorithm on two SD databases: the EPFL-PoliMI database [15, 16, 17], the LIVE Video database [18] and a HD database: LIVE Mobile VQA database [19, 20].”
MethodVSNR: A Wavelet-Based Visual Signal-to-Noise Ratio for Natural Images
Chandler, D. M., & Hemami, S. S. (2007) · IEEE Transactions on Image Processing · doi:10.1109/tip.2007.901820 · Cited by 1,147 · cited 1×
“Yet another popular approach is based on the observation that the HVS does not perceive all the distortions equally [12]–[14], [33], [37], [38].”
BackgroundVideo quality assessment based on structural distortion measurement
Wang, Z., Lu, L., & Bovik, A. C. (2004) · Signal Processing: Image Communication · doi:10.1016/s0923-5965(03)00076-6 · Cited by 1,024
STRUCTURE AND FUNCTION OF VISUAL AREA MT
Born, R. T., & Bradley, D. C. (2005) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.26.041002.131052 · Cited by 1,023 · cited 2×
“Visual area V5/MT plays a major role in the perception of motion and the guidance of some eye movements [8].”
BackgroundA model of neuronal responses in visual area MT
Simoncelli, E. P., & Heeger, D. J. (1998) · Vision Research · doi:10.1016/s0042-6989(97)00183-1 · Cited by 941 · cited 3×
“It is hypothesized that the optical flow field is computed and represented by neurons in area MT [3], [11], and is central to our perception of motion.”
Background
Cited by
Perceptual video quality assessment: a survey
Min, X., Duan, H., Sun, W., Zhu, Y., & Zhai, G. (2024) · Science China Information Sciences · doi:10.1007/s11432-024-4133-3 · Cited by 218
Spatiotemporal Feature Integration and Model Fusion for Full Reference Video Quality Assessment
Bampis, C. G., Li, Z., & Bovik, A. C. (2019) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2018.2868262 · Cited by 128
Recurrent and Dynamic Models for Predicting Streaming Video Quality of Experience
Bampis, C. G., Li, Z., Katsavounidis, I., & Bovik, A. C. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2815842 · Cited by 84
Metadata
- Authors
- Manasa, K., & Channappayya, S. S.
- Venue
- IEEE Transactions on Image Processing
- Published
- 2016-06
- Volume
- 25
- Issue
- 6
- Pages
- 2480-2492
- ISSN
- 1057-7149 (ISSN-L)
- 1941-0042
- OpenAlex
- W2314091824
- Cited by
- 93
- References
- 20
- FWCI
- 7.30
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.