An Optical Flow-Based Full Reference Video Quality Assessment Algorithm
The DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Abstract
We present a simple yet effective optical flow-based full-reference video quality assessment (FR-VQA) algorithm for assessing the perceptual quality of natural videos. Our algorithm is based on the premise that local optical flow statistics are affected by distortions and the deviation from pristine flow statistics is proportional to the amount of distortion. We characterize the local flow statistics using the mean, the standard deviation, the coefficient of variation (CV), and the minimum eigenvalue ( λ min ) of the local flow patches. Temporal distortion is estimated as the change in the CV of the distorted flow with respect to the reference flow, and the correlation between λ min of the reference and of the distorted patches. We rely on the robust multi-scale structural similarity index for spatial quality estimation. The computed temporal and spatial distortions, thus, are then pooled using a perceptually motivated heuristic to generate a spatio-temporal quality score. The proposed method is shown to be competitive with the state-of-the-art when evaluated on the LIVE SD database, the EPFL Polimi SD database, and the LIVE Mobile HD database. The distortions considered in these databases include those due to compression, packet-loss, wireless channel errors, and rate-adaptation. Our algorithm is flexible enough to allow for any robust FR spatial distortion metric for spatial distortion estimation. In addition, the proposed method is not only parameter-free but also independent of the choice of the optical flow algorithm. Finally, we show that the replacement of the optical flow vectors in our proposed method with the much coarser block motion vectors also results in an acceptable FR-VQA algorithm. Our algorithm is called the flow similarity index.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
Multiscale structural similarity for image quality assessment
Wang, Z., Simoncelli, E. P., & Bovik, A. C. (2004) · The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003 · doi:10.1109/acssc.2003.1292216 · Cited by 6,253
Structural and Functional Brain Networks: From Connections to Cognition
Park, H.-J., & Friston, K. (2013) · Science · doi:10.1126/science.1238411 · Cited by 2,106
Most apparent distortion: full-reference image quality assessment and the role of strategy
Larson, E. C., & Chandler, D. M. (2010) · Journal of Electronic Imaging · doi:10.1117/1.3267105 · Cited by 1,953 · cited 1×
“A video quality assessment model based on Most Apparent Distortion (MAD) [44] called Spatiotemporal MAD (ST-MAD) [45], is designed on the assumption that motion artifacts will manifest as spatial artifacts and the appearance-based model of MAD can measure these changes to agree with human perception.”
MethodA new standardized method for objectively measuring video quality
Pinson, M. H., & Wolf, S. (2004) · IEEE Transactions on Broadcasting · doi:10.1109/tbc.2004.834028 · Cited by 1,182 · cited 2×
“This is an extension to their previous work called video quality metric (VQM) [37].”
BackgroundMethodStudy of Subjective and Objective Quality Assessment of Video
Seshadrinathan, K., Soundararajan, R., Bovik, A. C., & Cormack, L. K. (2010) · IEEE Transactions on Image Processing · doi:10.1109/tip.2010.2042111 · Cited by 1,160 · cited 1×
“We have evaluated the performance of our algorithm on two SD databases: the EPFL-PoliMI database [15, 16, 17], the LIVE Video database [18] and a HD database: LIVE Mobile VQA database [19, 20].”
MethodVSNR: A Wavelet-Based Visual Signal-to-Noise Ratio for Natural Images
Chandler, D. M., & Hemami, S. S. (2007) · IEEE Transactions on Image Processing · doi:10.1109/tip.2007.901820 · Cited by 1,147 · cited 1×
“Yet another popular approach is based on the observation that the HVS does not perceive all the distortions equally [12]–[14], [33], [37], [38].”
BackgroundVideo quality assessment based on structural distortion measurement
Wang, Z., Lu, L., & Bovik, A. C. (2004) · Signal Processing: Image Communication · doi:10.1016/s0923-5965(03)00076-6 · Cited by 1,024
STRUCTURE AND FUNCTION OF VISUAL AREA MT
Born, R. T., & Bradley, D. C. (2005) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.26.041002.131052 · Cited by 1,023 · cited 2×
“Visual area V5/MT plays a major role in the perception of motion and the guidance of some eye movements [8].”
BackgroundA model of neuronal responses in visual area MT
Simoncelli, E. P., & Heeger, D. J. (1998) · Vision Research · doi:10.1016/s0042-6989(97)00183-1 · Cited by 941 · cited 3×
“It is hypothesized that the optical flow field is computed and represented by neurons in area MT [3], [11], and is central to our perception of motion.”
BackgroundMotion Tuned Spatio-Temporal Quality Assessment of Natural Videos
Seshadrinathan, K., & Bovik, A. C. (2010) · IEEE Transactions on Image Processing · doi:10.1109/tip.2009.2034992 · Cited by 708 · cited 16×
“In this work, we present a novel NR-VQA algorithm that attempts to quantify perceptual distortion by estimating statistical irregularities in optical flow using features at the patch level and the frame level.”
BackgroundMethodInfluentialBlind Prediction of Natural Video Quality
Saad, M. A., Bovik, A. C., & Charrier, C. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2014.2299154 · Cited by 471 · cited 4×
“We would like to note that Video BLIINDS was tested using the parameters supplied by its authors.”
MethodInfluentialVideo Quality Assessment by Reduced Reference Spatio-Temporal Entropic Differencing
Soundararajan, R., & Bovik, A. C. (2013) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2012.2214933 · Cited by 368
Objective video quality assessment
(2003) · Handbook of Video Databases · doi:10.1201/9780203489864-45 · Cited by 359
Neurons in monkey visual area V2 encode combinations of orientations
Anzai, A., Peng, X., & Van Essen, D. C. (2007) · Nature Neuroscience · doi:10.1038/nn1975 · Cited by 244 · cited 1×
“The neurons in the area V2 are tuned to simple visual characteristics such as orientation, spatial frequency, size, color, and shape [4]–[6].”
BackgroundVideo quality assessment using a statistical model of human visual speed perception
Wang, Z., & Li, Q. (2007) · Journal of the Optical Society of America A · doi:10.1364/josaa.24.000b61 · Cited by 210
A spatiotemporal most-apparent-distortion model for video quality assessment
Vu, P. V., Vu, C. T., & Chandler, D. M. (2011) · 2011 18th IEEE International Conference on Image Processing · doi:10.1109/icip.2011.6116171 · Cited by 180
Considering Temporal Variations of Spatial Visual Distortions in Video Quality Assessment
Ninassi, A., Le Meur, O., Le Callet, P., & Barba, D. (2009) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2009.2014806 · Cited by 159
Video Quality Pooling Adaptive to Perceptual Distortion Severity
Park, J., Seshadrinathan, K., Lee, S., & Bovik, A. C. (2013) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2219551 · Cited by 119 · cited 14×
“Importantly, this also ensures that frames that have uniform (or consistent) distortion are not classified as having high perceptual annoyance [14].”
BackgroundMethodInfluentialSpatio-temporal frequency separability in area 18 neurons of the cat
Friend, S. M., & Baker, C. L. (1993) · Vision Research · doi:10.1016/0042-6989(93)90167-u · Cited by 28 · cited 2×
“almost separable in the spatial and temporal fields [48].”
Background
Cited by
Perceptual video quality assessment: a survey
Min, X., Duan, H., Sun, W., Zhu, Y., & Zhai, G. (2024) · Science China Information Sciences · doi:10.1007/s11432-024-4133-3 · Cited by 218
Spatiotemporal Feature Integration and Model Fusion for Full Reference Video Quality Assessment
Bampis, C. G., Li, Z., & Bovik, A. C. (2019) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2018.2868262 · Cited by 128
Recurrent and Dynamic Models for Predicting Streaming Video Quality of Experience
Bampis, C. G., Li, Z., Katsavounidis, I., & Bovik, A. C. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2815842 · Cited by 84
Metadata
- Authors
- Manasa, K., & Channappayya, S. S.
- Venue
- IEEE Transactions on Image Processing
- Published
- 2016-06
- Volume
- 25
- Issue
- 6
- Pages
- 2480-2492
- ISSN
- 1057-7149 (ISSN-L)
- 1941-0042
- OpenAlex
- W2314091824
- Cited by
- 93
- References
- 20
- FWCI
- 7.30
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.