Video quality assessment accounting for temporal visual masking of local flicker
Open accessThe DOI matches its Crossref record; 28 of the 28 references it cites were found in a registry or an index.
Abstract
An important element of the design of video quality assessment (VQA) models that remains poorly understood is the effect of temporal visual masking on the visibility of temporal distortions. The visibility of temporal distortions like local flicker can be strongly reduced by motion. Based on a recently discovered visual change silencing illusion, we have developed a full reference VQA model that accounts for temporal visual masking of local flicker. The proposed model, called Flicker Sensitive-MOtion-based Video Integrity Evaluation (FS-MOVIE), augments the well-known MOVIE Index by combining motion tuned video integrity features with a new perceptual flicker visibility/masking index. FS-MOVIE captures the separated spectral signatures caused by local flicker distortions, by using a model of the responses of neurons in primary visual cortex to video flicker, an energy model of motion perception, and a divisive normalization stage. FS-MOVIE predicts the perceptual suppression of local flicker by the presence of motion and evaluates local flicker as it affects video quality. Experimental results show that FS-MOVIE significantly improves VQA performance against its predecessor and is highly competitive with top performing VQA algorithms when tested on the LIVE, IVP, EPFL, and VQEGHD5 VQA databases.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
Self-citationMultiscale structural similarity for image quality assessment
Wang, Z., Simoncelli, E. P., & Bovik, A. C. (2004) · The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003 · doi:10.1109/acssc.2003.1292216 · Cited by 6,253 · cited 6×
“For example, the first symbol value of ‘1’ at the second row and at the first column in Table 3 means that MS-SSIM is statistically better than PSNR on the LIVE VQA database.”
MethodInfluentialSelf-citationDaugman, J. G. (1985) · Journal of the Optical Society of America A · doi:10.1364/josaa.2.001160 · Cited by 3,219
Normalization of cell responses in cat striate cortex
Heeger, D. J. (1992) · Visual Neuroscience · doi:10.1017/s0952523800009640 · Cited by 1,835 · cited 7×
“Note that summing the energy outputs in (14) over all φ and θ yields the total Fourier energy of the stimulus [54], where the normalization could also be computed locally by summing over a limited region of space and a limited range of frequencies [54].”
BackgroundMethodInfluentialMultichannel texture analysis using localized spatial filters
Bovik, A. C., Clark, M., & Geisler, W. S. (1990) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.41384 · Cited by 1,382 · cited 2×
“The proposed model, called Flicker Sensitive-Motion-based Video Integrity Evaluation (FS-MOVIE), computes bandpass filter responses on reference and distorted videos using a spatiotemporal Gabor filter bank [51,52], then deploys a model of the responses of V1 neurons through an energy model of motion perception [53] a…”
MethodSelf-citationA new standardized method for objectively measuring video quality
Pinson, M. H., & Wolf, S. (2004) · IEEE Transactions on Broadcasting · doi:10.1109/tbc.2004.834028 · Cited by 1,182 · cited 13×
“VQM-VFD, ST-MAD, and Temporal FSMOVIE achieved higher PLCC (over 0.8) on the low bitrate videos, while ST-MAD obtained higher PLCC (over 0.7) on the high bitrate videos.”
BackgroundMethodInfluentialModel of human visual-motion sensing
Watson, A. B., & Ahumada, A. J. (1985) · Journal of the Optical Society of America A · doi:10.1364/josaa.2.000322 · Cited by 1,170 · cited 2×
“where u, v, and w are spatial and temporal frequency variables corresponding to (x, y) and t, respectively [69].”
BackgroundStudy of Subjective and Objective Quality Assessment of Video
Seshadrinathan, K., Soundararajan, R., Bovik, A. C., & Cormack, L. K. (2010) · IEEE Transactions on Image Processing · doi:10.1109/tip.2010.2042111 · Cited by 1,160 · cited 5×
“Signaocessing:Im ge om m unation 67 ((2)018) 18(2)–198 Table 2 Comparison of VQA algorithm performances on the LIVE [55], IVP [56], EPFL [57], and VQEGHD5 [58] VQA databases: (A) Spearman Rank Ordered Correlation Coefficient (SROCC) and (B) Pearson Linear Correlation Coefficient (PLCC) between the algorithm prediction…”
MethodResultInfluentialSelf-citationChange blindness: Past, present, and future
Simons, D. J., & Rensink, R. A. (2005) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2004.11.006 · Cited by 1,153
VSNR: A Wavelet-Based Visual Signal-to-Noise Ratio for Natural Images
Chandler, D. M., & Hemami, S. S. (2007) · IEEE Transactions on Image Processing · doi:10.1109/tip.2007.901820 · Cited by 1,147 · cited 5×
“Frame-based VQA algorithms such as PSNR, MS-SSIM, 189 L.K.Choi,A.C.Bovik SignalProcessing:Im age Com m unication 67 (2018) 182–198 and VSNR were applied frame-by-frame and the average score across all frame used as a final measure of quality.”
MethodInfluentialComputation of component image velocity from local phase information
Fleet, D. J., & Jepson, A. D. (1990) · International Journal of Computer Vision · doi:10.1007/bf00056772 · Cited by 1,085
The effects of a visual fidelity criterion of the encoding of images
Mannos, J., & Sakrison, D. (1974) · IEEE Transactions on Information Theory · doi:10.1109/tit.1974.1055250 · Cited by 1,063 · cited 1×
“Early human visual system (HVS) based VQA models include Mannos and Sakrison’s metric [9], the Visual Differences Predictor (VDP) [10], Sarnoff Just Noticeable Differences (JND) Vision Model [11], Moving Pictures Quality Metric (MPQM) [12], the Perceptual Distortion Metric (PDM) [13], and the Digital Video Quality (DV…”
MethodVideo quality assessment based on structural distortion measurement
Wang, Z., Lu, L., & Bovik, A. C. (2004) · Signal Processing: Image Communication · doi:10.1016/s0923-5965(03)00076-6 · Cited by 1,024
Self-citationA model of neuronal responses in visual area MT
Simoncelli, E. P., & Heeger, D. J. (1998) · Vision Research · doi:10.1016/s0042-6989(97)00183-1 · Cited by 941
Motion Tuned Spatio-Temporal Quality Assessment of Natural Videos
Seshadrinathan, K., & Bovik, A. C. (2010) · IEEE Transactions on Image Processing · doi:10.1109/tip.2009.2034992 · Cited by 708 · cited 10×
“The Temporal MOVIE Index measures motion quality relevant to temporal distortions, whereby errors are computed between the motion tuned responses of reference and distorted video sequences to evaluate temporal video integrity [24].”
BackgroundMethodInfluentialSelf-citationEnns, J. T., & Di Lollo, V. (2000) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(00)01520-5 · Cited by 684
Blind Prediction of Natural Video Quality
Saad, M. A., Bovik, A. C., & Charrier, C. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2014.2299154 · Cited by 471 · cited 1×
“Differences (STRRED) [26], Video-BLind Image Integrity Notator using DCT-Statistics (V-BLIINDS) [27], and VQM-Variable Frame Delays (VQM-VFD) [28] are examples that include more sophisticated temporal aspects.”
MethodSelf-citationTeo, P. C., & Heeger, D. J. (2002) · Proceedings of 1st International Conference on Image Processing · doi:10.1109/icip.1994.413502 · Cited by 471
Video Quality Assessment by Reduced Reference Spatio-Temporal Entropic Differencing
Soundararajan, R., & Bovik, A. C. (2013) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2012.2214933 · Cited by 368
Self-citationVisual Importance Pooling for Image Quality Assessment
Moorthy, A. K., & Bovik, A. C. (2009) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2009.2015374 · Cited by 340
Self-citationDigital video quality metric based on human vision
Watson, A. B., Hu, Q. J., & McGowan, J. F. (2001) · Journal of Electronic Imaging · doi:10.1117/1.1329896 · Cited by 317
A survey of hybrid MC/DPCM/DCT video coding distortions
Yuen, M., & Wu, H. R. (1998) · Signal Processing · doi:10.1016/s0165-1684(98)00128-5 · Cited by 254 · cited 2×
“Hence, videos suffer not only from spatial distortions such as blocking, blurring, ringing, mosaic patterns, and noise, but also from temporal distortions such as motion compensation mismatches, flicker, mosquito effects, ghosting, jerkiness, smearing, and so forth [3].”
BackgroundA spatiotemporal most-apparent-distortion model for video quality assessment
Vu, P. V., Vu, C. T., & Chandler, D. M. (2011) · 2011 18th IEEE International Conference on Image Processing · doi:10.1109/icip.2011.6116171 · Cited by 180
Considering Temporal Variations of Spatial Visual Distortions in Video Quality Assessment
Ninassi, A., Le Meur, O., Le Callet, P., & Barba, D. (2009) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2009.2014806 · Cited by 159
Motion Silences Awareness of Visual Change
Suchow, J. W., & Alvarez, G. A. (2011) · Current Biology · doi:10.1016/j.cub.2010.12.019 · Cited by 121
Adaptation from invisible flicker
Shady, S., MacLeod, D. I. A., & Fisher, H. S. (2004) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0303452101 · Cited by 111 · cited 2×
“The perceptual sensitivity to flicker distortions can be affected by prolonged exposure to flickering stimuli [71].”
BackgroundA scalable wavelet-based video distortion metric and applications
Masry, M., Hemami, S. S., & Sermadevi, Y. (2006) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2005.861946 · Cited by 94 · cited 1×
“Later models include Structural Similarity (SSIM) [15], Multiscale- SSIM (MS-SSIM) [16], motion-based SSIM [17], Visual Information Fidelity (VIF) [18], Visual Signal-to-Noise Ratio (VSNR) [19], Video Quality Metric (VQM) [20] and the Scalable Wavelet Based Video Distortion Index [21].”
MethodTemporal Trajectory Aware Video Quality Measure
Barkowsky, M., Bialkowski, J., Eskofier, B., Bitto, R., & Kaup, A. (2009) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2009.2015375 · Cited by 75
Cited by
Quality Assessment of In-the-Wild Videos
Li, D., Jiang, T., & Jiang, M. (2019) · Proceedings of the 27th ACM International Conference on Multimedia · doi:10.1145/3343031.3351028 · Cited by 387
Perceptual video quality assessment: a survey
Min, X., Duan, H., Sun, W., Zhu, Y., & Zhai, G. (2024) · Science China Information Sciences · doi:10.1007/s11432-024-4133-3 · Cited by 218
A Comparative Evaluation Of Temporal Pooling Methods For Blind Video Quality Assessment
Tu, Z., Chen, C.-J., Chen, L.-H., Birkbeck, N., Adsumilli, B., & Bovik, A. C. (2020) · 2020 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip40778.2020.9191169 · Cited by 71
“Another popular temporal memory modeling approach is hysteresis pooling [19], which has been justified in several video quality modeling papers [6, 9, 20].”
BackgroundSelf-citationVideo demoiréing with relation-based temporal consistency
Dai, P., Yu, X., Ma, L., Zhang, B., Li, J., Li, W., Shen, J., & Qi, X. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01710 · Cited by 26
“The reason might be that the mean of a patch reflects the brightness of that area, which is closely related to flickers [9].”
BackgroundStd-net: spatio-temporal decomposition network for video demoiréing with sparse transformers
Niu, Y., Xu, R., Lin, Z., & Liu, W. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3386604 · Cited by 12
Metadata
- Authors
- Choi, L. K., & Bovik, A. C.
- Venue
- Signal Processing: Image Communication
- Publisher
- Elsevier BV
- Published
- 2018-09
- Volume
- 67
- Pages
- 182-198
- ISSN
- 0923-5965 (ISSN-L)
- OpenAlex
- W2809703252
- Cited by
- 33
- References
- 28
- FWCI
- 1.99
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.