Video Quality Pooling Adaptive to Perceptual Distortion Severity
Abstract
It is generally recognized that severe video distortions that are transient in space and/or time have a large effect on overall perceived video quality. In order to understand this phenomena, we study the distribution of spatio-temporally local quality scores obtained from several video quality assessment (VQA) algorithms on videos suffering from compression and lossy transmission over communication channels. We propose a content adaptive spatial and temporal pooling strategy based on the observed distribution. Our method adaptively emphasizes "worst" scores along both the spatial and temporal dimensions of a video sequence and also considers the perceptual effect of large-area cohesive motion flow such as egomotion. We demonstrate the efficacy of the method by testing it using three different VQA algorithms on the LIVE Video Quality database and the EPFL-PoliMI video quality database.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892 · see citing works
Self-citation(1975) · CERN Document Server (European Organization for Nuclear Research) · Cited by 5,958 · see citing works
Mean squared error: Love it or leave it? A new look at Signal Fidelity Measures
Wang, Z., & Bovik, A. C. (2009) · IEEE Signal Processing Magazine · doi:10.1109/msp.2008.930649 · Cited by 3,175 · see citing works
Self-citationGordon, A. D., & Hartigan, J. A. (1976) · Journal of the American Statistical Association · doi:10.2307/2286880 · Cited by 2,600 · see citing works
Handbook of Image and Video Processing
Bovik, A. C. (2005) · Elsevier eBooks · doi:10.1016/b978-0-12-119792-6.x5062-1 · Cited by 1,585 · see citing works
Self-citationA new standardized method for objectively measuring video quality
Pinson, M. H., & Wolf, S. (2004) · IEEE Transactions on Broadcasting · doi:10.1109/tbc.2004.834028 · Cited by 1,182 · see citing works
Study of Subjective and Objective Quality Assessment of Video
Seshadrinathan, K., Soundararajan, R., Bovik, A. C., & Cormack, L. K. (2010) · IEEE Transactions on Image Processing · doi:10.1109/tip.2010.2042111 · Cited by 1,160 · see citing works
Self-citationVSNR: A Wavelet-Based Visual Signal-to-Noise Ratio for Natural Images
Chandler, D. M., & Hemami, S. S. (2007) · IEEE Transactions on Image Processing · doi:10.1109/tip.2007.901820 · Cited by 1,147 · see citing works
H.263: video coding for low-bit-rate communication
Rijkse, K. (1996) · IEEE Communications Magazine · doi:10.1109/35.556485 · Cited by 1,040 · see citing works
Video quality assessment based on structural distortion measurement
Wang, Z., Lu, L., & Bovik, A. C. (2004) · Signal Processing: Image Communication · doi:10.1016/s0923-5965(03)00076-6 · Cited by 1,024 · see citing works
Self-citationNoise characteristics and prior expectations in human visual speed perception
Stocker, A. A., & Simoncelli, E. P. (2006) · Nature Neuroscience · doi:10.1038/nn1669 · Cited by 780 · see citing works
Motion Tuned Spatio-Temporal Quality Assessment of Natural Videos
Seshadrinathan, K., & Bovik, A. C. (2010) · IEEE Transactions on Image Processing · doi:10.1109/tip.2009.2034992 · Cited by 708 · see citing works
Self-citationVisual Importance Pooling for Image Quality Assessment
Moorthy, A. K., & Bovik, A. C. (2009) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2009.2015374 · Cited by 340 · see citing works
Self-citationPerceptual quality measure using a spatiotemporal model of the human visual system
Lambrecht, C. J. V. D. B., & Verscheure, O. (1996) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.235440 · Cited by 340 · see citing works
The Essential Guide to Image Processing
Bovik, A. C. 1. (2009) · Elsevier eBooks · doi:10.1016/b978-0-12-374457-9.x0001-7 · Cited by 316 · see citing works
Self-citationA survey of hybrid MC/DPCM/DCT video coding distortions
Yuen, M., & Wu, H. R. (1998) · Signal Processing · doi:10.1016/s0165-1684(98)00128-5 · Cited by 254 · see citing works
Issues in vision modeling for perceptual video quality assessment
WINKLER, S. S. (1999) · Signal Processing · doi:10.1016/s0165-1684(99)00062-6 · Cited by 223 · see citing works
Video quality assessment using a statistical model of human visual speed perception
Wang, Z., & Li, Q. (2007) · Journal of the Optical Society of America A · doi:10.1364/josaa.24.000b61 · Cited by 210 · see citing works
Spatial Pooling Strategies for Perceptual Image Quality Assessment
Wang, Z., & Shang, X. (2006) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2006.313136 · Cited by 198 · see citing works
Foveated video quality assessment
Lee, S., Pattichis, M. S., & Bovik, A. C. (2002) · IEEE Transactions on Multimedia · doi:10.1109/6046.985561 · Cited by 165 · see citing works
Self-citationConsidering Temporal Variations of Spatial Visual Distortions in Video Quality Assessment
Ninassi, A., Le Meur, O., Le Callet, P., & Barba, D. (2009) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2009.2014806 · Cited by 159 · see citing works
Subjective assessment of H.264/AVC video sequences transmitted over a noisy channel
Simone, F., Naccari, M., Tagliasacchi, M., et al. (2009) · International Workshop on Quality of Multimedia Experience (QoMEX) · doi:10.1109/qomex.2009.5246952 · Cited by 141 · see citing works
Temporal hysteresis model of time varying subjective video quality
Seshadrinathan, K., & Bovik, A. C. (2011) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2011.5946613 · Cited by 137 · see citing works
Self-citationFoveated video compression with optimal rate control
Lee, S., Pattichis, M. S., & Bovik, A. C. (2001) · IEEE Transactions on Image Processing · doi:10.1109/83.931092 · Cited by 135 · see citing works
Self-citationMotion Silences Awareness of Visual Change
Suchow, J. W., & Alvarez, G. A. (2011) · Current Biology · doi:10.1016/j.cub.2010.12.019 · Cited by 121 · see citing works
On a unified architecture for video-on-demand services
Lee, J. Y. B. (2002) · IEEE Transactions on Multimedia · doi:10.1109/6046.985552 · Cited by 120 · see citing works
A fast VLSI architecture for full-search variable block size motion estimation in MPEG-4 AVC/H.264
Kim, M., Hwang, I., & Chae, S. (2005) · Asia and South Pacific Design Automation Conference (ASP-DAC) · doi:10.1145/1120725.1120980 · Cited by 96 · see citing works
No-Reference Quality Assessment for Networked Video via Primary Analysis of Bit Stream
Yang, F., Wan, S., Xie, Q., & Wu, H. R. (2010) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2010.2087433 · Cited by 89 · see citing works
Low-delay rate control for real-time H.264/AVC video coding
Jiang, M., & Ling, N. (2006) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2006.870713 · Cited by 69 · see citing works
A metric for continuous quality evaluation of compressed video with severe distortions
Masry, M. A., & Hemami, S. S. (2003) · Signal Processing Image Communication · doi:10.1016/j.image.2003.08.001 · Cited by 68 · see citing works
Real-time motion estimation and visualization on graphics cards
Strzodka, R., & Garbe, C. S. (2005) · IEEE Visualization · doi:10.1109/visual.2004.88 · Cited by 68 · see citing works
Low-Complexity Video Quality Assessment Using Temporal Quality Variations
Narwaria, M., Lin, W., & Liu, A. (2012) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2012.2190589 · Cited by 64 · see citing works
Fast algorithms for foveated video processing
Lee, S., & Bovik, A. C. (2003) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2002.808441 · Cited by 61 · see citing works
Self-citation<title>Viewer response to time-varying video quality</title>
Pearson, D. E. (1998) · SPIE Proceedings · doi:10.1117/12.320109 · Cited by 43 · see citing works
On the performance of human visual system based image quality assessment metric using wavelet domain
Ninassi, A., Meur, O. L., Callet, P. L., et al. (2008) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.766536 · Cited by 36 · see citing works
Influence of temporal pooling method on the objective video quality evaluation
Rimac-Drlje, S., Vranješ, M., & Žagar, D. (2009) · IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB) · doi:10.1109/isbmsb.2009.5133781 · Cited by 33 · see citing works
Attention modeling for video quality assessment: Balancing global quality and local quality
You, J., Korhonen, J., & Perkis, A. (2010) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2010.5583037 · Cited by 32 · see citing works
A fast VLSI architecture for full-search variable block size motion estimation in MPEG-4 AVC/H.264
Kim, M., Hwang, I., & Chae, S. (2005) · Asia and South Pacific Design Automation Conference (ASP-DAC) · doi:10.1109/aspdac.2005.1466240 · Cited by 29 · see citing works
Optimal Channel Adaptation of Scalable Video Over a Multicarrier-Based Multicell Environment
Park, J.-C., Lee, H., Lee, S., et al. (2009) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2009.2026084 · Cited by 28 · see citing works
Self-citationOptimal Carrier Loading Control for the Enhancement of Visual Quality over OFDMA Cellular Networks
Jang, U. J., Lee, H., & Lee, S. (2008) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2008.2001380 · Cited by 24 · see citing works
Self-citationImproved H.264 rate control by enhanced MAD-based frame complexity prediction
Yi, X., & Ling, N. (2005) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2005.04.005 · Cited by 19 · see citing works
`Mean Time Between Failures': A Subjectively Meaningful Video Quality Metric
Suresh, N., & Jayant, N. S. (2006) · doi:10.1109/icassp.2006.1660499 · Cited by 16 · see citing works
Perceptually motivated spatial and temporal integration of pixel based video quality measures
Barkowsky, M., Eskofier, B. M., Bitto, R., et al. (2007) · doi:10.1145/1577504.1577508 · Cited by 15 · see citing works
Influence of temporal pooling method on the objective video quality evaluation
Rimac-Drlje, S., Vranješ, M., & Žagar, D. (2009) · IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB) · doi:10.1109/isbmsb.2009.5133782 · Cited by 15 · see citing works
Spatio-temporal quality pooling accounting for transient severe impairments and egomotion
Park, J.-C., Seshadrinathan, K., Lee, S., & Bovik, A. C. (2011) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2011.6116172 · Cited by 13 · see citing works
Self-citationHigh quality, low delay foveated visual communications over mobile channels
Lee, S., Bovik, A. C., & Kim, Y. Y. (2004) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2004.06.001 · Cited by 11 · see citing works
Self-citationCiaramello, F. M., & Hemami, S. S. (2010) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2010.5653630 · Cited by 4 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Quality Assessment of In-the-Wild Videos
Li, D., Jiang, T., & Jiang, M. (2019) · Proceedings of the 27th ACM International Conference on Multimedia · doi:10.1145/3343031.3351028 · Cited by 387 · see citing works
UGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Content
Tu, Z., Wang, Y., Birkbeck, N., Adsumilli, B., & Bovik, A. C. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3072221 · Cited by 363 · see citing works
“Temporal pooling is one of the most important, unresolved problems for video quality prediction [42], [60], [76], [82], [83].”
BackgroundMethodSelf-citationBBAND INDEX: A NO-REFERENCE BANDING ARTIFACT PREDICTOR
Tu, Z., Lin, J., Wang, Y., Adsumilli, B., & Bovik, A. C. (2020) · ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp40776.2020.9053634 · Cited by 46 · see citing works
Self-citation
- Authors
- Park, J., Seshadrinathan, K., Lee, S., & Bovik, A. C.
- Affiliations
- Intel (United States) (US)
- Yonsei University (KR)
- The University of Texas at Austin (US)
- Venue
- IEEE Transactions on Image Processing
- Published
- 2013-02
- Volume
- 22
- Issue
- 2
- Pages
- 610-620
- ISSN
- 1057-7149 (ISSN-L)
- 1941-0042
- OpenAlex
- W2074883210
- Cited by
- 119
- References
- 47
- Citations by year
- 2025: 1 · 2024: 11 · 2023: 4 · 2022: 7 · 2021: 11 · 2020: 7 · 2019: 6 · 2018: 8 · 2017: 11 · 2016: 12 · 2015: 19 · 2014: 15 · 2013: 7
- FWCI
- 13.51
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Image Processing (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Video Quality Assessment
- Physical Sciences › Computer Science › Signal Processing › Video Coding and Compression Technologies
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
- Keywords
- video quality assessment, perceptual video quality, spatial pooling, temporal pooling, compression distortion, self-motion
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review