A scalable wavelet-based video distortion metric and applications
Abstract
Video distortion metrics based on models of the human visual system have traditionally used comparisons between the distorted signal and a reference signal to calculate distortions objectively. In video coding applications, this is not prohibitive. In quality monitoring applications, however, access to the reference signal is often limited. This paper presents a computationally efficient video distortion metric that can operate in full- or reduced-reference mode as required. The metric is based on a model of the human visual system implemented using the wavelet transform and separable filters. The visual model is parameterized using a set of video frames and the associated quality scores. The visual model's hierarchical structure, as well as the limited impact of fine scale distortions on quality judgments of severely impaired video, are exploited to build a framework for scaling the bitrate required to represent the reference signal. Two applications of the metric are also presented. In the first, the metric is used as the distortion measure in a rate-distortion optimized rate control algorithm for MPEG-2 video compression. The resulting compressed video sequences demonstrate significant improvements in visual quality over compressed sequences with allocations determined by the TM5 rate control algorithm operating with MPEG-2 at the same rate. In the second, the metric is used to estimate time series of objective quality scores for distorted video sequences using reference bitrates as low as 10 kb/s. The resulting quality scores more accurately model subjective quality recordings than do those estimated using the mean squared error as a distortion metric, while requiring a fraction of the bitrate used to represent the reference signal. The reduced-reference metric's performance is comparable to that of the full-reference metrics tested in the first Video Quality Experts Group evaluation.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
A Simplex Method for Function Minimization
Nelder, J. A., & Mead, R. (1965) · The Computer Journal · doi:10.1093/comjnl/7.4.308 · Cited by 28,905 · see citing works
Normalization of cell responses in cat striate cortex
Heeger, D. J. (1992) · Visual Neuroscience · doi:10.1017/s0952523800009640 · Cited by 1,835 · see citing works
Peli, E. (1990) · Journal of the Optical Society of America A · doi:10.1364/josaa.7.002032 · Cited by 1,310 · see citing works
An online algorithm for segmenting time series
Keogh, E., Chu, S., Hart, D., & Pazzani, M. (2002) · Proceedings 2001 IEEE International Conference on Data Mining · doi:10.1109/icdm.2001.989531 · Cited by 1,149 · see citing works
The steerable pyramid: a flexible architecture for multi-scale derivative computation
Simoncelli, E. P., & Freeman, W. T. (2002) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.1995.537667 · Cited by 1,077 · see citing works
Peli, E. (1988) · Annual Meeting Optical Society of America · doi:10.1364/oam.1988.fd8 · Cited by 940 · see citing works
The visible differences predictor: an algorithm for the assessment of image fidelity
Daly, S. J. (1993) · The MIT Press eBooks · Cited by 828 · see citing works
Visible differences predictor: an algorithm for the assessment of image fidelity
Daly, S. J. (1992) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.135952 · Cited by 718 · see citing works
Human luminance pattern-vision mechanisms: masking experiments require a new model
Foley, J. M. (1994) · Journal of the Optical Society of America A · doi:10.1364/josaa.11.001710 · Cited by 573 · see citing works
A no-reference perceptual blur metric
Marziliano, P., Frederic Dufaux, A., WINKLER, S. S., & Ebrahimi, T. (2003) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2002.1038902 · Cited by 568 · see citing works
Model of visual contrast gain control and pattern masking
Watson, A. B., & Solomon, J. A. (1997) · Journal of the Optical Society of America A · doi:10.1364/josaa.14.002379 · Cited by 472 · see citing works
Teo, P. C., & Heeger, D. J. (2002) · Proceedings of 1st International Conference on Image Processing · doi:10.1109/icip.1994.413502 · Cited by 471 · see citing works
Blind measurement of blocking artifacts in images
Wang, Z., Bovik, A. C., & Evan, B. L. (2002) · Proceedings 2000 International Conference on Image Processing (Cat. No.00CH37101) · doi:10.1109/icip.2000.899622 · Cited by 413 · see citing works
Marcellin, M. W., Gormish, M. J., Bilgin, A., & Boliek, M. P. (2002) · DCC · doi:10.1109/dcc.2000.838192 · Cited by 406 · see citing works
The cortex transform: Rapid computation of simulated neural images
Watson, A. B. (1987) · Computer Vision Graphics and Image Processing · doi:10.1016/s0734-189x(87)80184-6 · Cited by 392 · see citing works
A VISUAL DISCRIMINATION MODEL FOR IMAGING SYSTEM DESIGN AND EVALUATION
Lubin, J. S. (1995) · WORLD SCIENTIFIC eBooks · doi:10.1142/9789812831200_0010 · Cited by 326 · see citing works
A generalized block-edge impairment metric for video coding
Wu, H. R., & Yuen, M. (1997) · IEEE Signal Processing Letters · doi:10.1109/97.641398 · Cited by 316 · see citing works
Perceptual distortion metric for digital color video
WINKLER, S. S. (1999) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.348438 · Cited by 252 · see citing works
A vector-magnitude model of contrast detection
Quick, R. (2004) · Kybernetik · doi:10.1007/bf00271628 · Cited by 248 · see citing works
A perceptually based adaptive sampling algorithm
Bolin, M. R., & Meyer, G. W. (1998) · ACM SIGGRAPH Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH) · doi:10.1145/280814.280924 · Cited by 223 · see citing works
Temporal properties of human visual filters: number, shapes and spatial covariation
Hess, R. F., & Snowden, R. J. (1992) · Vision Research · doi:10.1016/0042-6989(92)90112-v · Cited by 205 · see citing works
Spatial-temporal distortion metric for in-service quality monitoring of any digital video system
Wolf, S., & Pinson, M. (1999) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.371210 · Cited by 169 · see citing works
A wavelet visible difference predictor
Bradley, A. P. (1999) · IEEE Transactions on Image Processing · doi:10.1109/83.760338 · Cited by 150 · see citing works
No-reference sharpness metric based on local edge kurtosis
Caviedes, J. E., & Gurbuz, S. (2003) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2002.1038901 · Cited by 141 · see citing works
Toward a perceptual video-quality metric
Watson, A. B. (1998) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.320105 · Cited by 110 · see citing works
<title>Perceptual image distortion</title>
Teo, P. C., & Heeger, D. J. (1994) · SPIE Proceedings · doi:10.1117/12.172664 · Cited by 104 · see citing works
Perceptual models and architectures for video coding applications
Lambrecht, C. V. D. B. (1996) · doi:10.5075/epfl-thesis-1520 · Cited by 101 · see citing works
Peli, E. (1997) · Vision Research · doi:10.1016/s0042-6989(96)00262-3 · Cited by 83 · see citing works
Minkowski-metrics as a combination rule for digital-image-coding impairments
de Ridder, H. (1992) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.135953 · Cited by 77 · see citing works
A hexagonal orthogonal-oriented pyramid as a model of image representation in visual cortex
Watson, A. B., & Ahumada, A. J. (1989) · IEEE Transactions on Biomedical Engineering · doi:10.1109/10.16453 · Cited by 73 · see citing works
A metric for continuous quality evaluation of compressed video with severe distortions
Masry, M. A., & Hemami, S. S. (2003) · Signal Processing Image Communication · doi:10.1016/j.image.2003.08.001 · Cited by 68 · see citing works
Self-citationEstimating multiple temporal mechanisms in human vision
Fredericksen, R. E., & Hess, R. F. (1998) · Vision Research · doi:10.1016/s0042-6989(97)00239-3 · Cited by 64 · see citing works
Generalized rate-distortion optimization for motion-compensated video coders
Yang, Y. Q., & Hemami, S. S. (2000) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/76.867931 · Cited by 51 · see citing works
Self-citationQuality metric design: a closer look
WINKLER, S. S. (2000) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.387175 · Cited by 45 · see citing works
Time-varying Image Quality: Modeling the Relation between Instantaneous and Overall Quality
Hamberg, R., & de Ridder, H. (1999) · SMPTE Journal · doi:10.5594/j04337 · Cited by 45 · see citing works
Frame rate preferences in low bit rate video
Yadavalli, G., Masry, M. A., & Hemami, S. S. (2004) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2003.1246993 · Cited by 36 · see citing works
Self-citationTemporal detection in human vision: dependence on stimulus energy
Fredericksen, R. E., & Hess, R. F. (1997) · Journal of the Optical Society of America A · doi:10.1364/josaa.14.002557 · Cited by 31 · see citing works
Barten, P. G. J. (1989) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.952705 · Cited by 29 · see citing works
Image quality measurement using the Haar wavelet
Lai, Y., & Kuo, C. ‐. J. (1997) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.292789 · Cited by 24 · see citing works
Subjective quality evaluation of low-bit-rate video
Masry, M. A., Hemami, S. S., Osberger, W. M., et al. (2001) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.429539 · Cited by 21 · see citing works
Self-citationCVQE: a metric for continuous video quality evaluation at low bit rates
Masry, M. A., & Hemami, S. S. (2003) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.477774 · Cited by 20 · see citing works
Self-citationLinear programming optimization for video coding under multiple constraints
Sermadevi, Y., & Hemami, S. S. (2003) · DCC · doi:10.1109/dcc.2003.1193996 · Cited by 10 · see citing works
Self-citationContrast of slightly complex patterns: computing the perceived contrast of Gabor patches
Peli, E. (1996) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.238713 · Cited by 3 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Motion Tuned Spatio-Temporal Quality Assessment of Natural Videos
Seshadrinathan, K., & Bovik, A. C. (2010) · IEEE Transactions on Image Processing · doi:10.1109/tip.2009.2034992 · Cited by 708 · see citing works
Blind Prediction of Natural Video Quality
Saad, M. A., Bovik, A. C., & Charrier, C. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2014.2299154 · Cited by 471 · see citing works
Video Quality Assessment by Reduced Reference Spatio-Temporal Entropic Differencing
Soundararajan, R., & Bovik, A. C. (2013) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2012.2214933 · Cited by 368 · see citing works
RRED Indices: Reduced Reference Entropic Differencing for Image Quality Assessment
Soundararajan, R., & Bovik, A. C. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2011.2166082 · Cited by 337 · see citing works
ViS3: an algorithm for video quality assessment via analysis of spatial and spatiotemporal slices
Vu, P. V., & Chandler, D. M. (2014) · Journal of Electronic Imaging · doi:10.1117/1.jei.23.1.013016 · Cited by 185 · see citing works
Considering Temporal Variations of Spatial Visual Distortions in Video Quality Assessment
Ninassi, A., Le Meur, O., Le Callet, P., & Barba, D. (2009) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2009.2014806 · Cited by 159 · see citing works
No-reference image quality assessment based on DCT domain statistics
Brandão, T., & Queluz, M. P. (2008) · Signal Processing · doi:10.1016/j.sigpro.2007.09.017 · Cited by 147 · see citing works
- Authors
- Masry, M., Hemami, S. S., & Sermadevi, Y.
- Affiliations
- Cornell University (US)
- Microsoft (United States) (US)
- ORCID
- Masry, M.
- Venue
- IEEE Transactions on Circuits and Systems for Video Technology
- Published
- 2006-02
- Volume
- 16
- Issue
- 2
- Pages
- 260-273
- ISSN
- 1051-8215 (ISSN-L)
- OpenAlex
- W2163160830
- Cited by
- 94
- References
- 43
- Citations by year
- 2024: 1 · 2022: 1 · 2020: 1 · 2019: 2 · 2018: 2 · 2017: 2 · 2016: 7 · 2015: 8 · 2014: 5 · 2013: 10 · 2012: 11
- FWCI
- 7.47
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Circuits and Systems for Video Technology (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Video Quality Assessment
- Physical Sciences › Computer Science › Signal Processing › Video Coding and Compression Technologies
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Data Compression Techniques
- Keywords
- human visual system model, wavelet transform, separable filters, reduced-reference quality assessment, full-reference quality assessment, rate-distortion optimization, rate control, low bit rate video, video quality monitoring
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
