CoChat

Motion Tuned Spatio-Temporal Quality Assessment of Natural Videos

Seshadrinathan, K., Bovik, A.C. · 2010 · IEEE Transactions on Image Processing

The DOI matches its Crossref record; 91 of the 91 references it cites were found in a registry or an index; 12 of the 89 sources its claims cite were checked for support.

Abstract

There has recently been a great deal of interest in the development of algorithms that objectively measure the integrity of video signals. Since video signals are being delivered to human end users in an increasingly wide array of applications and products, it is important that automatic methods of video quality assessment (VQA) be available that can assist in controlling the quality of video being delivered to this critical audience. Naturally, the quality of motion representation in videos plays an important role in the perception of video quality, yet existing VQA algorithms make little direct use of motion information, thus limiting their effectiveness. We seek to ameliorate this by developing a general, spatio-spectrally localized multiscale framework for evaluating dynamic video fidelity that integrates both spatial and temporal (and spatio-temporal) aspects of distortion assessment. Video quality is evaluated not only in space and time, but also in space-time, by evaluating motion quality along computed motion trajectories. Using this framework, we develop a full reference VQA algorithm for which we coin the term the MOtion-based Video Integrity Evaluation index, or MOVIE index. It is found that the MOVIE index delivers VQA scores that correlate quite closely with human subjective judgment, using the Video Quality Expert Group (VQEG) FRTV Phase 1 database as a test bed. Indeed, the MOVIE index is found to be quite competitive with, and even outperform, algorithms developed and submitted to the VQEG FRTV Phase 1 study, as well as more recent VQA algorithms tested on this database.

Overview

CoChat Summary

TL;DR

The paper develops MOVIE, a full-reference video quality assessment index that integrates spatial and temporal/motion quality using spatio-temporal Gabor decomposition and motion trajectories. On the VQEG FRTV Phase 1 database, MOVIE correlates closely with subjective scores and is competitive with or outperforms several leading VQA algorithms.

Key findings

  • On the entire VQEG FRTV Phase 1 database, MOVIE achieved SROCC 0.833, LCC 0.821, and OR 0.644, compared with PSNR SROCC 0.786, Proponent P8 SROCC 0.803, and SSIM with weighting SROCC 0.812.
  • On only the 16 natural videos and their distorted versions (256 videos), MOVIE achieved SROCC 0.860, LCC 0.858, OR 0.656, and RMSE 8.093; PSNR had SROCC 0.739 and SSIM without weighting had SROCC 0.802.
  • Temporal MOVIE alone outperformed Spatial MOVIE on the entire database (SROCC 0.816 vs. 0.793) and on natural videos (SROCC 0.835 vs. 0.825), and combining both improved over either separately.
  • The RMSE between subjective scores and MOVIE scores after nonlinear regression on the entire VQEG database was 8.76; the outlier ratio using the three-sigma rule was 0.488 on the entire database and 0.461 on natural videos.

Methods

  • MOVIE is a full-reference VQA framework: reference and test videos are decomposed with a Gabor filterbank; spatial quality uses divisive normalization/contrast masking inspired by SSIM and information-theoretic IQA; temporal quality uses optical flow from the reference via a multi-scale Fleet–Jepson algorithm and motion-tuned weighted Gabor responses; spatial and temporal scores are pooled using coefficient-of-variation frame descriptors and a product.
  • Evaluation used the VQEG FRTV Phase 1 database: 20 reference sequences and 16 distorted versions of each, totaling 320 videos; the algorithm ran on one field of interlaced video and ignored color. A separate natural-video analysis used 16 references and 256 videos after omitting 4 animated sources.
  • Performance was measured with SROCC, LCC after logistic nonlinear regression, outlier ratio, and RMSE, and compared against PSNR, SSIM variants, and VQEG proponent models.

Limitations

  • The paper states that MOVIE ignores color information and uses only one field of interlaced video; color is listed as a future improvement.
  • The paper states that temporal pooling is a simple mean of frame-level descriptors, with more advanced temporal pooling left for future work; it also notes that the model is implicitly geared toward natural videos, and 4 animated sequences are omitted from the natural-video table because they contain synthetic properties and text.
  • The paper states that evaluation is limited to the VQEG FRTV Phase 1 database, that VQM performance is not reported because VQM was trained on VQEG data, and that some compared algorithms’ reported performance is from the 2000 VQEG study and may have improved since.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1109/tip.2009.2034992

Cites

91 of 91 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 Block diagram of MOVIE index. Flow of the reference video through the MOVIE VQA system is color coded in red, while flow of the test video is shown in blue. Both reference and test videos undergo linear decomposition using a Gabor filter family. Spatial and temporal quality is estimated using the Gabor coefficients from the reference and test videos. Temporal quality computation additionally uses reference motion information, computed using the reference Gabor coefficients. Spatial and temporal quality indices are then combined to produce the overall MOVIE index.
Fig. 1 Block diagram of MOVIE index. Flow of the reference video through the MOVIE VQA system is color coded in red, while flow of the test video is shown in blue. Both reference and test videos undergo linear decomposition using a Gabor filter family. Spatial and temporal quality is estimated using the Gabor coefficients from the reference and test videos. Temporal quality computation additionally uses reference motion information, computed using the reference Gabor coefficients. Spatial and temporal quality indices are then combined to produce the overall MOVIE index.From the PDF
Fig. 2 (a) Geometry of the Gabor filterbank in the frequency domain. The figure shows iso-surface contours of all Gabor filters at the finest scale. The two horizontal axes denote the spatial frequency coordinates and the vertical axis denotes temporal frequency. (b) A slice of the Gabor filter bank along the plane of zero temporal frequency. The x-axis denotes horizontal spatial frequency and the y-axis denotes vertical spatial frequency. (c) A slice of the Gabor filter bank along the plane of zero vertical spatial frequency. The x-axis denotes horizontal spatial frequency and the y-axis denotes temporal frequency.
Fig. 2 (a) Geometry of the Gabor filterbank in the frequency domain. The figure shows iso-surface contours of all Gabor filters at the finest scale. The two horizontal axes denote the spatial frequency coordinates and the vertical axis denotes temporal frequency. (b) A slice of the Gabor filter bank along the plane of zero temporal frequency. The x-axis denotes horizontal spatial frequency and the y-axis denotes vertical spatial frequency. (c) A slice of the Gabor filter bank along the plane of zero vertical spatial frequency. The x-axis denotes horizontal spatial frequency and the y-axis denotes temporal frequency.From the PDF
Fig. 3 A slice of the Gabor filters and the spectral plane shown in 2 dimensions. The horizontal axis denotes horizontal spatial frequency and the vertical axis denotes temporal frequency. Each circle represents a Gabor filter and the centers of each filter are also marked. The radius ρ of the single scale of Gabor filters and the distance δ of the center frequency of one Gabor filter from the spectral plane are marked.
Fig. 3 A slice of the Gabor filters and the spectral plane shown in 2 dimensions. The horizontal axis denotes horizontal spatial frequency and the vertical axis denotes temporal frequency. Each circle represents a Gabor filter and the centers of each filter are also marked. The radius ρ of the single scale of Gabor filters and the distance δ of the center frequency of one Gabor filter from the spectral plane are marked.From the PDF
Image 4
From the PDF
Fig. 5 Scatter plot of the subjective DMOS scores against MOVIE scores on the VQEG database. Each point on the plot represents one video in the database. The best fitting logistic function used for non-linear regression is also shown. (a) On all sequences in the VQEG database (b) After omitting the animated videos.
Fig. 5 Scatter plot of the subjective DMOS scores against MOVIE scores on the VQEG database. Each point on the plot represents one video in the database. The best fitting logistic function used for non-linear regression is also shown. (a) On all sequences in the VQEG database (b) After omitting the animated videos.From the PDF
Image 6
From the PDF
Image 7
From the PDF

Verification

Claims the paper makes

10 Valid · 9 Not valid · 43 Not checked · 80 Own results and interpretations

61 more claims

Create a free account to see all 142 claims and to verify the 2 references still open.

See all claims — free account
  • Probably not a substantial claim. 48
  • It may state two separate claims as one. 41
  • The quote only partly states this claim. 2

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

1 of 1

Citations

Cites

Show all 91 references

Research works printed but not linked: 7

  1. webster, 2008, progress and future plans for vqeg, ETSI STQ Workshop Multimedia Quality of Service
  2. fox, 1978, Handbook of Sensory Physiology Volume VIII Perception
  3. lubin, 1993, Digital Images and Human Vision, 163
  4. seshadrinathan, 2007, an information theoretic video quality metric based on motion models, 1st Int Workshop Video Process Quality Metrics Consum Electron
  5. girod, 1993, Digital Images and Human Vision, 207
  6. 0, (2008) NTT News Release
  7. 2008, Objective Perceptual Multimedia Video Quality Measurement in the Presence of a Full Reference

Cited by

Items 401–500 of 717

Metadata

Authors
Seshadrinathan, K., & Bovik, A. C.
Affiliations
  • Intel (United States) (US)
  • The University of Texas at Austin (US)
Venue
IEEE Transactions on Image Processing
Published
2010-02
Volume
19
Issue
2
Pages
335-350
ISSN
  • 1057-7149 (ISSN-L)
  • 1941-0042
OpenAlex
W2165222134
Cited by
708
References
83
Citations by year
2026: 14 · 2025: 16 · 2024: 25 · 2023: 31 · 2022: 43 · 2021: 50 · 2020: 41 · 2019: 41 · 2018: 60 · 2017: 59 · 2016: 52 · 2015: 59 · 2014: 69 · 2013: 43 · 2012: 48
FWCI
38.55
Type
journal article
Language
en
Open access
closed
Locations
  • IEEE Transactions on Image Processing (publishedVersion)
  • PubMed (publishedVersion)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Video Quality Assessment
  • Physical Sciences › Computer Science › Signal Processing › Video Coding and Compression Technologies
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Retrieval and Classification Techniques
Keywords
video quality assessment, subjective quality correlation, movement trajectories
MeSH terms
Algorithms, Humans, Image Processing, Computer-Assisted, Movement, Video Recording, Visual Perception

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.