CoChat
Sign Up
PreprintCited by 69Image and Video Quality AssessmentVideo Coding and Compression TechnologiesSpeech and Audio Processing

The DOI matches its DataCite record.

Subjective and Objective Quality Assessment of High Frame Rate Videos

Open access
Madhusudana, Pavan C., Yu, Xiangxu, Birkbeck, Neil, Wang, Yilin, Adsumilli, Balu, Bovik, Alan C.·2020·arXiv
DOI 10.48550/arxiv.2007.11634arXiv 2007.11634
Open this paper in CoChat — free accountOpen PDF

Abstract

High frame rate (HFR) videos are becoming increasingly common with the tremendous popularity of live, high-action streaming content such as sports. Although HFR contents are generally of very high quality, high bandwidth requirements make them challenging to deliver efficiently, while simultaneously maintaining their quality. To optimize trade-offs between bandwidth requirements and video quality, in terms of frame rate adaptation, it is imperative to understand the intricate relationship between frame rate and perceptual video quality. Towards advancing progression in this direction we designed a new subjective resource, called the LIVE-YouTube-HFR (LIVE-YT-HFR) dataset, which is comprised of 480 videos having 6 different frame rates, obtained from 16 diverse contents. In order to understand the combined effects of compression and frame rate adjustment, we also processed videos at 5 compression levels at each frame rate. To obtain subjective labels on the videos, we conducted a human study yielding 19,000 human quality ratings obtained from a pool of 85 human subjects. We also conducted a holistic evaluation of existing state-of-the-art Full and No-Reference video quality algorithms, and statistically benchmarked their performance on the new database. The LIVE-YT-HFR database has been made available online for public use and evaluation purposes, with hopes that it will help advance research in this exciting video technology direction. It may be obtained at \url{https://live.ece.utexas.edu/research/LIVE_YT_HFR/LIVE_YT_HFR/index.html}

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2007.11634

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 49

  1. C. Ge, N. Wang, G. Foster, and M. Wilson, ''Toward QoE-assured 4K video-on-demand delivery through mobile edge virtualization with adaptive prefetching,'' IEEE Trans. Multimedia, vol. 19, no. 10, pp. 2222-2237, Oct. 2017.
  2. Z. Mai, H. Mansour, R. Mantiuk, P. Nasiopoulos, R. K. Ward, and W. Heidrich, ''Optimizing a tone curve for backward-compatible high dynamic range image and video compression,'' IEEE Trans. Image Pro- cess., vol. 20, no. 6, pp. 1558-1571, Jun. 2011.
  3. D. Kundu, D. Ghadiyaram, A. C. Bovik, and B. L. Evans, ''No-reference quality assessment of tone-mapped HDR pictures,'' IEEE Trans. Image Process., vol. 26, no. 6, pp. 2957-2971, Jun. 2017.
  4. A. Smolic, K. Mueller, N. Stefanoski, J. Ostermann, A. Gotchev, G. B. Akar, G. Triantafyllidis, and A. Koz, ''Coding algorithms for 3DTV-A survey,'' IEEE Trans. Circuits Syst. Video Technol., vol. 17, no. 11, pp. 1606-1621, Nov. 2007.
  5. V. De Silva, H. K. Arachchi, E. Ekmekcioglu, and A. Kondoz, ''Toward an impairment metric for stereoscopic video: A full-reference video quality metric to assess compressed stereoscopic video,'' IEEE Trans. Image Process., vol. 22, no. 9, pp. 3392-3404, Sep. 2013.
  6. R. M. Nasiri, J. Wang, A. Rehman, S. Wang, and Z. Wang, ''Perceptual quality assessment of high frame rate video,'' in Proc. IEEE 17th Int. Workshop Multimedia Signal Process. (MMSP), Oct. 2015, pp. 1-6.
  7. A. Mackin, F. Zhang, and D. R. Bull, ''A study of high frame rate video for- mats,'' IEEE Trans. Multimedia, vol. 21, no. 6, pp. 1499-1512, Jun. 2019.
  8. K. Seshadrinathan, R. Soundararajan, A. C. Bovik, and L. K. Cormack, ''Study of subjective and objective quality assessment of video,'' IEEE Trans. Image Process., vol. 19, no. 6, pp. 1427-1441, Jun. 2010.
  9. A. K. Moorthy, L. K. Choi, A. C. Bovik, and G. de Veciana, ''Video quality assessment on mobile devices: Subjective, behavioral and objective studies,'' IEEE J. Sel. Topics Signal Process., vol. 6, no. 6, pp. 652-671, Oct. 2012.
  10. P. V. Vu and D. M. Chandler, ''ViS3: An algorithm for video quality assessment via analysis of spatial and spatiotemporal slices,'' J. Electron. Imag., vol. 23, no. 1, Feb. 2014, Art. no. 013016.
  11. M. H. Pinson, ''The consumer digital video library [best of the web],'' IEEE Signal Process. Mag., vol. 30, no. 4, pp. 172-174, Jul. 2013.
  12. Z. Sinno and A. C. Bovik, ''Large-scale study of perceptual video quality,'' IEEE Trans. Image Process., vol. 28, no. 2, pp. 612-627, Feb. 2019.
  13. V. Hosu, F. Hahn, M. Jenadeleh, H. Lin, H. Men, T. Sziranyi, S. Li, and D. Saupe, ''The Konstanz natural video database (KoNViD-1k),'' in Proc. 9th Int. Conf. Qual. Multimedia Exper. (QoMEX), May 2017, pp. 1-6.
  14. Y. Wang, S. Inguva, and B. Adsumilli, ''YouTube UGC dataset for video compression research,'' in Proc. IEEE 21st Int. Workshop Multimedia Signal Process. (MMSP), Sep. 2019, pp. 1-5.
  15. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, ''Image quality assessment: From error visibility to structural similarity,'' IEEE Trans. Image Process., vol. 13, no. 4, pp. 600-612, Apr. 2004.
  16. Z. Wang, E. P. Simoncelli, and A. C. Bovik, ''Multiscale structural simi- larity for image quality assessment,'' in Proc. 37th Asilomar Conf. Signals, Syst. Comput., vol. 2. Nov. 2003, pp. 1398-1402.
  17. L. Zhang, L. Zhang, X. Mou, and D. Zhang, ''FSIM: A feature similarity index for image quality assessment,'' IEEE Trans. Image Process., vol. 20, no. 8, pp. 2378-2386, Aug. 2011.
  18. M. H. Pinson and S. Wolf, ''A new standardized method for objec- tively measuring video quality,'' IEEE Trans. Broadcast., vol. 50, no. 3, pp. 312-322, Sep. 2004.
  19. K. Seshadrinathan and A. C. Bovik, ''A structural similarity metric for video based on motion models,'' in Proc. IEEE Int. Conf. Acoust., Speech Signal Process. (ICASSP), Apr. 2007, p. 869.
  20. K. Seshadrinathan and A. C. Bovik, ''Motion tuned spatio-temporal quality assessment of natural videos,'' IEEE Trans. Image Process., vol. 19, no. 2, pp. 335-350, Feb. 2010.
  21. P. V. Vu, C. T. Vu, and D. M. Chandler, ''A spatiotemporal most-apparent- distortion model for video quality assessment,'' in Proc. 18th IEEE Int. Conf. Image Process., Brussels, Belgium, Sep. 2011, pp. 2505-2508.
  22. D. M. Chandler, ''Most apparent distortion: Full-reference image quality assessment and the role of strategy,'' J. Electron. Imag., vol. 19, no. 1, Jan. 2010, Art. no. 011006.
  23. R. Soundararajan and A. C. Bovik, ''Video quality assessment by reduced reference spatio-temporal entropic differencing,'' IEEE Trans. Circuits Syst. Video Technol., vol. 23, no. 4, pp. 684-694, Apr. 2013.
  24. C. G. Bampis, P. Gupta, R. Soundararajan, and A. C. Bovik, ''SpEED-QA: Spatial efficient entropic differencing for image and video quality,'' IEEE Signal Process. Lett., vol. 24, no. 9, pp. 1333-1337, Sep. 2017.
  25. H. R. Sheikh and A. C. Bovik, ''Image information and visual quality,'' IEEE Trans. Image Process., vol. 15, no. 2, pp. 430-444, Feb. 2006.
  26. S. Li, F. Zhang, M. Lin, and K. N. Ngan, ''Image quality assessment by separately evaluating detail losses and additive impairments,'' IEEE Trans. Multimedia, vol. 13, no. 5, pp. 935-949, Oct. 2011.
  27. W. Kim, J. Kim, S. Ahn, J. Kim, and S. Lee, ''Deep video quality assessor: From spatio-temporal visual sensitivity to a convolutional neural aggre- gation network,'' in Proc. Eur. Conf. Comput. Vis. (ECCV), Sep. 2018, pp. 219-234.
  28. R. M. Nasiri and Z. Wang, ''Perceptual aliasing factors and the impact of frame rate on video quality,'' in Proc. IEEE Int. Conf. Image Process. (ICIP), Sep. 2017, pp. 3475-3479.
  29. R. M. Nasiri, Z. Duanmu, and Z. Wang, ''Temporal motion smoothness and the impact of frame rate variation on video quality,'' in Proc. 25th IEEE Int. Conf. Image Process. (ICIP), Oct. 2018, pp. 1418-1422.
  30. F. Zhang, A. Mackin, and D. R. Bull, ''A frame rate dependent video quality metric based on temporal wavelet decomposition and spatiotempo- ral pooling,'' in Proc. IEEE Int. Conf. Image Process. (ICIP), Sep. 2017, pp. 300-304.
  31. P. C. Madhusudana, N. Birkbeck, Y. Wang, B. Adsumilli, and A. C. Bovik, ''Capturing video frame rate variations via entropic differencing,'' IEEE Signal Process. Lett., vol. 27, pp. 1809-1813, 2020.
  32. A. Mittal, A. K. Moorthy, and A. C. Bovik, ''No-reference image quality assessment in the spatial domain,'' IEEE Trans. Image Process., vol. 21, no. 12, pp. 4695-4708, Dec. 2012.
  33. M. A. Saad, A. C. Bovik, and C. Charrier, ''Blind prediction of natural video quality,'' IEEE Trans. Image Process., vol. 23, no. 3, pp. 1352-1365, Mar. 2014.
  34. X. Li, Q. Guo, and X. Lu, ''Spatiotemporal statistics for video quality assessment,'' IEEE Trans. Image Process., vol. 25, no. 7, pp. 3329-3342, Jul. 2016.
  35. J. Korhonen, ''Two-level approach for no-reference consumer video quality assessment,'' IEEE Trans. Image Process., vol. 28, no. 12, pp. 5923-5938, Dec. 2019.
  36. Y. Zhang, X. Gao, L. He, W. Lu, and R. He, ''Blind video quality assessment with weakly supervised learning and resampling strategy,'' IEEE Trans. Circuits Syst. Video Technol., vol. 29, no. 8, pp. 2244-2255, Aug. 2019.
  37. S. Ahn and S. Lee, ''Deep blind video quality assessment based on tem- poral human perception,'' in Proc. 25th IEEE Int. Conf. Image Process. (ICIP), Oct. 2018, pp. 619-623.
  38. D. Li, T. Jiang, and M. Jiang, ''Quality assessment of in-the-wild videos,'' in Proc. 27th ACM Int. Conf. Multimedia, Oct. 2019, pp. 2351-2359.
  39. A. Mackin, F. Zhang, and D. R. Bull, ''A study of subjective video quality at various frame rates,'' in Proc. IEEE Int. Conf. Image Process. (ICIP), Sep. 2015, pp. 3407-3411.
  40. D. Hasler and S. E. Suesstrunk, ''Measuring colorfulness in natural images,'' Proc. SPIE, vol. 5007, pp. 87-95, Jun. 2003.
  41. A. B. Watson, ''High frame rates and human vision: A view through the window of visibility,'' SMPTE Motion Imag. J., vol. 122, no. 2, pp. 18-32, Mar. 2013.
  42. B.-D. Choi, J.-W. Han, C.-S. Kim, and S.-J. Ko, ''Motion-compensated frame interpolation using bilateral motion estimation and adaptive over- lapped block motion compensation,'' IEEE Trans. Circuits Syst. Video Technol., vol. 17, no. 4, pp. 407-416, Apr. 2007.
  43. D. Mukherjee, J. Han, J. Bankoski, R. Bultje, A. Grange, J. Koleszar, P. Wilkins, and Y. Xu, ''A technical overview of VP9-the latest open- source video codec,'' SMPTE Motion Imag. J., vol. 124, no. 1, pp. 44-54, Jan. 2015.
  44. A. M. van Dijk, J.-B. Martens, and A. B. Watson, ''Quality asessment of coded images using numerical category scaling,'' Proc. SPIE, vol. 2451, pp. 90-101, Feb. 1995.
  45. T. Hossfeld, C. Keimel, M. Hirth, B. Gardlo, J. Habigt, K. Diepold, and P. Tran-Gia, ''Best practices for QoE crowdtesting: QoE assessment with crowdsourcing,'' IEEE Trans. Multimedia, vol. 16, no. 2, pp. 541-558, Feb. 2014.
  46. Z. Li and C. G. Bampis, ''Recover subjective quality scores from noisy measurements,'' in Proc. Data Compress. Conf. (DCC), Apr. 2017, pp. 52-61.
  47. Y. Nehme, F. Dupont, J.-P. Farrugia, P. Le Callet, and G. Lavoue, ''Visual quality of 3D meshes with diffuse colors in virtual reality: Subjective and objective evaluation,'' IEEE Trans. Vis. Comput. Graphics, vol. 27, no. 3, pp. 2202-2219, Mar. 2021.
  48. Z. Wang and A. C. Bovik, ''Mean squared error: Love it or leave it? A new look at signal fidelity measures,'' IEEE Signal Process. Mag., vol. 26, no. 1, pp. 98-117, Jan. 2009.
  49. A. Mittal, R. Soundararajan, and A. C. Bovik, ''Making a 'completely blind' image quality analyzer,'' IEEE Sig. Process. Lett., vol. 20, no. 3, pp. 209-212, Mar. 2013.

Cited by

Sorted by citations

Metadata

Authors
Madhusudana, P. C., Yu, X., Birkbeck, N., Wang, Y., Adsumilli, B., & Bovik, A. C.
Affiliations
  • The University of Texas at Austin (US)
  • Google (United States) (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2020
Volume
9
Pages
108069-108082
ISSN
2169-3536 (ISSN-L)
DOI
10.48550/arxiv.2007.11634
arXiv
2007.11634
OpenAlex
W3045430385
Cited by
69
References
0
Citations by year
2026: 10 · 2025: 16 · 2024: 15 · 2023: 11 · 2022: 8 · 2021: 8 · 2020: 1
FWCI
3.76
Type
journal article
Language
en
Open access
Licence
Locations
  • IEEE Access (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • arXiv (Cornell University)
Funders
Google
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Video Quality Assessment
  • Physical Sciences › Computer Science › Signal Processing › Video Coding and Compression Technologies
  • Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
Keywords
high frame rate video, perceptual video quality, subjective quality assessment, full-reference quality metrics, no-reference quality metrics, video compression, frame rate adaptation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review