The DOI matches its Crossref record; 3 of the 5 references it cites were found in a registry or an index.
VMAF reproducibility: Validating a perceptual practical video quality metric
Abstract
Measuring video quality with standard metrics ensures that operators can deliver to consumers the desired quality of experience (QoE) at an optimal cost. Such metrics also allow CODEC engineers to optimize the performance of their encoding algorithms. This paper briefly surveys existing video quality metrics and then presents results of the new Video Multi-Method Assessment Fusion (VMAF) metric [1] proposed by Netflix. The author and colleagues used VMAF to measure the quality of a 4K dataset encoded with the RealMedia video CODEC at a range of bitrates. They also gathered subjective quality assessments from a group of viewers for the same dataset. The paper presents findings of correlation between subjective and objective results.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
ON BETWEEN-COEFFICIENT CONTRAST MASKING OF DCT BASIS FUNCTIONS
Ponomarenko, N. N., Silvestri, F., Egiazarian, K., Carli, M., Astola, J. T., & Лукин, В. В. (2007) · Cited by 364
Measuring video quality in the network: from quality of service to user experience
Staelens, N. (2015) · Ghent University Academic Bibliography (Ghent University) · Cited by 15
Research works printed but not linked: 3
- 0, Elementa
- 0, Methodology for the subjective assessment of the quality of television pictures, ITU-R Recommendation BT 500
- li, 2016, Toward A Practical Perceptual Video Quality Metric, Netflix TechBlog June
Cited by
Two-Level Approach for No-Reference Consumer Video Quality Assessment
Korhonen, J. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2923051 · Cited by 354
A Paradigm Shift toward Semantic Communications
Niu, K., Dai, J., Yao, S., Wang, S., Si, Z., Qin, X., & Zhang, P. (2022) · IEEE Communications Magazine · doi:10.1109/mcom.001.2200099 · Cited by 138
“For image and video sources, human perception of image quality is taken into account [14, 15], including peak SNR (PSNR), the multi-scale extension of the structural similarity index (MS-SSIM), the learned perceptual image patch similarity model (LPIPS), and the video multi-method assessment fusion…”
MethodComyco: Quality-Aware Adaptive Video Streaming via Imitation Learning
Huang, T., Zhou, C., Zhang, R.-X., Wu, C., Yao, X., & Sun, L. (2019) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3343031.3351014 · Cited by 153
“trate. To achieve this, we first integrate the information of video contents, network status, and video playback states into the Comyco’s NN for bitrate selection (§4.1). Next, we consider using VMAF [38], an objective full-reference perceptual video quality metric, to measure the video quality. Concurrently, we also…”
MethodResultInfluentialHuang, T., Zhang, R.-X., Zhou, C., & Sun, L. (2018) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3240508.3240545 · Cited by 110
“dynamic video scene (MV) respectively. In our experiment, we use Video Multi-Method Assessment Fusion(VMAF), a smart perceptual video quality assessment algorithm based on support vector machine(SVM) [23]. We compare the video quality score of each video in different encoders. In detail, we use three video encoders in…”
MethodVideo Super-Resolution and Caching—An Edge-Assisted Adaptive Video Streaming Solution
Zhang, A., Li, Q., Chen, Y., Ma, X., Zou, L., Jiang, Y., Xu, Z., & Muntean, G. (2021) · IEEE Transactions on Broadcasting · doi:10.1109/tbc.2021.3071010 · Cited by 81
ABR streaming of VBR-encoded videos
Qin, Y., Hao, S., Pattipati, K. R., Qian, F., Sen, S. B., Wang, B., & Yue, C. (2018) · ACM International Conference on emerging Networking EXperiments and Technologies (CoNEXT) · doi:10.1145/3281411.3281439 · Cited by 71
“VMAF is a recently proposed perceptual quality metric that correlates quality strongly with subjective scores and has been validated independently in [34].”
BackgroundVideo Multimethod Assessment Fusion (VMAF) on 360VR Contents
Orduna, M., Díaz, C., Munoz, L., Perez, P., Benito, I., & Garcia, N. (2019) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2019.2957987 · Cited by 94
“Concretely, Rassol et al. [25] carried out subjective quality tests to validate the application of VMAF to traditional contents with 4K resolution, a resolution for which the metric is not trained, obtaining good results when trying to predict the VMAF score.”
MethodHuang, T., Zhou, C., Zhang, R.-X., Wu, C., & Sun, L. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2022.3180804 · Cited by 67
“…where R n represents the each chunk’s video bitrate, T n reflects the rebuffering time for each chunk n , q ( R n ) means the quality metric such as video bitrate [8] and VMAF [22] (state-of-the-art quality assessment), µ and σ are the weight of rebuffering and smoothness penalty, respectively.”
MethodMa, X., Li, Q., Zou, L., Peng, J., Zhou, J., Chai, J., Jiang, Y., & Muntean, G. (2022) · IEEE Transactions on Broadcasting · doi:10.1109/tbc.2022.3171131 · Cited by 67
“Perceptual Quality: The Video Multi-Method Assessment Fusion (VMAF) [41] is used to evaluate the perceptual quality of a video chunk.”
MethodQuality-Aware Neural Adaptive Video Streaming With Lifelong Imitation Learning
Huang, T., Zhou, C., Yao, X., Zhang, R.-X., Wu, C., Yu, B., & Sun, L. (2020) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3000363 · Cited by 73
“…and the bitrate-aware ABR scheme, we report the trajectory generated by the two methods in Figure 5, in which the perceptual quality is measured by Video Multi-Method Assessment Fusion (VMAF) [20], a smart perceptual video quality assessment algorithm based on support vector machine (SVM), Fig.”
BackgroundMethodInfluentialMultimodal Inputs Driven Talking Face Generation With Spatial–Temporal Dependency
Yu, L., Yu, J., Li, M., & Ling, Q. (2020) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2020.2973374 · Cited by 79
“Currently, three image fidelity metrics and one temporal signal (i.e, Anti-noise SNR (AN-SNR), Detail Loss Measure (DLM), Visual Information Fidelity (VIF) and Mean Co-Located Pixel Difference (MCPD)) have been chosen as features to the SVM [45].”
MethodVideo Multimethod Assessment Fusion (VMAF) on 360VR Contents
Orduna, M., Díaz, C., Muñoz, L., Pérez, P., Benito, I., & Garcia, N. N. (2020) · Hispana · Cited by 61
Making Sense of Meaning: A Survey on Metrics for Semantic and Goal-Oriented Communication
Getu, T. M., Kaddoum, G., & Bennis, M. (2023) · IEEE Access · doi:10.1109/access.2023.3269848 · Cited by 90
AVT-VQDB-UHD-1: A Large Scale Video Quality Database for UHD-1
Rao, R. R. R., Göring, S., Robitza, W., Feiten, B., & Raake, A. (2019) · IEEE International Symposium on Multimedia (ISM) · doi:10.1109/ism46123.2019.00012 · Cited by 74
“A study to validate VMAF as a perceptual practical video quality metric is presented in Rassool [29].”
BackgroundDNeRV: Modeling Inherent Dynamics via Difference Neural Representation for Videos
Zhao, Q., Asif, M. S., & Ma, Z. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00202 · Cited by 67
Quality-aware strategies for optimizing ABR video streaming QoE and reducing data usage
Qin, Y., Hao, S., Pattipati, K. R., et al. (2019) · ACM Multimedia Systems Conference (MMSys) · doi:10.1145/3304109.3306231 · Cited by 46
“VMAF is a recently proposed perceptual quality metric that correlates quality strongly with subjective scores, and has been validated independently in [41].”
BackgroundA Comprehensive Video Codec Comparison
Laude, T., Adhisantoso, Y. G., Voges, J., Munderloh, M., & Östermann, J. (2019) · APSIPA Transactions on Signal and Information Processing · doi:10.1017/atsip.2019.23 · Cited by 41
“A metric called Video Multimethod Assessment Fusion (VMAF) for the approximation of perceptual quality, which combines multiple methods by machine learning, gained attention recently [38–40].”
MethodDeep Perceptual Preprocessing for Video Coding
Chadha, A., & Andreopoulos, Y. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01461 · Cited by 38
“Multiple independent studies have shown that VMAF is significantly more correlated to human opinion scores than SSIM and PSNR [30, 4, 45].”
BackgroundA Survey on Intelligent Solutions for Increased Video Delivery Quality in Cloud–Edge–End Networks
Shi, W., Li, Q., Qian, Y., Wang, F., Shen, G., Jiang, Y., Xu, Y., Ma, L., & Muntean, G. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3427360 · Cited by 39
“In addition, Yeo et al. proposed that PSNR did not reflect human perception well and more reasonable evaluation metrics such as VMAF [78] should be used.”
Optimizing Adaptive Video Streaming with Human Feedback
Huang, T., Zhang, R.-X., Wu, C., & Sun, L. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3611771 · Cited by 38
“Similar abnormal improvements are observed with VMAF valued at 80, with noticeable changes occurring at rebuffering times of 0.8.”
MethodInfluentialAttacking O-RAN Interfaces: Threat Modeling, Analysis and Practical Experimentation
Baguer, P., Yilma, G. M., Municio, E., et al. (2024) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2024.3431681 · Cited by 27
Viewport-Based Omnidirectional Video Quality Assessment: Database, Modeling and Inference
Yu, M., & Ma, Z. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3057368 · Cited by 31
Zwei: A Self-Play Reinforcement Learning Framework for Video Transmission Services
Huang, T., Zhang, R.-X., & Sun, L. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3063620 · Cited by 28
“In this work, the perceptual quality is measured by Video Multi-Method Assessment Fusion (VMAF) [30], which stands for the state-of-the-art video quality assessment metric.”
MethodReview of Image Quality Assessment Methods for Compressed Images
Jamil, S. (2024) · Journal of Imaging · doi:10.3390/jimaging10050113 · Cited by 44
“Therefore, despite the fact that its initial intent was to evaluate movies, it is an invaluable tool for evaluating and comparing the quality of photos [33].”
Video Compression Artifacts Removal With Spatial-Temporal Attention-Guided Enhancement
Jiang, N., Chen, W., Lin, J., Zhao, T., & Lin, C. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3338087 · Cited by 31
Deep reinforced bitrate ladders for adaptive video streaming
Huang, T., Zhang, R.-X., & Sun, L. (2021) · International Workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV) · doi:10.1145/3458306.3458873 · Cited by 29
“Moreover, we measure the VMAF [28] metric for each video chunk with different levels and resolutions, yielding 254,529 samples.”
MethodInfluentialDecoder-Side Cross Resolution Synthesis for Video Compression Enhancement
Lu, M., Chen, T., Zhen-yu, D., Wang, D., Ding, D., & Ma, Z. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3142414 · Cited by 24
“Other distortion measures, such as the multiscale structural similarity, and VMAF [25], present the similar performance lead for our proposed CRS.”
InfluentialAdaptive Computation Offloading for Mobile Augmented Reality
Ren, J., Gao, L. C., Wang, X., Ma, M., Qiu, G., Wang, H., Zheng, J., & Wang, Z. (2021) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3494958 · Cited by 29
“A recent study on the 4K VMAF model shows that the VMAF score has a strong correlation with the user’s subjective score on 4K video quality [62, 72].”
MethodThe Bjøntegaard Bible Why Your Way of Comparing Video Codecs May Be Wrong
Herglotz, C., Och, H., Meyer, A. P., Ramasubbu, G., Eichermüller, L., Kränzler, M., Brand, F., Fischer, K., Nguyen, D. T., Regensky, A., et al. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3346695 · Cited by 34
“Saturation was also observed in other studies using the SSIM metric [69].”
ResultVoronoi-based Objective Quality Metrics for Omnidirectional Video
Croci, S., Ozcinar, C., Zerman, E., Cabrera, J., & Smolić, A. (2019) · International Conference on Quality of Multimedia Experience (QoMEX) · doi:10.1109/qomex.2019.8743345 · Cited by 24
Zhao, Z., He, X., Xiong, S., et al. (2023) · IEEE Transactions on Broadcasting · doi:10.1109/tbc.2023.3268946 · Cited by 21
“As a novel metric, VMAF is designed to integrate visual quality fidelity (VIF), detail loss measure (DLM) and temporal information (TI) to achieve a comprehensive evaluation of video in both the spatial and temporal domains [52].”
MethodA Perceptual Quality Metric for Video Frame Interpolation
Hou, Q., Ghildyal, A., & Liu, F. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19784-0_14 · Cited by 24
Experimental QoE Evaluation of Multicast Video Delivery over IEEE 802.11aa WLANs
Gringoli, F., Serrano, P., Úcar, I., et al. (2018) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2018.2876000 · Cited by 19
“theresultingVMAFvaluevaries thanSRC1),”
Wang, Y., Li, J., Li, Z., Shang, S., & Liu, Y. (2024) · IEEE Transactions on Broadcasting · doi:10.1109/tbc.2024.3374119 · Cited by 21
Affective Content-aware Adaptation Scheme on QoE Optimization of Adaptive Streaming over HTTP
Hu, S., Xu, M. G., Zhang, H., et al. (2019) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3328997 · Cited by 18
A Viewport-Driven Multi-Metric Fusion Approach for 360-Degree Video Quality Assessment
Azevedo, R. G. D. A., Birkbeck, N., Janatra, I., Adsumilli, B., & Frossard, P. (2020) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme46284.2020.9102936 · Cited by 25
“Indeed, previous work in both traditional 2D [8, 9, 10] and 360-degree [7] VQA have recognized that even with the multitude of available objective IQA metrics, there is no single one that always performs best for all distortions.”
BackgroundMethodVMAF Oriented Perceptual Coding Based on Piecewise Metric Coupling
Luo, Z., Zhu, C., Huang, Y., Xie, R., Song, L., & Kuo, C. ‐. J. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3078622 · Cited by 17
“It has been shown that VMAF has higher accuracy than conventional metrics [15]–[18] and has demonstrated its practicality for the industry [19]–[22].”
BackgroundLearning-Based Joint QoE Optimization for Adaptive Video Streaming Based on Smart Edge
Ma, X., Li, Q., Jiang, Y., Muntean, G., & Zou, L. (2022) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2022.3145619 · Cited by 17
“Recent work [33], [39] proves that Video Multi-method Assessment Fusion (VMAF) [40] can better characterize clients’ perceptual quality.”
Scalable MDC-Based Volumetric Video Delivery for Real-Time One-to-Many WebRTC Conferencing
Fré, M. D., van der Hooft, J., Wauters, T., & Turck, F. D. (2024) · ACM Multimedia Systems Conference (MMSys) · doi:10.1145/3625468.3647617 · Cited by 29
Croci, S., Ozcinar, C., Zerman, E., Knorr, S. B., Cabrera, J., & Smolić, A. (2020) · Quality and User Experience · doi:10.1007/s41233-020-00032-3 · Cited by 20
“The work in [36] created a dataset of ODVs in ERP compressed using constant quantization parameters, and showed that VMAF can be used as a metric also for ODVs without modifications.”
MethodBitrate Adaptation and Guidance With Meta Reinforcement Learning
Bentaleb, A., Lim, M. L., Akcay, M. N., Begen, A. C., & Zimmermann, R. (2024) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2024.3376560 · Cited by 20
“In this study, we use an objective full-reference perceptual video quality metric known as Video Multi-method Assessment Fusion (VMAF) [42].”
MethodVideo Quality Assessment with Texture Information Fusion for Streaming Applications
Menon, V. V., Rajendran, P. T., Farahani, R., et al. (2024) · doi:10.1145/3638036.3640798 · Cited by 16
“It often does not correlate well with perceived visual quality, especially at lower bitrates or scenarios with complex video content [10].”
BackgroundInfluentialHardware and software video encoding comparison
Safin, R., Garipova, E., Lavrenov, R., et al. (2020) · Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE) · doi:10.23919/sice48898.2020.9240439 · Cited by 16
“Peak Signal-to-noise Ratio (PSNR) is a ratio be-tween a maximum possible power of a signal and a power of corrupting noise that affects its representation fidelity [17, 18].”
BackgroundReal-time video super resolution network using recurrent multi-branch dilated convolutions
Zeng, Y., Xiao, Z., Hung, K.-W., & Lui, S. (2021) · Signal Processing Image Communication · doi:10.1016/j.image.2021.116167 · Cited by 15
Selective Encryption of the Versatile Video Coding Standard
Farajallah, M., Gautier, G., Hamidouche, W., Olivier Déforges, A., & Assad, S. E. (2022) · IEEE Access · doi:10.1109/access.2022.3149599 · Cited by 17
“Finally, VMAF is a video quality metric that predicts the perceived quality score of a video sequence [37], where a score of 100 indicates a good perceptual video quality and 0 refers to a very low perceived video quality.”
BackgroundPerceptual Evaluation on Audio-Visual Dataset of 360 Content
Fela, R. F., Pastor, A., Callet, P. L., et al. (2022) · IEEE International Conference on Multimedia and Expo workshops · doi:10.1109/icmew56448.2022.9859426 · Cited by 18
“For video quality metrics, VMAF 4K, and the bootstrapped version (VMAF B) outperforms other metrics.”
MethodInfluentialMamba: Bringing Multi-Dimensional ABR to WebRTC
Li, Y., Zhang, Z., Chen, H., & Ma, Z. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3611915 · Cited by 16
VMAF Oriented Perceptual Optimization for Video Coding
Luo, Z., Huang, Y., Wang, X., Xie, R., & Song, L. (2019) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.2019.8702513 · Cited by 16
“It has been shown that VMAF has higher accuracy than conventional metrics [5]–[7] and has demonstrated its practicality for the industry [8]–[10].”
BackgroundBhat, M., Thiesse, J.-M., & Callet, P. L. (2021) · Proceedings - International Conference on Image Processing · doi:10.1109/icip42928.2021.9506310 · Cited by 15
Hu, X., Pan, Y., Wang, Y., Zhang, L., & Shirmohammadi, S. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3129918 · Cited by 22
“In order to give more comparisons of subjective and objective quality assessment metrics, the results of Structural SIMilarity index (SSIM) and Netflix’s VMAF [39] are also given in Table II.”
BackgroundRubel, A., Єремеєв, О., Лукин, В. В., Fastowicz, J., & Okarma, K. (2022) · Applied Sciences · doi:10.3390/app12041986 · Cited by 21
Hacking VMAF and VMAF NEG: Vulnerability to Different Preprocessing Methods
Siniukov, M., Antsiferova, A., Kulikov, D., et al. (2021) · Artificial Intelligence and Cloud Computing Conference (AICCC) · doi:10.1145/3508259.3508272 · Cited by 14
“correlation with subjective quality, as proven by independent researchers [3].”
BackgroundMeta Reinforcement Learning for Rate Adaptation
Bentaleb, A., Lim, M. L., Akcay, M. N., Begen, A. C., & Zimmermann, R. (2023) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom53939.2023.10228951 · Cited by 17
Deeplive: QoE Optimization for Live Video Streaming through Deep Reinforcement Learning
Tian, Z., Zhao, L., Nie, L., Chen, P., & Chen, S. (2019) · IEEE International Conference on Parallel and Distributed Systems (ICPADS) · doi:10.1109/icpads47876.2019.00122 · Cited by 13
“But it relies on predictions of future video quality that is unconvincing for different user real experience of video quality [13].”
BackgroundCost-Effective, Quality-Oriented Transcoding of Live-Streamed Video on Edge-Servers
Lee, D., Kim, Y., & Song, M. (2023) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2023.3256425 · Cited by 13
“We use the video multi-method assessment fusion (VMAF) index developed by Netflix, which is known [35] to express QoE more accurately than other video quality indices.”
BackgroundMethodImproving Adaptive Real-Time Video Communication via Cross-Layer Optimization
Li, Y., Chen, H., Xu, B., Zhang, Z., & Ma, Z. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3331946 · Cited by 14
“3%–15.2% band-widthatthesamevideoqualitymeasuredbyprevalentVMAF index(VideoMulti-MethodAssessmentFusion)[18].Asforthe Asforthe real-worldtestsheldinavarietyofapplicationscenarios,Palette notonlyreduces3.”
MethodInfluentialChen, Z., Sun, H., Zhang, L., & Zhang, F. (2024) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2024.3403524 · Cited by 22
“Various quality metrics, including MS-SSIM, Video Multimethod Assessment Fusion (VMAF) metric [142], Visual Information Fidelity (VIF) [143], are employed to evaluate the coding distortion of the reconstructed image, emphasizing the perceptual quality of human vision.”
MethodLALISA: Adaptive Bitrate Ladder Optimization in HTTP-based Adaptive Live Streaming
Tashtarian, F., Bentaleb, A., Amirpour, H., Taraghi, B., Timmerer, C., Hellwagner, H., & Zimmermann, R. (2023) · IEEE/IFIP Network Operations and Management Symposium (NOMS) · doi:10.1109/noms56928.2023.10154347 · Cited by 17
Machine Learning for Per-Title Encoding
Silhavy, D., Krauss, C., Chen, A., et al. (2022) · SMPTE Motion Imaging Journal · doi:10.5594/jmi.2022.3154836 · Cited by 12
“Furthermore, a VMAF score of 93 is sufficient in producing a video that is either indistinguishable from the original or with noticeable but not annoying distortion.(8) While the title-based encoding ladder is capped at approximately 93 VMAF points, the conventional ladder goes up to a score of 95.”
BackgroundFang, H., Zhao, H., Shi, J., Zhang, M., Wu, G., Chou, Y. C., Wang, F., & Liu, J. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3680712 · Cited by 17
A Preliminary Study on Quality of Experience Assessment of Compressed Audio File Format
Baloch, J. A., Jumani, A. K., Laghari, A. A., Estrela, V. V., & Lopes, R. T. (2021) · IEEE URUCON · doi:10.1109/urucon53396.2021.9647114 · Cited by 11
SJTU-TMQA: A Quality Assessment Database for Static Mesh with Texture Map
Cui, B., Yang, Q., Yang, K., et al. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10445942 · Cited by 11
HEVC VMAF-oriented Perceptual Rate Distortion Optimization using CNN
Zhu, C., Huang, Y., Xie, R., et al. (2021) · Picture Coding Symposium · doi:10.1109/pcs50896.2021.9477459 · Cited by 11
Fast Video Visual Quality and Resolution Improvement using SR-UNet
Vaccaro, F., Bertini, M., Uricchio, T., & Bimbo, A. D. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475683 · Cited by 14
“Table2 reports the results in terms of SSIM, LPIPS and VMAF, reporting also the frames per second processed by each method, as obtained on a NVIDIA GTX 1080Ti.”
MethodInfluential360° video quality assessment based on saliency-guided viewport extraction
Yang, F., Yang, C., An, P., & Huang, X. (2024) · Multimedia Systems · doi:10.1007/s00530-024-01285-0 · Cited by 10
“In this section, we evaluate the proposed method on the commonly used 360 VQA database VQA-ODV [15] and 2D VQA database CSIQ-VQA [52].”
On the accuracy of video quality measurement techniques
Nandakumar, D., Wu, Y., Wei, H., et al. (2019) · IEEE International Workshop on Multimedia Signal Processing (MMSP) · doi:10.1109/mmsp.2019.8901796 · Cited by 10
“V. ANALYSIS BASED ON QUALITY REGIONS The results shown in TABLE II above appear to be consistent with previous analysis performed on the correlation of objective metrics with Mean Opinion Scores [20].”
ResultQuality-Aware Transcoding Task Allocation Under Limited Power in Live-Streaming Systems
Lee, D., & Song, M. (2021) · IEEE Systems Journal · doi:10.1109/jsyst.2021.3103526 · Cited by 9
“We use the video multimethod assessment fusion (VMAF) index developed by Netflix, which is known to be more accurate than other popular quality indices [38], [39].”
MethodPer-title and per-segment CRF estimation using DNNs for quality-based video coding
Micó-Enguídanos, F., Moina-Rivera, W., Aguado, J., & García, M. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.120289 · Cited by 11
Vlaović, J., Žagar, D., Rimac-Drlje, S., & Vranješ, M. (2021) · International journal of electrical and computer engineering systems · doi:10.32985/ijeces.12.1.1 · Cited by 10
“6. present test results for VQA methods PSNR, SSIM, VQM, and Netflix VMAF.”
MethodInfluentialTang, K., Kan, N., Jiang, Y., Li, C., Dai, W., Zou, J., & Xiong, H. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3331487 · Cited by 10
“Some subjective experiments [33] show that VMAF can better reflect the subjective visual quality perceived by users.”
BackgroundPsyQoE: Improving Quality-of-Experience Assessment With Psychological Effects in Video Streaming
Sheng, D., Qi, Q., Wang, J., Li, L., Yu, W., & Liao, J. (2024) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2024.3451215 · Cited by 8
Comparative survey of image compression methods across different pixel formats and bit depths
Chlubna, T., & Zemčík, P. (2025) · Signal Image and Video Processing · doi:10.1007/s11760-025-04579-6 · Cited by 8
Energy-aware images: Quality of Experience vs Energy Reduction
Meur, O. L., Demarty, C.-H., Aumont, F., et al. (2023) · Mile-High Video Conference (MHV) · doi:10.1145/3588444.3591003 · Cited by 8
“The VMAF score is computed between some original video sequences from the JVET test sequences [1] and their energy aware processed correspondences.”
MethodInfluentialQoE-oriented Adaptive Video Streaming with Edge-Client Collaborative Super-Resolution
Liu, X., Ke, Z., Zhou, X., et al. (2022) · Globecom · doi:10.1109/globecom48099.2022.10000659 · Cited by 8
“detail loss measure and temporal information [16].”
Surprise-based JND estimation for perceptual quantization in H.265/HEVC codecs
Wang, H., Yu, L., Yang, H., Xu, H., Yin, H., Zhai, G., Li, T., & Kuang, Z. (2023) · Signal Processing Image Communication · doi:10.1016/j.image.2023.117019 · Cited by 8
A Real-Time Video Quality Metric for HTTP Adaptive Streaming
Amirpour, H., Zhu, J., Callet, P. L., et al. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10446839 · Cited by 7
“Among objective quality metrics, VMAF [10] has gained significant attention in recent years due to its high correlation with subjective testing results [11, 12, 13].”
BackgroundRate-constrained learning-based image compression
Guerin, N. D., da Silva, R. C., de Oliveira, M. C., et al. (2021) · Signal Processing Image Communication · doi:10.1016/j.image.2021.116544 · Cited by 7
Duan, H., Liao, J., Lin, L., et al. (2022) · doi:10.1145/3534088.3534349 · Cited by 7
“On the other hand, most existing video quality assessment works mainly focus on the objective quality loss after compression, encoding or transmission [24, 30, 32, 44], while few pay attention to the subjective sense of viewers caused by the content of videos.”
BackgroundMethodSEPE Dataset: 8K Video Sequences and Images for Analysis and Development
Shoura, T. A., Dehaghi, A. M., Razavi, R., et al. (2023) · ACM Multimedia Systems Conference (MMSys) · doi:10.1145/3587819.3592560 · Cited by 7
“VMAF is a video quality evaluation method that was developed by Netflix using machine learning methods to correlate strongly with subjective mean opinion score (MOS) [26], where this metric aims to measure the degradation due to rescaling and compression of videos in a matter that approximates the human perception by…”
MethodVisual Saliency Guided Foveated Video Compression
Zhang, S., & Basu, A. (2023) · IEEE Access · doi:10.1109/access.2023.3286577 · Cited by 7
“Thus, we conduct an objective image quality assessment using perceptual quality metrics, specifically using the Eye-tracking Weighted PSNR (EWPSNR) [35], Perceptual Similarity (LPIPS) [36], and Video Multi-Method Assessment Fusion (VMAF) [37] metrics.”
MethodSepahi, A., Cai, L., Yang, W., & Pan, J. (2024) · IEEE Transactions on Cognitive Communications and Networking · doi:10.1109/tccn.2024.3502512 · Cited by 8
“VMAF, developed by Netflix, employs machine learning for content-dependent assessments, providing scores on a 0 to 100 scale [44].”
Improved EDVR Model for Robust and Efficient Video Super-Resolution
Huang, Y., & Chen, J. (2022) · IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw54805.2022.00016 · Cited by 8
“This experiment uses Vimeo90k dataset to train our model and test the model in Vid4 dataset, and we will compare the PSNR, SSIM and VMAF [19] metrics with widely-used models.”
MethodMulti-Feature 360 Video Quality Estimation
Azevedo, R. G. D. A., Birkbeck, N., Janatra, I., Adsumilli, B., & Frossard, P. (2021) · IEEE Open Journal of Circuits and Systems · doi:10.1109/ojcas.2021.3073891 · Cited by 10
“We compare the performance of our method against: PSNR, MS-SSIM, and VMAF (state-of-the-art metrics for 2D quality assessment) when the features are computed in both the projection and the viewports domain; and S-PSNR and WS-PSNR, metrics specifically developed for 360 VQA.”
BackgroundInfluentialExploring Spatial Frequency Information for Enhanced Video Prediction Quality
Lai, J., Gan, L., Zhu, J., Liu, H., & Gao, L. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3384062 · Cited by 8
“Another category of image/video metrics is DNN-based perceptual metrics, such as LPIPS [39], VMAF [40], and VFIPS [41], etc., which require the construction of DNN models and dedicated datasets for training.”
BackgroundTSBG: A Two-Stage Stackelberg Game Algorithm for QoE-Awareness Video Streaming Transmission
Xiang, S., Rong, H., Chen, J., Liu, D., & Jiang, H. (2024) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2024.3412860 · Cited by 6
“VMAF [8] serves as a representation of video quality and employs alargesampleMeanOpinionScore(MOS) as afounda-tional truth value to train its quality evaluation model.”
MethodData Orchestration for Accelerating GPU-Based Light Field Rendering Aiming at a Wide Virtual Space
Lee, S., Jung, H., & Rhee, C. E. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3121010 · Cited by 6
An Intelligent Advertisement Short Video Production System via Multi-Modal Retrieval
Wei, Y., Huang, L., Zhang, Y., Zheng, Y. P., & Pan, P. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3536323 · Cited by 6
“This module adopts the most popular video quality assessment model, VMAF[10], to score video quality from the perspectives of clarity and supervised viewing.”
MethodNezhivleva, K., Davydova, A., Drebuzhan, A. M., Mozhaeva, A., & Balobanov, A. V. (2022) · 2018Systems of Signal Synchronization, Generating and Processing in Telecommunications (SYNCHROINFO) · doi:10.1109/synchroinfo55067.2022.9840983 · Cited by 6
A VMAF Directed Perceptual Rate Distortion Optimization for Video Coding
Tong, X., Zhu, C. H., Xie, R., et al. (2020) · IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB) · doi:10.1109/bmsb49480.2020.9379915 · Cited by 6
“Developed by Netflix, Video Multimethod Assessment Fusion (VMAF) [10] has better ability to evaluate perceptual distortion than PSNR and SSIM [11]—[13].”
MethodAntsiferova, A., Yakovenko, A., Safonov, N., Kulikov, D., Gushin, A., & Vatolin, D. S. (2021) · International Conference on Computer Graphics and Vision (GraphiCon) · doi:10.20948/graphicon-2021-3027-199-210 · Cited by 5
Blind quality scalable video compression algorithm for low bit-rate coding
Cheggoju, N., & Satpute, V. R. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-022-12061-5 · Cited by 5
“Along with these standard metrics, latest metric Video Multimethod Assessment Fusion (VMAF) [7, 10, 14] is also included in Table 7 for the bit rates 512kbps and 1 Mbps.”
MethodChmieliauskas, D., & Paulikas, Š. (2025) · IEEE Access · doi:10.1109/access.2025.3554340 · Cited by 13
“As described by [24], VMAF is specifically formulated to correlate strongly with subjective MOS scores.”
Quality-Oriented Task Allocation and Scheduling in Transcoding Servers With Heterogeneous Processors
Lee, D., & Song, M. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3074158 · Cited by 5
Improving whatsapp Video Statuses using FFMPEG and Software based encoding
Gupta, M., Shah, S., & Salmani, S. B. (2021) · International Conference on Communication Information and Computing Technology (ICCICT) · doi:10.1109/iccict50803.2021.9510129 · Cited by 5
VMAF based quantization parameter prediction model for low resolution video coding
Huu, T. V., Cong, H. P., Sisouvong, T., et al. (2022) · International Conference on Advanced Technologies for Communications · doi:10.1109/atc55345.2022.9942982 · Cited by 5
“The accuracy performance of VMAF metric is proved higher than conventional metrics [8 – 10] and applied widely in the industry.”
BackgroundAssessment of the Quality of Video Sequences Performed by Viewers at Home and in the Laboratory
Klink, J. H., Brachmański, S., & Łuczyński, M. (2023) · Applied Sciences · doi:10.3390/app13085025 · Cited by 5
Color-Sensitivity-Based Rate-Distortion Optimization for H.265/HEVC
Shang, X., Wang, G., Liang, J., et al. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3062402 · Cited by 5
Analysis of the Perceptual Quality Performance of Different HEVC Coding Tools
Atencia, J. R., López-Granado, O., Malumbres, M. P., Martínez-Rach, M., & Wallendael, G. V. (2021) · IEEE Access · doi:10.1109/access.2021.3062938 · Cited by 4
Recurrent Neural Network-Based Video Compression
Montajabi, Z., Ghassab, V. K., & Bouguila, N. (2022) · International Conference on Machine Learning and Applications · doi:10.1109/icmla55696.2022.00154 · Cited by 3
“VMAF is a subjective measure of the human eye perception and so it is a pivotel measure in real-world applications [28].”
BackgroundLearning Adaptive Quantization Parameter for Consistent Quality Oriented Video Coding
Huu, T. V., Do, M. N., Nguyen, S. Q., et al. (2023) · Electronics · doi:10.3390/electronics12244905 · Cited by 4
Metadata
- Authors
- Rassool, R.
- Affiliations
- RealNetworks (United States) (US)
- Corresponding author
- Rassool, Reza
- Venue
- 2017 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB)
- Publisher
- IEEE
- Published
- 2017-06
- Pages
- 1-2
- OpenAlex
- W2737019897
- Cited by
- 235
- References
- 5
- Citations by year
- 2026: 23 · 2025: 39 · 2024: 37 · 2023: 36 · 2022: 30 · 2021: 37 · 2020: 17 · 2019: 12 · 2018: 4
- FWCI
- 2.64
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Video Quality Assessment
- Physical Sciences › Computer Science › Signal Processing › Video Coding and Compression Technologies
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Keywords
- VMAF, video quality metrics, perceptual video quality, reproducibility, subjective quality assessment, objective quality measurement, 4K video, data rate, quality of experience, subjective-objective correlation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.