Mean opinion score (MOS) revisited: methods and applications, limitations and alternatives
The DOI matches its Crossref record; 76 of the 78 references it cites were found in a registry or an index.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Metrics that mirror subjective human judgment, such as the mean opinion scores (MOS) and differential mean opinion scores (DMOS), continue to serve as the benchmark for validating objective algorithms [1-3].”
From A Survey and Tutorial on Image Quality Assessment with a Contrast-Weighted Structural Similarity Framework (2026) · cites this paper as [1]
Citations
Cites
The magical number seven, plus or minus two: Some limits on our capacity for processing information.
Miller, G. A. (1956) · Psychological Review · doi:10.1037/h0043158 · Cited by 17,812
The magical number seven, plus or minus two: Some limits on our capacity for processing information.
Miller, G. A. (1994) · Psychological Review · doi:10.1037/0033-295x.101.2.343 · Cited by 17,161
(1966) · Medical Entomology and Zoology · Cited by 10,293
Osgood, C. E., Suci, G. J., & Tannenbaum, P. H. (1957) · Cited by 9,433
Signal detection theory and psychophysics
Ingleby, J. D. (1967) · Journal of Sound and Vibration · doi:10.1016/0022-460x(67)90197-6 · Cited by 8,509
Hearing lips and seeing voices
MCGURK, H., & MACDONALD, J. (1976) · Nature · doi:10.1038/264746a0 · Cited by 6,183 · cited 1×
“One example they offer is that they can measure the degradation in performance by wrong answers such as those introduced by McGurk effects [91], whereas a MOS score might miss this degradation.”
BackgroundModern Multidimensional Scaling: Theory and Applications
Borg, I., & Groenen, P. (2003) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.2003.tb01108.x · Cited by 2,197 · cited 1×
“Scaling perceptual differences of pairwise stimuli, which are then mapped into a multidimensional space using multi-dimensional scaling (MDS) procedures [66, 67].”
BackgroundModern Multidimensional Scaling: Theory and Applications
O'Connell, A. A., Borg, I., & Groenen, P. (1999) · Journal of the American Statistical Association · doi:10.2307/2669710 · Cited by 1,048
Perceptual visual quality metrics: A survey
Lin, W., & Jay Kuo, C.-C. (2011) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2011.01.005 · Cited by 922 · cited 2×
“Large numbers of subjective MOS data [94] and objective quality metrics [37-39] have been made available.”
BackgroundMobile computation: Spatiotemporal integration of the properties of objects in motion
Cavanagh, P., Holcombe, A. O., & Chou, W. (2008) · Journal of Vision · doi:10.1167/8.12.1 · Cited by 715
Research works printed but not linked: 5
- Guilford, J.P.: Psychometric Methods, 2nd edn. McGraw-Hill, New York (1954)
- Mullin, J., Smallwood, L., Watson, A., Wilson, G.M.: New techniques for assessing audio and video quality in real-time interactive communication. In: Proceedings of IHM-HCI, Lille. (2001)
- Sen, D.: Determining the dimensions of speech quality from PCA and MDS analysis of the diagnostic acceptability measure. In: Proceedings of MESAQIN, Prague (2011)
- Virtanen, M.T., Gleiss, N., Goldstein, M.: On the use of evaluative category scales in telecommunications. In: Proceedings of Human Factors in Telecommunications, pp. 253–260 (1995)
- Egger, S., Ries, M., Reichl, P.: Quality-of-experience beyond MOS: experiences with a holistic user test methodology for interactive video services. In: Proceedings of 21st ITC Specialist Seminar, Miyazaki (2010)
Cited by
Structural similarity index (SSIM) revisited: A data-driven approach
Bakurov, I., Buzzelli, M., Schettini, R., Castelli, M., & Vanneschi, L. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.116087 · Cited by 421
A Comprehensive Review on Generative AI for Education
Mittal, U., Sai, S., Chamola, V., & Sangwan, D. (2024) · IEEE Access · doi:10.1109/access.2024.3468368 · Cited by 383
“These models have been trained over multiple deep learning models and are then evaluated using metrics such as signal-to-noise ratio [17] and mean-opinion-score [18].”
MethodAn Overview of Voice Conversion and Its Challenges: From Statistical Modeling to Deep Learning
Sisman, B., Yamagishi, J., King, S., & Li, H. (2021) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2020.3038524 · Cited by 357
“(MOS) [2], [251]–[253], preference tests [18], [254] and bestworst scaling [255] could represent the intrinsic naturalness and similarity to the target.”
BackgroundOn the use of deep learning for blind image quality assessment
Bianco, S., Celona, L., Napoletano, P., & Schettini, R. (2018) · Signal, Image and Video Processing · doi:10.1007/s11760-017-1166-8 · Cited by 277
“The CNN is discriminatively fine-tuned to classify image sub-regions into five classes according to the 5-points MOS scale [43].”
MethodLossy Image Compression with Compressive Autoencoders
Huszár, F., Theis, L., Shi, W., & Cunningham, A. (2020) · Apollo (University of Cambridge) · doi:10.17863/cam.51995 · Cited by 237
Lossy Image Compression with Compressive Autoencoders
Theis, L., Shi, W., Cunningham, A., & Huszár, F. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.00395 · Cited by 230
“While MOS tests have their limitations, they are a widely used standard for evaluating perceptual quality (Streijl et al., 2014).”
MethodEmotion Intensity and its Control for Emotional Voice Conversion
Zhou, K., Şişman, B., Rana, R. K., Schuller, B. W., & Li, H. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3175578 · Cited by 181
“We adopt two subjective metrics: 1) a mean opinion score (MOS) test for emotion similarity evaluation, and 2) a bestworst scaling (BWS) test to evaluate speech quality, emotion intensity, and emotion similarity.”
MethodInfluentialHaptic Codecs for the Tactile Internet
Steinbach, E., Strese, M., Eid, M., Liu, X., Bhardwaj, A., Liu, Q., Al-Ja'Afreh, M., Mahmoodi, T., Hassen, R., El Saddik, A., et al. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2018.2867835 · Cited by 166
Quality of Experience for Streaming Services: Measurements, Challenges and Insights
Bouraqia, K., Sabir, E., Sadik, M., & Ladid, L. (2020) · IEEE Access · doi:10.1109/access.2020.2965099 · Cited by 167
“good and accustomed practices, for certain test applications, standards do not provide the best or most advanced method available, but it gives solid, common basis which is accessible to all, like ITU - International Telecommunication Union [75] (Table II).”
BackgroundCMGAN: Conformer-Based Metric-GAN for Monaural Speech Enhancement
Abdulatif, S., Cao, R., & Yang, B. (2024) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2024.3393718 · Cited by 162
“Therefore, finding an objective measure that can highly correlate with the subjective quality score is still an open research topic [120].”
BackgroundAn Overview of Affective Speech Synthesis and Conversion in the Deep Learning Era
Triantafyllopoulos, A., Schuller, B. W., İymen, G., Sezgin, T. M., He, X., Yang, Z., Tzirakis, P., Liu, S., Mertes, S., André, E., et al. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2023.3250266 · Cited by 151
QoE beyond the MOS: an in-depth look at QoE via better metrics and their relation to MOS
Hoßfeld, T., Heegaard, P. E., Varela, M., & Möller, S. (2016) · Quality and User Experience · doi:10.1007/s41233-016-0002-1 · Cited by 135
“In [9], the authors discuss the limitations of MOS and other related issues.”
BackgroundDeep Architectures for Image Compression: A Critical Review
Mishra, D., Singh, S. K., & Singh, R. K. (2021) · Signal Processing · doi:10.1016/j.sigpro.2021.108346 · Cited by 130
A systematic review of pedagogical agent research: Similarities, differences and unexplored aspects
Dai, L., Jung, M. M., Postma, M., & Louwerse, M. M. (2022) · Computers & Education · doi:10.1016/j.compedu.2022.104607 · Cited by 128
Towards an efficient and a more accurate web service selection using MCDM methods
Serrai, W., Abdelli, A., Mokdad, L., & Hammal, Y. (2017) · Journal of Computational Science · doi:10.1016/j.jocs.2017.05.024 · Cited by 126
Reversible data hiding with automatic contrast enhancement for medical images
Gao, G., Tong, S., Xia, Z., Wu, B., Xu, L., & Zhao, Z. (2020) · Signal Processing · doi:10.1016/j.sigpro.2020.107817 · Cited by 112
Semi-Supervised Neural Architecture SearchPreprint
Luo, R., Tan, X., Wang, R., Qin, T., Chen, E., & Liu, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.10389 · Cited by 98
“We also use MOS (mean opinion score) [28] to evaluate the naturalness of the synthesized speech.”
MethodVideo Multimethod Assessment Fusion (VMAF) on 360VR Contents
Orduna, M., Díaz, C., Munoz, L., Perez, P., Benito, I., & Garcia, N. (2019) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2019.2957987 · Cited by 94
“Moreover, 95% Confidence Intervals (CI) are included to properly measure the agreement between subjects [41], according to the Recommendation ITU-R BT.500-13 [33].”
BackgroundFinancial Technology Adoption – A Case of Indian MSMEs
Gupta, U., Agarwal, B., & Nautiyal, N. (2022) · Finance: Theory and Practice · doi:10.26794/2587-5671-2022-26-6-192-211 · Cited by 85
Soccer Video Summarization Using Deep Learning
Agyeman, R., Rafiq, M. T., & Choi, G. S. (2019) · IEEE International Conference on Multimedia Information Processing and Retrieval (MIPR) · doi:10.1109/mipr.2019.00055 · Cited by 76
Video stabilization: A comprehensive survey
Wang, Y., Huang, Q., Jiang, C., Liu, J., Shang, M., & Miao, Z. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.10.008 · Cited by 75
A review on quality of experience (QoE) in cloud computing
Laghari, A. A., Zhang, X., Shaikh, Z. A., Khan, A., Estrela, V. V., & Izadi, S. (2023) · Journal of Reliable Intelligent Environments · doi:10.1007/s40860-023-00210-y · Cited by 75
“The ITU provides standards for the sake of sieving authentic information from inaccurate information, standards such as Mean Opinion Score [MOS] for the sake of collecting responses from users based on image/video/network quality, which are subjective, as is given in Table 1 [59, 73].”
A review on subjective and objective evaluation of synthetic speech
Cooper, E., Huang, W.-C., Tsao, Y., Wang, H., Toda, T., & Yamagishi, J. (2024) · Nippon Onkyo Gakkaishi/Acoustical science and technology/Nihon Onkyo Gakkaishi · doi:10.1250/ast.e24.12 · Cited by 75
AVT-VQDB-UHD-1: A Large Scale Video Quality Database for UHD-1
Rao, R. R. R., Göring, S., Robitza, W., Feiten, B., & Raake, A. (2019) · IEEE International Symposium on Multimedia (ISM) · doi:10.1109/ism46123.2019.00012 · Cited by 74
“Studies have shown that there is no significant difference between DSIS and ACR test methodologies [35, 7].”
BackgroundChai, L., Du, J., Liu, Q., & Lee, C. (2020) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2020.3036783 · Cited by 72
“It has a high correlation with the perceptual mean opinion score [47].”
BackgroundEMOQ-TTS: Emotion Intensity Quantization for Fine-Grained Controllable Emotional Text-to-Speech
Im, C.-B., Lee, S.-H., Kim, S.-B., & Lee, S. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747098 · Cited by 72
Eficient image denoising using deep learning: A brief survey
Jiang, B., Li, J., Lu, Y., Cai, Q., Song, H., & Lu, G. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103013 · Cited by 73
The limits of the Mean Opinion Score for speech synthesis evaluation
Maguer, S. L., King, S., & Harte, N. (2023) · Computer Speech & Language · doi:10.1016/j.csl.2023.101577 · Cited by 67
“It is the only dimension with a median MOS di ff erent from 5.”
BackgroundInfluentialSpeech Enhancement—A Review of Modern Methods
O’Shaughnessy, D. D. (2024) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2023.3339663 · Cited by 66
“It uses an auditory transform, comparing loudness spectra of a clean reference signal and a separated signal, with a score in a range of − 0.5 to 4.5, which corresponds to the perceptual metric mean opinion score (MOS) [34], which rates speech quality on a scale from bad (1) to excellent (5).”
MethodInfluentialWu, Q., Li, H., Ngan, K. N., & Ma, K. (2017) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2017.2710419 · Cited by 65
Reliability of Human Evaluation for Text Summarization: Lessons Learned and Challenges Ahead
Iskender, N., Polzehl, T., & Möller, S. (2021) · Cited by 58
Wang, L., Song, Z., Zhu, L., & Jiang, J. (2023) · iScience · doi:10.1016/j.isci.2023.106463 · Cited by 56
“24 Through reasonable control of the amplitude, linear analysis can still be used in this nonlinear change process.”
Rabie, T. F., Baziyad, M. S., & Kamel, I. (2018) · Multimedia Tools and Applications · doi:10.1007/s11042-018-5713-2 · Cited by 54
An Attention Enhanced Multi-Task Model for Objective Speech Assessment in Real-World Environments
Dong, X., & Williamson, D. S. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053366 · Cited by 54
“Subjective listening studies are the most accurate forms of assessing speech quality and intelligibility, but this form of assessment is general costly and time-consuming to perform when large-scale assessment is needed [1].”
BackgroundMichel, S., Kaur, S., Gillespie, S. E., Gleason, J. L., Wilson, C. L., & Ghosh, A. (2025) · Conference on Fairness, Accountability and Transparency · doi:10.1145/3715275.3732018 · Cited by 45
“MOS is a widely used [73] and validated measure that has been used in other studies to compare voices from TTS systems with human voices [15, 76].”
Speech Quality Assessment through MOS using Non-Matching References
Manocha, P., & Kumar, A. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-407 · Cited by 52
A novel approach for audio file encryption using hand geometry
Al‐Kateeb, Z. N., & Mohammed, S. J. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-020-08869-8 · Cited by 52
“Most usually judged on a scale of 1 (bad) to 5 (excellent) as shown below [30].”
BackgroundConversational Agents: Theory and ApplicationsPreprint
Wahde, M., & Virgolin, M. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2202.03164 · Cited by 53
Liu, R., Hu, Y., Ren, Y., Yin, X., & Li, H. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29833 · Cited by 49
When IoT met Augmented Reality: Visualizing the Source of the Wireless Signal in AR View
Park, Y., Yun, S., & Kim, K.-H. (2019) · ACM SIGMOBILE International Conference on Mobile Systems, Applications, and Services · doi:10.1145/3307334.3326079 · Cited by 48
“MOS has been widely used as an indicator of the media quality such as VoIP and compressed video [31].”
MethodSpeech Quality Assessment Over Lossy Transmission Channels Using Deep Belief Networks
Affonso, E. T., Rosa, R. L., & Rodríguez, D. Z. (2017) · IEEE Signal Processing Letters · doi:10.1109/lsp.2017.2773536 · Cited by 46
“of limitations [12], such as a minimum number of participants, the cost, and the time required to carry out the tests.”
BackgroundRefining the evaluation of speech synthesis: A summary of the Blizzard Challenge 2023
Perrotin, O., Stephenson, B., Gerber, S., Bailly, G., & King, S. (2024) · Computer Speech & Language · doi:10.1016/j.csl.2024.101747 · Cited by 45
“…the type of listener also has a potentially large effect on the MOS, but their demographics, expertise, and language competency are rarely reported by authors, presumably because they were not controlled (Wester et al., 2015; Streijl et al., 2016; Chiang et al., 2023; Kirkland et al., 2023).”
Sun, H., Shi, W., Liang, X., & Yu, Y. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2936504 · Cited by 45
“In addition to 4232 cameras online cameras, there are 201 cameras that are unable to run, which is attributed to: 1) failure of a charge-couple device (CCD) in a camera; 2) CMOS imaging sensor damage; and 3) a camera being past its useful life.”
BackgroundAccented Text-to-Speech Synthesis With Limited Data
Zhou, X., Zhang, M., Zhou, Y., Wu, Z., & Li, H. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3363414 · Cited by 43
“2) Subjective Evaluation: We evaluate the speech quality in terms of naturalness by mean opinion score (MOS) [63] and Authorized licensed use limited to the terms of the applicable license agreement with IEEE.”
MethodEnvGAN: a GAN-based augmentation to improve environmental sound classification
Madhu, A., & Suresh, K. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10153-0 · Cited by 42
“Figure 1 shows the MOS evaluation of the quality of generated samples from the three datasets with a 95% confidence interval (CI).”
MethodInfluentialBaker, A., Hammerling, D., & Turton, T. L. (2019) · Computer graphics forum (Print) · doi:10.1111/cgf.13707 · Cited by 36
“In image quality measures, subjective evaluation – in the form of mean opinion scores (MOS) – is often considered the ground truth [SWH16,SSB06] to measure the effect of a “deformation" or change to an image.”
BackgroundMeasuring the Quality of Text-to-Video Model Outputs: Metrics and DatasetPreprint
Chivileva, I., Lynch, P., Ward, T. E., & Smeaton, A. F. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.08009 · Cited by 42
“Despite its popularity across various media including speech, audio, images, video, multi-media, etc. it does have issues with its acceptability [21] because a single number cannot capture a diversity of opinion scores.”
BackgroundPasquini, L., Napolitano, A., Pignatelli, M., Tagliente, E., Parrillo, C., Nasta, F., Romano, A., Bozzao, A., & Napoli, A. D. (2022) · Pharmaceutics · doi:10.3390/pharmaceutics14112378 · Cited by 41
“An example of a rating sc re is the mean opinion score (MOS), an index score of quality ranging between 1 and 5 (1 as the most inferior value) [52]:”
BackgroundAudio Large Language Models Can Be Descriptive Speech Quality EvaluatorsPreprint
Chen, C., Hu, Y., Wang, S., Wang, H., Chen, Z., Zhang, C., Yang, C.-H. H., & Chng, E. S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2501.17202 · Cited by 41
“Mainstream efforts for speech quality evaluation primarily focus on MOS prediction–the subjective score usually used for speech quality evaluation (Streijl et al., 2016).”
QoE-Driven IoT Architecture: A Comprehensive Review on System and Resource Management
Saovapakhiran, B., Naruephiphat, W., Charnsripinyo, C., Baydere, Ş., & Özdemir, S. (2022) · IEEE Access · doi:10.1109/access.2022.3197585 · Cited by 40
IRWArt: Levering Watermarking Performance for Protecting High-quality Artwork Images
Luo, Y., Zhou, T., Liu, F., & Cai, Z. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543507.3583489 · Cited by 38
“Meanwhile, as expected, IRWArt reaches the lowest MOS value, indicating that the watermark embedded through IRWArt is the most imperceptibility.”
BackgroundInfluentialHuang, J., Wu, Y., Wang, F., Fang, Y., Yang, N., Alkan, C., Abraham, D. P., Liao, C., Xu, L., Gao, Z., et al. (2024) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2024.3485022 · Cited by 37
“This subjectivity also makes MOS sensitive to the specifics of the experimental setup and the varying experience levels of the raters [176].”
BackgroundMethodAnalysis of the Effect of QoS on Video Conferencing QoE
Rao, N. B., Maleki, A., Chen, F., Chen, W., Zhang, C., Kaur, N., & Haque, A. (2019) · International Wireless Communications and Mobile Computing Conference (IWCMC) · doi:10.1109/iwcmc.2019.8766591 · Cited by 37
“Furthermore, we need to analyze each pair of videos to get a Differential Mean Opinion Score (DMOS) for that pair [18].”
MethodUsing acoustic perception to water sounds in the planning of urban gardens
Patón, D., Delgado, P., Galet, C., Muriel, J., Méndez-Suárez, M., & Hidalgo‐Sánchez, M. (2019) · Building and Environment · doi:10.1016/j.buildenv.2019.106510 · Cited by 36
Cloud Gaming: A QoE Study of Fast-paced Single-player and Multiplayer Gaming
Lindstrom, S. F., Wetterberg, M., & Carlsson, N. F. (2020) · IEEE/ACM International Conference on Utility and Cloud Computing (UCC) · doi:10.1109/ucc48980.2020.00023 · Cited by 36
“We note that both 5 and 7 fall within the criteria for the number of rating levels that a person generally can distinguish between [24] and empathize that a MOS score should not be interpreted as a precise number but as a statistical measurement [25].”
BackgroundA Survey of AI-Generated Video EvaluationPreprint
Liu, X., Xiang, X., Li, Z., Wang, Y., Li, Z., Liu, Z., Zhang, W., Ye, W., & Zhang, J. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.19884 · Cited by 35
GVSUM: generic video summarization using deep visual features
Basavarajaiah, M., & Sharma, P. (2021) · Multimedia Tools and Applications · doi:10.1007/s11042-020-10460-0 · Cited by 34
A QoS-Based routing algorithm over software defined networks
Elbasheer, M. O., Aldegheishem, A., Lloret, J., & Alrajeh, N. A. (2021) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2021.103215 · Cited by 34
Shao, H., Zhao, H., Sun, Y., Zhang, J., & Xu, Y. (2016) · IEEE Access · doi:10.1109/access.2016.2628382 · Cited by 34
“Mean Opinion Score (MOS) is one of the main subjective service quality evaluation methods to characterize QoE and has emerged as the most popular descriptor of perceived multiple media quality [17].”
BackgroundAn automated model for the assessment of QoE of adaptive video streaming over wireless networks
Taha, M., Ali, A., Lloret, J., Gondim, P. R. D. L., & Cánovas, A. (2021) · Multimedia Tools and Applications · doi:10.1007/s11042-021-10934-9 · Cited by 33
“P.911: Subjective audiovisual quality assessment methods for multimedia applications [38], the possible number of subjective evaluations in tests (as well as in usability tests on terminals or services) ranges from 4 to 40 participants.”
BackgroundSVoice: Enabling Voice Communication in Silence via Acoustic Sensing on Commodity Devices
Fu, Y., Wang, S., Zhong, L., Chen, L., Ren, J., & Zhang, Y. (2022) · ACM Conference on Embedded Networked Sensor Systems (SenSys) · doi:10.1145/3560905.3568530 · Cited by 33
Machine learning-based QoE prediction for video streaming over LTE network
Begluk, T., Husić, J. B., & Baraković, S. (2018) · International Symposium INFOTEH-JAHORINA (INFOTEH) · doi:10.1109/infoteh.2018.8345519 · Cited by 33
“Interpretation of MOS rates has been presented to the participants before the survey, as stated in [25]: 5-excellent, 4good, 3-fair, 2-poor, and 1-bad.”
BackgroundNeural Text-to-Speech Adaptation from Low Quality Public Recordings
Hu, Q., Marchi, E., Winarsky, D., Stylianou, Y., Naik, D., & Kajarekar, S. (2019) · 10th ISCA Workshop on Speech Synthesis (SSW 10) · doi:10.21437/ssw.2019-5 · Cited by 31
“Synthesized voices are evaluated from three aspects: 1) the speech naturalness Mean Opinion Score (MOS) (from 1 to 5) [34] 2) speaker similarity MOS (from 1 to 5) compared to the target speaker voice 3) Signal Noise Ratio (SNR).”
BackgroundDeep learning‐based automatic inpainting for material microscopic images
Ma, B., Ma, B., Gao, M., Wang, Z., Ban, X., Huang, H., & WU, W. (2020) · Journal of Microscopy · doi:10.1111/jmi.12960 · Cited by 31
Image Compression Using Deep Learning: Methods and Techniques
Abd-Alzhra, A. S., & Al-Tamimi, M. S. H. (2022) · Iraqi Journal of Science · doi:10.24996/ijs.2022.63.3.34 · Cited by 30
“To train the CAE network, 5500 images were used from Image Net database [31].”
MethodRupp, M. A. (2023) · Applied Ergonomics · doi:10.1016/j.apergo.2023.104128 · Cited by 30
QoE beyond the MOS: Added value using quantiles and distributions
Hoßfeld, T., Heegaard, P. E., & Varela, M. (2015) · International Conference on Quality of Multimedia Experience (QoMEX) · doi:10.1109/qomex.2015.7148142 · Cited by 30
“Such limitations of MOS are recently discussed [6], [7]; assessment results should contain more details on the variation of opinions among users [8] .”
BackgroundOn Human-in-the-Loop CPS in Healthcare: A Cloud-Enabled Mobility Assistance Service
de Mello, R. C., Jiménez, M. F., Ribeiro, M. R. N., Guimarães, R. L., & Frizera, A. (2019) · Robotica · doi:10.1017/s0263574719000079 · Cited by 29
Incremental Text-to-Speech Synthesis Using Pseudo Lookahead With Large Pretrained Language Model
Saeki, T., Takamichi, S., & Saruwatari, H. (2021) · IEEE Signal Processing Letters · doi:10.1109/lsp.2021.3073869 · Cited by 29
“To evaluate the quality of output speech, we conducted a mean opinion score (MOS) evaluation test [25] on naturalness.”
MethodText Region Conditional Generative Adversarial Network for Text Concealment in the Wild
Keserwani, P., & Roy, P. P. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3103922 · Cited by 28
A Full-Reference Image Quality Assessment Method via Deep Meta-Learning and Conformer
Lang, S., Liu, X., Zhou, M., Luo, J. T., Pu, H., Zhuang, X., Wang, J., Wei, X., Zhang, T., Feng, Y., et al. (2023) · IEEE Transactions on Broadcasting · doi:10.1109/tbc.2023.3308349 · Cited by 28
“We use the mean square error (MSE) as the loss function for both the support set and the query set, where e represents the mean opinion score (MOS) [30], s represents the predicted score, and n represents the number of images.”
MethodNo silver bullet: QoE metrics, QoE fairness, and user diversity in the context of QoE management
Hoßfeld, T., Heegaard, P. E., Skorin‐Kapov, L., & Varela, M. (2017) · International Conference on Quality of Multimedia Experience (QoMEX) · doi:10.1109/qomex.2017.7965671 · Cited by 28
“Previous studies have argued that objective QoE models based on Mean Opinion Scores (MOS) may be ill-suited for QoE management purposes, as variations between users are averaged out [4], [5].”
BackgroundQuality evaluation of no‐reference MR images using multidirectional filters and image statistics
Jang, J., Bang, K., Jang, H., Hwang, D., & Initiative, A. D. N. (2018) · Magnetic Resonance in Medicine · doi:10.1002/mrm.27084 · Cited by 28
“PLCC and SROCC were used to compare the human-rated MOSs of authentically distorted images with the scores obtained by Mortamet’s method, NIQE, and QEMDIM.”
MethodInfluentialVisualCritic: Making LMMs Perceive Visual Quality Like HumansPreprint
Huang, Z., Zhang, Z., Lu, Y., Zha, Z.-J., Chen, Z., & Guo, B. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2403.12806 · Cited by 28
“…quantitatively assesses the perceptual quality of given images by estimating the most commonly used metric for subjective quality, i . e ., Mean Opinion Score (MOS) [46], and a series of quality attribute indicators including the brightness, colorfulness, contrast, noisiness, and sharpness.”
BackgroundReview of methods for coding of speech signals
O’Shaughnessy, D. D. (2023) · EURASIP Journal on Audio Speech and Music Processing · doi:10.1186/s13636-023-00274-x · Cited by 27
“One subjective measures are the MUltiple Stimuli with Hidden Reference and Anchor test [43]; the common mean opinion score is typically above 4 for good waveform coders and above 3 for vocoders [121].”
BackgroundVAW-GAN for Singing Voice Conversion with Non-parallel Training DataPreprint
Lu, J., Zhou, K., Şişman, B., & Li, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.03992 · Cited by 27
“We conduct mean opinion score (MOS) [40], [41] to assess the voice quality of the converted singing voices.”
MethodA Small-Scale Image U-Net-Based Color Quality Enhancement for Dense Point Cloud
Xing, J., Yuan, H., Zhang, W., Guo, T., & Chen, C. (2024) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2024.3367539 · Cited by 26
“The mean opinion score (MOS) [42] represents the average score assigned by the participants, providing an overall measure of perceived quality.”
BackgroundSemantic Summarization of Egocentric Photo Stream Events
Lidon, A., Bolaños, M., Dimiccoli, M., Radeva, P. I., Garolera, M., & Giró-i-Nieto, X. (2017) · doi:10.1145/3133202.3133204 · Cited by 26
Auditory Measures for the Next Billion Users
Slaney, M., Lyon, R. F., Garcia, R. A., Kemler, B., Gnegy, C. N., Wilson, K., Kanevsky, D., Savla, S., & Cerf, V. G. (2020) · Ear and Hearing · doi:10.1097/aud.0000000000000955 · Cited by 26
Charting the path forward: CT image quality assessment - an in-depth review
Xun, S., Li, Q., Liu, X., Huang, P., Zhai, G., Sun, Y., de With, P. H. N., Wu, M., & Tan, T. (2025) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-025-00085-4 · Cited by 26
“The assessment results are generally represented by the Mean Opinion Score (MOS) or Differential Mean Opin-ion Score (DMOS) (Streijl et al. 2016).”
A QoE-maximization-based vertical handover scheme for VLC heterogeneous networks
Bao, X., Adjardjah, W., Okine, A. A., Zhang, W., & Dai, J. (2018) · EURASIP Journal on Wireless Communications and Networking · doi:10.1186/s13638-018-1284-1 · Cited by 25
“Basically, UE’s QoE can be obtained from the mean opinion score (MOS) of data traffic [19].”
BackgroundForty years of color quantization: a modern, algorithmic survey
Celebi, M. E. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10406-6 · Cited by 25
Speech coding techniques and challenges: a comprehensive literature survey
Nagaraja, B. G., Anees, M., & Thimmaraja Yadava, G. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-16665-3 · Cited by 25
“• Mean opinion score (MOS): is another subjective metric that measures the perceived quality of the coded speech signal [58].”
Ahn, J., Lee, H., Kim, Y., Lee, S., & Chung, J. (2019) · Japanese Journal of Applied Physics · doi:10.7567/1347-4065/ab14d2 · Cited by 25
Full-Reference Image Quality Expression via Genetic Programming
Bakurov, I., Buzzelli, M., Schettini, R., Castelli, M., & Vanneschi, L. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3244662 · Cited by 13
“However, in the context of a digital society where hundreds of millions of photos are being generated and uploaded to social media every day, subjective evaluation becomes impractical as it is time-consuming, expensive, and highly sensible to the experimental design [2].”
BackgroundQoE Prediction for Enriched Assessment of Individual Video Viewing Experience
Zhu, Y., Hanjalić, A., & Redi, J. (2016) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2964284.2964330 · Cited by 24
“That is, PQ predictions are the same for all users, mimicking the perception of an “average user”, typically measured subjectively through Mean Opinion Scores (MOS) [32].”
BackgroundInfluentialIskender, N., Polzehl, T., & Möller, S. (2020) · Workshop on Evaluation and Comparison of NLP Systems (Eval4NLP) · doi:10.18653/v1/2020.eval4nlp-1.16 · Cited by 24
Direct Speech to Speech Translation: A ReviewPreprint
Sarim, M., Shakeel, S., Javed, L., Jamaluddin, & Nadeem, M. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.04799 · Cited by 24
Ghosh, S., Sarkar, S., Ghosh, S., Zalkow, F., & Jana, N. D. (2024) · Applied Intelligence · doi:10.1007/s10489-024-05380-7 · Cited by 23
“To assess the quality of the generated audio, we have employed the Mean Opinion Score (MOS) [74] as a subjective evaluation metric.”
ControlVC: Zero-Shot Voice Conversion with Time-Varying Controls on Pitch and Speed
Chen, M., & Duan, Z. (2023) · Interspeech (USB) · doi:10.21437/interspeech.2023-1788 · Cited by 23
“The differences of MOS ratings between the single-control (pitch-only or speed-only) and pitch+speed control is not statistically significant (p = 0.07 for pitch; p = 0.43 for speed).”
MethodInfluentialde Souza, J., & Tartz, R. (2024) · Frontiers in Virtual Reality · doi:10.3389/frvir.2024.1368721 · Cited by 22
Kumcu, A., Bombeke, K., Platiša, L., Jovanov, L., LOOY, J. V., & Philips, W. R. (2016) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2016.2638681 · Cited by 22
“Regression models may offer additional advantages such as better handling of violations of normality in rating scores [38], [39] and controlling for within-subjects correlation across conditions and scenes.”
BackgroundMethodA Blockchain-Assisted Trusted Clustering Mechanism for IoT-Enabled Smart Transportation System
Awan, K. A., Din, I. U., & Almogren, A. (2022) · Sustainability · doi:10.3390/su142214889 · Cited by 21
Luo, Y., Zhou, T., Cui, S., Ye, Y., Liu, F., & Cai, Z. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3295895 · Cited by 21
“We count the final Mean Opinion Scores (MOS) [57] are results.”
BackgroundTuncer, D., Koch, R. A., Badonnel, R., & Stiller, B. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-60774-0 · Cited by 21
“IFMAP [3,30,31] is not capable of reflecting interfaces or routing information.”
FRAMEWORK FOR MODELLING MOBILE NETWORK QUALITY OF EXPERIENCE THROUGH BIG DATA ANALYTICS APPROACH
Yusuf-Asaju, A. W., Dahalin, Z. M., & Ta’a, A. (2017) · Journal of Information and Communication Technology · doi:10.32890/jict2018.17.1.8246 · Cited by 21
Yin, J., Chen, Y., Sang, G., Liao, B. Y., & Wang, X. (2019) · IEEE Access · doi:10.1109/access.2019.2908681 · Cited by 21
Retrieval-Augmented Dialogue Knowledge Aggregation for expressive conversational speech synthesis
Liu, R., Jia, Z., Bao, F., & Li, H. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.102948 · Cited by 20
Ismail-Fawaz, A., Devanne, M., Berretti, S., Weber, J., & Forestier, G. (2025) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2025.104337 · Cited by 20
Jara, E. P., Mezher, A. M., Igartua, M. A., Redondo, R. P. D., & Vilas, A. F. (2021) · Sensors · doi:10.3390/s21030901 · Cited by 20
“We have chosen five values following the same structure traditionally done in the mean opinion score (MOS) metric used to assess subjective quality of service (QoS) in multimedia applications [Streijl et al., 2016].”
Method
Metadata
- Authors
- Streijl, R. C., Winkler, S., & Hands, D. S.
- Affiliations
- AT&T (United States) (US)
- Advanced Digital Sciences Center (SG)
- Microsoft (United Kingdom) (GB)
- Corresponding author
- Winkler, Stefan
- Venue
- Multimedia Systems
- Published
- 2014-12-28
- Volume
- 22
- Issue
- 2
- Pages
- 213-227
- ISSN
- 0942-4962 (ISSN-L)
- 1432-1882
- OpenAlex
- W1975163393
- Cited by
- 506
- References
- 78
- Citations by year
- 2026: 34 · 2025: 55 · 2024: 78 · 2023: 76 · 2022: 53 · 2021: 40 · 2020: 52 · 2019: 40 · 2018: 36 · 2017: 30 · 2016: 10 · 2015: 2
- FWCI
- 12.52
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Multimedia Systems (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Video Quality Assessment
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Physical Sciences › Computer Science › Computer Networks and Communications › Network Traffic and Congestion Control
- Keywords
- mean opinion score, subjective quality assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.