Mean squared error: Love it or leave it? A new look at Signal Fidelity Measures
Abstract
In this article, we have reviewed the reasons why we (collectively) want to love or leave the venerable (but perhaps hoary) MSE. We have also reviewed emerging alternative signal fidelity measures and discussed their potential application to a wide variety of problems. The message we are trying to send here is not that one should abandon use of the MSE nor to blindly switch to any other particular signal fidelity measure. Rather, we hope to make the point that there are powerful, easy-to-use, and easy-to-understand alternatives that might be deployed depending on the application environment and needs. While we expect (and indeed, hope) that the MSE will continue to be widely used as a signal fidelity measure, it is our greater desire to see more advanced signal fidelity measures being used, especially in applications where perceptual criteria might be relevant. Ideally, the performance of a new signal processing algorithm might be compared to other algorithms using several fidelity criteria. Lastly, we hope that we have given further motivation to the community to consider recent advanced signal fidelity measures as design criteria for optimizing signal processing algorithms and systems. It is in this direction that we believe that the greatest benefit eventually lies.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
PSNR is a poor measure of perceptual quality.
“Although PSNR is a poor measure of perceptual quality, it is often used to benchmark for QA algorithms [41], [42].”
From No-Reference Image Quality Assessment in the Spatial Domain (2012) · cites this paper as [41]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892 · see citing works
Self-citationMultiscale structural similarity for image quality assessment
Wang, Z., Simoncelli, E. P., & Bovik, A. C. (2004) · The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003 · doi:10.1109/acssc.2003.1292216 · Cited by 6,253 · see citing works
Self-citationNatural Image Statistics and Neural Representation
Simoncelli, E. P., & Olshausen, B. A. (2001) · Annual Review of Neuroscience · doi:10.1146/annurev.neuro.24.1.1193 · Cited by 2,460 · see citing works
The effects of a visual fidelity criterion of the encoding of images
Mannos, J., & Sakrison, D. (1974) · IEEE Transactions on Information Theory · doi:10.1109/tit.1974.1055250 · Cited by 1,063 · see citing works
Quality assessment of motion rendition in video coding
van den Branden Lambrecht, C. J., Costantini, D. M., Sicuranza, G. L., & Kunt, M. (1999) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/76.780365 · Cited by 38 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
No-Reference Image Quality Assessment in the Spatial Domain
Mittal, A., Moorthy, A. K., & Bovik, A. C. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2214050 · Cited by 6,064 · see citing works
Self-citationImage Quality Metrics: PSNR vs. SSIM
Hore, A., & Ziou, D. (2010) · 2010 20th International Conference on Pattern Recognition · doi:10.1109/icpr.2010.579 · Cited by 4,943 · see citing works
Blind Image Quality Assessment: From Natural Scene Statistics to Perceptual Quality
Moorthy, A. K., & Bovik, A. C. (2011) · IEEE Transactions on Image Processing · doi:10.1109/tip.2011.2147325 · Cited by 1,763 · see citing works
Self-citationPros and cons of GAN evaluation measures
Borji, A. (2019) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2018.10.009 · Cited by 963 · see citing works
Waterloo Exploration Database: New Challenges for Image Quality Assessment Models
Ma, K., Duanmu, Z., Wu, Q., Wang, Z., Yong, H., Li, H., & Zhang, L. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2016.2631888 · Cited by 789 · see citing works
Self-citationNo-Reference Image Quality Assessment via Transformers, Relative Ranking, and Self-Consistency
Golestaneh, S. A., Dadsetan, S., & Kitani, K. M. (2022) · 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv51458.2022.00404 · Cited by 465 · see citing works
Visual Importance Pooling for Image Quality Assessment
Moorthy, A. K., & Bovik, A. C. (2009) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2009.2015374 · Cited by 340 · see citing works
Self-citationRRED Indices: Reduced Reference Entropic Differencing for Image Quality Assessment
Soundararajan, R., & Bovik, A. C. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2011.2166082 · Cited by 337 · see citing works
Self-citationReduced- and No-Reference Image Quality Assessment
Wang, Z., & Bovik, A. (2011) · IEEE Signal Processing Magazine · doi:10.1109/msp.2011.942471 · Cited by 280 · see citing works
Self-citationA comprehensive assessment of the structural similarity index
Dosselmann, R., & Yang, X. D. (2011) · Signal, Image and Video Processing · doi:10.1007/s11760-009-0144-1 · Cited by 275 · see citing works
A survey of image quality measures
Thung, K.-H., & Raveendran, P. (2009) · 2009 International Conference for Technical Postgraduates (TECHPOS) · doi:10.1109/techpos.2009.5412098 · Cited by 217 · see citing works
Applications of Objective Image Quality Assessment Methods [Applications Corner]
Wang, Z. (2011) · IEEE Signal Processing Magazine · doi:10.1109/msp.2011.942295 · Cited by 167 · see citing works
Self-citationVideo Quality Pooling Adaptive to Perceptual Distortion Severity
Park, J., Seshadrinathan, K., Lee, S., & Bovik, A. C. (2013) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2219551 · Cited by 119 · see citing works
Self-citationZhang, Y., Moorthy, A. K., Chandler, D. M., & Bovik, A. C. (2014) · Signal Processing: Image Communication · doi:10.1016/j.image.2014.05.004 · Cited by 102 · see citing works
Self-citationProspects of Structural Similarity Index for Medical Image Analysis
Mudeng, V., Kim, M., & Choe, S.-W. (2022) · Applied Sciences · doi:10.3390/app12083754 · Cited by 101 · see citing works
Martini, M. G. (2025) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2025.3529045 · Cited by 47 · see citing works
Stochastic Image Denoising by Sampling from the Posterior Distribution
Kawar, B., Vaksman, G., & Elad, M. (2021) · 2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW) · doi:10.1109/iccvw54120.2021.00213 · Cited by 42 · see citing works
Perceptually significant spatial pooling techniques for image quality assessment
Moorthy, A. K., & Bovik, A. C. (2009) · SPIE Proceedings · doi:10.1117/12.810166 · Cited by 37 · see citing works
Self-citationDisruptive Innovations and Disruptive Assurance: Assuring Machine Learning and Autonomy
Bloomfield, R., Khlaaf, H., Ryan Conmy, P., & Fletcher, G. (2019) · Computer · doi:10.1109/mc.2019.2914775 · Cited by 23 · see citing works
Perceptual Video Processing: Seeing the Future [Point of View]
Bovik, A. C. (2010) · Proceedings of the IEEE · doi:10.1109/jproc.2010.2068371 · Cited by 19 · see citing works
Self-citation
- Authors
- Wang, Z., & Bovik, A. C.
- Affiliations
- Howard Hughes Medical Institute (US)
- The University of Texas at Austin (US)
- ORCID
- Venue
- IEEE Signal Processing Magazine
- Published
- 2009-01
- Volume
- 26
- Issue
- 1
- Pages
- 98-117
- ISSN
- 1053-5888 (ISSN-L)
- OpenAlex
- W1973207880
- Cited by
- 3,175
- References
- 5
- Citations by year
- 2026: 181 · 2025: 283 · 2024: 255 · 2023: 236 · 2022: 217 · 2021: 269 · 2020: 201 · 2019: 189 · 2018: 174 · 2017: 136 · 2016: 159 · 2015: 178 · 2014: 192 · 2013: 129 · 2012: 131
- FWCI
- 138.33
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Signal Processing Magazine (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Video Quality Assessment
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Data Compression Techniques
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Keywords
- mean squared error, perceptual evaluation, algorithm comparison
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
