The DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
Analysis of Public Image and Video Databases for Quality Assessment
Abstract
Databases of images or videos annotated with subjective ratings constitute essential ground truth for training, testing, and benchmarking algorithms for objective quality assessment. More than two dozen such databases are now available in the public domain; they are presented and analyzed in this paper. We propose several criteria for quantitative comparisons of source content, test conditions, and subjective ratings, which are used as the basis for the ensuing analyses and discussion. This information will allow researchers to make more well-informed decisions about databases, and may also guide the creation of additional test material and the design of future experiments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Statistical Evaluation of Recent Full Reference Image Quality Assessment Algorithms
Sheikh, H. R., Sabir, M. F., & Bovik, A. C. (2006) · IEEE Transactions on Image Processing · doi:10.1109/tip.2006.881959 · Cited by 2,966
Most apparent distortion: full-reference image quality assessment and the role of strategy
Larson, E. C., & Chandler, D. M. (2010) · Journal of Electronic Imaging · doi:10.1117/1.3267105 · Cited by 1,953
Live image quality assessment database release 2
Sheikh, H. R. (2005) · Medical Entomology and Zoology · Cited by 1,275
Study of Subjective and Objective Quality Assessment of Video
Seshadrinathan, K., Soundararajan, R., Bovik, A. C., & Cormack, L. K. (2010) · IEEE Transactions on Image Processing · doi:10.1109/tip.2010.2042111 · Cited by 1,160
TID2008 – A database for evaluation of full-reference visual quality assessment metrics
Ponomarenko, N. N., Лукин, В. В., & Zelensky, A. A. (2004) · Iris (Roma Tre University) · Cited by 992
Measuring colorfulness in natural images
Hasler, D., & Suesstrunk, S. E. (2003) · SPIE Proceedings · doi:10.1117/12.477378 · Cited by 729
Quality Assessment of Stereoscopic Images
Benoit, A., Le Callet, P., Campisi, P., & Cousseau, R. (2008) · EURASIP Journal on Image and Video Processing · doi:10.1155/2008/659024 · Cited by 374
Video Quality Assessment on Mobile Devices: Subjective, Behavioral and Objective Studies
Moorthy, A. K., Choi, L. K., Bovik, A. C., & de Veciana, G. (2012) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2012.2212417 · Cited by 303
Subjective quality assessment IRCCyN/IVC database
Callet, P. L., & Autrusseau, F. (2005) · HAL · Cited by 261
Brooks, A. C., Zhao, X., & Pappas, T. N. (2008) · IEEE Transactions on Image Processing · doi:10.1109/tip.2008.926161 · Cited by 175
Assessing Quality of Experience of IPTV and Video on Demand Services in Real-Life Environments
Staelens, N., Moens, S., Van den Broeck, W., Marien, I., Vermeulen, B., Lambert, P., Van de Walle, R., & Demeester, P. (2010) · IEEE Transactions on Broadcasting · doi:10.1109/tbc.2010.2067710 · Cited by 166
Subjective assessment of H.264/AVC video sequences transmitted over a noisy channel
Francesca De Simone, A., Naccari, M., Tagliasacchi, M., Frederic Dufaux, A., Tubaro, S., & Ebrahimi, T. (2009) · International Conference on Quality of Multimedia Experience (QoMEX) · doi:10.1109/qomex.2009.5246952 · Cited by 142
Lutz Goldmann, A., Francesca, D. S., & Ebrahimi, T. (2010) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.839438 · Cited by 103
Subjective Quality Evaluation via Paired Comparison: Application to Scalable Video Coding
Lee, J., Francesca De Simone, A., & Ebrahimi, T. (2011) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2011.2157333 · Cited by 102
Reduced-reference metric design for objective perceptual quality assessment in wireless imaging
Engelke, U., Kusuma, T. M., Zepernick, H., & Caldera, M. (2009) · Signal Processing Image Communication · doi:10.1016/j.image.2009.06.005 · Cited by 95
Pseudo no reference image quality metric using perceptual data hiding
Ninassi, A., Callet, P. L., & Autrusseau, F. (2006) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.650780 · Cited by 83
Modeling the impact of frame rate on perceptual quality of video
Ou, Y.-F., Liu, T., Zhao, Z., et al. (2008) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2008.4711848 · Cited by 82
On the properties of subjective ratings in video quality experiments
WINKLER, S. S. (2009) · International Conference on Quality of Multimedia Experience (QoMEX) · doi:10.1109/qomex.2009.5246961 · Cited by 77
Self-citationImpact of subjective dataset on the performance of image quality metrics
Tourancheau, S., Autrusseau, F., Sazzad, Z. M. P., & Horita, Y. (2008) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2008.4711767 · Cited by 58
Interactions of visual attention and quality perception
Redi, J., Liu, H., Zunino, R., et al. (2011) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.876712 · Cited by 52
Péchard, S., Pépion, R., & Callet, P. L. (2008) · HAL · Cited by 49
Subjective evaluation of scalable video coding for content distribution
Lee, J., Francesca De Simone, A., Ramzan, N., Zhao, Z., Kurutepe, E., Sikora, T., Östermann, J., Ebroul Izquierdo, M., & Ebrahimi, T. (2010) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1873951.1873981 · Cited by 48
Saliency based objective quality assessment of decoded video affected by packet losses
Feng, X., Liu, T., Yang, D., & Wang, Y. (2008) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2008.4712316 · Cited by 46
Visual attention modelling for subjective image quality databases
Engelke, U., Maeder, A., & Zepernick, H. (2009) · IEEE International Workshop on Multimedia Signal Processing (MMSP) · doi:10.1109/mmsp.2009.5293248 · Cited by 38
Evaluation of standard watermarking techniques
Marini, E., Autrusseau, F., Callet, P. L., et al. (2007) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.704359 · Cited by 34
Perceptual quality of video with frame rate variation: A subjective study
Ou, Y.-F., Zhou, Y., & Wang, Y. (2010) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2010.5496300 · Cited by 25
Systematic stress testing of image quality estimators
Ciaramello, F. M., & Reibman, A. R. (2011) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2011.6116322 · Cited by 18
Toward a simplified perceptual quality metric for watermarking applications
Carosi, M., Pankajakshan, V., & Autrusseau, F. (2010) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.838627 · Cited by 14
Pyykko, S. J., & Häkkinen, J. P. (2008) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.765697 · Cited by 13
Improving the verification process of video quality metrics
Keimel, C., Oelbaum, T., & Diepold, K. J. (2009) · International Workshop on Quality of Multimedia Experience (QoMEX) · doi:10.1109/qomex.2009.5246966 · Cited by 13
Subjective and objective quality evaluation of lar coded art images
Strauss, C., Pasteau, F., Autrusseau, F., Marie Babel, A., Bédat, L., & Olivier Déforges, A. (2009) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2009.5202586 · Cited by 13
Hanhart, P., Francesca, D. S., Řeřábek, M., et al. (2013) · doi:10.1002/9781118583593.ch19 · Cited by 7
Streijl, R. C., WINKLER, S. S., & Hands, D. S. (2010) · IEEE Signal Processing Magazine · doi:10.1109/msp.2010.936776 · Cited by 7
Self-citationPerceptual Quality Measurement—Towards a More Efficient Process for Validating Objective Models
Streijl, R. C., WINKLER, S. S., & Hands, D. S. (2010) · Cited by 6
Self-citation
Cited by
Image database TID2013: Peculiarities, results and perspectives
Ponomarenko, N., Jin, L., Ieremeiev, O., Lukin, V., Egiazarian, K., Astola, J., Vozel, B., Chehdi, K., Carli, M., Battisti, F., et al. (2015) · Signal Processing: Image Communication · doi:10.1016/j.image.2014.10.009 · Cited by 1,241
Waterloo Exploration Database: New Challenges for Image Quality Assessment Models
Ma, K., Duanmu, Z., Wu, Q., Wang, Z., Yong, H., Li, H., & Zhang, L. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2016.2631888 · Cited by 789
Perceptual image quality assessment: a survey
Zhai, G., & Min, X. (2020) · Science China Information Sciences · doi:10.1007/s11432-019-2757-1 · Cited by 655
Mean opinion score (MOS) revisited: Methods and applications, limitations and alternatives
Streijl, R. C., Winkler, S., & Hands, D. S. (2016) · Multimedia Systems · doi:10.1007/s00530-014-0446-1 · Cited by 506
Self-citationSeven Challenges in Image Quality Assessment: Past, Present, and Future Research
Chandler, D. M. (2013) · ISRN Signal Processing · doi:10.1155/2013/905685 · Cited by 427
Metadata
- Authors
- Winkler, S.
- Venue
- IEEE Journal of Selected Topics in Signal Processing
- Published
- 2012-10
- Volume
- 6
- Issue
- 6
- Pages
- 616-625
- ISSN
- 1932-4553 (ISSN-L)
- 1941-0484
- OpenAlex
- W2026139930
- Cited by
- 299
- References
- 34
- FWCI
- 24.59
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.