CoChat
Sign Up
Cited by 14Image and Video Quality AssessmentDigital Media Forensic DetectionAdvanced Image Processing Techniques

Helping Visually Impaired People Take Better Quality Pictures

Open access
Mandal, Maniratnam, Ghadiyaram, Deepti, Gurari, Danna, Bovik, Alan C.·2023·IEEE Transactions on Image Processing
DOI 10.1109/tip.2023.3282067
Open this paper in CoChat — free accountOpen PDF

Abstract

Perception-based image analysis technologies can be used to help visually impaired people take better quality pictures by providing automated guidance, thereby empowering them to interact more confidently on social media. The photographs taken by visually impaired users often suffer from one or both of two kinds of quality issues: technical quality (distortions), and semantic quality, such as framing and aesthetic composition. Here we develop tools to help them minimize occurrences of common technical distortions, such as blur, poor exposure, and noise. We do not address the complementary problems of semantic quality, leaving that aspect for future work. The problem of assessing, and providing actionable feedback on the technical quality of pictures captured by visually impaired users is hard enough, owing to the severe, commingled distortions that often occur. To advance progress on the problem of analyzing and measuring the technical quality of visually impaired user-generated content (VI-UGC), we built a very large and unique subjective image quality and distortion dataset. This new perceptual resource, which we call the LIVE-Meta VI-UGC Database, contains 40K real-world distorted VI-UGC images and 40K patches, on which we recorded 2.7M human perceptual quality judgments and 2.7M distortion labels. Using this psychometric resource we also created an automatic limited vision picture quality and distortion predictor that learns local-to-global spatial quality relationships, achieving state-of-the-art prediction performance on VI-UGC pictures, significantly outperforming existing picture quality models on this unique class of distorted picture data. We also created a prototype feedback system that helps to guide users to mitigate quality issues and take better quality pictures, by creating a multi-task learning framework. The dataset and models can be accessed at: https://github.com/mandal-cv/visimpaired.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/tip.2023.3282067

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Numerous studies have examined how VLMs can support BLV people's visual access needs, but they largely sidestep image quality issues by asking for better photos or leaving users to triangulate facts across multiple models.

    “While numerous studies have examined how VLMs can support BLV people's visual access needs, they largely sidestep image quality issues by asking for better photos (e.g., Seeing AI, Be My AI, [92]) or leaving users to triangulate facts across multiple models [27].”

    From "It's trained by non-disabled people": Evaluating How Image Quality Affects Product Captioning with Vision-Language Models (2026) · cites this paper as [92]

    The Abstract and Section B, “Feedback to Assist Visually Impaired Users” (pages unavailable), say this paper directly addresses image-quality issues: it predicts distortions and offers specific guidance on blur, lighting, and camera movement. It is therefore contrary to presenting this paper as an example of work that sidesteps image quality by merely asking users for better photos. (LLM, conf=0.88)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 56 · Cited by 14

Research works printed but not linked: 45

  1. J. P. Bigham, C. Jayant, H. Ji, G. Little, A. Miller, R. C. Miller, R. Miller, A. Tatarowicz, B. White, S. White, and T. Yeh, "Vizwiz: Nearly real-time answers to visual questions," ser. UIST '10. New York, NY, USA: Association for Computing Machinery, 2010, p. 333-342. [Online]. Available: https://doi.org/10.1145/1866029.1866080 1, 2, 3 · DOI · Paper page
  2. Y. Zhong, P. J. Garrigues, and J. P. Bigham, "Real time object scanning using a mobile phone and cloud-based visual search engine," in Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility, ser. ASSETS '13. New York, NY, USA: Association for Computing Machinery, 2013. [Online]. Available: https://doi.org/10.1145/2513383.2513443 1, 3 · DOI
  3. C. L. Bennett, E. Jane, M. E. Mott, E. Cutrell, and M. R. Morris, How Teens with Visual Impairments Take, Edit, and Share Photos on Social Media. New York, NY, USA: Association for Computing Machinery, 2018, p. 1-12. [Online]. Available: https://doi.org/10.1145/3173574.3173650 1 · DOI
  4. H. MacLeod, C. L. Bennett, M. R. Morris, and E. Cutrell, "Understanding blind people's experiences with computer-generated captions of social media images," in Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, ser. CHI '17. New York, NY, USA: Association for Computing Machinery, 2017, p. 5988-5999. [Online]. Available: https://doi.org/10.1145/ 3025453.3025814 1 · DOI · Paper page
  5. S. Wu, J. Wieland, O. Farivar, and J. Schiller, "Automatic alt- text: Computer-generated image descriptions for blind users on a social network service," in Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing, ser. CSCW '17. New York, NY, USA: Association for Computing Machinery, 2017, p. 1180-1192. [Online]. Available: https://doi.org/10.1145/2998181.2998364 1 · DOI
  6. A. Mittal, A. K. Moorthy, and A. C. Bovik, "No-reference image quality assessment in the spatial domain," IEEE Transactions on Image Processing, vol. vol. 21, no. 12, pp. 4695-4708, 2012. 1, 3, 7
  7. Z. Ying, H. Niu, P. Gupta, D. Mahajan, D. Ghadiyaram, and A. C. Bovik, "From patches to pictures (paq-2-piq): Mapping the perceptual space of picture quality," in 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020, pp. 3572-3582. 1, 2, 3, 4, 5, 6, 7, 8, 9
  8. H. Talebi and P. Milanfar, "NIMA: Neural image assessment," IEEE Transactions on Image Processing, vol. vol. 27, no. 8, pp. 3998-4011, Aug 2018. 1, 2, 7, 8
  9. L. Kang, P. Ye, Y. Li, and D. Doermann, "Convolutional neural networks for no-reference image quality assessment," in IEEE Int'l Conf. on Comput. Vision and Pattern Recogn. (CVPR), June 2014, pp. 1733- 1740. 1, 7, 8
  10. M. Vázquez and A. Steinfeld, "An assisted photography framework to help visually impaired users properly aim a camera," vol. 21, no. 5, Nov. 2014. [Online]. Available: https://doi.org/10.1145/2651380 2, 3 · DOI
  11. C. Jayant, H. Ji, S. White, and J. P. Bigham, "Supporting blind photography," in The Proceedings of the 13th International ACM SIGACCESS Conference on Computers and Accessibility. New York, NY, USA: Association for Computing Machinery, 2011, p. 203-210. [Online]. Available: https://doi.org/10.1145/2049536.2049573 2, 3 · DOI
  12. Z. Ying, M. Mandal, D. Ghadiyaram, A. B. U. of Texas at Austin, and A. Facebook, "Patch-vq: 'patching up' the video quality problem," 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 14 014-14 024, 2021. 2, 4, 5, 6
  13. D. Ghadiyaram and A. C. Bovik, "Massive online crowdsourced study of subjective and objective picture quality," IEEE Transactions on Image Processing, vol. vol. 25, no. 1, pp. 372-387, Jan 2016. 2, 3, 4, 5
  14. H. R. Sheikh, M. F. Sabir, and A. C. Bovik, "A statistical evaluation of recent full reference image quality assessment algorithms," IEEE Transactions on Image Processing, vol. vol. 15, no. 11, pp. 3440-3451, Nov 2006. 2, 3
  15. N. Ponomarenko, V. Lukin, A. Zelensky, K. Egiazarian, M. Carli, and F. Battisti, "TID2008-a database for evaluation of full-reference visual quality assessment metrics," Advances of Modern Radioelectronics, vol. vol. 10, no. 4, pp. 30-45, 2009. 2, 3
  16. N. Ponomarenko, O. Ieremeiev, V. Lukin, K. Egiazarian, L. Jin, J. Astola, B. Vozel, K. Chehdi, M. Carli, F. Battisti, and C. . J. Kuo, "Color image database TID2013: Peculiarities and preliminary results," in European Workshop on Visual Information Processing, vol. vol. 30, pp. 106-111, June 2013, pp. 106-111. 2, 3
  17. Z. Wang and A. C. Bovik, "Mean squared error: Love it or leave it? A new look at signal fidelity measures," IEEE Signal Process. Mag., vol. vol. 26, no. 1, pp. 98-117, Jan 2009. 2
  18. J. Park, S. Lee, and A. Bovik, "VQpooling: Video quality pooling adaptive to perceptual distortion severity," IEEE Transactions on Image Processing, vol. vol. 22, no. 2, pp. 610-620, Feb. 2013. 2
  19. H. Lin, V. Hosu, and D. Saupe, "Koniq-10K: Towards an ecologically valid and large-scale IQA database," arXiv preprint arXiv:1803.08489, March 2018. 2, 3, 4, 5 · arXiv
  20. N. Ahn, B. Kang, and K. Sohn, "Image distortion detection using convolutional neural network," 2017 4th IAPR Asian Conference on Pattern Recognition (ACPR), pp. 220-225, 2017. 2, 8
  21. A. Mittal, R. Soundararajan, and A. C. Bovik, "Making a "Completely blind" image quality analyzer," IEEE Signal Processing Letters, vol. vol. 20, pp. 209-212, 2013. 3, 7
  22. P. Ye, J. Kumar, L. Kang, and D. Doermann, "Unsupervised feature learning framework for no-reference image quality assessment," in IEEE Int'l Conf. on Comput. Vision and Pattern Recogn. (CVPR), June 2012, pp. 1098-1105. 3
  23. D. Ghadiyaram and A. C. Bovik, "Perceptual quality prediction on authentically distorted images using a bag of features approach," Journal of Vision, vol. vol. 17, no. 1, art. 32, pp. 1-25, January 2017. 3, 7
  24. --, "Blind image quality assessment on real distorted images using deep belief nets," in IEEE Global Conference on Signal and Information processing, vol. pp. 946-950, Atlanta, GA, 2014, pp. 946-950. 3
  25. S. Bosse, D. Maniry, T. Wiegand, and W. Samek, "A deep neural network for image quality assessment," in 2016 IEEE Int'l Conf. Image Process. (ICIP), Sep. 2016, pp. 3773-3777. 3
  26. J. Kim and S. Lee, "Fully deep blind image quality predictor," IEEE J. of Selected Topics in Signal Process., vol. vol. 11, no. 1, pp. 206-220, Feb 2017. 3
  27. L. Kang, P. Ye, Y. Li, and D. Doermann, "Simultaneous estimation of image quality and distortion via multi-task convolutional neural networks," in 2015 IEEE International Conference on Image Processing (ICIP), 2015, pp. 2791-2795. 3, 8, 9, 10, 11
  28. J. Deng, W. Dong, R. Socher, L. Li, Kai Li, and Li Fei-Fei, "ImageNet: A large-scale hierarchical image database," in IEEE Conf. Comput. Vision and Pattern Recogn., June 2009, pp. 248-255. 3, 7, 8
  29. J. Fritz and K. Barner, "Design of a haptic data visualization system for people with visual impairments," IEEE Transactions on Rehabilitation Engineering, vol. 7, no. 3, pp. 372-384, 1999. 3
  30. T. Way and K. Barner, "Automatic visual to tactile translation. I. human factors, access methods and image manipulation," IEEE Transactions on Rehabilitation Engineering, vol. 5, no. 1, pp. 81-94, 1997. 3
  31. --, "Automatic visual to tactile translation. II. evaluation of the TACTile image creation system," IEEE Transactions on Rehabilitation Engineering, vol. 5, no. 1, pp. 95-105, 1997. 3
  32. B. D. Lucas and T. Kanade, "An iterative image registration technique with an application to stereo vision," in Proceedings of the 7th Inter- national Joint Conference on Artificial Intelligence -Volume 2, ser. IJCAI'81. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 1981, p. 674-679. 3
  33. T. Zhao and X. Wu, "Pyramid feature attention network for saliency detection," in 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 3080-3089. 4
  34. Z. Sinno and A. Bovik, "Large-scale study of perceptual video quality," IEEE Transactions on Image Processing, vol. vol. 28, no. 2, pp. 612-627, Feb. 2019, [Online] LIVE VQC Database: http://live.ece.utexas.edu/ research/LIVEVQC/index.html. 4, 5
  35. R. Likert, "A technique for the measurement of attitudes," Archives of Psychology, vol. vol. 140, pp. 1-55, 1932. 5
  36. Measure of Kurtosis. New York, NY: Springer New York, 2008, pp. 343-343. [Online]. Available: https://doi.org/10.1007/978-0-387- 32833-1 255 5 · DOI
  37. J. Tukey, Exploratory data analysis. Reading, Mass: Addison-Wesley Pub. Co, 1977. 5
  38. J. L. Rodgers and W. A. Nicewander, "Thirteen ways to look at the correlation coefficient," The American Statistician, vol. 42, no. 1, pp. 59-66, 1988. [Online]. Available: https://doi.org/10.1080/ 00031305.1988.10475524 6 · DOI
  39. J. C. Peterson, R. M. Battleday, T. L. Griffiths, and O. Russakovsky, "Human uncertainty makes classification more robust," 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 9616-9625, 2019. 7
  40. K. He, X. Zhang, S. Ren, and J. Sun, "Identity mappings in deep residual networks," ArXiv, vol. abs/1603.05027, 2016. 7, 8, 9 · arXiv
  41. F. Chollet, "Xception: Deep learning with depthwise separable con- volutions," 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1800-1807, 2017. 7, 8, 9
  42. K. Hara, H. Kataoka, and Y. Satoh, "Can spatiotemporal 3d cnns retrace the history of 2d cnns and imagenet?" in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 6546-6555. 7, 8
  43. K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," CoRR, vol. abs/1409.1556, 2015. 7, 8 · arXiv
  44. L. C. Chen, G. Papandreou, I. Kokkinos, K. P. Murphy, and A. L. Yuille, "Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 40, pp. 834-848, 2018.
  45. M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L. Chen, "Mo- bilenetv2: Inverted residuals and linear bottlenecks," in IEEE Int'l Conf. on Comput. Vision and Pattern Recogn. (CVPR), June 2018, pp. 4510- 4520. 9

Authors
Mandal, M., Ghadiyaram, D., Gurari, D., & Bovik, A. C.
Affiliations
  • The University of Texas at Austin (US)
  • Meta (United States) (US)
  • University of Colorado Boulder (US)
ORCID
Venue
IEEE Transactions on Image Processing
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Published
2023
Volume
32
Pages
3873-3884
ISSN
  • 1057-7149 (ISSN-L)
  • 1941-0042
DOI
10.1109/tip.2023.3282067
OpenAlex
W4383890030
Cited by
14
References
56
Citations by year
2026: 4 · 2025: 6 · 2024: 2 · 2023: 2
FWCI
1.27
Type
journal article
Language
en
Open access
Licence
Locations
  • IEEE Transactions on Image Processing (publishedVersion)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by-nc-nd)
  • arXiv (Cornell University)
Funders
  • Meta AI Research
  • National Science Foundation AI Institute for Foundations of Machine Learning (2019844)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Video Quality Assessment
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image Processing Techniques
Keywords
visually impaired users, user-generated content, image quality assessment, image distortion detection, blur, exposure, noise, quality prediction, multi-task learning, feedback systems, photographic composition, social media
MeSH terms
Color Perception, Humans, Image Processing, Computer-Assisted, Semantics, Visual Acuity, Persons with Visual Disabilities, Visually Impaired Persons

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review