CoChat
Sign Up
PreprintCited by 1Multimodal Machine Learning ApplicationsAssistive Technology in Communication and MobilityAdvanced Neural Network Applications

Explaining CLIP's performance disparities on data from blind/low vision users

Open access
Massiceti, Daniela, Longden, Camilla, Słowik, Agnieszka, Wills, Samuel, Grayson, Martin, Morrison, Cecily·2023·arXiv
DOI 10.48550/arxiv.2311.17315arXiv 2311.17315
Open this paper in CoChat — free accountOpen PDF

Abstract

Large multi-modal models (LMMs) hold the potential to usher in a new era of automated visual assistance for people who are blind or low vision (BLV). Yet, these models have not been systematically evaluated on data captured by BLV users. We address this by empirically assessing CLIP, a widely-used LMM likely to underpin many assistive technologies. Testing 25 CLIP variants in a zero-shot classification task, we find that their accuracy is 15 percentage points lower on average for images captured by BLV users than web-crawled images. This disparity stems from CLIP's sensitivities to 1) image content (e.g. not recognizing disability objects as well as other objects); 2) image quality (e.g. not being robust to lighting variation); and 3) text content (e.g. not recognizing objects described by tactile adjectives as well as visual ones). We delve deeper with a textual analysis of three common pre-training datasets: LAION-400M, LAION-2B and DataComp-1B, showing that disability content is rarely mentioned. We then provide three examples that illustrate how the performance disparities extend to three downstream models underpinned by CLIP: OWL-ViT, CLIPSeg and DALL-E2. We find that few-shot learning with as few as 5 images can mitigate CLIP's quality-of-service disparities for BLV users in some scenarios, which we discuss alongside a set of other possible mitigations.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2311.17315

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Existing research predominantly focuses on understanding, defining or evaluating representational 'harms' and often uses poor AI-generated images to elicit feedback from marginalized communities.

    “Our approach to defining and evaluating 'good' representation differs from existing research that predominantly focuses on understanding, defining or evaluating representational 'harms' [24, 28, 38, 44, 67, 77, 91, 116] and often uses poor AI-generated images to elicit feedback from marginalized communities (e.g., [65, 77, 79]).”

    From Engaging Communities Meaningfully in Defining Disability Representation for AI Image Generation (2026) · cites this paper as [79]

    Section 5.2 reports that DALL-E2 often generated incorrect images of disability-related objects, supporting the point about poor AI-generated images. But the images were evaluated by three annotators; our copy does not say they were used to elicit feedback from marginalized communities. The paper also studies performance disparities rather than establishing that existing research predominantly focuses on representational harms. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • Web-scraped data for AI model pre-training is skewed towards an underrepresentation of disability data.

    “Web-scraped data for AI model pre-training is skewed towards an underrepresentation of disability data [69, 79].”

    From Engaging Communities Meaningfully in Defining Disability Representation for AI Image Generation (2026) · cites this paper as [79]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 57

  1. Sandhini Agarwal, Gretchen Krueger, Jack Clark, Alec Rad- ford, Jong Wook Kim, and Miles Brundage. Evaluating clip: towards characterization of broader capabilities and downstream implications. arXiv preprint arXiv:2108.02818, 2021. 1, 2 · arXiv
  2. Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. In Advances in Neural Information Processing Systems, 2022. 2
  3. Reza Akbarian Bafghi and Danna Gurari. A New Dataset Based on Images Taken by Blind People for Testing the Ro- bustness of Image Classification Models Trained for Ima- geNet Categories. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023. 1, 2, 3
  4. Solon Barocas, Anhong Guo, Ece Kamar, Jacquelyn Kro- nes, Meredith Ringel Morris, Jennifer Wortman Vaughan, W Duncan Wadsworth, and Hanna Wallach. Designing dis- aggregated evaluations of ai systems: Choices, considera- tions, and tradeoffs. In AAAI/ACM Conference on AI, Ethics, and Society, 2021. 2
  5. Manuele Barraco, Marcella Cornia, Silvia Cascianelli, Lorenzo Baraldi, and Rita Cucchiara. The unreasonable ef- fectiveness of CLIP features for image captioning: an ex- perimental analysis. In IEEE/CVF conference on computer vision and pattern recognition, 2022. 2
  6. Samyadeep Basu, Daniela Massiceti, Shell Xu Hu, and So- heil Feizi. Strong Baselines for Parameter Efficient Few-Shot Fine-tuning. arXiv preprint arXiv:2304.01917, 2023. 8 · arXiv
  7. Jeffrey P Bigham, Chandrika Jayant, Hanjie Ji, Greg Lit- tle, Andrew Miller, Robert C Miller, Robin Miller, Aubrey Tatarowicz, Brandyn White, Samual White, et al. VizWiz: Nearly real-time answers to visual questions. In Annual ACM Symposium on User Interface Software and Technol- ogy, 2010. 3
  8. Joy Buolamwini and Timnit Gebru. Gender shades: Inter- sectional accuracy disparities in commercial gender classifi- cation. In Conference on fairness, accountability and trans- parency, pages 77-91. PMLR, 2018. 2
  9. Ting Chen, Simon Kornblith, Mohammad Norouzi, and Ge- offrey Hinton. A simple framework for contrastive learn- ing of visual representations. In International Conference on Machine Learning. PMLR, 2020. 2
  10. Xi Chen, Xiao Wang, Soravit Changpinyo, AJ Piergiovanni, Piotr Padlewski, Daniel Salz, Sebastian Goodman, Adam Grycner, Basil Mustafa, Lucas Beyer, et al. PaLI: A jointly- scaled multilingual language-image model. arXiv preprint arXiv:2209.06794, 2022. 2 · arXiv
  11. Tai-Yin Chiu, Yinan Zhao, and Danna Gurari. Assessing im- age quality issues for real-world problems. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020. 1, 2, 3, 5
  12. Sam Corbett-Davies, J Gaebler, Hamed Nilforoshan, Ravi Shroff, and Sharad Goel. The measure and mismeasure of fairness. Journal Machine Learning Research, 2023. 2
  13. Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. InstructBLIP: Towards General- purpose Vision-Language Models with Instruction Tuning. arXiv preprint arXic:2305.06500, 2023. 1, 2
  14. Han Fang, Pengfei Xiong, Luhui Xu, and Yu Chen. CLIP2Video: Mastering video-text retrieval via image CLIP. arXiv preprint arXiv:2106.11097, 2021. 2 · arXiv
  15. Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model- agnostic meta-learning for fast adaptation of deep networks. In International Conference on Machine Learning, 2017. 2
  16. Samir Yitzhak Gadre, Gabriel Ilharco, Alex Fang, Jonathan Hayase, Georgios Smyrnis, Thao Nguyen, Ryan Marten, Mitchell Wortsman, Dhruba Ghosh, Jieyu Zhang, et al. Dat- aComp: In search of the next generation of multimodal datasets. arXiv preprint arXiv:2304.14108, 2023. 1, 2, 3, 4, 13, 14, 16 · arXiv
  17. Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jen- nifer Wortman Vaughan, Hanna Wallach, Hal Daumé Iii, and Kate Crawford. Datasheets for datasets. Communications of the ACM, 64(12):86-92, 2021. 8
  18. Danna Gurari, Qing Li, Abigale J Stangl, Anhong Guo, Chi Lin, Kristen Grauman, Jiebo Luo, and Jeffrey P Bigham. VizWiz Grand Challenge: Answering visual questions from blind people. In IEEE/CVR conference on computer vision and pattern recognition, 2018. 3
  19. Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation met- ric for image captioning. arXiv preprint arXiv:2104.08718, 2021. 1, 3, 7, 20 · arXiv
  20. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen- Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRa: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021. 8 · arXiv
  21. Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representa- tion learning with noisy text supervision. In International conference on machine learning, 2021. 2, 3, 13
  22. Hernisa Kacorri, Kris M Kitani, Jeffrey P Bigham, and Chieko Asakawa. People with visual impairment training personal object recognizers: Feasibility and challenges. In CHI Conference on Human Factors in Computing Systems, 2017. 4, 5, 8
  23. Alina Kuznetsova, Hassan Rom, Neil Alldrin, Jasper Ui- jlings, Ivan Krasin, Jordi Pont-Tuset, Shahab Kamali, Stefan Popov, Matteo Malloci, Alexander Kolesnikov, et al. The open images dataset v4: Unified image classification, object detection, and visual relationship detection at scale. Interna- tional Journal of Computer Vision, 128(7), 2020. 3, 12
  24. Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13. Springer, 2014. 3, 4, 12
  25. Zhiqiu Lin, Jia Shi, Deepak Pathak, and Deva Ramanan. The clear benchmark: Continual learning on real-world im- agery. In Neural Information Processing Systems Datasets and Benchmarks Track, 2021. 1
  26. Ziyi Lin, Shijie Geng, Renrui Zhang, Peng Gao, Gerard de Melo, Xiaogang Wang, Jifeng Dai, Yu Qiao, and Hongsheng Li. Frozen clip models are efficient video learners. In Euro- pean Conference on Computer Vision, 2022. 1, 2
  27. Alexandra Sasha Luccioni, Christopher Akiki, Margaret Mitchell, and Yacine Jernite. Stable Bias: Analyzing so- cietal representations in diffusion models. arXiv preprint arXiv:2303.11408, 2023. 1, 2 · arXiv
  28. Timo Lüddecke and Alexander Ecker. Image segmentation using text and image prompts. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022. 1, 2, 7, 25
  29. Daniela Massiceti, Luisa Zintgraf, John Bronskill, Lida Theodorou, Matthew Tobias Harris, Edward Cutrell, Cecily Morrison, Katja Hofmann, and Simone Stumpf. Orbit: A real-world few-shot dataset for teachable object recognition. In IEEE/CVF International Conference on Computer Vision, 2021. 3, 4, 8
  30. Sachit Menon and Carl Vondrick. Visual classification via description from large language models. arXiv preprint arXiv:2210.07183, 2022. 5 · arXiv
  31. George A Miller. WordNet: a lexical database for English. Communications of the ACM, 38(11):39-41, 1995. 4, 14
  32. Matthias Minderer, Alexey Gritsenko, Austin Stone, Maxim Neumann, Dirk Weissenborn, Alexey Dosovitskiy, Aravindh Mahendran, Anurag Arnab, Mostafa Dehghani, Zhuoran Shen, et al. Simple open-vocabulary object detection. In European Conference on Computer Vision, 2022. 1, 2, 7, 8
  33. Matthias Minderer, Alexey Gritsenko, and Neil Houlsby. Scaling open-vocabulary object detection. arXiv preprint arXiv:2306.09683, 2023. 1, 7 · arXiv
  34. Ron Mokady, Amir Hertz, and Amit H Bermano. CLIP- Cap: CLIP prefix for image captioning. arXiv preprint arXiv:2111.09734, 2021. 2 · arXiv
  35. Cecily Morrison, Rita Marques, Martin Grayson, Daniela Massiceti, Camilla Longden, Linda Yilin Wen, and Edward Cutrell. Understanding Personalized Accessibility through Teachable AI: Designing and Evaluating Find My Things for People who are Blind or Low Vision. In International ACM SIGACCESS Conference on Computers and Accessi- bility, 2023. 1, 3, 4, 6
  36. Ranjita Naik and Besmira Nushi. Social biases through the text-to-image generation lens. arXiv preprint arXiv:2304.06034, 2023. 1, 2 · arXiv
  37. Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. GLIDE: Towards photorealistic image gener- ation and editing with text-guided diffusion models. arXiv preprint arXiv:2112.10741, 2021. 2 · arXiv
  38. Besmira Nushi, Ece Kamar, and Eric Horvitz. Towards Ac- countable AI: Hybrid human-machine analyses for charac- terizing system failure. In AAAI Conference on Human Com- putation and Crowdsourcing, 2018. 2
  39. Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Repre- sentation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748, 2018. 2 · arXiv
  40. Dong Huk Park, Samaneh Azadi, Xihui Liu, Trevor Darrell, and Anna Rohrbach. Benchmark for compositional text-to- image synthesis. In Neural Information Processing Systems Datasets and Benchmarks Track, 2021. 1
  41. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. In International Conference on Machine Learning. PMLR, 2021. 1, 2, 3, 13
  42. Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image gener- ation with CLIP latents. arXiv preprint arXiv:2204.06125, 2022. 2, 7, 8 · arXiv
  43. Tal Ridnik, Emanuel Ben-Baruch, Asaf Noy, and Lihi Zelnik-Manor. Imagenet-21k pretraining for the masses. arXiv preprint arXiv:2104.10972, 2021. 4 · arXiv
  44. Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image syn- thesis with latent diffusion models. In IEEE/CVF conference on computer vision and pattern recognition, 2022. 1
  45. Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, San- jeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Chal- lenge. International Journal of Computer Vision, 115(3): 211-252, 2015. 4
  46. Christoph Schuhmann, Richard Vencu, Romain Beaumont, Robert Kaczmarczyk, Clayton Mullis, Aarush Katta, Theo Coombes, Jenia Jitsev, and Aran Komatsuzaki. LAION- 400M: Open dataset of CLIP-filtered 400 million image-text pairs. arXiv preprint arXiv:2111.02114, 2021. 1, 2, 3, 4, 13, 14, 16 · arXiv
  47. Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Worts- man, et al. LAION-5B: An open large-scale dataset for train- ing next generation image-text models. Advances in Neural Information Processing Systems Datasets and Benchmarks Track, 2022. 1, 2, 3, 4, 13, 14, 16
  48. Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. Advances in neural informa- tion processing systems, 2017. 5, 14, 19
  49. Haoyu Song, Li Dong, Wei-Nan Zhang, Ting Liu, and Furu Wei. CLIP models are few-shot learners: Empiri- cal studies on VQA and visual entailment. arXiv preprint arXiv:2203.07190, 2022. 5 · arXiv
  50. Yixuan Su, Tian Lan, Yahui Liu, Fangyu Liu, Dani Yo- gatama, Yan Wang, Lingpeng Kong, and Nigel Collier. Lan- guage models can see: Plugging visual controls in text gen- eration. arXiv preprint arXiv:2205.02655, 2022. 1, 2 · arXiv
  51. Mingkang Tang, Zhanyu Wang, Zhenhua Liu, Fengyun Rao, Dian Li, and Xiu Li. CLIP4caption: CLIP for video caption. In ACM International Conference on Multimedia, 2021.
  52. Yoad Tewel, Yoav Shalev, Idan Schwartz, and Lior Wolf. Zerocap: Zero-shot image-to-text generation for visual- semantic arithmetic. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022. 1, 2
  53. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems, 2017. 3
  54. Mingyang Wan, Daochen Zha, Ninghao Liu, and Na Zou. Modeling techniques for machine learning fairness: A sur- vey. arXiv preprint arXiv:2111.03015, 2021. 2 · arXiv
  55. Angelina Wang, Solon Barocas, Kristen Laird, and Hanna Wallach. Measuring representational harms in image cap- tioning. In ACM Conference on Fairness, Accountability, and Transparency, 2022. 1, 2
  56. Zhaoqing Wang, Yu Lu, Qiang Li, Xunqiang Tao, Yandong Guo, Mingming Gong, and Tongliang Liu. CRIS: CLIP- driven referring image segmentation. In IEEE/CVF confer- ence on computer vision and pattern recognition, 2022. 2
  57. Lu Yuan, Dongdong Chen, Yi-Ling Chen, Noel Codella, Xiyang Dai, Jianfeng Gao, Houdong Hu, Xuedong Huang, Boxin Li, Chunyuan Li, et al. Florence: A new foundation model for computer vision. arXiv preprint arXiv:2111.11432, 2021. 2 · arXiv

Cited by

Sorted by citations

Authors
Massiceti, D., Longden, C., Słowik, A., Wills, S., Grayson, M., & Morrison, C.
Affiliations
  • Microsoft (United States) (US)
  • World Bank (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2023
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2311.17315
arXiv
2311.17315
OpenAlex
W4389217382
Cited by
1
References
0
Citations by year
2024: 1
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
  • Health Sciences › Health Professions › Occupational Therapy › Assistive Technology in Communication and Mobility
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Keywords
CLIP, blind and visually impaired users, assistive technology, multimodal models, zero-shot classification, image quality robustness, illumination variation, pre-training dataset, few-shot learning, quality of service

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review