Rich Representations of Visual Content for Screen Reader Users
The DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Abstract
Alt text (short for "alternative text") is descriptive text associated with an image in HTML and other document formats. Screen reader technologies speak the alt text aloud to people who are visually impaired. Introduced with HTML 2.0 in 1995, the alt attribute has not evolved despite significant changes in technology over the past two decades. In light of the expanding volume, purpose, and importance of digital imagery, we reflect on how alt text could be supplemented to offer a richer experience of visual content to screen reader users. Our contributions include articulating the design space of representations of visual content for screen reader users, prototypes illustrating several points within this design space, and evaluations of several of these new image representations with people who are blind. We close by discussing the implications of our taxonomy, prototypes, and user study findings.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“BVI people most often rely on labels and alternative text (alt text) on web images and charts [24,84]”
From The Accessibility of Data Visualizations on the Web for Screen Reader Users: Practices and Experiences During COVID-19 (2023) · cites this paper as [84]
In the citing paper
“Moreover, BVI users are likely to encounter help-seeking situations related to accessing the content of visual documents (e.g., images and graphs). These situations occur mainly due to a lack of alternative (ALT) text or descriptions or inconsistent ALT text for visual documents [7,16,22,23,24,25].”
From Help-Seeking Situations Related to Visual Interactions on Mobile Platforms and Recommended Designs for Blind and Visually Impaired Users (2024) · cites this paper as [24]
Citations
Cites
From captions to visual concepts and back
Fang, H., Gupta, S., Iandola, F., Srivastava, R. K., Deng, L., Dollar, P., Gao, J., He, X., Mitchell, M., Platt, J. C., et al. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298754 · Cited by 1,287
VizWiz: nearly real-time answers to visual questions
Bigham, J. P., Jayant, C., Ji, H., Little, G., Miller, A., Miller, R. C., Miller, R., Tatarowicz, A., White, B., White, S., et al. (2010) · Proceedings of the 23nd annual ACM symposium on User interface software and technology · doi:10.1145/1866029.1866080 · Cited by 601 · cited 4×
“In this paper, we consider how modern computing capabilities such as interactivity, high-fidelity audio capabilities, touch interaction, and real-time crowdsourcing [4] and friendsourcing [7] can provide a new experience of visual content for screen reader users.”
BackgroundMethodInfluentialVizWiz: nearly real-time answers to visual questions
Bigham, J. P., Jayant, C., Ji, H., Little, G., Miller, A., Miller, R. C., Tatarowicz, A., White, B., White, S., & Yeh, T. (2010) · Proceedings of the 2010 International Cross Disciplinary Conference on Web Accessibility (W4A) · doi:10.1145/1805986.1806020 · Cited by 536
Guidelines are only half of the story: accessibility problems encountered by blind users on the web
Power, C., Freire, A., Petrie, H., & Swallow, D. (2012) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2207676.2207736 · Cited by 327 · cited 2×
“Currently, missing or low-quality alt text is a pervasive problem [5, 13, 17, 18, 21, 22, 25, 29, 34], and it is unclear how our prototype supplements to alt text might impact compliance with alt text guidelines.”
Background(2017) · doi:10.1145/2998181.2998364 · Cited by 258 · cited 1×
“, “this image may contain a dog, trees, the sun”) [41].”
BackgroundThe design space of input devices
Card, S. K., Mackinlay, J. D., & Robertson, G. G. (1990) · Proceedings of the SIGCHI conference on Human factors in computing systems Empowering people - CHI '90 · doi:10.1145/97243.97263 · Cited by 238 · cited 2×
“goal in proposing the taxonomy as a foundation upon which others might expand, is inspired by prior work such as “The Design Space of Input Devices,” [8] in which Card et al.”
BackgroundUnderstanding Blind People's Experiences with Computer-Generated Captions of Social Media Images
MacLeod, H., Bennett, C. L., Morris, M. R., & Cutrell, E. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025814 · Cited by 200 · cited 1×
“However, computergenerated captions are often inaccurate, and users who are blind tend to place more trust in such systems than is warranted by the state of the art [19].”
BackgroundSelf-citationHow Blind People Interact with Visual Content on Social Networking Services
Voykinska, V., Azenkot, S., Wu, S., & Leshed, G. (2016) · Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing · doi:10.1145/2818048.2820013 · Cited by 174 · cited 2×
“Currently, missing or low-quality alt text is a pervasive problem [5, 13, 17, 18, 21, 22, 25, 29, 34], and it is unclear how our prototype supplements to alt text might impact compliance with alt text guidelines.”
Background(2003) · Network Tutorial · doi:10.1201/9781482280876-57 · Cited by 172
WebInSight: making web images accessible
Bigham, J. P., Kaminsky, R. S., Ladner, R. E., Danielsson, O. M., & Hempton, G. L. (2006) · Proceedings of the 8th international ACM SIGACCESS conference on Computers and accessibility · doi:10.1145/1168987.1169018 · Cited by 161 · cited 2×
“Currently, missing or low-quality alt text is a pervasive problem [5, 13, 17, 18, 21, 22, 25, 29, 34], and it is unclear how our prototype supplements to alt text might impact compliance with alt text guidelines.”
BackgroundImproving accessibility of the web with a computer game
von Ahn, L., Ginosar, S., Kedia, M., Liu, R., & Blum, M. (2006) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1124772.1124785 · Cited by 156 · cited 1×
“Researchers have explored the applicability of human computation techniques to produce near-real-time image labels via either crowdsourcing [4, 33, 42], friendsourcing [6], or social microvolunteering [7].”
BackgroundOlalere, A., & Lazar, J. (2011) · Government Information Quarterly · doi:10.1016/j.giq.2011.02.002 · Cited by 148 · cited 2×
“Currently, missing or low-quality alt text is a pervasive problem [5, 13, 17, 18, 21, 22, 25, 29, 34], and it is unclear how our prototype supplements to alt text might impact compliance with alt text guidelines.”
BackgroundInvestigating the appropriateness of social network question asking as a resource for blind users
Brady, E. L., Zhong, Y., Morris, M. R., & Bigham, J. P. (2013) · Proceedings of the 2013 conference on Computer supported cooperative work · doi:10.1145/2441776.2441915 · Cited by 147 · cited 3×
“Alternatively, questions could be sent to and answered by the crowd [4] or friends [6, 7].”
BackgroundMethodSelf-citationMaximum accessibility : making your Web site more usable for everyone
Slatin, J. M., & Rush, S. (2002) · International Linear Collider · Cited by 133
Morris, M. R., Zolyomi, A., Yao, C., Bahram, S., Bigham, J. P., & Kane, S. K. (2016) · Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858116 · Cited by 131 · cited 3×
“Currently, missing or low-quality alt text is a pervasive problem [5, 13, 17, 18, 21, 22, 25, 29, 34], and it is unclear how our prototype supplements to alt text might impact compliance with alt text guidelines.”
BackgroundSelf-citationRich Image Captioning in the Wild
Tran, K., He, X., Zhang, L., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2016.61 · Cited by 103
Salisbury, E., Kamar, E. S., & Morris, M. R. (2017) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v5i1.13301 · Cited by 86 · cited 4×
“Further, automated captions do not yet include the amount and types of detail desired by many screen reader users [27].”
BackgroundInfluentialSelf-citationRegionSpeak: Quick Comprehensive Spatial Descriptions of Complex Images for Blind Users
Zhong, Y., Lasecki, W. S., Brady, E. L., & Bigham, J. P. (2015) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702437 · Cited by 83
The state of corporate website accessibility
Loiacono, E. T., Romano, N., & McCoy, S. (2009) · Communications of the ACM · doi:10.1145/1562164.1562197 · Cited by 77 · cited 2×
“Currently, missing or low-quality alt text is a pervasive problem [5, 13, 17, 18, 21, 22, 25, 29, 34], and it is unclear how our prototype supplements to alt text might impact compliance with alt text guidelines.”
BackgroundGlobal Web Accessibility Analysis of National Government Portals and Ministry Web Sites
Goodwin, M., Susar, D., Nietzio, A., Snaprud, M., & Jensen, C. S. (2011) · Journal of Information Technology & Politics · doi:10.1080/19331681.2010.508011 · Cited by 74 · cited 2×
“Currently, missing or low-quality alt text is a pervasive problem [5, 13, 17, 18, 21, 22, 25, 29, 34], and it is unclear how our prototype supplements to alt text might impact compliance with alt text guidelines.”
BackgroundGauging Receptiveness to Social Microvolunteering
Brady, E. L., Morris, M. R., & Bigham, J. P. (2015) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702329 · Cited by 69 · cited 3×
“In this paper, we consider how modern computing capabilities such as interactivity, high-fidelity audio capabilities, touch interaction, and real-time crowdsourcing [4] and friendsourcing [7] can provide a new experience of visual content for screen reader users.”
BackgroundSelf-citationGuiding Novice Web Workers in Making Image Descriptions Using Templates
Morash, V. S., Siu, Y.-T., Miele, J. A., Hasty, L., & Landau, S. (2015) · ACM Transactions on Accessible Computing · doi:10.1145/2764916 · Cited by 69 · cited 3×
“For instance, highly structured alt text (exemplified by our categories prototype) might benefit human authors or AI algorithms (structured templates have been shown to increase alt text quality for STEM diagrams [20]).”
BackgroundMethodInfluentialEyes-Free Art: Exploring Proxemic Audio Interfaces For Blind and Low Vision Art Engagement
Rector, K., Salmon, K., Thornton, D., Joshi, N., & Morris, M. R. (2017) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3130958 · Cited by 66
Self-citationE-Government Web Site Accessibility in Australia and China: A Longitudinal Study
Shi, Y. (2006) · Social Science Computer Review · doi:10.1177/0894439305283707 · Cited by 61 · cited 2×
“Currently, missing or low-quality alt text is a pervasive problem [5, 13, 17, 18, 21, 22, 25, 29, 34], and it is unclear how our prototype supplements to alt text might impact compliance with alt text guidelines.”
BackgroundHarada, S., Sato, D., Adams, D., Kurniawan, S., Takagi, H., & Asakawa, C. (2013) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2481292 · Cited by 50 · cited 1×
“Such sounds might be curated by the author from personal libraries or publicly available sources, suggested (or even composed) by future AI algorithms based on visual or metadata properties of the image, or even captured in-situ by novel photography tools (prototype photo applications for blind users like VizSnap [1]…”
BackgroundAudiophotography: practice and prospects
Frohlich, D. M., & Tallyn, E. (1999) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/632716.632897 · Cited by 40
Blind Photographers and VizSnap: A Long-Term Study
Adams, D., Kurniawan, S., Herrera, C., Kang, V. Y., & Friedman, N. (2016) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/2982142.2982169 · Cited by 30 · cited 2×
“Such sounds might be curated by the author from personal libraries or publicly available sources, suggested (or even composed) by future AI algorithms based on visual or metadata properties of the image, or even captured in-situ by novel photography tools (prototype photo applications for blind users like VizSnap [1]…”
BackgroundMethodAccessibility for the blind: An automated audio/tactile description of pictures in digital documents
Issa, Y. B., Mojahid, M., Oriola, B., & Vigouroux, N. (2009) · International Conference on Advances in Computational Tools for Engineering Applications · doi:10.1109/actea.2009.5227855 · Cited by 10
ABA resolution and report on website accessibility.
(2007) · PubMed · Cited by 3
Research works printed but not linked: 5
- Edwards, J. "Planet Selfie: We're Now Posting a Staggering 1.8 Billion Photos Every Day." Business Insider, May 28, 2014.
- Korpela, J. (September 1998). Guidelines on ALT texts in IMG elements. Retrieved 26 August, 2017 from http://www.cs.tut.fi/~jkorpela/html/alt.html.
- Petrie, H., Harrison, C., and Dev, S. Describing Images on the Web: A Survey of current Practice and Prospects for the Future. Proceedings of HCI International 2005.
- 10.5555/560695 · doi:10.5555/560695
- Stolley, K. Image Accessibility Part I: Beyond alt Attributes. Digital Rhetoric Collaborative. June 2016. http://www.digitalrhetoriccollaborative.org/2016/06/15 /image-accessibility-part-i-beyond-alt-attributes/
Cited by
Captioning Images Taken by People Who Are Blind
Gurari, D., Zhao, Y., Zhang, M., & Bhattacharya, N. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58520-4_25 · Cited by 175
Accessible Visualization via Natural Language Descriptions: A Four-Level Model of Semantic Content
Lundgard, A., & Satyanarayan, A. (2022) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2021.3114770 · Cited by 155
Stangl, A., Morris, M. R., & Gurari, D. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376404 · Cited by 139
“The absence of alt text from content authors has motivated scholars and practitioners to innovate, by introducing a variety of more scalable image description services that are powered by humans [4, 5, 7, 6, 45], computers [14, 24, 35, 37, 38, 43], and a mixture of their efforts [17, 28, 32, 33].”
BackgroundInfluentialSelf-citationAccessible Visualization: Design Space, Opportunities, and Challenges
Kim, N. W., Joyner, S. C., Riegelhuth, A., & Kim, Y. (2021) · Computer Graphics Forum · doi:10.1111/cgf.14298 · Cited by 137
Bennett, C. L., Gleason, C., Scheuerman, M. K., Bigham, J. P., Guo, A., & To, A. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445498 · Cited by 128
Self-citation
Metadata
- Authors
- Morris, M. R., Johnson, J., Bennett, C. L., & Cutrell, E.
- Affiliations
- Microsoft (United States) (US)
- Vanderbilt University (US)
- Venue
- Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2018-04-19
- Pages
- 1-11
- OpenAlex
- W2789636240
- Cited by
- 159
- References
- 29
- Citations by year
- 2026: 18 · 2025: 17 · 2024: 25 · 2023: 25 · 2022: 28 · 2021: 25 · 2020: 12 · 2019: 7 · 2018: 2
- FWCI
- 19.85
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Topics
- Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
- Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
- Health Sciences › Health Professions › Occupational Therapy › Assistive Technology in Communication and Mobility
- Keywords
- alt text, screen readers, blind users, visually impaired users, image accessibility, HTML, accessible web design, picture description, design space
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.