Toward Automatic Audio Description Generation for Accessible Videos
The DOI matches its Crossref record; 53 of the 54 references it cites were found in a registry or an index.
Abstract
Video accessibility is essential for people with visual impairments. Audio descriptions describe what is happening on-screen, e.g., physical actions, facial expressions, and scene changes. Generating high-quality audio descriptions requires a lot of manual description generation [50]. To address this accessibility obstacle, we built a system that analyzes the audiovisual contents of a video and generates the audio descriptions. The system consisted of three modules: AD insertion time prediction, AD generation, and AD optimization. We evaluated the quality of our system on five types of videos by conducting qualitative studies with 20 sighted users and 12 users who were blind or visually impaired. Our findings revealed how audio description preferences varied with user types and video types. Based on our study’s analysis, we provided recommendations for the development of future audio description generation technologies.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · cited 1×
“Starting from word embeddings [7, 32, 59], recent popular neural network methods evolved to use sentence-level embedding for distance metrics [5, 30, 36].”
MethodLearning Spatiotemporal Features with 3D Convolutional Networks
Tran, D., Bourdev, L., Fergus, R., Torresani, L., & Paluri, M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.510 · Cited by 9,593 · cited 1×
“Research on dense video captioning, such as C3D [53] and I3D [11], showed that multiple features can improve video captioning models [37, 38, 54].”
BackgroundQuo Vadis, Action Recognition? A New Model and the Kinetics Dataset
Carreira, J., & Zisserman, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.502 · Cited by 9,570
Transformers: State-of-the-Art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2020.emnlp-demos.6 · Cited by 8,741
A Neural Probabilistic Language Model
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C., Jaz, K., Hofmann, T., Poggio, T., & Shawe-Taylor, J. S. (2000) · Cited by 3,486
HuggingFace's Transformers: State-of-the-art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.03771 · Cited by 3,347
A neural probabilistic language model
BengioYoshua, DucharmeRéjean, VincentPascal, & JanvinChristian (2003) · Journal of Machine Learning Research · doi:10.5555/944919.944966 · Cited by 2,644
Sequence to Sequence -- Video to Text
Venugopalan, S., Rohrbach, M., Donahue, J., Mooney, R., Darrell, T., & Saenko, K. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.515 · Cited by 1,352 · cited 1×
“Research on dense video captioning, such as C3D [53] and I3D [11], showed that multiple features can improve video captioning models [37, 38, 54].”
BackgroundDense-Captioning Events in Videos
Krishna, R., Hata, K., Ren, F., Fei-Fei, L., & Niebles, J. C. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.83 · Cited by 1,251
Attention on Attention for Image Captioning
Huang, L., Wang, W., Chen, J., & Wei, X.-Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00473 · Cited by 1,092 · cited 1×
“Similar to image captioning, it requires not only recognizing salient objects, but also understanding actions and interactions [26].”
Background
Research works printed but not linked: 7
- Yusuf Aytar, Carl Vondrick, and Antonio Torralba. 2016. Soundnet: Learning sound representations from unlabeled video. In Advances in neural information processing systems. 892–900.
- Pat Fletcher. 2002. Seeing with sound: A journey into sight. Retrieved September 21(2002), 2015.
- 10.4324/9781315707228 · doi:10.4324/9781315707228
- R Kingett. 2014. The Accessible Netflix Project Advocates Taking Steps to Ensure Netflix Accessibility for Everyone. The Accessible Netflix Project 26 (2014).
- Aline Remael and Gert Vercauteren. 2011. Basisprincipes voor audiobeschrijving voor televisie en film [Basics of audio description for television and film]. Antwerp: Departement Vertalers and Tolken, Artesis Hogeschool (2011).
- Emilie Schmeidler and Corinne Kirchner. 2001. Adding audio description: Does it make a difference?Journal of Visual Impairment & Blindness 95, 4 (2001), 197–212.
- Joel Snyder. 2014. The visual made verbal: A comprehensive training manual and guide to the history and applications of audio description. American Council of the Blind, Incorporated.
Cited by
Mukhiddinov, M., Abdusalomov, A. B., & Cho, J. (2022) · Sensors · doi:10.3390/s22093307 · Cited by 102
“Assistive systems aid BVI people with daily activities such as distinguishing banknotes [2], crossing a road [3,4], video media accessibility [5,6], image sonification for navigation [7], recognizing people [8], recognizing private visual information [9], selecting clothing [10], and navigating both outdoors and indoo…”
BackgroundA Literature Review of Video-Sharing Platform Research in HCI
Bartolome, A., & Niu, S. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581107 · Cited by 83
“video navigation and recommendation [14, 23, 24, 84, 187, 191, 207], live streaming support [38, 55, 107, 108, 127, 152, 179, 211], closed captions [69], user protection techniques [74, 118, 129] accessibility features [125, 193]”
BackgroundInfluentialAVscript: Accessible Video Editing with Audio-Visual Scripts
Huh, M., Yang, S., Peng, Y.-H., Chen, X. '., Kim, Y.-H., & Pavel, A. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581494 · Cited by 45
“Beyond helping creators author accessible videos, prior work makes inaccessible videos accessible on demand by generating automatic [83] or interactive [29, 59] visual descriptions.”
AutoAD: Movie Description in Context
Han, T., Bain, M., Nagrani, A., Varol, G., Xie, W., & Zisserman, A. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01815 · Cited by 43
“This is a relatively unexplored task in the vision community with previous work targeting ActivityNet videos [91], a very different domain to long-term feature films with storylines, and the LSMDC challenge [71], where the descriptions and character names are treated separately.”
Making Short-Form Videos Accessible with Hierarchical Video Summaries
Van Daele, T., Iyer, A., Zhang, Y., Derry, J. C., Huh, M., & Pavel, A. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642839 · Cited by 38
Self-citationRong, E. Z., Zhou, M. M., Lu, Z., & Fan, M. (2022) · Designing Interactive Systems Conference · doi:10.1145/3532106.3533514 · Cited by 41
“Although previous works introduced algorithms to generate video descriptions [40, 46, 77], such algorithms were designed or trained to recognize videos taken by sighted people.”
MethodAutoAD II: The Sequel – Who, When, and What in Movie Audio Description
Han, T., Bain, M., Nagrani, A., Varol, G., Xie, W., & Zisserman, A. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01255 · Cited by 74
“Initial work [58] applies heuristic cost-based filtering to video captioning on ActivityNet to generate diverse and relevant captions more akin to AD.”
MethodNing, Z., Wimer, B. L., Jiang, K., Chen, K., Ban, J., Tian, Y., Zhao, Y., & Li, T. J.-J. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642632 · Cited by 49
“Recent advancements in deep learning and computer vision technologies have paved the way for automated approaches in AD generation [1, 52].”
BackgroundJiang, L., Phutane, M., & Azenkot, S. (2023) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3597638.3608381 · Cited by 46
“Increase AD Quality and Quantity Researchers have studied how AI can make human processes for creating AD more efcient by automating parts of the workfow [12, 16, 17, 32, 49, 66, 86, 87].”
BackgroundMethodThe Efficacy of Collaborative Authoring of Video Scene Descriptions
Natalie, R., Loh, J., Tan, H. S., Tseng, J., Chan, I. L. Y.-R., Jarjue, E., Kacorri, H., & Hara, K. (2021) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3441852.3471201 · Cited by 36
“The number of approvals changed from (49, 43, 46) in session 1 to (52, 49, 56) in session 2 .”
BackgroundMethodInfluentialJiang, L., Jung, C., Phutane, M., Stangl, A. J., & Azenkot, S. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642238 · Cited by 59
“…organizations serving BLV people [3, 26–28, 117 ], and industry practitioners [ 34, 73, Additional insights about which details, and levels of detail, that BLV users want in AD have emerged through the development of tools to streamline the AD creation process [11, 61, 66, 93, 126, 128, 134].”
BackgroundInfluentialSlidecho: Flexible Non-Visual Exploration of Presentation Videos
Peng, Y.-H., Bigham, J. P., & Pavel, A. (2021) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3441852.3471234 · Cited by 40
“tecting key visual content [21, 22], generating descriptions of the visual content [22, 56, 70, 73], and editing descriptions to ft within”
BackgroundCrossA11y: Identifying Video Accessibility Issues via Cross-modal Grounding
Liu, X., Wang, R., Li, D., Chen, X., & Pavel, A. (2022) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3526113.3545703 · Cited by 33
“We also predicted accessibility issues using two baselines for comparison: (1) mark each segment as inaccessible with 50% chance (Random) , and (2) mark each segment as inaccessible if it did not include speech (Gaps) following prior work [27, 31, 46, 48].”
BackgroundMethodInfluentialSelf-citationAutoAD III: The Prequel - Back to the Pixels
Han, T., Bain, M., Nagrani, A., Varol, G., Xie, W., & Zisserman, A. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.01720 · Cited by 60
“With the current power of visual-to-text generative models, generating AD automatically is now becoming possible [66], and there has been a recent flurry of interest in this goal [20, 21], kick-started by the availability of films and AD provided in the MAD dataset [54].”
MethodAudio Description Customization
Natalie, R., Chang, R.-C., Sheshadri, S., Guo, A., & Hara, K. (2024) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3663548.3675617 · Cited by 38
“Tone [7, 14, 80] Tone of AD, which could be monotonous or dynamic.”
InfluentialSupporting Novices Author Audio Descriptions via Automatic Feedback
Natalie, R., Tseng, J., Kacorri, H., & Hara, K. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581023 · Cited by 26
“built a tool to automatically generate AD by combining computer vision and natural language processing techniques [54].”
BackgroundMethodInfluentialDang, K., Korreshi, H., Iqbal, Y., & Lee, S. (2023) · ACM Symposium on Spatial User Interaction (SUI) · doi:10.1145/3607822.3614540 · Cited by 25
“The strategies to enhance visual content accessibility extend further into video accessibility, focusing on methods like automatic AD generation systems [99, 148], web-based authoring tools exemplified by ViScene [100], and collaborative video description techniques using human-in-the-loop machine learning [158].”
BackgroundThe Potential of a Visual Dialogue Agent In a Tandem Automated Audio Description System for Videos
Stangl, A. J., Ihorn, S. M., Siu, Y.-T., Bodi, A., Castanon, M., Narins, L. D., & Yoon, I. (2023) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3597638.3608402 · Cited by 31
“Unlike other recent work (e.g., [77]), we also employ both inline and extended descriptions and verify/revise AI-generated captions using a human-in-the-loop system.”
BackgroundInfluentialJain, G., Hindi, B., Courtien, C., Xu, X. Y. T., Wyrick, C., Malcolm, M., & Smith, B. A. (2023) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3586183.3606830 · Cited by 25
“Many researchers have studied ways to create efective descriptions for BLV people, by proposing methods and guidelines for authoring descriptions [3, 6, 19, 42, 45, 72, 78] as well as by introducing tools to support and automate the process [9, 20, 63, 77, 78, 82].”
BackgroundNevsky, A., Neate, T., Simperl, E. P. B., & Vatavu, R.-D. (2023) · ACM International Conference on Interactive Media Experiences · doi:10.1145/3573381.3596159 · Cited by 24
“9%), such as AI and ML systems to automate existing interventions [219, 231].”
BackgroundMethodSilva, M. T. D., Goodwin, S., Holloway, L. M., & Butler, M. (2023) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3597638.3608409 · Cited by 23
“However, the needs raised from this study are focused towards the awareness of the movement of other people in a space and in a fast-moving pace.”
BackgroundLozano, J. D. L. R., & Mejías-Climent, L. (2023) · Linguistica Antverpiensia New Series – Themes in Translation Studies · doi:10.52034/lans-tts.v22i.790 · Cited by 21
Sattari, S., & Yazıcı, A. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-19312-7 · Cited by 16
Xu, S., Jin, X., Qu, H., & Yan, Y. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713496 · Cited by 24
“• Table 5 outlines the uncompleted trials in the controlled comparison.”
InfluentialDescribe Now: User-Driven Audio Description for Blind and Low Vision Individuals
Cheema, M. S., Seifi, H., & Fazli, P. (2025) · Designing Interactive Systems Conference · doi:10.1145/3715336.3735685 · Cited by 34
“Red crosses and the words "Stop wearing your top like this" emphasize her message.”
Vid2Coach: Transforming How-To Videos into Task Assistants
Huh, M., Xue, Z., Das, U., Ashutosh, K., Grauman, K. L., & Pavel, A. (2025) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3746059.3747612 · Cited by 35
“To support task assistance through a more common consumer device, Arakawa et al. developed smart watch systems that deliver context-aware support during physical tasks [6, 7].”
A Recipe for Success? Exploring Strategies for Improving Non-Visual Access to Cooking Instructions
Li, F. M., Wang, A., Carrington, P., & Kane, S. K. (2024) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3663548.3675662 · Cited by 18
Lights, Camera, Access: A Closeup on Audiovisual Media Accessibility and Aphasia
Nevsky, A., Neate, T., Simperl, E. P. B., & Cruice, M. N. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641893 · Cited by 17
Dang, K., Burke, G., Korreshi, H., & Lee, S. (2024) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3663548.3675618 · Cited by 16
Ji, Z., An, P., & Zhao, J. (2025) · Designing Interactive Systems Conference · doi:10.1145/3715336.3735762 · Cited by 14
PEANUT: A Human-AI Collaborative Tool for Annotating Audio-Visual Data
Zhang, Z., Ning, Z., Xu, C., Tian, Y., & Li, T. J.-J. (2023) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3586183.3606776 · Cited by 19
“…learning problems and applications, such as representation learning [7, 10, 85], audio-visual sound separation [31, 36, 85, 125], sounding object localization [49, 85, 98, 106], audio-visual event localization [67, 106, 118], audio-visual captioning [89, 103], and multimodal embodied AI [20, 35].”
BackgroundImproving the Accessibility of Screen-Shared Presentations by Enabling Concurrent Exploration
Fan, D., Junuzovic, S., Tang, J. C., & Jaeger, T. F. (2023) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3597638.3608411 · Cited by 9
MIMOSA: Human-AI Co-Creation of Computational Spatial Audio Effects on Videos
Ning, Z., Zhang, Z., Ban, J., Jiang, K., Gan, R., Tian, Y., & Li, T. J.-J. (2024) · Creativity and Cognition · doi:10.1145/3635636.3656189 · Cited by 18
Dang, K., & Lee, S. (2024) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3663548.3688492 · Cited by 12
“AI-driven improvements in AD generation [29, 44] and web-based authoring tools such as ViScene [30] and collaborative description techniques [45] have enhanced content accessibility.”
Sattari, S., Kalkan, S., & Yazıcı, A. (2025) · Fuzzy Sets and Systems · doi:10.1016/j.fss.2025.109419 · Cited by 10
Uncovering Patterns in Reviewers’ Feedback to Scene Description Authors
Natalie, R., Loh, J., Tan, H. S., Tseng, J., Kacorri, H., & Hara, K. (2021) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3441852.3476550 · Cited by 8
“[20, 23, 24].”
Yu, Y., Ryskeldiev, B., Tsutsui, A., Gillingham, M., & Wang, Y. (2025) · Proceedings of the 27th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3663547.3746353 · Cited by 8
“However, there is limited research on the specific challenges that blind or low vision users face throughout their shopping process due to website design and development, especially on new or unfamiliar websites.”
Self-citationGenerative AI and Accessibility Workshop: Surfacing Opportunities and Risks
Glazko, K. S., Huh, M., Johnson, J., Pavel, A., & Mankoff, J. C. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3706734 · Cited by 9
Xu, S., Jin, X., Zhang, W., Qu, H., & Yan, Y. (2025) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3746059.3747791 · Cited by 16
“BLV users have diverse content preferences when consuming 2D videos [29, 62], yet providing such varied information within the limited video time remains challenging [66, 83].”
Cost-effective and Collaborative Methods to Author Video’s Scene Description for Blind People.
Natalie, R. (2022) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3491101.3503814 · Cited by 6
“[33] were the frst to build and evaluate a system to automate AD for blind people fully.”
BackgroundSmartLearn: Visual-Temporal Accessibility for Slide-based e-learning Videos
Sechayk, Y., Shamir, A., & Igarashi, T. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3650883 · Cited by 8
“Prior work explored systems and methods to improve AD content creation [25, 45, 48], static visual filters [38] and gaze-estimated video magnification [3].”
MethodNevsky, A., Birčanin, F., Cruice, M. N., Wilson, S., Simperl, E. P. B., & Neate, T. (2024) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3663548.3675598 · Cited by 6
“Additionally, novel technologies have changed the way these interventions are produced, such as using machine learning tools to facilitate the creation of subtitles [111] and audio descriptions [44, 104].”
Ning, Z., Li, L., Killough, D., Seo, J., Carrington, P., Tian, Y., Zhao, Y., Li, F. M., & Li, T. J.-J. (2025) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3746059.3747650 · Cited by 14
Sechayk, Y., Shamir, A., Pavel, A., & Igarashi, T. (2025) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3663547.3746372 · Cited by 12
“Research in AD focused on improving the creation process [30, 57], automating the creation altogether [10, 27, 35, 39, 80, 84], and supporting AD consumption through spatial audio [22] or user exploration [53, 68].”
Zhu, X., Qiu, Z., Ye, W., Wang, J., & Jiao, Y. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2526569 · Cited by 15
“…of accessibility software that can be used on the computer, such as video description (Campos et al., 2023; Liu et al., 2021; Packer et al., 2015; Wang et al., 2021), image description (Lee et al., 2022a; Shrestha, 2022; Sreedhar et al., 2022; Thapa et al., 2017) game playing (Sanchez et al.,…”
DescribePro: Collaborative Audio Description with Human-AI Interaction
Cheema, M. S., Elahimanesh, S., Martin, S., Fazli, P., & Seifi, H. (2025) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3663547.3746320 · Cited by 13
“We review prior work on existing approaches and practices for AD and collaboration when authoring AD.”
Impacts of Presenting Extra Information in Short Videos via Text and Voice on User Experience
Gao, W., Xiang, W. D., Liu, X., Wang, X., & Sun, L. (2022) · International Conference on Quality of Multimedia Experience (QoMEX) · doi:10.1109/qomex55416.2022.9900918 · Cited by 4
“This element is helpful for users to understand video content [8].”
BackgroundInfluentialSocial media communication management in the media sector
Brigas, J. M. F., Gonçalves, F. B., & Gonçalves, J. (2023) · Revista Venezolana de Gerencia · doi:10.52080/rvgluz.28.e10.24 · Cited by 4
“Social media has allowed short videos to become profoundly important digital resources in our society (Wang et al, 2021).”
BackgroundDang, K., & Lee, S. (2025) · IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct) · doi:10.1109/ismar-adjunct68609.2025.00268 · Cited by 3
“Recent work demonstrates that real-time AD on Meta Quest headsets enhances spatial awareness for BLV users [5], and that BLV audiences’ preferences for description detail vary widely [11].”
Kim, S., Lee, S., Kim, K., & Oh, U. (2024) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3640543.3645154 · Cited by 7
An Approach for Automatic Description of Characters for Blind People
Filho, I. R., Honorato, F., Lucena, J. W., Teixeira, J. P., & Maritan, T. (2021) · Brazilian Symposium on Multimedia and the Web · doi:10.1145/3470482.3479617 · Cited by 3
“Wujia Yang et al. [17] presents a system divided into three models.”
BackgroundImpact of short videos on digital journalism in social media
Brigas, J., Gonçalves, F., Marques, H., & Gonçalves, J. (2023) · doi:10.31876/rcs.v29i.40935 · Cited by 2
Tiwari, A., Bera, S., Saha, S., Bhattacharyya, P., & Ghosh, S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-56063-7_14 · Cited by 2
Netizen A11y: Engaging Internet Users in Making Visual Media Accessible
Wang, R. (2024) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3640544.3645247 · Cited by 2
Professional and novice audio describers: quality assessments and audio interactions
Nakajima, S., & Mitobe, K. (2024) · The Journal of Specialised Translation · doi:10.26034/cm.jostrans.2024.5980 · Cited by 3
Hammad, N., Elavsky, F., Moharana, S., Chen, J., Lee, S., Carrington, P., Moritz, D., Hammer, J., & Harpstead, E. (2024) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3663548.3675665 · Cited by 3
“We know from other work on accessibility that metadata is critical for access, whether it is automatically generated [40] or created by human participants [24].”
Enrichment of Product Presentation Video: Methods and Impacts on User Experience
Gao, W., Xiang, W. D., Liu, X., Huang, Y., & Sun, L. (2022) · International Conference on Quality of Multimedia Experience (QoMEX) · doi:10.1109/qomex55416.2022.9900910 · Cited by 2
Design considerations for photosensitivity warnings in visual media
South, L., Yildirim, C., Pavel, A., & Borkin, M. A. (2024) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3663548.3675643 · Cited by 1
Li, C., Liang, G., Lin, J., Chen, L., He, W., & Yang, J. (2025) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2025.104573 · Cited by 1
MVAuLT: Authoring Interpretive Alt Text for Music Videos
Zhong, V., Chen, A. S., Rajesh, D., Gruender, M., & Ringland, K. E. (2025) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3663547.3759691 · Cited by 1
“For instance, users preferred more background description in music videos than in comedy, and that both sighted and BLV participants wanted deeper interpretations that captured subtle relationships beyond surface-level audiovisual content [13].”
Bennett, C. L., Pavel, A., Bryant, A., Eberly, L. J. M., Shelby, R. M., & Kane, S. K. (2025) · ACM Transactions on Accessible Computing · doi:10.1145/3777475 · Cited by 5
“AI-generated AI-generated feedback on AD was also ranked lower as compared to that from humans [113].”
InfluentialCoSight: Exploring Viewer Contributions to Online Video Accessibility Through Descriptive Commenting
Wang, R., Liu, X., Wang, B., Zhang, W., Liao, Z., Li, Z., Pavel, A., & Chen, X. (2025) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3746059.3747747 · Cited by 4
“To support and expand the accessibility ecosystem alongside professional AD, prior research has introduced methods to broaden participation in description efforts.”
Introduction to the First Workshop on Vision Foundation Models and Generative AI for Accessibility
Tian, Y., Zhao, Y., Froehlich, J. E., Li, C., & Wu, Y. (2025) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw69036.2025.00697 · Cited by 3
“Modern systems [46, 47, 63, 94] integrate segmentation, recognition, captioning, and visual augmentation to provide richer and more flexible assistance.”
Xu, S., Jin, X., Jain, G., Zhang, W., Qu, H., Smith, B. A., & Yan, Y. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798425 · Cited by 6
“In response, researchers have proposed manual [15, 19], semi-automated [6, 20, 25, 28], and fully automated approaches [16, 34–36] for optimizing the content and timing of AD.”
InfluentialMyWay: a 3D and audio-enhanced transportation learning kit for the visually impaired teenagers
Gu, Q., Li, X., Cai, Q., Zhu, K., Dai, C., Zhang, Y., Sun, L., Tao, Y., & Wang, G. (2024) · CCF Transactions on Pervasive Computing and Interaction · doi:10.1007/s42486-024-00163-y · Cited by 3
“…(1) Auditory description of visual content: This involves verbal explanations for textual information (Guerreiro and Gonçalves 2015), images (Wörtwein et al. 2015), graphics (Holloway et al. 2022), photograph (White et al. 2010; Jayant et al. 2011) and videos (Wang et al. 2021; Alonzo et al. 2022).”
InfluentialSelf-citationEnhancing movie experience by speech rate design of audio description
Nakajima, S., Okochi, N., & Mitobe, K. (2024) · Universal Access in the Information Society · doi:10.1007/s10209-024-01178-z · Cited by 2
“Therefore, ADs employing speech synthesis [11, 12, 16, 21, 29, 32] and automatic AD generation technology using machine learning [2, 5, 6, 14, 17, 22, 26, 28, 33] are being researched and developed to reduce production costs.”
Bernareggi, C., Armano, T., Brunetto, E., Capietto, A., Coriasco, S., Baratto, G., Gabrielli, M. L., & Guadalupi, M. (2024) · International Conference on Education Technology and Computer · doi:10.1145/3702163.3702458 · Cited by 1
“Several automated tools to adapt multimedia materials in accessible formats have been investigated: for automatic captioning [28–31], for automatic generation of audio descriptions through recognition of scenes [27, 32] and for automatic transcriptions [31, 33].”
Das, S., Ghosh, S., Tiwari, A., Lyngkhoi, R. E. Z. M., Saha, S., Murad, Z., & Maurya, A. (2025) · IEEE Access · doi:10.1109/access.2025.3551124 · Cited by 1
“2) Video Features: To extract salient visual information from financial discussions, we employ the Katna [42] library for keyframe extraction.”
YouDescribe: Bridging AI Efficiency and Human Insight for Scalable Audio Description
Do, L., Mirani, S., Pitcher-Cooper, C., Nguyen, X. D. A., Bairaboina, A., & Yoon, I. (2025) · Annual Symposium on Human-Computer Interaction for Work (CHIWORK) · doi:10.1145/3707640.3731916 · Cited by 3
“Wang et al. (2021) proposed a system analyzing multimodal video content for automatic AD [19], while ShortScribe, LLM-AD, and MMAD leverage vision-language models like BLIP-2, GPT-4V, and LLaMA2 to produce scene-aware descriptions [7, 18, 20].”
Convolutional Neural Network-based Acoustic Scene Classification using Litis Rouen Dataset
Haider, A., Shah, S. M. A., Irtaza, A., Arshad, S., Ahmad, M. W., & Zafar, S. (2021) · Mohammad Ali Jinnah University International Conference on Computing (MAJICC) · doi:10.1109/majicc53071.2021.9526265
South, L. (2024) · Digital Repository Service (Northeastern University) · doi:10.17760/d20653004
The Current Landscape of Audio Description Research: A Survey
Rajiv, L. G. A. (2023) · International Journal of Science and Research (IJSR) · doi:10.21275/sr23209131618
Audio Description of Videos Using Machine Learning
Roy, R., Manepalli, S., Rajan, S., Mishra, S., & Sampatrao, G. S. (2024) · IEEE International Conference for Convergence in Technology (I2CT) · doi:10.1109/i2ct61223.2024.10544216 · Cited by 1
“In 200 milliseconds, Tiresias polishes descriptions, utilizing FFmpeg to smoothly integrate them into videos, and Google Cloud Text-to-Speech to convert them to audio while minimizing repetition and grammatical errors[1].”
MethodWang, X., Cheng, A., Namkung, K., & Pan, Y. (2024) · Journal of Information Systems Engineering & Management · doi:10.55267/iadt.07.14774
Shen, M., Huang, G., Wu, Y., Song, S., Zhou, S., Li, L., Yu, Z., Wang, W., & Bu, J. (2024) · International Web for All Conference (W4A) · doi:10.1145/3677846.3677855 · Cited by 2
Audio-Visual LLM for Augmenting Accessibility of 360° Video
Wang, Y., Deng, Q., & Liang, W. Y. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3588530 · Cited by 1
“Many works have been conducted to augment such visual media with spatial audio assignment [21], emotional background music synthesis [22], and audio captioning [23].”
Self-citationAutomatic Audio Description: A Training-Free Approach Using Foundation Models
Țapu, R., & Mocanu, B. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-05060-1_15
Automatic Accessible Multimodal Translation of Graphics Using A Refreshable Pin Array
Kim, S.-G., Kim, A., Kim, E., Chung, M., & Yoo, Y. (2025) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3746027.3754586 · Cited by 1
Mapping the Evolving Landscape of Audiovisual Translation and Media Accessibility
Zabrodzka, M. (2025) · Anglica An International Journal of English Studies · doi:10.7311/0860-5734.34.2.01
Balaji, V., Sonowal, G., Ramachandran, V., & Rangasamy, R. (2026) · Universal Access in the Information Society · doi:10.1007/s10209-025-01294-4 · Cited by 1
“Vrindavanam et al. [37] and Wang et al. [39] both focus on the use of machine learning and computer vision technologies to automatically generate audio descriptions for images and videos, respectively.”
Radha, D. S., & Kavitha, M. G. (2025) · BENTHAM SCIENCE PUBLISHERS eBooks · doi:10.2174/9798898810337125010008
InclusiVision: Exploring Deep Learning Techniques for Enhanced Audio Description Generation
Chettri, S., Kerketta, A. C., Banu, P. K. N., & Goyal, A. (2026) · BENTHAM SCIENCE PUBLISHERS eBooks · doi:10.2174/9798898812881126010007
Heuer, H., Goller, S., & Buschek, D. (2026) · i-com · doi:10.1515/icom-2025-0024
“33 36”
MCAD: Multimodal Context-Aware Audio Description Generation for Soccer
Chaudhary, L., Mittal, T., Gopalakrishnan, S., Nwogu, I., & Pytlarz, J. (2025) · International Symposium on Multimedia · doi:10.1109/ism66958.2025.00060 · Cited by 2
“Some prior multimodal models detect characters, actions, and context while capturing subtle cues and emotions [4].”
Amorim, E., Nascimento, T. H., Fonseca, A. U., Ferreira, C., Juárez‐Ramírez, R., Félix, J. P., & Soares, F. A. A. D. M. N. (2026) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac69091.2026.00261
“The adoption of audio description as the primary output is grounded in the reduction of cognitive load [6] and the effectiveness of interactive sonification [19].”
Making Videos Accessible for Blind and Low Vision Users Using a Multimodal Agent Video Player
Sinha, A. K., Olmos, A., Santos, R. D., Rodriguez, R. R., Landay, J. A., Sepah, S., Nelson, P., & Kane, S. K. (2026) · Companion Publication of the International Conference on Multimodal Interaction (ICMI Companion) · doi:10.1145/3776591.3833853
Multimodal Artificial Intelligence for Accessible Picture-Book Narration
-, A. J., & Bhimaraju, G. (2026) · International Symposium on Computer Science and Educational Technology (ISCSET) · doi:10.1109/iscset70652.2026.11722781
“Automatic audio description (AD) describes visual information in video, such as actions, expressions, and scene changes, so blind viewers can follow visual content [8], [9].”
Metadata
- Authors
- Wang, Y., Liang, W., Huang, H., Zhang, Y., Li, D., & Yu, L.-F.
- Affiliations
- Beijing Institute of Technology (CN)
- George Mason University (US)
- University of Massachusetts Boston (US)
- Adobe Systems (United States) (US)
- Venue
- Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2021-05-06
- Pages
- 1-12
- OpenAlex
- W3160237369
- Cited by
- 87
- References
- 54
- Citations by year
- 2026: 7 · 2025: 26 · 2024: 26 · 2023: 17 · 2022: 6 · 2021: 5
- FWCI
- 121.88
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Funders
- NSF CAREER Award (1942531)
- Topics
- Social Sciences › Arts and Humanities › Language and Linguistics › Subtitles and Audiovisual Media
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Analysis and Summarization
- Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
- Keywords
- video accessibility, visual impairment, audiovisual content analysis, blind and visually impaired users
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.