Making Short-Form Videos Accessible with Hierarchical Video Summaries
Open accessThe DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Abstract
Short videos on platforms such as TikTok, Instagram Reels, and YouTube Shorts (i.e. short-form videos) have become a primary source of information and entertainment. Many short-form videos are inaccessible to blind and low vision (BLV) viewers due to their rapid visual changes, on-screen text, and music or meme-audio overlays. In our formative study, 7 BLV viewers who regularly watched short-form videos reported frequently skipping such inaccessible content. We present ShortScribe, a system that provides hierarchical visual summaries of short-form videos at three levels of detail to support BLV viewers in selecting and understanding short-form videos. ShortScribe allows BLV users to navigate between video descriptions based on their level of interest. To evaluate ShortScribe, we assessed description accuracy and conducted a user study with 10 BLV participants comparing ShortScribe to a baseline interface. When using ShortScribe, participants reported higher comprehension and provided more accurate summaries of video content.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Li, J., Li, D., Savarese, S., & Hoi, S. (2023) · arXiv · doi:10.48550/arxiv.2301.12597 · Cited by 908
Women Also Snowboard: Overcoming Bias in Captioning Models
Hendricks, L. A., Burns, K., Saenko, K., Darrell, T., & Rohrbach, A. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01219-9_47 · Cited by 404
Using Affinity Diagrams to Evaluate Interactive Prototypes
Lucero, A. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-22668-2_19 · Cited by 376
(2017) · doi:10.1145/2998181.2998364 · Cited by 258 · cited 1×
“Prior work has investigated ways to support BLV social media users as authors [14, 34, 40] and consumers [25–27, 40, 45, 74] of new types of images and videos used on social media.”
Understanding Blind People's Experiences with Computer-Generated Captions of Social Media Images
MacLeod, H., Bennett, C. L., Morris, M. R., & Cutrell, E. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025814 · Cited by 200 · cited 1×
“Future work could alternatively use the CLIP score directly to reveal confidence [48].”
Toward Automatic Audio Description Generation for Accessible Videos
Wang, Y., Liang, W., Huang, H., Zhang, Y., Li, D., & Yu, L.-F. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445347 · Cited by 87
Self-citationStangl, A., Verma, N., Fleischmann, K. R., Morris, M. R., & Gurari, D. (2021) · Proceedings of the 23rd International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3441852.3471233 · Cited by 81
Gleason, C., Pavel, A., Liu, X., Carrington, P., Chilton, L. B., & Bigham, J. P. (2019) · Proceedings of the 21st International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3308561.3353792 · Cited by 69
Self-citationSceneSkim: Searching and Browsing Movies Using Synchronized Captions, Scripts and Plot Summaries
Pavel, A., Goldman, D. B., Hartmann, B., & Agrawala, M. (2015) · Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology · doi:10.1145/2807442.2807502 · Cited by 67 · cited 3×
“Video Digests provides chapter and section summaries for learners skimming and browsing lecture videos [59].”
Self-citationBrowseWithMe: An Online Clothes Shopping Assistant for People with Visual Impairments
Stangl, A. J., Kothari, E., Jain, S. D., Yeh, T., Grauman, K., & Gurari, D. (2018) · Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3234695.3236337 · Cited by 66 · cited 1×
“For instance, prior work revealed that BLV people want adaptive length descriptions for digital comics [32], shopping images [67], and urban scenes [31].”
Cited by
Everyday Uncertainty: How Blind People Use GenAI Tools for Information Access
Tang, X., Abdolrahmani, A., Gergle, D., & Piper, A. M. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713433 · Cited by 30
Surfacing Variations to Calibrate Perceived Reliability of MLLM-generated Image Descriptions
Chen, M., Iyer, A., & Pavel, A. (2025) · Proceedings of the 27th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3663547.3746393 · Cited by 6
Self-citationYu, Y., Ryskeldiev, B., Tsutsui, A., Gillingham, M., & Wang, Y. (2025) · Proceedings of the 27th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3663547.3746353 · Cited by 8
Garg, K., Tang, X., Heo, J., Morgan, D. R., Gergle, D., Sudderth, E. B., & Piper, A. M. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791309 · Cited by 3
“Understanding how pervasive image quality issues affect the cap-tions generated by state-of-the-art VLMs is critical, given that these models are used in a wide range of assistive technologies and research prototypes [24, 25, 58, 62, 96, 126].”
Singh, P., Kumari, D., & Sahu, D. (2025) · Social Science Computer Review · doi:10.1177/08944393251405480 · Cited by 4
Metadata
- Authors
- Van Daele, T., Iyer, A., Zhang, Y., Derry, J. C., Huh, M., & Pavel, A.
- Venue
- Proceedings of the CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2024-05-11
- Pages
- 1-17
- OpenAlex
- W4391940741
- Cited by
- 38
- References
- 13
- FWCI
- 22.50
- Type
- conference paper
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Making Short-Form Videos Accessible with Hierarchical Video Summaries. https://app.cochat.ai/papers/10.1145/3613904.3642839
- The paper
- Van Daele, T., Iyer, A., Zhang, Y., Derry, J. C., Huh, M., & Pavel, A. (2024). Making Short-Form Videos Accessible with Hierarchical Video Summaries. Proceedings of the CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3613904.3642839
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.