The DOI matches its Crossref record; 83 of the 86 references it cites were found in a registry or an index.
Accessible Visualization via Natural Language Descriptions: A Four-Level Model of Semantic Content
Open accessAbstract
Natural language descriptions sometimes accompany visualizations to better communicate and contextualize their insights, and to improve their accessibility for readers with disabilities. However, it is difficult to evaluate the usefulness of these descriptions, and how effectively they improve access to meaningful information, because we have little understanding of the semantic content they convey, and how different readers receive this content. In response, we introduce a conceptual model for the semantic content conveyed by natural language descriptions of visualizations. Developed through a grounded theory analysis of 2,147 sentences, our model spans four levels of semantic content: enumerating visualization construction properties (e.g., marks and encodings); reporting statistical concepts and relations (e.g., extrema and correlations); identifying perceptual and cognitive phenomena (e.g., complex trends and patterns); and elucidating domain-specific insights (e.g., social and political context). To demonstrate how our model can be applied to evaluate the effectiveness of visualization descriptions, we conduct a mixed-methods evaluation with 30 blind and 90 sighted readers, and find that these reader groups differ significantly on which semantic content they rank as most useful. Together, our model and findings suggest that access to meaningful information is strongly reader-specific, and that research in automatic visualization captioning should orient toward descriptions that more richly communicate overall trends and statistics, sensitive to reader preferences. Our work further opens a space of research on natural language as a data interface coequal with visualization.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248
Show and tell: A neural image caption generator
Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298935 · Cited by 6,318
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
Krishna, R., Zhu, Y., Groth, O., Johnson, J., Hata, K., Kravitz, J., Chen, S., Kalantidis, Y., Li, L.-J., Shamma, D. A., et al. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-016-0981-7 · Cited by 5,107
García, S., Fernández, A., Luengo, J., & Herrera, F. (2010) · Information Sciences · doi:10.1016/j.ins.2009.12.010 · Cited by 2,229
Deep Visual-Semantic Alignments for Generating Image Descriptions
Karpathy, A., & Fei-Fei, L. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2598339 · Cited by 1,048
Vega-Lite: A Grammar of Interactive Graphics
Satyanarayan, A., Moritz, D., Wongsuphasawat, K., & Heer, J. (2017) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2016.2599030 · Cited by 883
Self-citationA Multi-Level Typology of Abstract Visualization Tasks
Brehmer, M., & Munzner, T. (2013) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2013.124 · Cited by 747
What Makes a Visualization Memorable?
Borkin, M. A., Vo, A. A., Bylinskii, Z., Isola, P., Sunkavalli, S., Oliva, A., & Pfister, H. (2013) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2013.234 · Cited by 666
Instrumental interaction: an interaction model for designing post-WIMP user interfaces
Beaudouin-Lafon, M. (2000) · Proceedings of the SIGCHI conference on Human Factors in Computing Systems · doi:10.1145/332040.332473 · Cited by 614
Research works printed but not linked: 25
- gould, 2008, Effective Practices for Description of Science Content within Digital Talking Books
- geveci, 2012, VTK, The Architecture of Open Source Applications
- fleet, 2021, Things which garner a ton of glowing reviews from mainstream outlets without being of much use to disabled people, For instance Facebook's auto image descriptions much loved by sighted journos but famously useless in the Blind community Twitter
- fisher, 2019, Creating Accessible SVGs
- elzer, 0, A Browser Extension For Providing Visually Impaired Users Access To The Content Of Bar Charts On The Web, WEBIS
- gao, 0, Data-Tone: Managing Ambiguity in Natural Language Interfaces for Data Visualization, UIST
- fossheim, 2020, An Introduction To Accessible Data Visualizations With D3 js
- ehrenkranz, 2020, Vital Coronavirus Information Is Failing the Blind and Visually Impaired, Vice
- elavsky, 2021, Chartability
- watson, 2017, Accessible SVG Line Graphs
- demir, 0, Generating textual summaries of bar chartsGenerating Textual Summaries Of Bar Charts, INLG
- 0, Making Images Accessible
- hasty, 2011, Guidelines and Standards for Tactile Graphics
- lee, 0, Vlat: Development Of A Visualization Literacy Assessment Test, TVCG
- littlefield, 2020, COVID-19 Statistics Tracker
- livingston, 0, Position: Visual Sentences: Definitions and Applications, VIS
- ackland, 2017, World Blindness and Visual Impairment, Community Eye Health Journal
- wilkinson, 2005, The Grammar of Graphics, Statistics and Computing
- watson, 2018, Accessible SVG Flowcharts
- 2019, WAI Web Accessibility Tutorials Complex Images
- team, 2014, Bokeh Python library for interactive visualization
- bergstrom, 2020, SARS-CoV-2 Coronavirus
- bertin, 1983, Semiology of Graphics
- cesal, 2020, Writing Alt Text for Data Visualization
- schepers, 2020, Why Accessibility Is at the Heart of Data Visualization
Cited by
Chart-to-Text: A Large-Scale Benchmark for Chart Summarization
Kantharaj, S., Leong, R. T. K., Lin, X., Masry, A., Thakkar, M., Hoque, E., & Joty, S. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.277 · Cited by 308
Rich Screen Reader Experiences for Accessible Data Visualization
Zong, J., Lee, C., Lundgard, A., Jang, J., Hajas, D., & Satyanarayan, A. (2022) · Computer Graphics Forum · doi:10.1111/cgf.14519 · Cited by 149
Self-citationUniChart: A Universal Vision-language Pretrained Model for Chart Comprehension and Reasoning
Masry, A., Kavehzadeh, P., Long, X., Hoque, E., & Joty, S. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.906 · Cited by 279
VoxLens: Making Online Data Visualizations Accessible with an Interactive JavaScript Plug-In
Sharif, A., Wang, O. H., Muongchan, A. T., Reinecke, K., & Wobbrock, J. O. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517431 · Cited by 86
How accessible is my visualization? Evaluating visualization accessibility with Chartability
Elavsky, F., Bennett, C., & Moritz, D. (2022) · Computer Graphics Forum · doi:10.1111/cgf.14522 · Cited by 68
Metadata
- Authors
- Lundgard, A., & Satyanarayan, A.
- Affiliations
- MIT Computer Science and Artificial Intelligence Laboratory (US)
- ORCID
- Satyanarayan, Arvind
- Venue
- IEEE Transactions on Visualization and Computer Graphics
- Published
- 2022-01
- Volume
- 28
- Issue
- 1
- Pages
- 1073-1083
- ISSN
- 1077-2626 (ISSN-L)
- 1941-0506
- 2160-9306
- OpenAlex
- W3202687543
- Cited by
- 155
- References
- 86
- Citations by year
- 2026: 30 · 2025: 47 · 2024: 36 · 2023: 26 · 2022: 14
- FWCI
- 7.01
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Visualization and Computer Graphics (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- National Science Foundation (GRFP-1122374, III-1900991)
- Topics
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Data Visualization and Analytics
- Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
- Social Sciences › Arts and Humanities › Language and Linguistics › Subtitles and Audiovisual Media
- Keywords
- blind readers
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Accessible Visualization via Natural Language Descriptions: A Four-Level Model of Semantic Content. https://app.cochat.ai/papers/10.1109/tvcg.2021.3114770
- The paper
- Lundgard, A., & Satyanarayan, A. (2022). Accessible Visualization via Natural Language Descriptions: A Four-Level Model of Semantic Content. IEEE Transactions on Visualization and Computer Graphics. https://doi.org/10.1109/tvcg.2021.3114770
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.