Supporting Accessible Data Visualization Through Audio Data Narratives
The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
Abstract
Online data visualizations play an important role in informing public opinion but are often inaccessible to screen reader users. To address the need for accessible data representations on the web that provide direct, multimodal, and up-to-date access to the data, we investigate audio data narratives –which combine textual descriptions and sonification (the mapping of data to non-speech sounds). We conduct two co-design workshops with screen reader users to define design principles that guide the structure, content, and duration of a data narrative. Based on these principles and relevant auditory processing characteristics, we propose a dynamic programming approach to automatically generate an audio data narrative from a given dataset. We evaluate our approach with 16 screen reader users. Findings show with audio narratives, users gain significantly more insights from the data. Users describe data narratives help them better extract and comprehend the information in both the sonification and description.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Paas, F. G. W. C. (1992) · Journal of Educational Psychology · doi:10.1037/0022-0663.84.4.429 · Cited by 2,150 · cited 1×
“To assess cognitive load in understanding the information, we measured users’ self-perceived mental efort, on a 9-point Paas Likert scale [57].”
Method(2003) · Computer Graphics Forum · doi:10.1111/1467-8659.00652 · Cited by 1,436
Narrative Visualization: Telling Stories with Data
Segel, E., & Heer, J. (2010) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2010.179 · Cited by 1,334
Lo, S., & Andrews, S. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2015.01171 · Cited by 973
The intelligibility of speech as a function of the context of the test materials.
Miller, G. A., Heise, G. A., & Lichten, W. (1951) · Journal of Experimental Psychology · doi:10.1037/h0062491 · Cited by 769
CONSIDERATIONS IN THE CHOICE OF INTEROBSERVER RELIABILITY ESTIMATES
Hartmann, D. P. (1977) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.1977.10-103 · Cited by 762
Visualization of Time-Oriented Data
Aigner, W., Miksch, S., Schumann, H., & Tominski, C. (2011) · Human-Computer Interaction Series · doi:10.1007/978-0-85729-079-3 · Cited by 688
Empirical Studies in Information Visualization: Seven Scenarios
Lam, H., Bertini, E., Isenberg, P., Plaisant, C., & Carpendale, S. (2012) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2011.279 · Cited by 623
Exploratory Analysis of Spatial and Temporal Data
(2006) · doi:10.1007/3-540-31190-4 · Cited by 530
Auditory Scene Analysis: The Perceptual Organization of Sound
Smoliar, S., & Bregman, A. S. (1991) · Computer Music Journal · doi:10.2307/3680919 · Cited by 502
Research works printed but not linked: 22
- Benjamin Bach , Moritz Stefaner , Jeremy Boy , Steven Drucker , Lyn Bartram , Jo Wood , Paolo Ciuccarelli , Yuri Engelhardt , Ulrike Koeppen , and Barbara Tversky . 2018. Narrative design patterns for data-driven storytelling. In Data-driven storytelling , pp. 107 - 133 . AK Peters/CRC Press . Benjamin Bach, Moritz Stefaner, Jeremy Boy, Steven Drucker, Lyn Bartram, Jo Wood, Paolo Ciuccarelli, Yuri Engelhardt, Ulrike Koeppen, and Barbara Tversky. 2018. Narrative design patterns for data-driven storytelling. In Data-driven storytelling, pp. 107-133. AK Peters/CRC Press.
- Matthew Brehmer , and Tamara Munzner . 2013. A multi-level typology of abstract visualization tasks. IEEE transactions on visualization and computer graphics, 19(12) , pp. 2376 - 2385 . Matthew Brehmer, and Tamara Munzner. 2013. A multi-level typology of abstract visualization tasks. IEEE transactions on visualization and computer graphics, 19(12), pp.2376-2385.
- Lorna M. Brown , Stephen Brewster , Ramesh Ramloll , Wai Yu , and Beate Riedel . 2002 . Browsing modes for exploring sonified line graphs . In Proceedings of the British Human-Computer Interface Conference. 6–9. Lorna M. Brown, Stephen Brewster, Ramesh Ramloll, Wai Yu, and Beate Riedel. 2002. Browsing modes for exploring sonified line graphs. In Proceedings of the British Human-Computer Interface Conference. 6–9.
- Lorna M. Brown , Stephen A. Brewster , S. A. Ramloll , R. Burton , and Beate Riedel . 2003 . Design guidelines for audio presentation of graphs and tables . International Conference on Auditory Display (ICAD). Lorna M. Brown, Stephen A. Brewster, S. A. Ramloll, R. Burton, and Beate Riedel. 2003. Design guidelines for audio presentation of graphs and tables. International Conference on Auditory Display (ICAD).
- Stuart T. Card , Jock D. Mackinlay , and Ben Scheiderman . 1999. Readings in Information Visualization , using vision to think . San Francisco : Morgan Kaufmann . Stuart T. Card, Jock D. Mackinlay, and Ben Scheiderman. 1999. Readings in Information Visualization, using vision to think. San Francisco: Morgan Kaufmann.
- Jinho Choi , Sanghun Jung , Deok Gun Park , Jaegul Choo, and Niklas Elmqvist. 2019 . Visualizing for the non-visual: Enabling the visually impaired to use visualization. In Computer Graphics Forum , volume 38 , pages 249– 260 . Wiley Online Library. Jinho Choi, Sanghun Jung, Deok Gun Park, Jaegul Choo, and Niklas Elmqvist. 2019. Visualizing for the non-visual: Enabling the visually impaired to use visualization. In Computer Graphics Forum, volume 38, pages 249–260. Wiley Online Library.
- Diana Deutsch . 1999. Grouping mechanisms in music . In The psychology of music , pages 299– 348 . Elsevier. Diana Deutsch. 1999. Grouping mechanisms in music. In The psychology of music, pages 299–348. Elsevier.
- Mounya Elhilali , Juanjuan Xiang , Shihab A. Shamma , and Jonathan Z . Simon . 2009 . Interaction between attention and bottom-up saliency mediates the representation of foreground and back- ground in an auditory scene. PLoS biology, 7(6):e1000129. Mounya Elhilali, Juanjuan Xiang, Shihab A. Shamma, and Jonathan Z. Simon. 2009. Interaction between attention and bottom-up saliency mediates the representation of foreground and back- ground in an auditory scene. PLoS biology, 7(6):e1000129.
- John H. Flowers , Laura E. Whitwer , Douglas C. Grafel , and Cheryl A . Kotan . 2001 . Sonification of daily weather records: Issues of perception, attention and memory in design choices. Faculty Publications, Department of Psychology , page 432 . John H. Flowers, Laura E. Whitwer, Douglas C. Grafel, and Cheryl A. Kotan. 2001. Sonification of daily weather records: Issues of perception, attention and memory in design choices. Faculty Publications, Department of Psychology, page 432.
- Nicholas A. Giudice . 2018. Navigating without vision: Principles of blind spatial cognition. In Handbook of behavioral and cognitive geography . Edward Elgar Publishing . Nicholas A. Giudice. 2018. Navigating without vision: Principles of blind spatial cognition. In Handbook of behavioral and cognitive geography. Edward Elgar Publishing.
- Jenna L. Gorlewicz , Jennifer L. Tennison , Hari P. Palani , and Nicholas A . Giudice . 2018 . The graphical access challenge for people with visual impairments: Positions and pathways forward. In Interactive Multimedia-Multimedia Production and Digital Storytelling. IntechOpen . Jenna L. Gorlewicz, Jennifer L. Tennison, Hari P. Palani, and Nicholas A. Giudice. 2018. The graphical access challenge for people with visual impairments: Positions and pathways forward. In Interactive Multimedia-Multimedia Production and Digital Storytelling. IntechOpen.
- Michael Held , and Richard M . Karp . 1962 . A dynamic programming approach to sequencing problems. Journal of the Society for Industrial and Applied mathematics 10, no. 1: 196-210. Michael Held, and Richard M. Karp. 1962. A dynamic programming approach to sequencing problems. Journal of the Society for Industrial and Applied mathematics 10, no. 1: 196-210.
- Thomas Hermann , Andy Hunt , and John G . Neuhoff . 2011 . The sonification handbook. Logos Verlag Berlin . Thomas Hermann, Andy Hunt, and John G. Neuhoff. 2011. The sonification handbook. Logos Verlag Berlin.
- Johan Kildal , and Stephen A. Brewster . 2006. Providing a size-independent overview of non-visual tables . In Proceedings of the International Conference on Auditory Display (ICAD). Johan Kildal, and Stephen A. Brewster. 2006. Providing a size-independent overview of non-visual tables. In Proceedings of the International Conference on Auditory Display (ICAD).
- Edward Kim , and Kathleen F . McCoy. 2018. Multimodal deep learning using images and text for information graphic classification . In Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility (pp. 143-148) . Edward Kim, and Kathleen F. McCoy. 2018. Multimodal deep learning using images and text for information graphic classification. In Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility (pp. 143-148).
- Michael A. Nees and Bruce N. Walker. 2006. Relative intensity of auditory context for auditory graph design . In Proceedings of the International Conference on Auditory Display (ICAD). Michael A. Nees and Bruce N. Walker. 2006. Relative intensity of auditory context for auditory graph design. In Proceedings of the International Conference on Auditory Display (ICAD).
- Michael A. Nees and Bruce N. Walker. 2007. Listener, task, and auditory graph: Toward a conceptual model of auditory graph comprehension . In Proceedings of the International Conference on Auditory Display (ICAD). Michael A. Nees and Bruce N. Walker. 2007. Listener, task, and auditory graph: Toward a conceptual model of auditory graph comprehension. In Proceedings of the International Conference on Auditory Display (ICAD).
- Rameshsharma Ramloll , Stephen Brewster , Wai Yu , and Beate Riedel . 2001. Using non-speech sounds to improve access to 2D tabular numerical information for visually impaired users. In People and Computers XV—Interaction without Frontiers , pp. 515 - 529 . Springer , London . Rameshsharma Ramloll, Stephen Brewster, Wai Yu, and Beate Riedel. 2001. Using non-speech sounds to improve access to 2D tabular numerical information for visually impaired users. In People and Computers XV—Interaction without Frontiers, pp. 515-529. Springer, London.
- 10.5555/3235128 · doi:10.5555/3235128
- Tony Stockman , Christopher Frauenberger , and Greg Hind . 2005 . Interactive sonification of spreadsheets . In Proceedings of the International Conference on Auditory Display (ICAD). Tony Stockman, Christopher Frauenberger, and Greg Hind. 2005. Interactive sonification of spreadsheets. In Proceedings of the International Conference on Auditory Display (ICAD).
- Charles D. Stolper , Bongshin Lee , Nathalie Henry Riche, and John Stasko . 2016 . Emerging and recurring data-driven storytelling techniques: Analysis of a curated collection of recent stories. Charles D. Stolper, Bongshin Lee, Nathalie Henry Riche, and John Stasko. 2016. Emerging and recurring data-driven storytelling techniques: Analysis of a curated collection of recent stories.
- Qiyu Zhi , Alvitta Ottley , and Ronald Metoyer . 2019. Linking and layout: Exploring the integration of text and visualization in storytelling . In Computer Graphics Forum , volume 38(3), pages 675– 685 . Wiley Online Library . Qiyu Zhi, Alvitta Ottley, and Ronald Metoyer. 2019. Linking and layout: Exploring the integration of text and visualization in storytelling. In Computer Graphics Forum, volume 38(3), pages 675–685. Wiley Online Library.
Cited by
Understanding Visual Arts Experiences of Blind People
Li, F. M., Zhang, L., Bandukda, M., Stangl, A. J., Shinohara, K., Findlater, L., & Carrington, P. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580941 · Cited by 92
A survey of immersive visualization: Focus on perception and interaction
Zhang, Y., Wang, Z., Zhang, J., Shan, G., & Tian, D. (2023) · Visual Informatics · doi:10.1016/j.visinf.2023.10.003 · Cited by 78
Chart Reader: Accessible Visualization Experiences Designed with Screen Reader Users
Thompson, J. R., Martinez, J. J., Sarıkaya, A., Cutrell, E. B., & Lee, B. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581186 · Cited by 74
Jiang, L., Jung, C., Phutane, M., Stangl, A. J., & Azenkot, S. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642238 · Cited by 59
Alam, M. Z. I., Islam, S., & Hoque, E. (2023) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3581641.3584099 · Cited by 57
Metadata
- Authors
- Siu, A., S-H Kim, G., O'Modhrain, S., & Follmer, S.
- Affiliations
- Stanford University (US)
- Stanford University, Stanford University, United States
- University of Michigan, United States
- Venue
- CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2022-04-29
- Pages
- 1-19
- OpenAlex
- W4225163156
- Cited by
- 64
- References
- 51
- Citations by year
- 2026: 12 · 2025: 13 · 2024: 18 · 2023: 18 · 2022: 3
- FWCI
- 18.95
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Funders
- National science foundation (2016789, 2016363)
- Topics
- Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Data Visualization and Analytics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
- Keywords
- accessible data visualization, sonification, screen readers, multimodal representation, dynamic programming, co-design workshops, auditory processing, web accessibility
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Supporting Accessible Data Visualization Through Audio Data Narratives. https://app.cochat.ai/papers/10.1145/3491102.3517678
- The paper
- Siu, A., S-H Kim, G., O'Modhrain, S., & Follmer, S. (2022). Supporting Accessible Data Visualization Through Audio Data Narratives. CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3491102.3517678
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.