Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 77 of the 77 references it cites were found in a registry or an index; 9 of the 25 sources its claims cite were checked for support.

Understanding and Improving Drilled-Down Information Extraction from Online Data Visualizations for Screen-Reader Users

Open access
Sharif, Ather, Zhang, Andrew M., Reinecke, Katharina, Wobbrock, Jacob O. · 2023 · 20th International Web for All Conference

Abstract

Inaccessible online data visualizations can significantly disenfranchise screen-reader users from accessing critical online information. Current accessibility measures, such as adding alternative text to visualizations, only provide a high-level overview of data, limiting screen-reader users from exploring data visualizations in depth. In this work, we build on prior research to develop taxonomies of information sought by screen-reader users to interact with online data visualizations granularly through role-based and longitudinal studies with screen-reader users. Utilizing these taxonomies, we extended the functionality of VoxLens—an open-source multi-modal system that improves the accessibility of data visualizations—by supporting drilled-down information extraction. We assessed the performance of our VoxLens enhancements through task-based user studies with 10 screen-reader and 10 non-screen-reader users. Our enhancements “closed the gap” between the two groups by enabling screen-reader users to extract information with approximately the same accuracy as non-screen-reader users, reducing interaction time by 22% in the process.

Overview

CoChat Summary

TL;DR

This paper develops taxonomies of holistic and drilled-down information that screen-reader users (SRUs) seek from online data visualizations, then uses them to extend VoxLens for granular extraction from multi-series line graphs and geospatial maps. In a task-based study, SRUs using the enhanced VoxLens extracted information 5.6% more accurately on average than non-SRUs, closing a previously reported accuracy gap, though they still took 92.8% more interaction time.

Key findings

  • Role-based (12 SRUs) and longitudinal (7 SRUs) studies produced taxonomies of holistic and drilled-down information needs. The most issued command was Value (28.6%); by visualization type, the most issued command was Factor for single-series bar graphs (26.7%), Average for multi-series line graphs (22.0%), and Value for geospatial maps (46.4%).
  • In the task-based study with 10 SRUs using enhanced VoxLens and 10 non-SRUs using no tools, there was no significant effect of SRU on accuracy of extracted information (p ≈ .906); SRUs were 5.6% more accurate on average than non-SRUs, whereas prior work reported non-SRUs outperforming SRUs by 62%. Visualization type significantly affected accuracy (χ²(2, N=300)=9.35, p<.05, Cramer’s V=0.12); single-series bar graphs had the best performance (92% SRU, 98% non-SRU), followed by multi-series line graphs (88%, 84%) and geospatial maps (86%, 70%).
  • SRU had a significant main effect on interaction time (F(1,18)=36.94, p<.001, η²p=0.69): SRUs averaged 42.4 seconds (SD=21.2) versus 22.0 seconds (SD=16.5) for non-SRUs, 92.8% higher. Visualization type also significantly affected interaction time (F(2,275.3)=6.14, p<.05, η²p=0.04), and age had a significant effect (F(1,16.9)=8.21, p<.05, η²p=0.33), with participants over 50 having 44.0% higher interaction times.
  • NASA-TLX ratings showed no significant effect of SRU on any of the six workload dimensions, suggesting comparable perceived workload levels.

Methods

  • The authors conducted a Wizard-of-Oz role-based study with 12 SRUs (M=50.3 years, SD=13.6) using explorer, teacher, and news reporter roles, and a 12-day longitudinal study with 7 SRUs (M=48.1 years, SD=8.7). Participants interacted with visualizations drawn from 30 curated examples (10 each of single-series bar graphs, multi-series line graphs, and geospatial maps); data included interaction logs and semi-structured interviews.
  • Quantitative analysis used a mixed Poisson regression for command counts; qualitative analysis used semantic thematic analysis, combining 33 initial codes into 18 axial codes, with 90.1% inter-rater agreement.
  • VoxLens was extended with chartType and dataModule parameters, factor-level categorization, regional categorization, expanded ranking vocabulary, factor-level and dependent-variable range queries, and visually hidden tables. Evaluation used a 2×3 mixed-factorial task-based study (10 SRUs using enhanced VoxLens vs. 10 non-SRUs without tools; 15 tasks per participant; 300 total trials), analyzed with mixed logistic regression for accuracy and a linear mixed model for interaction time, plus NASA-TLX analyzed with Aligned Rank Transform.

Limitations

  • The paper states that exploration was limited to single-series bar graphs, multi-series line graphs, and geospatial maps; future work should cover other complex visualizations such as three-dimensional graphs and stacked bar charts, and physical interfaces such as tactile maps.
  • The paper states that enhancements were limited by the keyword-matching algorithm and Web Speech API voice recognition, and that factor-level categorization struggled with prepositions or adverbs of time (e.g., “five years ago,” “between 2017 and 2019”).
  • Evident from the text: sample sizes were small (12, 7, and 10+10 participants), and the authors note all participants were from the United States, which may limit generalization of regional categorization and findings.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1145/3587281.3587284

Cites

77 of 77 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

9 Valid · 1 Not valid · 12 Not checked · 88 Own results and interpretations

50 more claims

Create a free account to see all 110 claims and to verify the 1 reference still open.

See all claims — free account
  • Probably not a substantial claim. 36
  • It may state two separate claims as one. 21
  • The quote only partly states this claim. 1

Create a free account to see these claims.

Statistics recomputed

Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.

  • ConsistentF(1, 18) = 36.94reported p < .001 · recomputed p < .001

    “F (1,18)=36.94,p < .001”

  • ConsistentF(2, 275.3) = 6.14reported p < .05 · recomputed p = .002

    “F (2,275.3)=6.14, p < .05”

  • ConsistentF(1, 16.9) = 8.21reported p < .05 · recomputed p = .011

    “F (1,16.9)=8.21,p < .05”

3 of 3

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

3 of 3

Cited works that were retracted

Works this paper cites that were later retracted, from the retraction notices publishers deposit. Major when the notice came out before this paper was published.

1 of 1

Citations

Cites

1 retracted

Show all 53 references

Research works printed but not linked: 7

  1. Lorna M. Brown , Stephen A. Brewster , Ramesh Ramloll , Mike Burton , and Beate Riedel . 2003 . Design guidelines for audio presentation of graphs and tables . In Proceedings of the 9th International Conference on Auditory Display. Citeseer , Boston University, USA, 284–287. Lorna M. Brown, Stephen A. Brewster, Ramesh Ramloll, Mike Burton, and Beate Riedel. 2003. Design guidelines for audio presentation of graphs and tables. In Proceedings of the 9th International Conference on Auditory Display. Citeseer, Boston University, USA, 284–287.
  2. Brigitte N. Frederick . 1999. Fixed- , random-, and mixed-effects ANOVA models: A user-friendly guide for increasing the generalizability of ANOVA results . In Advances in Social Science Methodology, B. Thompson (Ed.) . JAI Press , Stamford, Connecticut , 111–122. Brigitte N. Frederick. 1999. Fixed-, random-, and mixed-effects ANOVA models: A user-friendly guide for increasing the generalizability of ANOVA results. In Advances in Social Science Methodology, B. Thompson (Ed.). JAI Press, Stamford, Connecticut, 111–122.
  3. Melita Hajdinjak and France Mihelic . 2004. Conducting the Wizard-of-Oz Experiment.Informatica (Slovenia) 28, 4 ( 2004 ), 425–429. Melita Hajdinjak and France Mihelic. 2004. Conducting the Wizard-of-Oz Experiment.Informatica (Slovenia) 28, 4 (2004), 425–429.
  4. Jake Holland . 2017. New York Times ’ Upshot editor discusses data visualization, storytelling. https://dailynorthwestern.com/ 2017 /05/03/campus/new-york-times-upshot-editor-discusses-data-visualization-storytelling/. ( Accessed on 03/05/2022). Jake Holland. 2017. New York Times’ Upshot editor discusses data visualization, storytelling. https://dailynorthwestern.com/2017/05/03/campus/new-york-times-upshot-editor-discusses-data-visualization-storytelling/. (Accessed on 03/05/2022).
  5. Michal Kubovics and Pavel Bielik . 2021 . Visualization of Data And Keywords in Online Journalism . Marketing Identity 9 , 1 (2021), 121 – 133 . Michal Kubovics and Pavel Bielik. 2021. Visualization of Data And Keywords in Online Journalism. Marketing Identity 9, 1 (2021), 121–133.
  6. Ather Sharif , Andrew M Zhang , Anna Shih , Jacob O Wobbrock , and Katharina Reinecke . 2022 . Understanding and Improving Information Extraction From Online Geospatial Data Visualizations for Screen-Reader Users. In The 24th International ACM SIGACCESS Conference on Computers and Accessibility ( Athens, Greece) (ASSETS ’22). Association for Computing Machinery, New York, NY, USA, To appear pages. Ather Sharif, Andrew M Zhang, Anna Shih, Jacob O Wobbrock, and Katharina Reinecke. 2022. Understanding and Improving Information Extraction From Online Geospatial Data Visualizations for Screen-Reader Users. In The 24th International ACM SIGACCESS Conference on Computers and Accessibility (Athens, Greece) (ASSETS ’22). Association for Computing Machinery, New York, NY, USA, To appear pages.
  7. 10.5555/645443.652687 · doi:10.5555/645443.652687

Cited by

See all 21

Metadata

Authors
Sharif, A., Zhang, A. M., Reinecke, K., & Wobbrock, J. O.
Affiliations
University of Washington (US)
Venue
20th International Web for All Conference
Publisher
ACM
Published
2023-04-30
Pages
18-31
OpenAlex
W4366762618
Cited by
21
References
53
Citations by year
2026: 5 · 2025: 8 · 2024: 6 · 2023: 2
FWCI
8.79
Type
conference paper
Language
en
Open access
Locations
International Web for All Conference (W4A) (publishedVersion, cc-by)
Funders
University of Washington
Topics
  • Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
  • Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Data Visualization and Analytics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
Keywords
multimodal accessibility, alternative text, web accessibility

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Understanding and Improving Drilled-Down Information Extraction from Online Data Visualizations for Screen-Reader Users. https://app.cochat.ai/papers/10.1145/3587281.3587284
The paper
Sharif, A., Zhang, A. M., Reinecke, K., & Wobbrock, J. O. (2023). Understanding and Improving Drilled-Down Information Extraction from Online Data Visualizations for Screen-Reader Users. 20th International Web for All Conference. https://doi.org/10.1145/3587281.3587284

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.