The Accessibility of Data Visualizations on the Web for Screen Reader Users: Practices and Experiences During COVID-19

The general inaccessibility of high-visibility websites across multiple organizations and visualization types during the COVID-19 pandemic supports prior reports of data inaccessibility.

“The general inaccessibility of these high-visibility websites across multiple organizations and visualization types during the COVID-19 pandemic supports prior reports of data inaccessibility [39,61,66,73,73].”

• cites [39] 39. Frank Elavsky, Cynthia Bennett, and Dominik Moritz. 2022. How accessible is my visualization? Evaluating visualiza- tion accessibility with Chartability. Computer Graphics Forum 41, 3 (2022), 57-70. sourceNot checked — no full text of the source
• cites [61] 61. Leona Holloway, Matthew Butler, Samuel Reinders, and Kim Marriott. 2020. Non-visual access to graphical information on COVID-19. In Proceedings of the 22nd International ACM SIGACCESS Conference on Computers and Accessibility. 1-3. sourceNot checked — no full text of the source
• cites [66] 66. Crescentia Jung, Shubham Mehta, Atharva Kulkarni, Yuhang Zhao, and Yea-Seul Kim. 2021. Communicating visu- alizations without visuals: Investigation of visualization alternative text for people with visual impairments. IEEE Trans. Visualiz. Comput. Graph. 28, 1 (2021), 1095-1105. sourceSupported
• cites [73] 73. Barbara Leporini and Fabio Paternò. 2008. Applying web usability criteria for vision-impaired users: Does it really improve task performance? Int. J. Hum.-comput. Interact. 24, 1 (2008), 17-47. sourcePartly supported

  The Introduction (p. 1) discusses barriers that can prevent users with disabilities from accessing information on websites, and §2.1 (p. 3) describes navigation difficulties for blind and low-vision users. This supports the broad point that web information can be inaccessible. The paper does not report inaccessibility across COVID-19 websites, organizations, or visualization types; those are findings of the citing document, not this source. (LLM, conf=0.82)

  Experimental — accuracy not yet measured

Both our audit and other recent work found that data-tracking tools, libraries, and services used by dashboards are mostly inaccessible to screen reader users, even two years after the beginning of the pandemic.

“While these tools, libraries, and services provide convenient ways to organize and convey large quantities of data and visualizations, both our audit and other recent work [102] found that they are mostly inaccessible to screen reader users, even two years after the beginning of the pandemic.”

• cites [102] 102. Ather Sharif, Olivia H. Wang, Alida T. Muongchan, Katharina Reinecke, and Jacob O. Wobbrock. 2022. VoxLens: Making online data visualizations accessible with an interactive JavaScript plug-in. In Proceedings of the CHI Conference on Human Factors in Computing Systems. 1-19. sourcePartly supported

  The Introduction (page not provided) says screen-reader users commonly cannot access online data visualizations, and §2.3 describes accessibility limitations in visualization libraries. The paper does not establish that dashboard data-tracking tools and services are mostly inaccessible, or assess their accessibility two years into the pandemic. (LLM, conf=0.85)

  Experimental — accuracy not yet measured

In the Top Results group, the pass rate for visualizations being detectable was 68% (52/76).

“whether visualizations are detectable (PR = 68%, 52/76)”

In the Top Results group, the majority of applicable visualizations received passing grades only in 3 of 26 criteria.

“In the Top Results group, the majority of applicable visualizations received passing grades only in 3 of 26 criteria.”

82% (9/11) of Born Accessible visualizations were rated as very or extremely accessible.

“82% (9/11) of Born Accessible visualizations were rated as very or extremely accessible”

In the Top Results group, 18% of visualizations were difficult to detect through the screen reader.

“18% of the visualizations were difcult to detect through the screen reader”

In the Born Accessible group, pass rates were over 50% for 21 of the same 26 criteria.

“In comparison, pass rates were over 50% for 21 of the same 26 criteria in the Born Accessible group.”

The proportion of pass grades for Born Accessible visualizations was significantly greater than for Top Results visualizations across navigation criteria (χ2(1)=33.35, P<.001).

“navigation (χ 2 (1) = 33.35,P < .001)”

In the Top Results group, the pass rate for not containing custom keyboard commands that override screen reader settings was 100% (76/76).

“do not contain custom keyboard commands that override screen reader settings (PR = 100%, 76/76)”

The ordinal logistic regression model predicted nearly a 30-fold likelihood of rating increase for Born Accessible visualizations over Top Results visualizations (OR=29.06, 95% CI=[125.16,7.31]).

“The ordinal logistic regression model predicted nearly a 30-fold likelihood of rating increase between for Born Accessible visualizations over Top Results visualizations (OR = 29.06,95% CI = [125.16,7.31]).”

The proportion of pass grades for Born Accessible visualizations was significantly greater than for Top Results visualizations across content criteria (χ2(1)=32.84, P<.001).

“The proportion of pass grades for Born Accessible visualizations was also signifcantly greater than the proportion of pass grades for Top Results visualizations across the content (χ 2 (1) = 32.84,P < .001)”

In the Top Results group, the pass rate for providing access to the title was 55% (42/76).

“provide access to the title (PR = 55%, 42/76)”

There was a significant difference between Born Accessible and Top Results visualization ratings (χ2(1)=26.31, P<.001).

“There was a signifcant diference between Born Accessible and Top Results visualization ratings (χ 2 (1) = 26.31,P < .001).”

The proportion of pass grades for Born Accessible visualizations was significantly greater than for Top Results visualizations across context criteria (χ2(1)=16.59, P<.001).

“context (χ 2 (1) = 16.59,P < .001)”

14% (11/76) of Top Results visualizations were rated as very or extremely accessible.

“only 14% (11/76) of Top Results visualizations achieved the same ratings”

The proportion of pass grades for Born Accessible visualizations was significantly greater than for Top Results visualizations across interactivity criteria (χ2(1)=8.76, P=.003).

“interactivity criteria (χ 2 (1) = 8.76, P = .003)”

Several visualizations in Top Results websites could not be detected by at least one auditor through their screen reader and browser combination.

“However, several visualizations in the Top Results websites could not be detected by at least one auditor through their screen reader and browser combination”

There were strong linear relationships between the proportion of pass rates and the overall accessibility rating auditors provided to each visualization.

“Figure 2 shows strong linear relationships between the proportion of pass rates and the overall accessibility rating on a 5-Point Likert scale”

Post hoc pairwise comparisons showed marginally significant differences in overall accessibility ratings between tables and line graphs (z = 2.81, P = .005).

“Post hoc pairwise comparisons (with a Bonferroni adjusted alpha level of 0.0033) showed signifcant diferences in overall accessibility ratings comparing tables with bar graphs (z.ratio = 2.97, P = .003),maps (z.ratio = 2.97, P < .003),and marginally signifcant diferences between tables and line graphs (z.ratio = 2.81, P = .005).”

In Born Accessible visualizations, data points accessibility pass rate was 60% (6/10).

“In contrast to the Top Results visualizations, for Born Accessible visualizations, most content-based criteria (9/11) had majority pass rates, including ones evaluating the accessibility of trends (PR = 100%, 8/8), visual features (PR = 88%, 7/8), and data points (PR = 60%, 6/10).”

55% (42/76) of visualizations in the Born Accessible group did not have interactive features.

“55% (42/76) of visualizations in the Born Accessible group did not have interactive features.”

Auditors found all tables (n = 3) in the Born Accessible group to be easy to use.

“In comparison, auditors found all tables (n = 3) in the Born Accessible group to be easy to use.”

In the Born Accessible group, all visualizations indicated the title and type of visualization through screen reader accessible text.

“In the Born Accessible group, all visualizations indicated the title and type of visualization through screen reader accessible text, provided an adequate summary when visual information was present, and the majority of visualizations provided information about the source of information (PR = 64%, 49/76).”

Auditors observed that SVGs in Top Results visualizations were not labeled or organized in ways that support understanding through screen readers.

“However, auditors observed that those SVGs were not labeled or organized in ways that support understanding through screen readers.”

For several visualizations, the screen reader focus jumped from data values of one visualization to that of another without providing any indication.

“For several visualizations, the screen reader focus jumped from data values of one visualization to that of another without providing any indication for several dashboards.”

Visualization type was not a significant predictor of the proportion of pass grades across interactivity criteria (χ2(5) = 0.78, P = .979) for major groups of visualizations in the Top Results group.

“Visualization type was also a signifcant predictor on the proportion of pass grades across content (χ 2 (5) = 30.05,P = .001)and navigation criteria (χ 2 (5) = 17.27,P = .004),but not context (χ 2 (5) = 5.22, P = .389)and interactivity (χ 2 (5) = 0.78, P = .979)criteria for major groups of visualization.”

Users could not access specific data points when navigating maps using keyboards.

“Users could not access specifc data points when navigating maps using keyboards.”

For significant pairwise comparisons, ordinal logistic regressions predicted a 9-fold likelihood of rating increase for tables compared with maps (OR = 9.10, 95% CI (39.04, 2.12)).

“For signifcant pairwise comparisons, ordinal logistic regressions predicted a 9-fold likelihood of rating increase for tables compared bar graphs (OR = 9.26, 95% CI (40.28, 2.13)) and a 9-fold likelihood of rating increase for tables compared to maps (OR = 9.10, 95% CI (39.04, 2.12)).”

The majority of Top Results visualizations did not provide easy screen reader access to the data source (PR = 26%, 20/76).

“Almost half of the visualizations in the Top Results group did not provide screen reader access to the visualization title (PR = 55%, 42/76), and the majority of visualizations did not provide easy screen reader access to the visualization type (PR = 34%, 23/68), data source (PR = 26%, 20/76), or when the data had last been updated (PR = 49%, 37/76).”

In the Top Results group, pictorial fraction charts were rated very accessible.

“In the Top Results group, several visualizations were only represented once in the group, including pictorial fraction charts (rated very accessible), bubble maps (rated not at all accessible), and pie charts (rated not at all accessible).”

In the Top Results group, 18% (3/17) of bar graphs were rated moderately to extremely accessible.

“50% (3/6) of bar graphs with line overlays, 50% (4/8) of summary panels of spatially oriented statistics, 21% (3/14) of line graphs, 18% (3/17) of bar graphs, and 7% (1/15) of maps were rated moderately to extremely accessible.”

Only 35% (24/68) of Top Results visualizations with interactive features exposed those features to the screen readers auditors used.

“In the Top Results group, 89% (68/76) of the visualizations had interactive features, such as sorting, fltering, panning, zooming, highlighting, and changing the display of groups and encodings, but only a fraction of those visualizations (PR = 35%, 24/68) exposed interactive features to the screen readers auditors used.”

The details in summaries were not consistent across all audited Born Accessible websites.

“However, the details in summaries were not consistent across all audited Born Accessible websites.”

Post hoc pairwise comparisons showed significant differences between tables and maps for navigation criteria (t.ratio(66.8) = 3.03, P = .004).

“Post hoc pairwise comparisons (with a Bonferroni adjusted alpha level of 0.0033) showed signifcant diferences between tables and maps for both content (t.ratio(65.5)= 3.99, P < .001)and navigation (t.ratio(66.8)= 3.03, P = .004)criteria.”

For significant pairwise comparisons, ordinal logistic regressions predicted a 9-fold likelihood of rating increase for tables compared with bar graphs (OR = 9.26, 95% CI (40.28, 2.13)).

“For signifcant pairwise comparisons, ordinal logistic regressions predicted a 9-fold likelihood of rating increase for tables compared bar graphs (OR = 9.26, 95% CI (40.28, 2.13)) and a 9-fold likelihood of rating increase for tables compared to maps (OR = 9.10, 95% CI (39.04, 2.12)).”

Auditors described having access to only detailed numerical values for a few websites and trends without data values in others.

“Auditors described having access to only detailed numerical values for a few websites and trends without data values in others.”

In the Top Results group, 21% (3/14) of line graphs were rated moderately to extremely accessible.

“50% (3/6) of bar graphs with line overlays, 50% (4/8) of summary panels of spatially oriented statistics, 21% (3/14) of line graphs, 18% (3/17) of bar graphs, and 7% (1/15) of maps were rated moderately to extremely accessible.”

Among tables provided in the Top Results group, 43% (17/40) were difficult to find and were encountered accidentally.

“Among tables that were provided, 43% (17/40) of them were difcult to fnd and were encountered accidentally, 40% (16/40) could not be sorted or downloaded in an accessible format, and 38% (15/40) did not articulate row and column headers during navigation.”

For navigation criteria, marginally significant differences were observed between tables and line graphs (t.ratio(65.1) = 3.01, P = .004).

“For navigation criteria, marginally signifcant diferences were observed between tables and line graphs (t.ratio(65.1)= 3.01, P = .004).”

For Born Accessible visualizations that had interactive features, auditors found it easy to follow changes for all of the visualizations.

“However, for those that did, auditors found it easy to follow changes for all of the visualizations.”

Visualization type was not a significant predictor of the proportion of pass grades across context criteria (χ2(5) = 5.22, P = .389) for major groups of visualizations in the Top Results group.

“Visualization type was also a signifcant predictor on the proportion of pass grades across content (χ 2 (5) = 30.05,P = .001)and navigation criteria (χ 2 (5) = 17.27,P = .004),but not context (χ 2 (5) = 5.22, P = .389)and interactivity (χ 2 (5) = 0.78, P = .979)criteria for major groups of visualization.”

For content criteria, marginally significant differences were observed between tables and line graphs (t.ratio(65.2) = 2.88, P = .005).

“For content criteria, marginally signifcant diferences was observed between tables and bar graphs (t.ratio(66.8)= 3.03, P = .004)and between tables and line graphs (t.ratio(65.2) = 2.88, P = .005).”

In the Born Accessible group, all visualizations provided an adequate summary when visual information was present.

“In the Born Accessible group, all visualizations indicated the title and type of visualization through screen reader accessible text, provided an adequate summary when visual information was present, and the majority of visualizations provided information about the source of information (PR = 64%, 49/76).”

In the Top Results group, 50% (4/8) of summary panels of spatially oriented statistics were rated moderately to extremely accessible.

“50% (3/6) of bar graphs with line overlays, 50% (4/8) of summary panels of spatially oriented statistics, 21% (3/14) of line graphs, 18% (3/17) of bar graphs, and 7% (1/15) of maps were rated moderately to extremely accessible.”

In Born Accessible visualizations, trends accessibility pass rate was 100% (8/8).

“In contrast to the Top Results visualizations, for Born Accessible visualizations, most content-based criteria (9/11) had majority pass rates, including ones evaluating the accessibility of trends (PR = 100%, 8/8), visual features (PR = 88%, 7/8), and data points (PR = 60%, 6/10).”

Auditors generally found it easier to navigate to tables because some screen readers provide default shortcuts for navigating to tables.

“Auditors generally found it easier to navigate to tables, because some screen readers provide default shortcuts for navigating to tables.”

Among tables provided in the Top Results group, 38% (15/40) did not articulate row and column headers during navigation.

“Among tables that were provided, 43% (17/40) of them were difcult to fnd and were encountered accidentally, 40% (16/40) could not be sorted or downloaded in an accessible format, and 38% (15/40) did not articulate row and column headers during navigation.”

The majority of Top Results visualizations did not provide easy screen reader access to the visualization type (PR = 34%, 23/68).

“Almost half of the visualizations in the Top Results group did not provide screen reader access to the visualization title (PR = 55%, 42/76), and the majority of visualizations did not provide easy screen reader access to the visualization type (PR = 34%, 23/68), data source (PR = 26%, 20/76), or when the data had last been updated (PR = 49%, 37/76).”

In the Top Results group, specific data points were accessible through screen readers in 21% (16/76) of visualizations.

“Trends (PR = 5%, 3/58), visual features (PR = 12%, 7/60), and specifc data points (PR = 21%, 16/76) were largely inaccessible through screen readers.”

The majority of Top Results visualizations did not provide easy screen reader access to when the data had last been updated (PR = 49%, 37/76).

“Almost half of the visualizations in the Top Results group did not provide screen reader access to the visualization title (PR = 55%, 42/76), and the majority of visualizations did not provide easy screen reader access to the visualization type (PR = 34%, 23/68), data source (PR = 26%, 20/76), or when the data had last been updated (PR = 49%, 37/76).”

When context information was presented, it was often difficult to find on the page because it was located away from the visualization.

“When presented, this information was often difcult to fnd on the page, as it was located away from the visualization, such as in a heading after the visualization, on a diferent page, or in the summary at the start of the web page.”

Post hoc pairwise comparisons showed significant differences between tables and maps for content criteria (t.ratio(65.5) = 3.99, P < .001).

“Post hoc pairwise comparisons (with a Bonferroni adjusted alpha level of 0.0033) showed signifcant diferences between tables and maps for both content (t.ratio(65.5)= 3.99, P < .001)and navigation (t.ratio(66.8)= 3.03, P = .004)criteria.”

Of the Top Results visualizations with exposed interactive features, auditors found understanding changes difficult in 25% (6/24).

“Of the visualizations with interactive features that were exposed, auditors found understanding changes difcult in 25% (6/24) of those visualizations.”

Auditors were able to move around the grid using announced data, headers, and labels as references when reading specific data points.

“They were able to move around the grid using announced data, headers, and labels as references when reading specifc data points.”

In the Top Results group, trends were accessible through screen readers in 5% (3/58) of visualizations.

“Trends (PR = 5%, 3/58), visual features (PR = 12%, 7/60), and specifc data points (PR = 21%, 16/76) were largely inaccessible through screen readers.”

Auditors flagged occasions when data formats were overly verbose.

“Auditors also fagged occasions when data formats were overly verbose.”

In the Born Accessible group, the majority of visualizations provided information about the source of information (PR = 64%, 49/76).

“In the Born Accessible group, all visualizations indicated the title and type of visualization through screen reader accessible text, provided an adequate summary when visual information was present, and the majority of visualizations provided information about the source of information (PR = 64%, 49/76).”

In the Top Results group, pie charts were rated not at all accessible.

“In the Top Results group, several visualizations were only represented once in the group, including pictorial fraction charts (rated very accessible), bubble maps (rated not at all accessible), and pie charts (rated not at all accessible).”

Auditors found it easy to tell when the screen reader focus was inside a table.

“Auditors also found it easy to tell when the screen reader focus was inside of a table.”

In the Born Accessible group, 100% of maps (1/1), area graphs (2/2), and tables (3/3) were rated very or extremely accessible.

“With a much smaller sample size, 100% of maps (1/1), area graphs (2/2), and tables (3/3) in the Born Accessible group were rated to be very or extremely accessible.”

Auditors had to rely on the summaries and supporting data to learn about the details in the visualization.

“Auditors had to rely on the summaries and supporting data to learn about the details in the visualization.”

Among tables provided in the Top Results group, 40% (16/40) could not be sorted or downloaded in an accessible format.

“Among tables that were provided, 43% (17/40) of them were difcult to fnd and were encountered accidentally, 40% (16/40) could not be sorted or downloaded in an accessible format, and 38% (15/40) did not articulate row and column headers during navigation.”

Unexposed interactions often included click to highlight, hover to highlight, scroll zoom, and filter interactions.

“Unexposed interactions often included click to highlight, hover to highlight, scroll zoom, and flter interactions.”

Auditors appreciated the levels of detail in the summaries provided for Born Accessible visualizations.

“Auditors particularly appreciated the levels of detail in the summaries provided for these visualizations.”

In Born Accessible visualizations, visual features accessibility pass rate was 88% (7/8).

“In contrast to the Top Results visualizations, for Born Accessible visualizations, most content-based criteria (9/11) had majority pass rates, including ones evaluating the accessibility of trends (PR = 100%, 8/8), visual features (PR = 88%, 7/8), and data points (PR = 60%, 6/10).”

Exposed interactive features tended to include buttons, dropdown links, and sortable columns.

“Exposed features tended to include buttons, dropdown links, and sortable columns.”

In the Top Results group, 89% (68/76) of visualizations had interactive features.

“In the Top Results group, 89% (68/76) of the visualizations had interactive features, such as sorting, fltering, panning, zooming, highlighting, and changing the display of groups and encodings, but only a fraction of those visualizations (PR = 35%, 24/68) exposed interactive features to the screen readers auditors used.”

For bar graphs, auditors struggled to access bar elements of the graph.

“For bar graphs, auditors struggled to access bar elements of the graph.”

In the Born Accessible group, auditors felt it was easy to identify when the screen reader focus was in the visualizations for all audited visualizations.

“In the Born Accessible Group, auditors felt that it was easy to identify when the screen reader focus was in the visualizations for all audited visualizations.”

Distinguishing between visualizations was a challenge for the majority of Top Results visualizations (PR = 28%, 21/76).

“Distinguishing between visualizations was a challenge for the majority of Top Results visualizations (PR = 28%, 21/76).”

Auditors were often unable to distinguish between grouped visualizations, particularly if located within the same data dashboard.

“They were also often unable to distinguish between grouped visualizations, particularly if located within the same data dashboard.”

Almost half of Top Results visualizations did not provide screen reader access to the visualization title (PR = 55%, 42/76).

“Almost half of the visualizations in the Top Results group did not provide screen reader access to the visualization title (PR = 55%, 42/76), and the majority of visualizations did not provide easy screen reader access to the visualization type (PR = 34%, 23/68), data source (PR = 26%, 20/76), or when the data had last been updated (PR = 49%, 37/76).”

In Born Accessible visualizations, most content-based criteria (9/11) had majority pass rates.

“In contrast to the Top Results visualizations, for Born Accessible visualizations, most content-based criteria (9/11) had majority pass rates, including ones evaluating the accessibility of trends (PR = 100%, 8/8), visual features (PR = 88%, 7/8), and data points (PR = 60%, 6/10).”

Visualization type was a significant predictor of the proportion of pass grades across navigation criteria (χ2(5) = 17.27, P = .004) for major groups of visualizations in the Top Results group.

“Visualization type was also a signifcant predictor on the proportion of pass grades across content (χ 2 (5) = 30.05,P = .001)and navigation criteria (χ 2 (5) = 17.27,P = .004),but not context (χ 2 (5) = 5.22, P = .389)and interactivity (χ 2 (5) = 0.78, P = .979)criteria for major groups of visualization.”

Visualization type was a significant predictor of the proportion of pass grades across content criteria (χ2(5) = 30.05, P = .001) for major groups of visualizations in the Top Results group.

“Visualization type was also a signifcant predictor on the proportion of pass grades across content (χ 2 (5) = 30.05,P = .001)and navigation criteria (χ 2 (5) = 17.27,P = .004),but not context (χ 2 (5) = 5.22, P = .389)and interactivity (χ 2 (5) = 0.78, P = .979)criteria for major groups of visualization.”

Born Accessible websites provided information through combinations of textual summaries, tables, and/or sonification.

“The Born Accessible websites provided information through combinations of textual summaries, tables, and/or sonifcation.”

In the Top Results group, 7% (1/15) of maps were rated moderately to extremely accessible.

“50% (3/6) of bar graphs with line overlays, 50% (4/8) of summary panels of spatially oriented statistics, 21% (3/14) of line graphs, 18% (3/17) of bar graphs, and 7% (1/15) of maps were rated moderately to extremely accessible.”

Several visualizations provided all numerical values across different axes and categories before any axes titles or category labels.

“For example, several visualizations provided all of the numerical values across diferent axes and categories before any of the axes titles or category labels.”

In the Top Results group, the majority of visualizations did not provide screen reader access to many types of visualization content.

“Based on our audit, the majority of visualizations in the Top Results group did not provide screen reader access to many types of visualization content.”

No other pairwise comparisons were significant.

“No other pairwise comparisons were signifcant.”

Many Top Results visualizations did not provide accessible tables to complement visualizations (PR = 41%, 28/68).

“Many visualizations in this group did not provide accessible tables to complement visualizations (PR = 41%, 28/68).”

Many inaccessible tables did not follow standard HTML table tag structure, but used specific dashboard libraries or presented information in image format.

“Many of the inaccessible tables did not follow standard HTML table tag structure, but used specifc dashboard libraries or presented information in image format.”

For content criteria, marginally significant differences were observed between tables and bar graphs (t.ratio(66.8) = 3.03, P = .004).

“For content criteria, marginally signifcant diferences was observed between tables and bar graphs (t.ratio(66.8)= 3.03, P = .004)and between tables and line graphs (t.ratio(65.2) = 2.88, P = .005).”

In the Top Results group, visual features were accessible through screen readers in 12% (7/60) of visualizations.

“Trends (PR = 5%, 3/58), visual features (PR = 12%, 7/60), and specifc data points (PR = 21%, 16/76) were largely inaccessible through screen readers.”

If presented with an informative description in the summary and announcements that supported navigation in the grid, tables were described by auditors to be more accessible.

“However, if presented with an informative description in the summary and announcements that supported navigation in the grid, then tables were described by auditors to be more accessible.”

Auditors did not find most Top Results visualizations to have approachable designs (PR = 16%, 12/76).

“Auditors did not fnd most visualizations to have approachable designs (PR = 16%, 12/76), which we defned to either be following best practices or providing an adequate explanation of the design.”

Many Top Results visualizations relied on Scalable Vector Graphics (SVGs).

“Many of the visualizations relied on Scalable Vector Graphics (SVGs), a standard markup language that is used for defning two-dimensional graphics.”

In the Top Results group, bubble maps were rated not at all accessible.

“In the Top Results group, several visualizations were only represented once in the group, including pictorial fraction charts (rated very accessible), bubble maps (rated not at all accessible), and pie charts (rated not at all accessible).”

Of the remaining Top Results visualization types represented by at least 6 visualizations, tables (n = 13) were the only type in which the majority were rated moderately to extremely accessible (54%, 7/13).

“Of the remaining visualization types represented by over ≥ 6 visualizations, tables (n = 13) were the only type of representation in which the majority of visualization were rated moderately to extremely accessible (54%, 7/13).”

Visualization type was a significant predictor of auditor rating (χ2(5) = 17.04, P = .004) for the major groups of visualization types (n ≥ 6) in the Top Results group.

“Visualization type was a signifcant predictor of auditor rating (χ 2 (5) = 17.04,P = .004)for the major groups of visualization types (n ≥ 6) in the Top Results group.”

All but one Born Accessible visualization were indicated to have an approachable design.

“All but one visualization were indicated to have an approachable design.”

Controls provided to manipulate the maps lacked explanations about what they do.

“Additionally, controls provided to manipulate the maps lacked explanations about what they do.”

Only a small fraction of Top Results visualizations provided adequate summaries (PR = 28%, 15/53) due to a lack of detail describing data structures, overall trends, or important contextual information.

“Auditors also reported that only a small fraction of visualizations provided adequate summaries of the visualization (PR = 28% 15/53) due to a lack of detail describing data structures, overall trends, or important contextual information about the chart.”

In the Top Results group, 50% (3/6) of bar graphs with line overlays were rated moderately to extremely accessible.

“50% (3/6) of bar graphs with line overlays, 50% (4/8) of summary panels of spatially oriented statistics, 21% (3/14) of line graphs, 18% (3/17) of bar graphs, and 7% (1/15) of maps were rated moderately to extremely accessible.”

Auditors had to spend time reading over all the website elements until they reached the specific element to access the visualization.

“Auditors had to spend time reading over all the website elements until they reached the specifc element to access the visualization.”

Overall ratings and criteria performance for Born Accessible visualizations were significantly higher than for Top Results websites.

“We fnd that overall ratings and criteria performance for Born Accessible visualizations were signifcantly higher than for Top Results websites.”

Several other visualizations were hidden in hierarchical structures that were much more apparent visually.

“Several other visualizations were hidden in hierarchical structures that were much more apparent visually.”

Many of the Top Results visualizations do not convey any data content through screen reader access.

“Many of the Top Results visualizations do not convey any data content through screen reader access.”

Even when the information is found, its relationship to the visualizations is often unclear.

“Even when the information is found, its relationship to the visualizations are often unclear.”

Websites that conveyed information through extensive passages without an overall hierarchy posed additional barriers.

“However, websites that conveyed information through extensive passages without an overall hierarchy posed additional barriers.”

When asked about current methods of accessing data graphics, the most frequently indicated method was screen readers (n = 20), followed by Braille (n = 18), audio and sound (n = 15), sighted assistance (n = 14), and tactile graphics (n = 5).

“However, when asked about current methods of accessing data graphics, the most frequently indicated method was through screen readers (n = 20), followed by Braille (n = 18), then audio and sound (n = 15), sighted assistance (n = 14), and tactile graphics (n = 5) (Figure 9).”

Several visualizations were conveyed through dynamic dashboards produced from libraries without screen reader support.

“Several visualizations were conveyed through dynamic dashboards produced from libraries without screen reader support, and most do not provide adequate summaries or access to tables of the underlying data.”

Pairwise comparisons reveal marginally significant to significant differences of tables when compared to bar graphs, line graphs, and maps.

“Pairwise comparisons reveal marginally signifcant to signifcant diferences of tables when compared to bar graphs, line graphs, and maps.”

In content-dense web pages, knowing where contextual and supplementary information is located and what visualization they are associated with is often difficult, if not impossible.

“In content-dense web pages, knowing where contextual and supplementary information is located and what visualization they are associated with is often difcult, if not impossible.”

The proportion of survey respondents who strongly agreed that tactile graphics are helpful (75/126, 59%) is higher than for audio-based methods (33/127, 26%).

“The proportion of respondents who strongly agreed that tactile graphics are helpful (75/126, 59%) is higher than for audio-based methods (33/127, 26%).”

Of the major groups of visualization types audited, tables, particularly ones that used standard HTML tagging, were found to be most accessible.

“Of the major groups of visualization types audited, tables, particularly ones that used standard HTML tagging, were found to be most accessible.”

For several websites with multiple visualizations, no delimiter was provided to screen reader users, making it difficult for users to identify which visualization the data belongs to.

“Auditors described how for several websites with multiple visualizations, no delimiter was provided to screen reader users, making it difcult for users to identify which visualization the data belongs to.”

All survey users (127/127, 100%) relied on a screen reader as their primary assistive technology for accessing information on the web.

“All users (127/127, 100%) relied on a screen reader as their primary assistive technology for accessing information on the web.”

The location of the screen reader focus is often not conveyed.

“Additionally, the location of the screen reader focus is often not conveyed.”

Auditors often described using implicit information such as position and order of announcement to infer meaning that was not directly available for data points in bar and line graphs.

“Auditors often described using implicit information such as position and order of announcement to infer meaning that was not directly available for data points in bar and line graphs.”

A few visualizations were not perceivable through screen reader at all.

“A few visualizations were not perceivable through screen reader at all.”

In comparison, tables provided direct access to the data points and their spatial relation in the grid.

“In comparison, tables provided direct access to the data points and their spatial relation in the grid.”

Pop-up blockers prevented A1 from perceiving the accessibility overlay of one website.

“Pop-up blockers also prevented A1 from perceiving the accessibility overlay of one website.”

When asked about preferred methods for accessing data graphics, the most frequently indicated method was tactile graphics (n = 50), followed by Braille (n = 25), audio (n = 15), screen reader (n = 7), and sighted assistance (n = 4).

“When asked about preferred methods for accessing data graphics, the most frequently indicated method was through tactile graphics (n = 50), followed by Braille (n = 25), then audio (n = 15), screen reader (n = 7), and sighted assistance (n = 4).”

Navigation was especially challenging, as audited websites often contained multiple visualizations across many sections that may require hundreds of interactions to read through.

“Navigation was especially challenging, as audited websites often contained multiple visualizations across many sections that may require hundreds of interactions to read through.”

Information about the data source, update frequency, download options, contextual summaries, and tabular alternatives were often not grouped with the visualization, but scattered throughout the page and difficult to access and associate with the visualization when using a screen reader.

“Information about the data source, update frequency, download options, contextual summaries, and tabular alternatives were often not grouped with the visualization, but scattered throughout the page and difcult to access and associate with the visualization when using a screen reader.”

63% of survey respondents answered positively that having access to data-driven news articles is very important (26/126, 21%) to extremely important (54/126, 43%).

“Figure 7(a) shows that 63% of respondents answered positively that having access to data-driven news articles is very important (26/126, 21%) to extremely important (54/126, 43%).”

Auditors observed infrastructural challenges that increased the barrier for accessing and understanding visualizations.

“Auditors observed infrastructural challenges that also increased the barrier for accessing and understanding visualizations.”

Information about the presence of visualizations was often not explicitly made available to screen reader users.

“Information about the presence of visualizations was often not explicitly made available to screen reader users.”

Most Top Results visualizations do not provide adequate summaries or access to tables of the underlying data.

“Several visualizations were conveyed through dynamic dashboards produced from libraries without screen reader support, and most do not provide adequate summaries or access to tables of the underlying data.”

Over 77% of survey respondents somewhat or strongly agreed that tactile and audio-based methods are helpful.

“Over 77% somewhat or strongly agreed that these methods are helpful.”

No survey respondents reported interaction with data graphics using SVG elements.

“we did not hear from any survey respondents about their interaction with data graphics using SVG elements, which have become prevalent with interactive web visualizations.”

Several participants used linked lists or search functionality to access specific pieces of information relevant to the task.

“Several participants then made use of linked lists (P2-P4) or search functionality (P2, P7-P12) to access specifc pieces of information relevant to the task, such as U.S. states or questions that are relevant to making a comparison.”

Participants used a diverse number of access-technology and representation-based strategies to understand how data was presented, what data conveyed, and to make comparisons to complete tasks.

“Participants used a diverse number of access-technology and representation-based strategies to understand how the data was presented, what the data conveyed, and to make comparisons to complete the tasks.”

At the onset of the pandemic in the United States (March-April), most participants checked for daily data updates.

“At the onset of the pandemic in the United States (March-April), most participants checked for daily data updates (P7-P12).”

Many participants used headings to gain overviews of web pages or representations before visiting specific pieces of information.

“When approaching new websites, many participants used headings to gain overviews of web pages or representations (P1, P3-P6, P9, P10, P12) before visiting specifc pieces of information.”

Participants without prior knowledge of sonification drew false comparisons by directly comparing pitch values across different plots.

“Participants without prior knowledge of sonifcation drew false comparisons by directly comparing pitch values across diferent plots.”

In contrast, screen readers are the most commonly described access method.

“In contrast, screen readers are the most commonly described access method.”

Accessible data representations often did not provide all the utility participants were looking for.

“While participants were able to access the data and information in diferent ways across all the websites visited, accessible data representations often did not provide all the utility participants were looking for.”

Website coding revealed that none of the websites provided comprehensive descriptions as suggested by guidelines for data graphics and indicated as important by respondents.

“Our website coding revealed that none of the websites provided comprehensive descriptions as suggested by guidelines for data graphics [56] and indicated as important by respondents.”

• cites [56] 56. Joshua Hailpern, Loretta Guarino-Reid, Richard Boardman, and Srinivas Annam. 2009. Web 2.0: Blind to an accessible new world. In Proceedings of the 18th International Conference on World Wide Web. 821-830. sourceNot checked — no full text of the source

When asked to rank the accessibility of types of information, survey respondents ranked global trends and local trends as most commonly inaccessible, followed by information on the severity of the pandemic, day-to-day advice, and health and safety guidelines.

“When asked to rank the accessibility of types of information, respondents ranked global trends and local trends as most commonly inaccessible, followed by information on the severity of the pandemic, day-to-day advice, and health and safety guidelines (Figure 7(e)).”

Most participants read through all column headers to first understand table format and content before exploring specific values.

“For tables, most participants read through all of the column headers to frst understand the table format and content (P1, P3, P5, P6, P9, P10, P12) before exploring specifc values.”

In the other strategy, participants made individual comparisons across parameters and synthesized these comparisons.

“In the other strategy, participants made individual comparisons across parameters and synthesized these comparisons (P1, P3, P7, P10-P12).”

Many participants used multiple representations to confirm and clarify information.

“Many participants used these multiple representations to confrm and clarify information, such as using alt text to validate what they heard through sonifcation (P4-P7, P10, P11), tables to clarify trends provided in alt text summaries (P3, P6, P12), and summaries to understand the sonifed range (P8).”

Orientation and mobility-related tasks was the most popular response for both tactile and audio-based modalities, followed by data-related media, education related, work related, personal tasks, and finally tasks related to art and music.

“Orientation and mobility-related tasks was the most popular response for both modalities (e.g., understanding layouts, support route navigation), followed by datarelated media (e.g., news, fnance, public health, scientifc journals), then education related (e.g., understanding academic subjects, learning instructions), work related (e.g., managing timelines and people, completing work-related tasks”

Several participants described experiencing a large range of websites that provided COVID-19 data in search of the more accessible and useful ones.

“Several participants described how they had experienced a large range of websites that provided COVID-19 data”

We observed two types of strategies used for making comparisons between states.

“On a high level, we observed two types of strategies used for making the comparison.”

More than half of the study participants had specific websites they recurrently accessed to gain up-to-date information about the pandemic.

“More than half of the study participants (P1, P7-P12) had specifc websites they recurrently accessed to gain up-to-date information about the pandemic.”

The majority of survey responses related to improving access to data-driven graphs and statistics (77/98, 79%).

“The majority of responses related to improving access to data-driven graphs and statistics (77/98, 79%).”

Other participants described having to actively seek out and complement multiple sources of information to get a fuller picture.

“Other participants described having to actively seek out and complement multiple sources of information to get a fuller picture since”

At the time this study was conducted (September), some participants had reduced their frequency of checking data to a weekly to monthly basis.

“At the time this study was conducted (September), some had reduced their frequency to a weekly to monthly basis (P1, P8, P9, P12).”

Several participants used tables to explore trend progression by first sorting the table by date, then moving down a parameter column to track how that parameter changed over time.

“Several participants used tables to explore the progression of the trend by frst sorting the table by date, then moving down a parameter column to track how that parameter changed over time (P1, P3-P5, P9-P12).”

Gaps in knowledge and unfamiliar domain-specific terminology caused participants to misinterpret or limit their understanding of the data.

“However, gaps in knowledge and unfamiliar domain-specifc terminology caused participants to misinterpret or limit their understanding of the data.”

Participants leveraged their prior data literacy knowledge and experience to effectively navigate and understand data representations.

“Participants leveraged their prior data literacy knowledge and experience to efectively navigate and understand data representations.”

Born Accessible visualizations typically did not contain many of the interactive features that Top Results visualizations supported for visual consumption (55%, 42/76 vs 89%, 68/76).

“Yet, Born Accessible visualizations typically did not contain many of the interactive features (55%, 42/76) Top Results visualizations (89%, 68/76) supported for visual consumption.”

Our systematic review shows that only a few of the audited top-ranked Google search results visualizations on the web provide screen reader access to content, context, effective navigation, and interactive features.

“Our systematic review shows that only a few of the audited top-ranked Google search results visualizations on the web (in the Top Results group) provide screen reader access to content, context, efective navigation, and interactive features [RQ1].”

Webpages and dashboards with poor semantic structure were a barrier for discovering information related to the visualization.

“On the other hand, webpages and dashboards with poor semantic structure were a barrier for discovering information related to the visualization, such as source, update frequency, and download links, and potentially the visualization itself, especially when navigation through the entire page can take hundreds of commands.”

Most participants used textual descriptions to gather contextual information missing from sonification.

“Most participants also used textual descriptions to gather contextual information missing from the sonifcation (P2-P8, P11, P12) in two ways: by retrieving both trends and snapshot summaries of current statistics and by contextualizing what they heard using the alt text description of axes ranges.”

Many Top Results visualizations used visual cues to highlight broader patterns, drill down on specific regions, make comparisons, or gain additional details through click and hover-over actions.

“Many of the Top Results visualizations used visual cues to highlight broader patterns, drill down on specifc regions, make comparisons, or gain additional details through click and hover-over actions, which can help users gain broader levels of insight [20,68].”

Almost all participants leveraged insights from multiple representations to complement information gaps of each individual representation.

“We found that almost all participants leveraged insights from multiple representations to complement information gaps of each individual representation (P3-P12).”

Without ready access to these types of interactions for screen reader users even in Born Accessible visualizations, several contextual inquiry participants resorted to external workarounds such as frequently using find queries and opening multiple tabs to focus on and compare between data points and patterns.

“Without ready access to these types of interactions for screen reader users even in Born Accessible visualizations, we observed several contextual inquiry participants resorting to external workarounds such as frequently using “fnd” queries and opening multiple tabs to focus on and compare between data points and patterns.”

Using existing technologies to support this process often requires users to retain multiple pieces of information in their working memory as they navigate between the different representations.

“However, using existing technologies to support this process often requires users to retain multiple pieces of information in their working memory as they navigate between the diferent representations.”

Participants adopted coping strategies or workarounds such as using link lists to jump between table values tagged as links or opening multiple tabs.

“Participants adopted coping strategies or workarounds [77] by using link lists to jump between table values tagged as links or opening multiple tabs and leaving their screen reader focus at diferent parts of the same dataset so they could easily resume exploration (P3, P9, P12).”

• cites [77] 77. Narges Mahyar, Sung-Hee Kim, and Bum Chul Kwon. 2015. Towards a taxonomy for evaluating user engagement in information visualization. In Workshop on Personal Visualization: Exploring Everyday Life.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

A number of participants used broader-level representations such as sonification and alt text to contextualize and inform more detailed explorations of the table.

“A number of participants used representations of broader-level information, such as sonifcation and alt text, to contextualize and inform more detailed explorations of the table (P3, P6, P7, P9-P11).”

Both auditors and contextual inquiry respondents appreciated the proper use of headers to delineate between individual groups of visualizations.

“On one hand, both auditors and contextual inquiry respondents appreciated the proper use of headers to delineate between individual groups of visualizations; this helped auditors perceive individual representations and contextual inquiry participants navigate between multiple representations to synthesize information from each representation.”

Audit results highlight the onus still placed on screen reader users to install add-ons or learn specific sets of screen reader commands to gain access to visualizations.

“Audit results highlight the onus still placed on screen reader users to install add-ons or learn specifc sets of screen reader commands to gain access to visualizations.”

The majority of survey respondents (≥ 77%) agreed that tactile and audio-based methods are helpful for exploring data-driven graphics.

“The majority of respondents from our survey (≥ 77%) agreed that tactile and audiobased methods are helpful for exploring data-driven graphics.”

For web audio-based methods, participants described how the amount of information presented made comparisons difficult.

“However, for web audio-based methods, participants described how the amount of information presented made comparisons difcult (P1-P4, P6, P8-P12).”

Several contextual inquiry participants described how interpreting data through text summaries requires additional cognitive load.

“Text summaries provided through screen readers are inherently sequential in nature and several contextual inquiry participants described how interpreting data through them requires additional cognitive load.”

Participants often found summaries of key metrics and main takeaways to be the most accessible and digestible.

“Participants often found summaries of key metrics and main takeaways to be the most accessible and digestible (P2, P5-P7), which we recommend public service websites to prioritize at the top of pages populated with data content.”

Users leveraged other representations to fill in the gaps inherent in one, altogether gaining a more holistic understanding.

“We observed how users leveraged other representations to fll in the gaps inherent in one, altogether gaining a more holistic understanding.”

Participants were less confident using representations where they relied on somebody else's subjective interpretations, such as alt text of data graphics.

“Participants were less confdent using representations where they relied on somebody else's subjective interpretations, such as alt text of data graphics (P3, P5, P10, P12).”

Participants had diverse preferences for representations, modalities, and levels of abstraction to understand information and make a decision.

“Participants had diverse preferences for representations, modalities, and levels of abstraction to understand the information and make a decision.”

Unfamiliar representations such as sonification reduced participants' confidence in their interpretation.

“Unfamiliar representations such as sonifcation reduced participants' confdence in their interpretation (P1, P2, P8, P9, P10).”

We observed how the cognitive load of comparing sonifications sequentially can also be difficult.

“We observed how the cognitive load of comparing sonifcations sequentially can also be difcult.”

Contextual inquiry participants made use of the overview then explore process to understand information presented in Born Accessible websites.

“In this study, we observed how contextual inquiry participants made use of the overview then explore process to understand information presented in Born Accessible websites, and how auditors described ways in which poor reading order, scattered contextual information, and lack of information to “situate” interactions were barriers towards understanding.”

Confidence played a role in participants' exploration and use of data in their decision-making.

“Confdence played a role in participants' exploration and use of data in their decision-making.”

In the Top Results group, auditors rated tables significantly higher than maps.

“In the Top Results group, auditors rated tables significantly higher than maps.”

The majority of survey respondents reported they encounter data-driven media regularly or all the time.

“The majority of survey respondents reported (a) access to data-driven new articles is very or extremely important, (b) they encounter data-driven media regularly or all the time, (c) they encounter data-driven media that is typically not accessible with their use of technology, and (d) they have concerns about accessing local COVID-19 data and graphics in a timely manner.”

We observed from both the survey and contextual inquiry ways in which community groups, allies, and blindness organizations strove to fill important information access gaps for BVI users.

“We also observed from both the survey and contextual inquiry ways in which community groups, allies, and blindness organizations strove to fll important information access gaps for BVI users.”

When BVI users encountered inaccessible graphics in news articles, many complemented their understanding with information and data from community and blindness organizations.

“When BVI users encountered inaccessible graphics in news articles, many complemented their understanding with information and data from community and blindness organizations.”

Pass rates for content, context, navigation, and interactivity criteria were significantly higher for Born Accessible visualizations than for Top Results visualizations.

“Pass rates for content, context, navigation, and interactivity criteria were significantly higher for Born Accessible visualizations than for Top Results visualizations.”

Our audit reveals that interactive features that enable learning through consumption are currently not accessible through screen readers.

“However, our audit reveals that interactive features that enable learning through consumption are currently not accessible through screen readers.”

Respondents indicated a variety of use cases for maps, graphs, and charts if readily available and affordable.

“Uses of tactile and audio-based spatial graphics: respondents indicated a variety of use cases for maps, graphs, and charts if readily available and afordable.”

Linear regression shows a strong linear relationship between criteria pass rate and the overall accessibility rating auditors assigned to audited visualizations.

“Linear regression shows a strong linear relationship between criteria pass rate and the overall accessibility rating auditors assigned to audited visualizations.”

Audits of 12 shared visualizations show near-substantial to substantial agreement between auditors.

“Audits of 12 shared visualizations show near-substantial to substantial agreement between auditors.”

The majority of survey respondents reported that access to data-driven news articles is very or extremely important.

“The majority of survey respondents reported (a) access to data-driven new articles is very or extremely important, (b) they encounter data-driven media regularly or all the time, (c) they encounter data-driven media that is typically not accessible with their use of technology, and (d) they have concerns about accessing local COVID-19 data and graphics in a timely manner.”

Respondents reported local trends and global trends as most commonly inaccessible types of COVID-19 information.

“(e) Respondents reported local trends and global trends as most commonly inaccessible types of COVID-19 information.”

Differences in ratings between Born Accessible and Top Results visualizations were statistically significant.

“Diferences in ratings between the two groups were statistically significant.”

The majority of survey respondents indicate that tactile and audio-based methods are not regularly used to explore maps, graphs, and charts.

“The majority of survey respondents indicate that (a) tactile and audio-based methods are helpful for exploring data-driven graphics, (b) that their level of skill in interpreting data graphs and charts is competent or above for tactile but the same was not indicated for audio-based methods, and (c) tactile and audio-based methods are not regularly used to explore maps, graphs, and charts.”

The results from our audit and survey confirm that data visualization remains largely inaccessible to BVI users.

“However, the results from our audit and survey confrm that data visualization remains largely inaccessible to BVI users.”

The majority of survey respondents reported they have concerns about accessing local COVID-19 data and graphics in a timely manner.

“The majority of survey respondents reported (a) access to data-driven new articles is very or extremely important, (b) they encounter data-driven media regularly or all the time, (c) they encounter data-driven media that is typically not accessible with their use of technology, and (d) they have concerns about accessing local COVID-19 data and graphics in a timely manner.”

Three visualizations were detected by some auditors through their screen reader and browser combinations but not others.

“Three visualizations were detected by some auditors through their screen reader and browser combinations but not others.”

The majority of survey respondents reported they encounter data-driven media that is typically not accessible with their use of technology.

“The majority of survey respondents reported (a) access to data-driven new articles is very or extremely important, (b) they encounter data-driven media regularly or all the time, (c) they encounter data-driven media that is typically not accessible with their use of technology, and (d) they have concerns about accessing local COVID-19 data and graphics in a timely manner.”

There is a gap between what is deemed technically accessible and what is practically usable.

“In other words, there is a gap between what is deemed technically accessible and what is practically usable.”

BVI people most often rely on labels and alternative text (alt text) on web images and charts.

“BVI people most often rely on labels and alternative text (alt text) on web images and charts [24,84]”

• cites [24] 24. Yevgen Borodin, Jefrey P. Bigham, Glenn Dausch, and I. V. Ramakrishnan. 2010. More than meets the eye: A survey of screen-reader browsing strategies. In Proceedings of the International Cross Disciplinary Conference on Web Accessibility (W4A). 1-10. sourceNot checked — no full text of the source
• cites [84] 84. Meredith Ringel Morris, Jazette Johnson, Cynthia L. Bennett, and Edward Cutrell. 2018. Rich representations of visual content for screen reader users. In Proceedings of the CHI Conference on Human Factors in Computing Systems. 1-11. sourceNot checked — no full text of the source

There still lacks a systematic and detailed understanding of specific accessibility gaps in current, highly encountered visualizations and the types of content, context, navigation, and interactivity that these visualizations provide for screen reader users.

“However, there still lacks a systematic and detailed understanding of specifc accessibility gaps in current, highly encountered visualizations and the types of content, context, navigation, and interactivity that these visualizations provide for screen reader users.”

Holloway et al. found that online graphics used to communicate COVID-19 information was often missing alt text.

“Holloway et al. found that online graphics used to communicate COVID-19 information was often missing alt text [62].”

• cites [62] 62. Leona M. Holloway, Cagatay Goncu, Alon Ilsar, Matthew Butler, and Kim Marriott. 2022. Infosonics: Accessible infographics for people who are blind using sonifcation and voice. In Proceedings of the CHI Conference on Human Factors in Computing Systems. 1-13. sourceNot checked — no full text of the source

The audit exposed data accessibility gaps in the current implementations of websites.

“While the audit exposed data accessibility gaps in the current implementations of websites”

• AssumptionRests on an assumption: The 87 visualizations audited are representative of current implementations of data visualizations on the web.

Overall, the visual benefits of map-based data representations did not translate to screen readers.

“Overall, the visual benefts of map-based data representations did not translate to screen readers.”

Together, the limitations of screen reader focus make situating screen reader focus within the context of the web page and understanding changes outside of the screen reader focus difficult.

“Together, these limitations makes situating screen reader focus within the context of the web page and understanding changes outside of the screen reader focus difcult.”

• AssumptionRests on an assumption: The screen reader experience can be analogized to looking at a screen through a straw because only a small portion of the page is read at a time.

This type of presentation, which resembles screen reader experiences navigating through a table, does not provide the rich and expressive high-level information expressed through maps and graphs.

“We fnd that this type of presentation, which resembles screen reader experiences navigating through a table, does not provide the rich and expressive high-level information expressed through maps and graphs.”

The overall grades and criteria ratings that auditors provided of the visualizations likely represent between the worst-case and best-case screen-reader-to-browser combination for each visualization.

“The overall grades and criteria ratings that auditors provided of the visualizations likely represent between the worst-case and best-case screen-reader-to-browser combination for each visualization.”

The last consideration highlights how the burden still often falls on screen reader users themselves to request data in accessible formats.

“The last consideration highlights how the burden still often falls on screen reader users themselves to request data in accessible formats.”

Web accessibility is known to vary by screen reader and browser combination.

“First, web accessibility is known to vary by screen reader and browser combination [59], and auditors used diferent screen readers and browsers to perform the audit.”

• cites [59] 59. Mary Hegarty, Mike Stief, and Bonnie Dixon. 2015. Reasoning with diagrams: Towards a broad ontology of spatial thinking strategies. Space Mind: Conc. Spat. Learn. Educ. (2015), 75-98. sourceNot checked — no full text of the source

The experiences reported by BVI users in the survey provide insight into the broad impact that accessibility gaps identified in the audit can have.

“The experiences reported by BVI users in our survey provide insight into the broad impact that accessibility gaps identifed in the audit can have.”

The pandemic created a unique situation where numerous websites were developed to meet BVI users' needs by providing alternative representations to information otherwise more challenging to access through mainstream data trackers.

“The pandemic created a unique situation where numerous websites were developed to meet the needs of BVI users by providing alternative representations to information that was otherwise more challenging to access through mainstream data trackers [39].”

• cites [39] 39. Frank Elavsky, Cynthia Bennett, and Dominik Moritz. 2022. How accessible is my visualization? Evaluating visualiza- tion accessibility with Chartability. Computer Graphics Forum 41, 3 (2022), 57-70. sourceNot checked — no full text of the source

How accurately users map sound characteristics to data graphics insights depends heavily on domain-specific knowledge and websites providing sufficient context.

“While users may identify sound characteristics such as trend, frequency, interval, from the sonifcation, how accurately they map those characteristics to data graphics insights depends heavily on domain-specifc knowledge and websites providing sufcient context to easily make those associations.”

Tables that contain information about both states tend to facilitate comparisons across states.

“Tables that contain information about both states tend to facilitate these interactions.”

All survey respondents were screen reader users.

“All respondents were screen reader users.”

Usability issues in screen reader navigation can make search and retrieval difficult even when a data representation offers needed utility.

“While a data representation may ofer the utility that users need, usability issues in screen reader navigation can make search and retrieval difcult.”

For comparison tasks observed in the study, benefits of visualizations are undermined if screen reader users must keep large quantities of information in working memory to navigate between graphs.

“However, for comparison tasks observed in the study, these benefts are undermined if screen reader users are required to keep in their working memory large quantities of information just to navigate between graphs, especially when compounded with the cognitive load of retaining data and descriptions.”

Our observations may not capture how users might experience visualizations from more prominent websites that are less accessible.

“Therefore, our observations may not capture how users might experience visualizations from more prominent websites that are less accessible.”

The contextual inquiry findings may be under-representing the range of interactions BVI users may come across on the web because the study focused on data representations available based on standards and survey responses.

“Moreover since we focused on data representations that were available based on standards and survey responses, the contextual inquiry fndings may be under-representing the range of interactions BVI users may come across on the web.”

Consuming content through screen readers is largely sequential.

“Screen readers are the prevailing method for consuming information on the web, and consuming content through screen readers is largely sequential [72].”

• Also stated in: § 6.2 Towards Data Insights, Practically Useful vs. Technically Accessible Visualizations · plain

  “Text summaries provided through screen readers are inherently sequential in nature and several contextual inquiry participants described how interpreting data through them requires additional cognitive load.”

• cites [72] 72. Max Lee. 2020. Eight months into pandemic, blind Coloradans still cannot access some state and county COVID infor- mation. Retrieved from https://www.coloradoindependent.com/2020/10/15/colorado-covid-information-accessibility- blind/.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Visual cues and interactions can help users gain broader levels of insight.

“Many of the Top Results visualizations used visual cues to highlight broader patterns, drill down on specifc regions, make comparisons, or gain additional details through click and hover-over actions, which can help users gain broader levels of insight [20,68].”

• cites [20] 20. Rakesh Babu, Rahul Singh, and Jai Ganesh. 2010. Understanding blind users' web accessibility and usability problems. AIS Trans. Hum.-comput. Interact. 2, 3 (2010), 73-94.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [68] 68. Eun Mi Kim, Leslie Nabors Oláh, and Stephanie Peters. 2020. A learning progression for constructing and interpreting data display. ETS Res. Rep. Series (2020). sourceNot checked — no full text of the source

Gaining the same information is more difficult through audio-based representations than through visual and tactile graphs.

“Unlike visual and tactile graphs, where broad patterns can be readily retrieved and contextualized, gaining the same information is more difcult through audio-based representations [134].”

• cites [134] 134. Haixia Zhao, Catherine Plaisant, and Ben Shneiderman. 2005. iSonic: Interactive sonifcation for non-visual data exploration. In Proceedings of the 7th International ACM SIGACCESS Conference on Computers and Accessibility. 194-195. sourceNot checked — no full text of the source

The contextual inquiry investigated three Born Accessible websites that the audit showed to have higher accessibility ratings and criteria pass rates than visualizations in top Google search results.

“Third, the contextual inquiry investigated how participants might gain insights from three Born Accessible websites, which the audit showed to have higher accessibility ratings and criteria pass rates than visualizations that appear in the top results of Google Searches.”

Screen readers are the prevailing method for consuming information on the web.

“Screen readers are the prevailing method for consuming information on the web, and consuming content through screen readers is largely sequential [72].”

• cites [72] 72. Max Lee. 2020. Eight months into pandemic, blind Coloradans still cannot access some state and county COVID infor- mation. Retrieved from https://www.coloradoindependent.com/2020/10/15/colorado-covid-information-accessibility- blind/.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Several recent works found that users gain more insights and clearer mental images through audio data narratives that combine textual descriptions and sonification than sonification alone.

“Several recent works found that users gain more insights and clearer mental images through audio data narratives that combine textual descriptions and sonifcation than sonifcation alone [63,106].”

• cites [63] 63. Samuel Huron, Yvonne Jansen, and Sheelagh Carpendale. 2014. Constructing visual representations: Investigating the use of tangible tokens. IEEE Trans. Visualiz. Comput. Graph. 20, 12 (2014), 2102-2111. sourceNot checked — no full text of the source
• cites [106] 106. Alexa F. Siu. 2021. Advancing Access to Non-visual Graphics: Haptic and Audio Representations of 3D Information and Data. Stanford University.Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.