Evaluating a Tool for Improving Accessibility to Charts and Graphs
Open accessThe DOI matches its Crossref record; 61 of the 61 references it cites were found in a registry or an index.
Abstract
This article reports a case study of the iterative design and evaluation of a natural language-driven assistive technology, iGraph -Lite, providing people who are blind access to line graphs. Two laboratory-based usability studies involving blind and sighted people are presented with a discussion of the ensuing implementation of changes. Blind participants were found to adopt different graph interrogation strategies than sighted participants. A small field study is then reported in which a blind user who works with graphs took part to determine the degree to which the iGraph -Lite commands would meet the needs of blind graph experts. The final study invited sighted graph experts and novices to visually inspect and explain a set of line graphs comparable to those used in the usability studies. It aimed to highlight the concepts and the range of words sighted people use, to ascertain the appropriateness of the iGraph -Lite lexicon. A set of preliminary guidelines is presented.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Audio interactions can also be powerfully expressive [42], spatially situated [88], help users understand semantic content [76], and complement the spatial limitations inherent to haptic interfaces.”
From The Accessibility of Data Visualizations on the Web for Screen Reader Users: Practices and Experiences During COVID-19 (2023) · cites this paper as [42]
Citations
Cites
Categorization and Representation of Physics Problems by Experts and Novices*
Chi, M. T. H., Feltovich, P. J., & Glaser, R. (1981) · Cognitive Science · doi:10.1207/s15516709cog0502_2 · Cited by 5,248 · cited 1×
“[2010]) representation of important graph information (including geometrical properties such as width, and height, and visual properties, such as colors or line styles [Chi et al. 1981].”
BackgroundThe visual display of quantitative information
(1985) · General Pharmacology: The Vascular System · doi:10.1016/0306-3623(85)90230-7 · Cited by 5,070
The eyes have it: a task by data type taxonomy for information visualizations
Shneiderman, B. (2002) · Proceedings 1996 IEEE Symposium on Visual Languages · doi:10.1109/vl.1996.545307 · Cited by 4,535
Why a Diagram is (Sometimes) Worth Ten Thousand Words
Larkin, J. H., & Simon, H. A. (1987) · Cognitive Science · doi:10.1016/s0364-0213(87)80026-5 · Cited by 3,733
Building Natural-Language Generation Systems
Reiter, E. (1996) · arXiv (Cornell University) · doi:10.48550/arxiv.cmp-lg/9605002 · Cited by 1,228
Eastman, J. R., & Bertin, J. (1986) · Economic Geography · doi:10.2307/143508 · Cited by 1,097
Building Natural Language Generation Systems
Reiter, E., & Dale, R. (2000) · Cambridge University Press eBooks · doi:10.1017/cbo9780511519857 · Cited by 772
Droughts of the Late 1980s in the United States as Derived from NOAA Polar-Orbiting Satellite Data
Kogan, F. N. (1995) · Bulletin of the American Meteorological Society · doi:10.1175/1520-0477(1995)076<0711:uaaocf>2.0.co;2 · Cited by 747
ANVIL - a generic annotation tool for multimodal dialogue
Kipp, M. (2001) · 7th European Conference on Speech Communication and Technology (Eurospeech 2001) · doi:10.21437/eurospeech.2001-354 · Cited by 570
The Visual Display of Quantitative Information
Carr, D. (1987) · Technometrics · doi:10.1080/00401706.1987.10488194 · Cited by 541
Domain specificity of spatial expertise: the case of video game players
Sims, V. K., & Mayer, R. E. (2002) · Applied Cognitive Psychology · doi:10.1002/acp.759 · Cited by 284
Choosing words in computer-generated weather forecasts
Reiter, E., Sripada, S., Hunter, J., Yu, J., & Davy, I. (2005) · Artificial Intelligence · doi:10.1016/j.artint.2005.06.006 · Cited by 262
Transfer of pattern recall skills may contribute to the development of sport expertise
Abernethy, B., Baker, J., & Côté, J. (2005) · Applied Cognitive Psychology · doi:10.1002/acp.1102 · Cited by 242
An architecture for data-to-text systems
Reiter, E. (2007) · Proceedings of the Eleventh European Workshop on Natural Language Generation - ENLG '07 · doi:10.3115/1610163.1610180 · Cited by 223
Design of a knowledge-based report generator
Kukich, K. (1983) · Proceedings of the 21st annual meeting on Association for Computational Linguistics - · doi:10.3115/981311.981340 · Cited by 202
Real versus Template-Based Natural Language Generation: A False Opposition?
Deemter, K. V., Theune, M., & Krahmer, E. (2005) · Computational Linguistics · doi:10.1162/0891201053630291 · Cited by 199
Recent advances in expertise research: a commentary on the contributions to the special issue
Ericsson, K. A. (2005) · Applied Cognitive Psychology · doi:10.1002/acp.1111 · Cited by 192
Haptic graphs for blind computer users
Yu, W. M., Ramloll, R., & Brewster, S. A. (2001) · Lecture notes in computer science · doi:10.1007/3-540-44589-7_5 · Cited by 155
NLG vs. TemplatesPreprint
Reiter, E. (1995) · arXiv (Cornell University) · doi:10.48550/arxiv.cmp-lg/9504013 · Cited by 135
Evaluation of multimodal graphs for blind people
Yu, W., & Brewster, S. (2003) · Universal Access in the Information Society · doi:10.1007/s10209-002-0042-6 · Cited by 120 · cited 1×
“2007a, 2010; Kurze 1995], and hybrid interfaces [Grabowski and Barner 1998; Kennel 1996; Yu and Brewster 2003].”
BackgroundAutomatic Generation of Textual Summaries from Neonatal Intensive Care Data
Portet, F., Reiter, E., Jim Hunter, A., & Sripada, S. G. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-73599-1_30 · Cited by 112
Feeling what you hear: tactile feedback for navigation of audio graphs
Wall, S., & Brewster, S. (2006) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1124772.1124941 · Cited by 107
Cross-Modal Equivalence of Visual and Auditory Scatterplots for Exploring Bivariate Data Samples
Flowers, J. H., Buhman, D. C., & Turnage, K. D. (1997) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872097778827151 · Cited by 107
Kennel, A. R. (1996) · doi:10.1145/228347.228357 · Cited by 104
DRAWING BY EAR: INTERPRETING SONIFIED LINE GRAPHS
Brown, L. M., & Brewster, S. A. (2003) · ENLIGHTEN (Jurnal Bimbingan dan Konseling Islam) · Cited by 98
Law, A. S., Freer, Y., Jim Hunter, A., Logie, R. H., McIntosh, N., & Quinn, J. A. (2005) · Journal of Clinical Monitoring and Computing · doi:10.1007/s10877-005-0879-3 · Cited by 97
Proceedings of the 13th European Workshop on Natural Language Generation
Gardent, C., & Striegnitz, K. (2011) · Cited by 88
Describing complex charts in natural language: a caption generation system
Mittal, V. O., Carenini, G., Moore, J. D., & Roth, S. F. (1998) · Cited by 88
Comparing two haptic interfaces for multimodal graph rendering
Yu, W., & Brewster, S. (2003) · Proceedings 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. HAPTICS 2002 · doi:10.1109/haptic.2002.998934 · Cited by 83
Improving accessibility to statistical graphs
Ferres, L., Verkhogliad, P., Lindgaard, G., Boucher, L., Chretien, A., & Lachance, M. (2007) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/1296843.1296857 · Cited by 73
Self-citationProceedings 6th European Conference on Speech Communication and Technology (Eurospeech 99)
McInnes, F. R., Attwater, D., Edgington, M. D., Schmidt, M. S., & Jack, M. A. (1999) · Cited by 62
Graphics and natural language as components of automatic explanation
Roth, S. F., Mattis, J., & Mesnard, X. (1991) · Intelligent User Interfaces · doi:10.1145/107215.128701 · Cited by 57
Getting Computers to See Information Graphics So Users Do Not Have to
Chester, D. L., & Elzer, S. (2005) · Lecture notes in computer science · doi:10.1007/11425274_68 · Cited by 52
Data sonification from the desktop: Should sound be part of standard data analysis software?
Flowers, J. H., Buhman, D. C., & Turnage, K. D. (2005) · ACM Transactions on Applied Perception · doi:10.1145/1101530.1101544 · Cited by 49
PostGraphe: A System for the Generation of Statistical Graphics and Text
Fasciano, M., & Lapalme, G. (1996) · International Conference on Natural Language Generation · doi:10.18653/v1/w96-0406 · Cited by 47
SONART : THE SONIFICATION APPLICATION RESEARCH TOOLBOX
Ben‐Tal, O., Berger, J., Cook, B. G., Daniels, M., Scavone, G. P., & Cook, P. R. (2002) · SMARTech Repository (Georgia Institute of Technology) · Cited by 45
Green, N. L., Carenini, G., Kerpedjiev, S. M., Mattis, J., Moore, J. D., & Roth, S. F. (2004) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2003.10.007 · Cited by 44
Ferres, L., Parush, A., Roberts, S., & Lindgaard, G. (2006) · Lecture notes in computer science · doi:10.1007/11788713_163 · Cited by 42
Self-citationSoundBar: exploiting multiple views in multimodal graph browsing
McGookin, D. K., & Brewster, S. A. (2006) · Proceedings of the 4th Nordic conference on Human-computer interaction: changing roles · doi:10.1145/1182475.1182491 · Cited by 41
PLUMB: displaying graphs to the blind using an active auditory interface
Cohen, R. F., Yu, R., Meacham, A., & Skaff, J. (2005) · Proceedings of the 7th international ACM SIGACCESS conference on Computers and accessibility · doi:10.1145/1090785.1090820 · Cited by 37
Non-visual overviews of complex data sets
Kildal, J., & Brewster, S. A. (2006) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/1125451.1125634 · Cited by 37
Evaluating a tool for improving accessibility to charts and graphs
Ferres, L., Lindgaard, G., & Sumegi, L. (2010) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/1878803.1878820 · Cited by 37
Self-citationElzer, S., Schwartz, E. J., Carberry, S., Chester, D. L., Demir, Ş., & Wu, P. (2007) · International Conference on Web Information Systems and Technologies · doi:10.5220/0001274600590066 · Cited by 37
Data visualization methods for the blind using force feedback and sonification
Grabowski, N. A., & Barner, K. E. (1998) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.333677 · Cited by 33
Intentions in the Coordinated Generation of Graphics and Text from Tabular Data
Fasciano, M., & Lapalme, G. (2000) · Knowledge and Information Systems · doi:10.1007/pl00011645 · Cited by 33
Doush, I. A., Pontelli, E., Simon, D. A., Son, T. C., & Ma, O. (2009) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/1639642.1639669 · Cited by 33
Generating spatio-temporal descriptions in pollen forecasts
Turner, R., Sripada, S. G., Reiter, E., & Davy, I. P. (2006) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.3115/1608974.1608998 · Cited by 29
SumTime-Turbine: A Knowledge-Based System to Communicate Gas Turbine Time-Series Data
Yu, J. S., Reiter, E., Jim Hunter, A., & Sripada, S. G. (2007) · Springer eBooks · doi:10.1007/3-540-45034-3_38 · Cited by 26
Lessons from Deploying NLG Technology for Marine Weather Forecast Text Generation
Sripada, S. G., Reiter, E., Davy, I. P., & Nilssen, K. (2004) · Cited by 22
AutoBrief: a multimedia presentation system for assisting data analysis
Kerpedjiev, S. M., Carenini, G., Roth, S. F., & Moore, J. D. (1997) · Computer Standards & Interfaces · doi:10.1016/s0920-5489(97)00022-6 · Cited by 21
Describing complex charts in natural language
MittalVibhu, O., CareniniGiuseppe, MooreJohanna, D., & RothSteven (1998) · Computational Linguistics · doi:10.5555/972749.972754 · Cited by 12
Integrated generation of graphics and text- a corpus study
Corio, M., & Lapalme, G. (1998) · Cited by 9
Evreinova, T. G., Evreinov, G., Raisamo, R., & Vesterinen, L. K. (2007) · Universal Access in the Information Society · doi:10.1007/s10209-007-0105-9 · Cited by 9
Modeling and querying graphical representations of statistical data
Dumontier, M. J., Ferres, L., & Villanueva‐Rosales, N. (2010) · Journal of Web Semantics · doi:10.1016/j.websem.2009.12.002 · Cited by 9
Self-citationSemantic Query Answering with Time-Series Graphs
Ferres, L., Dumontier, M. J., & Villanueva‐Rosales, N. (2007) · doi:10.1109/edocw.2007.28 · Cited by 7
Self-citationA syntactic analysis of accessibility to a corpus of statistical graphs
Ferres, L., Verkhogliad, P., Sumegi, L., Boucher, L., Lachance, M., & Lindgaard, G. (2008) · International Cross-Disciplinary Conference on Web Accessibility (W4A) · doi:10.1145/1368044.1368053 · Cited by 4
Self-citationDemir, Ş., Oliver, D., Schwartz, E. J., Elzer, S., Carberry, S., & McCoy, K. F. (2010) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/1878803.1878864 · Cited by 4
Model-Driven, Assertion-Based Generation of Multimedia Weather Information
Kerpedjiev, S. M. (1995) · Bulletin of the American Meteorological Society · Cited by 4
Interactive generation of overview information using speech
Kildal, J., & Brewster, S. A. (2007) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/1240866.1241029 · Cited by 3
Research works printed but not linked: 10
- 10.1175/1520-0477(1995)076<1791:MDABGO>2.0.CO;2 · doi:10.1175/1520-0477(1995)076
- Kurze , M. 1995 . Giving blind people access to graphics (example: Business graphics) . In Proceedings of the Software-Ergonomics’95 Workshop. Kurze, M. 1995. Giving blind people access to graphics (example: Business graphics). In Proceedings of the Software-Ergonomics’95 Workshop.
- 10.1111/j.1551-6708.1987.tb00863.x · DOI · Paper page
- 10.5555/1610163.1610180 · doi:10.5555/1610163.1610180
- Riedel , B. 2001 . Perception of gradient in haptic graphs: A comparison of virtual and hysical stimuli . In Conference Proceedings to Eurohaptics. Riedel, B. 2001. Perception of gradient in haptic graphs: A comparison of virtual and hysical stimuli. In Conference Proceedings to Eurohaptics.
- 10.5555/832277.834354 · doi:10.5555/832277.834354
- Sribunruangrit , N. , Marque , C. , Lenay , C. , and Gapenne , O . 2003. Improving blind people’s spatial ability by bimodal-perception assistive device for accessing graphic information . In Proceedings of the 7th Biannual Conference for the Advancement of Assistive Technology in Europe. 476--480 . Sribunruangrit, N., Marque, C., Lenay, C., and Gapenne, O. 2003. Improving blind people’s spatial ability by bimodal-perception assistive device for accessing graphic information. In Proceedings of the 7th Biannual Conference for the Advancement of Assistive Technology in Europe. 476--480.
- Sripada , S. and Gao , F . 2007. Summarizing dive computer data: A case study in integrating textual and graphical presentations of numerical data . In Proceedings of Workshop on Multimodal Output Generation. Vol. CTIT, 149-- 157 . Sripada, S. and Gao, F. 2007. Summarizing dive computer data: A case study in integrating textual and graphical presentations of numerical data. In Proceedings of Workshop on Multimodal Output Generation. Vol. CTIT, 149--157.
- Sripada , S. , Reiter , E. , and Davy , I. 2003 . Sumtime-mousam: Configurable marine weather forecast generator . Expert Update 6 , 4 -- 10 . Sripada, S., Reiter, E., and Davy, I. 2003. Sumtime-mousam: Configurable marine weather forecast generator. Expert Update 6, 4--10.
- Wells , L. R. and Landau , S . 2003. Merging of tactile sensory input and audio data by means of the talking tactile tablet . In Proceedings of Eurohaptics’03 . 414--418. Wells, L. R. and Landau, S. 2003. Merging of tactile sensory input and audio data by means of the talking tactile tablet. In Proceedings of Eurohaptics’03. 414--418.
Cited by
Mobile assistive technologies for the visually impaired
Hakobyan, L., Lumsden, J., O’Sullivan, D., & Bartlett, H. (2013) · Survey of Ophthalmology · doi:10.1016/j.survophthal.2012.10.004 · Cited by 253
Visualizing for the Non‐Visual: Enabling the Visually Impaired to Use Visualization
Choi, J., Jung, S., Park, D. G., Choo, J., & Elmqvist, N. (2019) · Computer Graphics Forum · doi:10.1111/cgf.13686 · Cited by 143
“Ferres et al. [FLST13] proposed a system that verbally describes line charts.”
BackgroundAccessible Visualization: Design Space, Opportunities, and Challenges
Kim, N. W., Joyner, S. C., Riegelhuth, A., & Kim, Y. (2021) · Computer Graphics Forum · doi:10.1111/cgf.14298 · Cited by 137
Chart-to-Text: Generating Natural Language Descriptions for Charts by Adapting the Transformer Model
Obeid, J., & Hoque, E. (2020) · Proceedings of the 13th International Conference on Natural Language Generation · doi:10.18653/v1/2020.inlg-1.20 · Cited by 91
Chart Mining: A Survey of Methods for Automated Chart Analysis
Davila, K., Setlur, S., Doermann, D., Kota, B. U., & Govindaraju, V. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2992028 · Cited by 107
“During evaluation, it is common to include both sighted and visually impaired people to test the effectiveness of the systems including their interaction capabilities [151], [152].”
BackgroundInfluential
Metadata
- Authors
- Ferres, L., Lindgaard, G., Sumegi, L., & Tsuji, B.
- Affiliations
- University of Concepción (CL)
- Swinburne University of Technology (AU)
- Carleton University (CA)
- Venue
- ACM Transactions on Computer-Human Interaction
- Published
- 2013-11
- Volume
- 20
- Issue
- 5
- Pages
- 1-32
- ISSN
- 1073-0516 (ISSN-L)
- 1557-7325
- OpenAlex
- W2020776366
- Cited by
- 67
- References
- 61
- Citations by year
- 2026: 2 · 2025: 6 · 2024: 7 · 2023: 11 · 2022: 10 · 2021: 6 · 2020: 8 · 2019: 3 · 2018: 1 · 2017: 2 · 2015: 5 · 2014: 5 · 2013: 1
- FWCI
- 16.01
- Type
- journal article
- Language
- en
- Open access
- green
- figshare.com
- Locations
- ACM Transactions on Computer-Human Interaction (publishedVersion)
- Swinburne figshare (Swinburne University of Technology) (acceptedVersion)
- Figshare (submittedVersion, cc-by)
- Swinburne figshare (Swinburne University of Technology)
- Funders
- Natural Sciences and Engineering Research Council of Canada
- International Business Machines Corporation
- Comisión Nacional de Investigación Científica y Tecnológica
- Cognos Industry Research Chair (IRCSA 234087-05)
- Topics
- Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
- Life Sciences › Neuroscience › Cognitive Neuroscience › Tactile and Sensory Interactions
- Health Sciences › Health Professions › Occupational Therapy › Assistive Technology in Communication and Mobility
- Keywords
- assistive technology, blind users, line graphs, natural language commands, accessibility guidelines, visual impairment, usability evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.