CoChat
Sign Up
Cited by 38Digital Accessibility for DisabilitiesData Visualization and AnalyticsUsability and User Interface Design

Methodology for heuristic evaluation of the accessibility of statistical charts for people with low vision and color vision deficiency

Open access
Alcaraz Martínez, Rubén, Turró, Mireia Ribera, Granollers Saltiveri, Toni·2022·Universal Access in the Information Society
DOI 10.1007/s10209-021-00816-0
Open this paper in CoChat — free accountOpen PDF

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/s10209-021-00816-0

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Digital content and online learning platforms are regularly found to be inaccessible.

    “Furthermore, even if digital learning materials are often better than printed ones handed in classrooms (Fichten et al., 2020), digital content and online learning platforms are regularly found to be inaccessible (Alcaraz Martínez et al., 2021;García-González et al., 2021).”

    From Exploring the role of generative AI in higher education: Semi-structured interviews with students with disabilities (2025) · cites this paper as Alcaraz Martínez et al., 2021

    The Introduction (page unavailable) says accessible statistical charts for visually impaired users remain scarce, and the Validation stage reports multiple accessibility problems in the charts evaluated. This supports the point for a specific type of digital content, but the paper does not establish that digital content generally or online learning platforms are regularly inaccessible. (LLM, conf=0.87)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 64

  1. Sarsenbayeva Z Van Berkel N, Luo C, Kostakos V, Goncalves J (2017) Challenges of situational impairments during interaction with mobile devices. In Proceedings of the 29th Australian Conference on Computer-Human Interaction (OZCHI '17). ACM, New York, NY, USA, pp 477-481. http://dx.doi.org/10.1145/3152771.3156161 · DOI
  2. Reinecke K, Flatla DR, Brooks C (2016) Enabling designers to foresee which colors users cannot see. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, pp 2693-2704. http://dx.doi.org/10.1145/2858036.2858077 · DOI
  3. Yesilada Y, Brajnik G, Harper S (2011). Barriers common to mobile and disabled web users. Interact Comput 23, 5:525-542. http://dx.doi.org/10.1016/j.intcom.2011.05.005 · DOI
  4. Evergreen S (2018) Presenting data effectively: communicating your ndings for maximum impact. SAGE, Thousand Oaks, California
  5. Evergreen S, Metzner C (2013) Design principles for data visualization in evaluation. In Azzam T, Evergreen S (Eds.) Data visualization, part 2. New directions for evaluation 2013(140):5-20
  6. Sanjines SCP (2018) Does it make a difference? data visualizations and the use of research and evaluation reports. Dissertation, University of Hawai'i at Manoa. https://scholarspace.manoa.hawaii.edu/bitstream/10125/62374/2018-05-phd-sanjines.pdf
  7. Splendiani B (2015) A proposal for the inclusion of accessibility criteria in the authoring work ow of images for scienti c articles. Dissertation, University of Barcelona. http://hdl.handle.net/10803/386242
  8. Splendiani B, Ribera M (2014). Accessible images in computer science journals. Procedia computer science, 27, pp. 9-18. https://doi.org/10.1016/j.procs.2014.02.003 · DOI
  9. Simon S, Becker BA, Hamouda, S, McCartney R, Sanders k, Sheard J (2019) Visual portrayals of data and results at ITiCSE. In ITiCSE '19: Proceedings of the 2019 ACM Conference on Innovation and Technology in Computer Science Education, pp 51-57. https://doi.org/10.1145/3304221.3319742 · DOI
  10. Sim G, Read JC, Cockton G (2009) Evidence based design of heuristics for computer assisted assessment. In: Proceedings of Human-Computer Interaction INTERACT 2009, pp 204-216
  11. Nielsen J (1994). Enhancing the explanatory power of usability heuristics. Proc. ACM CHI'94 Conf. (Boston, MA, April 24-28), pp 152-158
  12. Perlman G (1997) Practical usability evaluation. In CHI '97 Conference on Human Factors in Computing Systems. Association for Computing Machinery, Atlanta, GA, pp 168-169
  13. Hub M, Čapková V (2010) Heuristic evaluation of usability of public administration portal. In Proceedings of the International Conference on Applied Computer Science, pp 234-239
  14. Van Greunen D, Yeratziotis A, Pottas D (2011) A three-phase process to develop heuristics. In Proceedings of the 13th Anual Conference on WWW Applications, Johannesburg
  15. H Rusu C, Roncagliolo S, Rusu V, Collazos C (2011) A methodology to establish usability heuristics. In Proceedings of the Fourth International Conference on Advances in Computer-Human Interactions, ACHI2011, pp 59-62
  16. H Lechner B, Fruhling A, Petter S, Siy H (2013) The chicken and the pig: user involvement in developing usability heuristics. In Proceedings of the Nineteenth Americas Conference on Information Systems
  17. H Franklin F, Breyer F, Kelner J (2014) Heurísticas de usabilidade para sistemas colaborativos remotos de realidade aumentada. In Proceedings of XVI Symposium on Virtual and Augmented Reality, pp 53-62
  18. Hermawati S, Lawson G (2015) A user-centric methodology to establish usability heuristics for speci c domains. In Proc of the International Conference on Ergonomics & Human Factors, pp 80-85. https://doi.org/10.1201/b18293 · DOI
  19. Jiménez C, Allende Cid H, Figueroa I (2017) PROMETHEUS: PROcedural METhodology for developing HEuristics of USability. IEEE Lat Am T, 15(3):541-549. https://doi.org/10.1109/TLA.2017.7867606 · DOI
  20. Quiñones D, Rusu C, Rusu V (2018) A methodology to develop usability/user experience heuristics. Comput Stand Inter 59:109-129. https://doi.org/10.1016/j.csi.2018.03.002 · DOI
  21. Guarino L, Snow-Weaver A (2009) WCAG 2.0 for designers: beyond screen readers and captions. In Stephanidis C. (eds) Universal Access in Human- Computer Interaction. Applications and Services. UAHCI 2009. Lecture Notes in Computer Science, vol 5616. Springer, Berlin, Heidelberg, pp 674-682. https://doi.org/10.1007/978-3-642-02713-0_71 · DOI
  22. H Szpiro SFA, Hashash S, Zhao Y, Azenkot S (2016) How People with low vision access computing devices: understanding challenges and opportunities. In ASSETS '16 Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility, pp 171-180. https://dl.acm.org/doi/10.1145/2982142.2982168 · DOI
  23. Elzer S, Carberry S, Chester D, Demir S, Green N, Zukerman I, Trnka K (2007) Exploring and exploiting the limited utility of captions in recognizing intention in information graphics. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL'05), pp. 223-230
  24. Yu H, Agarwal S, Johnston M, Cohen A (2009) Are gure legends su cient? evaluating the contribution of associated text to biomedical gure comprehension. J Biomed Discov Collab 4(1). https://dx.doi.org/10.1186%2F1747-5333-4-1 · DOI
  25. Cohen WW, Wang R, Murphy RF (2003) Understanding captions in biomedical publications. In Proc of the ninth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp 499-504. https://doi.org/10.1145/956750.956809 · DOI
  26. Kna ic CN (2015) Storytelling with data: a data visualization guide for business professionals. Wiley, New Jersey.
  27. Elzer S, Schwartz E, Carberry S, Chester D, Demir S, Wu P (2008) Accessible bar charts for visually impaired users. In Proc. of the IASTED International Conf. on Telehealth/Assistive Technologies, pp 55-60
  28. Elzer S, Carberry S, Chester D, Demir S, Green N, Zukerman I, Trnka K (2005) Exploring and exploiting the limited utility of captions in recognizing intention in information graphics. In Proc. of the 43rd Annual Meeting of the Assoc. for Computational Linguistics, pp 223-230. http://dx.doi.org/10.3115/1219840.1219868 · DOI
  29. Mittal VO, Carenini G, Moore JD, Roth, S (1998) Describing complex charts in natural language: a caption generation system. Comput Linguist, 24(3):431-467
  30. Agarwal, S, Yu H (2009) FigSum: automatically generating structured text summaries for gures in biomedical literature. Proc. of the 2009 Anual Symposium of the American Medical Information Association (AMIA). American Medical Information Association, San Francisco, CA, pp 6-10
  31. Ault HK, Deloge JW, Lapp, RW, Morgan MJ, Barnett JR (2002) Evaluation of long descriptions of statistical graphics for blind and low vision web users. In 8th International Conference, ICCHP 2002, pp 517-526
  32. Corio M, Lapalme G (1998) Integrated generation of graphics and text: a corpus study. In Proc of the COLING-ACL Workshop on Content Visualization and Intermedia Representations (CVIR'98), pp 63-68
  33. Corio M, Lapalme G (1999) Generation of texts for information graphics. In Proc of the 7th European Workshop on Natural Language Generation (EWNLG'99), pp 49-58
  34. Legge, GE (2016) Reading digital with low vision. Visible Lang 50(2):102-125
  35. Bernard, M, Liao CH, Mills M (2001) The effects of font type and size on the legibility and reading time of online text by older adults. In Proc of the ACM/SIGCHI Conference on Human Factors in Computing Systems (CHI2001), pp 175-176 https://doi.org/10.1145/634067.634173 · DOI
  36. Rubin GS, Feely M, Perera S, Ekstrom K, Williamson E (2006) The effect of font and line width on reading speed in people with mild to moderate vision loss. Ophthal Physl Opt 26(6):545-554
  37. Bernard JB, Aguilar C, Castet E (2016) A new font, speci cally designed for peripheral vision, improves peripheral letter and word recognition, but not eye-mediated reading performance. PLoS One 11(4). https://doi.org/10.1371/journal.pone.0152506 · DOI
  38. Tinker MA (1963) The legibility of print. Iowa State University Press.
  39. Calabrese A, Bernard JB, Hoffart L, Faure G, Barouch F, Conrath J, Castet E (2010). Small effect of interline spacing on maximal reading speed in low- vision patients with central eld loss irrespective of scotoma size. Invest Ophth Vis Sci 51(2):1247-1254
  40. Blackmore-Wright S, Georgeson MA, Anderson SJ (2013) Enhanced text spacing improves reading performance in individuals with macular disease. PLoS One 8(11). https://doi.org/10.1371/journal.pone.0080325 · DOI
  41. Russell-Minda E (2007) The legibility of typefaces for readers with low vision: a research review. J Vis Impair Blind 101(7):402-415. https://doi.org/10.1177/0145482X0710100703 · DOI
  42. Henry SL (2012) Developing text customisation functionality requirements of PDF reader and other user agents. In Proc of the 13th International Conference on Computers Helping People with Special Needs (ICCHP 2012), pp 602-609
  43. Pavazza S, Pap K (2012) The alternative way of creating infographics using SVG technology. Acta graphica 23(1-2):45-56
  44. Herman I, Dardailler D (2002) SVG linearization and accessibility. Comput Graph Forum 21(4):777-786
  45. Mackinlay J (1986) Automating the design of graphical presentations of relational information. ACM Transactions on Graphics (TOG) 5:110-141
  46. Olson JM, Brewer CA (1997) An evaluation of color selections to accommodate map users with color-vision impairments. Annals of the Association of American Geographers 87(1):103-134
  47. Brewer CA (2016) Designing better maps: a guide for GIS users. ESRI Press, Redlands (Calif.)
  48. Albers D, Correll M, Gleicher M (2014) Task-driven evaluation of aggregation in time series visualization. Proc of the 32nd annual ACM conference on Human factors in Computing Systems, pp 551-560
  49. Adnan M, Just M, Baillie L (2016) Investigating time series visualizations to improve the user experience. In Proc of the 2016 CHI Conf Hum Fact Comput Systems, pp 5444-5455
  50. Correll M, Albers D, Franconeri S (2012) Comparing averages in time series data. In Proc of the SIGCHI Conference on Human Factors in Computing Systems, pp 1095-1104
  51. Kramer G (1994) Auditory display: soni cation, audi cation, and auditory interfaces. Addison-Wesley, Reading, MA.
  52. Cohen RF, Yu R, Meacham A, Skaff J (2005) PLUMB: displaying graphs to the blind using an active auditory interface. In Proc of the 7th international ACM SIGACCESS Conference on Computers and Accessibility, pp 182-183
  53. Evreinova TG (2008) Non-visual interaction with graphs assisted with directional-predictive sounds and vibrations: a comparative study. Universal Access Inf 7(1-2):93-102 https://doi.org/10.1007/s10209-007-0105-9. · DOI
  54. Alty JL, Rigas D (2005) Exploring the use of structured musical stimuli to communicate simple diagrams: the role of context. Int J Hum-Compt St 62(1):21-40
  55. Doush IA, Pontelli E, Simon D, Cao ST, Ma O (2009). Making Microsoft ExcelTM accessible: multimodal presentation of charts. Proc of the 11th international ACM SIGACCESS conference on Computers and accessibility. ACM, New York, NY, pp 147-154
  56. Petrie H, Kheir O (2007) The relationship between accessibility and usability of websites. In Proc of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2007, ACM, New York (2007). http://doi.acm.org/10.1145/1240624.1240688 · DOI
  57. Moreno L, Martínez P, Ruiz-Mezcua, B (2009) A bridge to web accessibility from the usability heuristics. In Holzinger A, Miesenberger K (eds) HCI and Usability for e-Inclusion. USAB 2009. Lecture Notes in Computer Science 5889. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-10308-7_20 · DOI
  58. Casare AR, Silva CG, Martins PS, Moraes RLO (2016) Usability heuristics and accessibility guidelines: a comparison of heuristic evaluation and WCAG. In SAC '16 Proceedings of the 31st Annual ACM Symposium on Applied Computing Pages, pp 213-215 https://doi.org/10.1145/2851613.2851913 · DOI
  59. Likert R (1932). A technique for the measurement of attitudes. Arch. Psychol. 22 (140), 1-55.
  60. Sullivan GM, Artino AR (2013) Analyzing and interpreting data from Likert-type scales. J Grad Med Educ. 5(4):541-542
  61. Sauro J, Lewis JR (2016) Quantifying the user experience: practical statistics for user research. Morgan Kaufmann, Amsterdam
  62. Alcaraz Martínez R, Ribera M, Granollers T, Pascual A (2020) ¿Son accesibles los grá cos estadísticos para personas con baja visión en la prensa digital?: una propuesta metodológica basada en heurísticas. Prof inform 29(5). https://doi.org/10.3145/epi.2020.sep.15 · DOI
  63. Alcaraz Martínez R, Ribera M (2020). An evaluation of accessibility of COVID-19 statistical charts of governments and health organisations for people with low vision. Prof inform 29(5). https://doi.org/10.3145/epi.2020.sep.14 · DOI
  64. Paddison C, Engle eld P (2004) Applying heuristics to accessibility inspections. Interact Comput 16(3):507-521. https://doi.org/10.1016/j.intcom.2004.04.007 · DOI

Cited by

Sorted by citations

Authors
Alcaraz Martínez, R., Turró, M. R., & Granollers Saltiveri, T.
ORCID
Venue
Universal Access in the Information Society
Publisher
Springer Science and Business Media LLC
Published
2021-05-10
Volume
21
Issue
4
Pages
863-894
ISSN
  • 1615-5289 (ISSN-L)
  • 1615-5297
DOI
10.1007/s10209-021-00816-0
OpenAlex
W4239501324
Cited by
38
References
1
Citations by year
2026: 8 · 2025: 6 · 2024: 9 · 2023: 7 · 2022: 5 · 2021: 1 · 2020: 2
FWCI
10.27
Type
journal article
Language
en
Open access
Licence
Locations
  • Universal Access in the Information Society (publishedVersion)
  • Research Square (acceptedVersion, cc-by)
Topics
  • Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Data Visualization and Analytics
  • Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
Keywords
statistical graphics, accessibility, low vision, color vision deficiency, heuristic evaluation, contrast sensitivity, digital accessibility, visual impairment, chart design

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review