Guidelines are only half of the story: accessibility problems encountered by blind users on the web
The DOI matches its Crossref record; 29 of the 31 references it cites were found in a registry or an index.
Abstract
This paper describes an empirical study of the problems encountered by 32 blind users on the Web. Task-based user evaluations were undertaken on 16 websites, yielding 1383 instances of user problems. The results showed that only 50.4% of the problems encountered by users were covered by Success Criteria in the Web Content Accessibility Guidelines 2.0 (WCAG 2.0). For user problems that were covered by WCAG 2.0, 16.7% of websites implemented techniques recommended in WCAG 2.0 but the techniques did not solve the problems. These results show that few developers are implementing the current version of WCAG, and even when the guidelines are implemented on websites there is little indication that people with disabilities will encounter fewer problems. The paper closes by discussing the implications of this study for future research and practice. In particular, it discusses the need to move away from a problem-based approach towards a design principle approach for web accessibility.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“User evaluations of 16 websites based on WCAG 2.0 showed that a large part of the problems encountered are either not solved by the given criteria or not covered at all (Power et al., 2012).”
From How compliance with web accessibility standards shapes the experiences of users with and without disabilities (2023) · cites this paper as Power et al., 2012
In the citing paper
“These considerations are in line with previous discussions that conformance to standards should be seen as a first step in web accessibility (Power et al., 2012).”
From How compliance with web accessibility standards shapes the experiences of users with and without disabilities (2023) · cites this paper as Power et al., 2012
In the citing paper
“Other scholars affirm that the WCAG 2.0 guidelines have not increased the coverage of user problems, as one would have expected, compared to the WCAG 1.0 [50].”
From Web Accessibility for People with Disabilities in the European Union: Paving the Road to Social Inclusion (2018) · cites this paper as [50]
In the citing paper
“However, many blind or low vision users face significant challenges when navigating websites with screen readers, which are tools that convert text to speech for auditory access [41].”
From From Cluttered to Clear: Improving the Web Accessibility Design for Screen Reader Users in E-commerce With Generative AI (2025) · cites this paper as [41]
In the citing paper
“While accessible, non-visual techniques to present a variety of data graphics have largely been investigated in research [43,92,93,130,134], few have been translated to standard web tools for use in practice.”
From The Accessibility of Data Visualizations on the Web for Screen Reader Users: Practices and Experiences During COVID-19 (2023) · cites this paper as [93]
Citations
Cites
(1993) · Cited by 9,879
(2013) · Congress Online · doi:10.4324/9780203494035-16 · Cited by 664
Research-based web design & usability guidelines
Shneiderman, B. (2006) · Cited by 376
Improving web accessibility: a study of webmaster perceptions
Lazar, J., Dudley-Sponaugle, A., & Greenidge, K.-D. (2004) · Computers in Human Behavior · doi:10.1016/j.chb.2003.10.018 · Cited by 326
The relationship between accessibility and usability of websites
Petrie, H., & Kheir, O. (2007) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1240624.1240688 · Cited by 297
Self-citationWeb Accessibility: Web Standards and Regulatory Compliance
Kirkpatrick, A., Rutter, R. N., Heilmann, C., Thatcher, J., & Waddell, C. D. (2006) · Cited by 191
Kelly, B., Sloan, D., Phipps, L., Petrie, H., & Hamilton, F. (2005) · Proceedings of the 2005 International Cross-Disciplinary Workshop on Web Accessibility (W4A) - W4A '05 · doi:10.1145/1061811.1061820 · Cited by 134 · cited 2×
“It can be argued that WAI’s original goal of raising awareness accessibility was achieved [17], with WCAG 1.”
BackgroundSelf-citation2015 Americas Conference on Information Systems, AMCIS 2015
Singh, K., Janz, B. D., Richardson, S. M., Agrawal, D., & Amis, J. (2015) · Americas Conference on Information Systems · Cited by 130
A longitudinal evaluation of accessibility: higher education web sites
Hackett, S., & Parmanto, B. (2005) · Internet Research · doi:10.1108/10662240510602690 · Cited by 124
Leuthold, S., Bargas-Avila, J. A., & Opwis, K. (2008) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2007.10.006 · Cited by 109 · cited 1×
“…such as the widely used guidelines from the U.S. Department of Health and Human Sciences that cite empirical research to establish the strength of evidence supporting their application [1], and to accessibility guidelines that have been defined and validated for specific user groups [20, 21].”
BackgroundA web accessibility report card for top international university web sites
Kane, S. K., Shulman, J. A., Shockley, T. J., & Ladner, R. E. (2007) · Proceedings of the 2007 international cross-disciplinary conference on Web accessibility (W4A) - W4A '07 · doi:10.1145/1243441.1243472 · Cited by 107 · cited 1×
“Even worse, recent studies show that the Web is becoming less accessible to people with disabilities over time [16].”
BackgroundApplying Web Usability Criteria for Vision-Impaired Users: Does It Really Improve Task Performance?
Leporini, B., & Paternò, F. (2008) · International Journal of Human-Computer Interaction · doi:10.1080/10447310701771472 · Cited by 97
Web not for all: a large scale study of web accessibility
Lopes, R., Gomes, D., & Carriço, L. (2010) · Proceedings of the 2010 International Cross Disciplinary Conference on Web Accessibility (W4A) · doi:10.1145/1805986.1806001 · Cited by 67
Lazar, J. K., & Greenidge, K.-D. (2005) · Universal Access in the Information Society · doi:10.1007/s10209-003-0087-1 · Cited by 66
On the testability of WCAG 2.0 for beginners
Alonso, F., Fuertes, J. L., González, Á. L., & Martínez, L. (2010) · Proceedings of the 2010 International Cross Disciplinary Conference on Web Accessibility (W4A) · doi:10.1145/1805986.1806000 · Cited by 65
Testability and validity of WCAG 2.0: the expertise effect
Brajnik, G., Yesilada, Y., & Harper, S. (2010) · Proceedings of the 12th international ACM SIGACCESS conference on Computers and accessibility · doi:10.1145/1878803.1878813 · Cited by 61
Auditing accessibility of UK Higher Education web sites
Sloan, D., Gregor, P., Booth, P., & Gibson, L. (2002) · Interacting with Computers · doi:10.1016/s0953-5438(01)00056-x · Cited by 59 · cited 1×
“Evaluations using automated tools covering a small subset of the guidelines [13, 14, 33], expert evaluations using a combination of automated testing tools and human judgment [19, 30], and user evaluations with disabled participants [4, 10, 24] all found the level of accessibility of web pages to be extremely low in b…”
MethodAn exploratory study of the accessibility of state government Web sites
Goette, T., Collier, C., & White, J. D. (2006) · Universal Access in the Information Society · doi:10.1007/s10209-006-0023-2 · Cited by 40
Rømen, D., & Svanæs, D. (2008) · Proceedings of the 5th Nordic conference on Human-computer interaction: building bridges · doi:10.1145/1463160.1463238 · Cited by 36
Powell, T. A. (2000) · Cited by 33
Accessibility and design: a failure of the imagination
Regan, B. (2004) · Proceedings of the international cross-disciplinary workshop on Web accessibility - W4A · doi:10.1145/990657.990663 · Cited by 32
Web accessibility web standards and regulatory compliance
Thatcher, J. (2006) · Cited by 28
Evaluation of guidelines for designing accessible Web content
Colwell, C., & Petrie, H. (2001) · ACM SIGCAPH Computers and the Physically Handicapped · doi:10.1145/501078.501082 · Cited by 23
Self-citationAn assessment of Web accessibility of UK accountancy firms
Williams, R. B., & Rattray, R. (2003) · Managerial Auditing Journal · doi:10.1108/02686900310500460 · Cited by 22
GENERAL SERVICES ADMINISTRATION
(1963) · Elsevier eBooks · doi:10.1016/b978-0-12-395579-1.50026-7 · Cited by 17
Babu, R., & Singh, R. (2009) · Americas Conference on Information Systems · Cited by 13
Petrie, H., Badani, A., & Bhalla, A. (2005) · Electronic workshops in computing · doi:10.14236/ewic/ad2005.12 · Cited by 12
Self-citationMaking Accessibility Guidelines Usable
Donnelly, A., & Magennis, M. (2003) · Lecture notes in computer science · doi:10.1007/3-540-36572-9_4 · Cited by 11
Remote Evaluation of WCAG 2.0 Techniques by Web Users with Visual Disabilities
Power, C., Petrie, H., Freire, A. P., & Swallow, D. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-21672-5_32 · Cited by 9
Self-citation
Research works printed but not linked: 6
- Equality Act 201 0. Government Equalities Office , 2010 . Equality Act 2010. Government Equalities Office, 2010.
- Disability Rights Commission . The Web : access and inclusion for disabled people - A formal Investigation conducted by the Disability Rights Commission . Disability Rights Commission , 2004 , Disability Rights Commission. The Web: access and inclusion for disabled people - A formal Investigation conducted by the Disability Rights Commission. Disability Rights Commission, 2004,
- 10.5555/1765426.1765431 · doi:10.5555/1765426.1765431
- Freed , F. and Rothberg , M . Accessible Digital Media Guidelines , National Centre for Accessible Media Website , ( 2006 ). Retrieved from http://ncam.wgbh.org/invent_build/web_multimedia/accessible-digital-media-guide on Jan. 15, 2012. Freed, F. and Rothberg, M. Accessible Digital Media Guidelines, National Centre for Accessible Media Website, (2006). Retrieved from http://ncam.wgbh.org/invent_build/web_multimedia/accessible-digital-media-guide on Jan. 15, 2012.
- 10.5555/2227734.2227751 · doi:10.5555/2227734.2227751
- 10.5555/2022591.2022625 · doi:10.5555/2022591.2022625
Cited by
GenderMag: A Method for Evaluating Software's Gender Inclusiveness
Burnett, M., Stumpf, S., Macbeth, J., Makri, S., Beckwith, L., Kwan, I., Peters, A., & Jernigan, W. (2016) · Interacting with Computers · doi:10.1093/iwc/iwv046 · Cited by 293
(2017) · doi:10.1145/2998181.2998364 · Cited by 258
“RELATED WORK There has been a long line of research addressing web accessibility issues, mostly focusing on the usability of specific websites, products, and platforms [6, 13, 15, 16, 17, 18, 21, 23, 25].”
BackgroundVigo, M., Brown, J., & Conway, V. (2013) · Proceedings of the 10th International Cross-Disciplinary Conference on Web Accessibility · doi:10.1145/2461121.2461124 · Cited by 215
“on guidelines alone leaves out a number of issues as it was found that guidelines do not cover the whole spectrum of accessibility problems, as they only cover around 50% of accessibility problems encountered by users [19].”
BackgroundHow Blind People Interact with Visual Content on Social Networking Services
Voykinska, V., Azenkot, S., Wu, S., & Leshed, G. (2016) · Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing · doi:10.1145/2818048.2820013 · Cited by 174
“Further, even applying web accessibility guidelines does not necessarily improve accessibility [30].”
BackgroundThe Current Status of Accessibility in Mobile Apps
Yan, S., & Ramachandran, P. G. (2019) · ACM Transactions on Accessible Computing · doi:10.1145/3300176 · Cited by 165
“The effectiveness of an automated tool depends on its coverage and correctness [5, 20].”
BackgroundMethodResult
Metadata
- Authors
- Power, C., Freire, A., Petrie, H., & Swallow, D.
- Affiliations
- University of York (GB)
- Venue
- Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2012-05-05
- Pages
- 433-442
- OpenAlex
- W2068293943
- Cited by
- 327
- References
- 31
- Citations by year
- 2026: 18 · 2025: 27 · 2024: 32 · 2023: 23 · 2022: 25 · 2021: 31 · 2020: 29 · 2019: 20 · 2018: 29 · 2017: 24 · 2016: 17 · 2015: 22 · 2014: 14 · 2013: 15 · 2012: 1
- FWCI
- 28.37
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Topics
- Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Health Sciences › Health Professions › Occupational Therapy › Assistive Technology in Communication and Mobility
- Keywords
- web accessibility, blind users, visual impairment, WCAG 2.0, accessibility guidelines, user problems, design principles
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.