CoChat

A Heuristic Method to Evaluate Web Accessibility for Users With Low Vision

Open access
Acosta-Vargas, Patricia, Antonio Salvador-Ullauri, Luis, Lujan-Mora, Sergio · 2019 · IEEE Access

The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index; none of the 7 sources its claims cite is checked for support yet.

Abstract

Checking the accessibility of a website is a significant challenge for accessibility experts. Users who suffer from age-related changes, such as low vision, poor hearing, and diminishing motor skills, among others, have problems accessing the services offered by the web. Currently, there are qualitative and quantitative methods to check if a website is accessible. Most methods apply automatic tools because they are low cost, but they do not present an ideal solution. Instead, heuristic methods require manual support that will help the expert to assess accessibility by establishing severity ranges. This research used a modification of the Barrier Walkthrough method proposed by Giorgio Brajnik considering the Web Content Accessibility Guidelines 2.1. The modification consisted of including persistence to determine the severity of an accessibility barrier. This method enabled the measurement of the accessibility of websites to test a new heuristic process and to obtain sample data for analysis. The method was applied to 40 websites, including those of 30 universities in Latin America, according to the Webometrics ranking, and 10 websites among the most visited, according to Alexa ranking. With this heuristic method, the evaluators concluded that although a website is in a high-ranking position, this does not imply that it is accessible and inclusive. However, the manual method takes too long, and it is therefore too costly to solve accessibility problems. This research can serve as a starting point for future studies related to web accessibility heuristics.

Overview

CoChat Summary

TL;DR

The paper proposes and tests a modified Barrier Walkthrough heuristic, incorporating persistence, to evaluate web accessibility for users with low vision against WCAG 2.1. Across 40 websites, it finds that high ranking does not imply accessibility, with critical barriers common and no sampled site reaching an acceptable level.

Key findings

  • Of 40 websites evaluated, severity totals were 520 null (48.1%), 215 minor (19.9%), 104 significant (9.6%), and 241 critical (22.3%); excluding null severity, critical severity was 43.0%.
  • Critical-severity barriers violated WCAG 2.1 principles: 58.9% “perceivable,” 40.7% “operable,” and 0.4% “robust.”
  • Ranking correlations with severity were negative and weak for null (−0.23), minor (−0.10), and significant (−0.08), but positive and moderate for critical (0.34).
  • The most severe barriers were “color is necessary,” “insufficient visual contrast,” and “too many links”; the authors conclude no sampled website reached an acceptable level of accessibility.

Methods

  • The study modified Brajnik’s Barrier Walkthrough method with WCAG 2.1, adding persistence and widening the impact/persistence scale from 0 to 3 to assign severity.
  • Two web accessibility experts evaluated 40 home pages: 30 Latin American university websites from Webometrics and 10 most-visited websites from Alexa; five users with low vision (average age 40.8 years) participated using magnifiers.
  • The method had 10 phases, used 27 barriers for low vision users, applied UX Check, and analyzed results with descriptive statistics and correlation coefficients.

Limitations

  • The paper states the manual method takes too much time and is too costly to solve accessibility problems, and that analytical evaluation methods depend on evaluator experience and adopted guidelines.
  • The paper states the method is long and expensive for finding all accessibility barriers and suggests combining automatic tools with heuristic methods, while noting no tool can replace expert evaluation.
  • Evident from the text: the evaluation is limited to home pages of 40 websites, only users with low vision, two expert evaluators, and five users, so findings may not generalize to other pages, disabilities, or larger evaluator/user samples.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1109/access.2019.2939068

Cites

40 of 40 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

FIGURE 1 Diagram of evaluation website accessibility with a heuristic method.
FIGURE 1 Diagram of evaluation website accessibility with a heuristic method.CC-BY · From the PDF
FIGURE 2 Screenshot of UX Check when evaluating accessibility barriers.
FIGURE 2 Screenshot of UX Check when evaluating accessibility barriers.CC-BY · From the PDF
FIGURE 3 Websites with analysis of severities.
FIGURE 3 Websites with analysis of severities.CC-BY · From the PDF
Image 4
CC-BY · From the PDF
FIGURE 5 Ranking vs. null severity.
FIGURE 5 Ranking vs. null severity.CC-BY · From the PDF
FIGURE 6 Ranking vs. minor severity.
FIGURE 6 Ranking vs. minor severity.CC-BY · From the PDF
FIGURE 7 Ranking vs. significant severity.
FIGURE 7 Ranking vs. significant severity.CC-BY · From the PDF
FIGURE 8 Ranking vs. critical severity.
FIGURE 8 Ranking vs. critical severity.CC-BY · From the PDF
Image 9
CC-BY · From the PDF
Image 10
CC-BY · From the PDF
Image 11
CC-BY · From the PDF

Verification

Claims the paper makes

0 Valid · 0 Not valid · 6 Not checked · 41 Own results and interpretations

8 more claims

Create a free account to see all 47 claims and to verify the 7 references still open.

See all claims — free account
  • It may state two separate claims as one. 22
  • Probably not a substantial claim. 23
  • The quote only partly states this claim. 1

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

2 of 2

Citations

Cites

Research works printed but not linked: 10

  1. alroobaea, 2013, Generating a domain specific inspection evaluation method through an adaptive framework: A comparative study on educational Websites, Int J Hum Comput Interact, 4, 88
  2. consortium, 2016, Accessibility Usability and Inclusion
  3. brajnik, 2011, Barrier Walkthrough
  4. 2019, How Many Websites Are There Around the World?
  5. 2019, WebAIM Accessibility of AJAX Applications
  6. 2019, World Report on Disability
  7. 2019, Introduction to Web Accessibility|Web Accessibility Initiative (WAI)|W3C
  8. 2019, We are social - digital 2019 Global internet use accelerates
  9. 2019, Why You Only Need to Test with 5 Users
  10. lunn, 2009, Barriers Faced by Older Users On Static Web Pages Criteria Used In The Barrier Walkthrough Method (Technical Report), 1

Cited by

See all 94

Metadata

Authors
Acosta-Vargas, P., Antonio Salvador-Ullauri, L., & Lujan-Mora, S.
Affiliations
Universidad de Las Américas (EC)
Venue
IEEE Access
Published
2019
Volume
7
Pages
125634-125648
ISSN
2169-3536 (ISSN-L)
OpenAlex
W2972056002
Cited by
78
References
40
Citations by year
2026: 5 · 2025: 5 · 2024: 12 · 2023: 14 · 2022: 17 · 2021: 13 · 2020: 12
FWCI
30.74
Type
journal article
Language
en
Open access
Locations
  • IEEE Access (publishedVersion, cc-by)
  • LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • RUA, Repositorio Institucional de la Universidad de Alicante (Universidad de Alicante) (submittedVersion, cc-by)
  • Greater South Information System
Funders
Universidad de Las Américas, Ecuador (FGE.PAV.18.10)
Topics
  • Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
  • Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
  • Health Sciences › Health Professions › Occupational Therapy › Assistive Technology in Communication and Mobility
Keywords
web accessibility, low vision, heuristic evaluation, access barriers, university websites, Webometrics ranking

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.