A Heuristic Method to Evaluate Web Accessibility for Users With Low Vision
Open accessThe DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index; none of the 7 sources its claims cite is checked for support yet.
Abstract
Checking the accessibility of a website is a significant challenge for accessibility experts. Users who suffer from age-related changes, such as low vision, poor hearing, and diminishing motor skills, among others, have problems accessing the services offered by the web. Currently, there are qualitative and quantitative methods to check if a website is accessible. Most methods apply automatic tools because they are low cost, but they do not present an ideal solution. Instead, heuristic methods require manual support that will help the expert to assess accessibility by establishing severity ranges. This research used a modification of the Barrier Walkthrough method proposed by Giorgio Brajnik considering the Web Content Accessibility Guidelines 2.1. The modification consisted of including persistence to determine the severity of an accessibility barrier. This method enabled the measurement of the accessibility of websites to test a new heuristic process and to obtain sample data for analysis. The method was applied to 40 websites, including those of 30 universities in Latin America, according to the Webometrics ranking, and 10 websites among the most visited, according to Alexa ranking. With this heuristic method, the evaluators concluded that although a website is in a high-ranking position, this does not imply that it is accessible and inclusive. However, the manual method takes too long, and it is therefore too costly to solve accessibility problems. This research can serve as a starting point for future studies related to web accessibility heuristics.
Overview
CoChat Summary
TL;DR
The paper proposes and tests a modified Barrier Walkthrough heuristic, incorporating persistence, to evaluate web accessibility for users with low vision against WCAG 2.1. Across 40 websites, it finds that high ranking does not imply accessibility, with critical barriers common and no sampled site reaching an acceptable level.
Key findings
- Of 40 websites evaluated, severity totals were 520 null (48.1%), 215 minor (19.9%), 104 significant (9.6%), and 241 critical (22.3%); excluding null severity, critical severity was 43.0%.
- Critical-severity barriers violated WCAG 2.1 principles: 58.9% “perceivable,” 40.7% “operable,” and 0.4% “robust.”
- Ranking correlations with severity were negative and weak for null (−0.23), minor (−0.10), and significant (−0.08), but positive and moderate for critical (0.34).
- The most severe barriers were “color is necessary,” “insufficient visual contrast,” and “too many links”; the authors conclude no sampled website reached an acceptable level of accessibility.
Methods
- The study modified Brajnik’s Barrier Walkthrough method with WCAG 2.1, adding persistence and widening the impact/persistence scale from 0 to 3 to assign severity.
- Two web accessibility experts evaluated 40 home pages: 30 Latin American university websites from Webometrics and 10 most-visited websites from Alexa; five users with low vision (average age 40.8 years) participated using magnifiers.
- The method had 10 phases, used 27 barriers for low vision users, applied UX Check, and analyzed results with descriptive statistics and correlation coefficients.
Limitations
- The paper states the manual method takes too much time and is too costly to solve accessibility problems, and that analytical evaluation methods depend on evaluator experience and adopted guidelines.
- The paper states the method is long and expensive for finding all accessibility barriers and suggests combining automatic tools with heuristic methods, while noting no tool can replace expert evaluation.
- Evident from the text: the evaluation is limited to home pages of 40 websites, only users with low vision, two expert evaluators, and five users, so findings may not generalize to other pages, disabilities, or larger evaluator/user samples.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures











Verification
Claims the paper makes
0 Valid · 0 Not valid · 6 Not checked · 41 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The evaluators identified several violations of the WCAG 2.1 guidelines, such as images without textual alternatives, broken links, and low color contrasts.”
§ A Heuristic Method to Evaluate Web Accessibility for Users With Low Vision > I. INTRODUCTION
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“With this heuristic method, the evaluators concluded that although a website is in a high-ranking position, this does not imply that it is accessible and inclusive.”
- Also stated in: § I. INTRODUCTION · hedged
“With this heuristic method, the evaluators concluded that although an institution may be in a high-ranking position, this does not necessarily imply that it has an accessible and inclusive website.”
§ A Heuristic Method to Evaluate Web Accessibility for Users With Low Vision > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on 2 assumptions not shown here.
- Also stated in: § I. INTRODUCTION · hedged
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In this research, it was observed that the pages analyzed with critical severity violate some WCAG 2.1 principles”
§ A Heuristic Method to Evaluate Web Accessibility for Users With Low Vision > IV. RESULTS ANALYSIS
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In Figure 7, showing the ranking and the significant severity, the correlation coefficient is -0.08; this implies that the correlation is very weak and negative.”
§ A Heuristic Method to Evaluate Web Accessibility for Users With Low Vision > IV. RESULTS ANALYSIS
hedgedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In Figure 6, showing the ranking and minor severity, the correlation coefficient is -0.10; this implies that the correlation is negative and weak.”
§ A Heuristic Method to Evaluate Web Accessibility for Users With Low Vision > IV. RESULTS ANALYSIS
hedgedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
8 more claims
Create a free account to see all 47 claims and to verify the 7 references still open.
- It may state two separate claims as one. 22
- Probably not a substantial claim. 23
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“One paper [68] presented a heuristic review of university websites using the UX Check Tool and a manual evaluation with two experts in web accessibility”
From Accessibility of university websites worldwide: a systematic literature review (2023) · cites this paper as [68]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ A Heuristic Method to Evaluate Web Accessibility for Users With Low Vision > III. METHOD AND CASE STUDY A. METHOD
“In the sixth phase, the evaluators defined the list of barriers according to the type of disability of the users.”
§ A Heuristic Method to Evaluate Web Accessibility for Users With Low Vision > 6) PHASE 6: APPLY THE UX CHECK TOOL
“## 6) PHASE 6: APPLY THE UX CHECK TOOL”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ A Heuristic Method to Evaluate Web Accessibility for Users With Low Vision > C. HEURISTIC METHOD BASED ON THE BARRIER WALKTHROUGH
“Brajnik [41], and Brajnik and Englefield [42] argued that analytical evaluation methods, based on manual heuristic inspection of the code, do not guarantee full accessibility; instead, they depend on the experience of the evaluator and the guidelines adopted.”
§ A Heuristic Method to Evaluate Web Accessibility for Users With Low Vision > References
“42. G. Brajnik and R. Lomuscio, ''SAMBA: A semi-automatic method for measuring barriers of accessibility,''”
Citations
Cites
Heuristic evaluation of user interfaces
Nielsen, J., & Molich, R. (1990) · Proceedings of the SIGCHI conference on Human factors in computing systems Empowering people - CHI '90 · doi:10.1145/97243.97281 · Cited by 3,533 · cited 3×
“[38] J. Nielsen and M. Rolf, ‘‘Heuristic evaluation of user interfaces,’’ in Proc.”
MethodInfluentialShakespeare, T., & Officer, A. (2011) · Disability and Rehabilitation · doi:10.3109/09638288.2011.590392 · Cited by 1,792
Web content accessibility guidelines 1.0
Chisholm, W., Vanderheiden, G., & Jacobs, I. (2001) · Interactions · doi:10.1145/379537.379550 · Cited by 984
Heuristic Walkthroughs: Finding the Problems Without the Noise
Sears, A. (1997) · International Journal of Human-Computer Interaction · doi:10.1207/s15327590ijhc0903_2 · Cited by 258 · cited 2×
“The method proposes the heuristic walkthroughs of Sears [45], considering the context of the use of the website.”
MethodInfluentialVigo, M., Brown, J., & Conway, V. (2013) · Proceedings of the 10th International Cross-Disciplinary Conference on Web Accessibility · doi:10.1145/2461121.2461124 · Cited by 215 · cited 1×
“Automatic tools cannot detect all accessibility barriers [29]; therefore, to complement this process of evaluation, applying a heuristic method is suggested, which, according to the definition, is based on the use of empirical rules to arrive at a solution.”
MethodMankoff, J., Fait, H., & Tran, T. (2005) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1054972.1054979 · Cited by 207 · cited 1×
“[37] presented a comparison of different methods to find accessibility problems that affect users with blindness.”
BackgroundAccessibility of Internet websites through time
Hackett, S., Parmanto, B., & Zeng, X. (2003) · Proceedings of the 6th international ACM SIGACCESS conference on Computers and accessibility · doi:10.1145/1028630.1028638 · Cited by 145 · cited 1×
“It included the web accessibility barrier (WAB) [36] metric to objectively summarize the results and amplify the use of this method to cover all types of evaluation tasks.”
MethodAccessibility of Indian universities’ homepages: An exploratory study
Ismail, A., & Kuppusamy, K. S. (2018) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2016.06.006 · Cited by 114 · cited 1×
“Ismail andKuppusamy [17] presented an exploratory study on the accessibility of the websites of Indian universities.”
BackgroundBeyond Conformance: The Role of Accessibility Evaluation Methods
Brajnik, G. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-85200-1_9 · Cited by 114 · cited 2×
“In this phase, it was essential to consider a list of potential barriers for low vision users [41] (See Table 5).”
BackgroundAcosta-Vargas, P., Acosta, T., & Lujan-Mora, S. (2018) · IEEE Access · doi:10.1109/access.2018.2848978 · Cited by 92 · cited 1×
“[18] described the barriers of web accessibility identified in 348 websites of Latin American universities according to the Webometrics ranking.”
BackgroundSelf-citation
Research works printed but not linked: 10
- alroobaea, 2013, Generating a domain specific inspection evaluation method through an adaptive framework: A comparative study on educational Websites, Int J Hum Comput Interact, 4, 88
- consortium, 2016, Accessibility Usability and Inclusion
- brajnik, 2011, Barrier Walkthrough
- 2019, How Many Websites Are There Around the World?
- 2019, WebAIM Accessibility of AJAX Applications
- 2019, World Report on Disability
- 2019, Introduction to Web Accessibility|Web Accessibility Initiative (WAI)|W3C
- 2019, We are social - digital 2019 Global internet use accelerates
- 2019, Why You Only Need to Test with 5 Users
- lunn, 2009, Barriers Faced by Older Users On Static Web Pages Criteria Used In The Barrier Walkthrough Method (Technical Report), 1
Cited by
Empirical Studies on Web Accessibility of Educational Websites: A Systematic Literature Review
Campoverde-Molina, M., Lujan-Mora, S., & Garcia, L. V. (2020) · IEEE Access · doi:10.1109/access.2020.2994288 · Cited by 94
“It should be noted that the papers [52], [59] published by Ecuador refer to Universities websites in Latin America.”
BackgroundMethodInfluentialAccessibility engineering in web evaluation process: a systematic literature review
Ara, J., Sik‐Lányi, C., & Kelemen, A. (2023) · Universal Access in the Information Society · doi:10.1007/s10209-023-00967-2 · Cited by 96
“…potential errors and violations, even without prior knowledge [43]; (9) proposed a framework for website accessibility barrier measurement according to several variable magnitude techniques [50]; 10) proposed a heuristic method to determine the level of accessibility of high ranked websites [66].”
MethodAccessibility of university websites worldwide: a systematic literature review
Campoverde-Molina, M., Luján-Mora, S., & Valverde, L. (2023) · Universal Access in the Information Society · doi:10.1007/s10209-021-00825-z · Cited by 84
Visualization Accessibility in the Wild: Challenges Faced by Visualization Designers
Joyner, S. C. S., Riegelhuth, A., Garrity, K., Kim, Y., & Kim, N. W. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517630 · Cited by 53
“Some studies investigate ways to derive quality metrics as an effort to derive an objective, comparative evaluation framework [2, 65], such as using WCAG [67] as a basis.”
BackgroundCombined Method for Evaluating Accessibility in Serious Games
Salvador-Ullauri, L., Acosta-Vargas, P., González, M., & Luján‐Mora, S. (2020) · Applied Sciences · doi:10.3390/app10186324 · Cited by 45
“In this study, we consider web accessibility as a starting point, which implies how users perceive, navigate, understand and interact on the web [13]; therefore, it is essential to keep in mind that the level of accessibility is the fundamental basis to facilitate access to serious games,…”
BackgroundSelf-citation
Metadata
- Authors
- Acosta-Vargas, P., Antonio Salvador-Ullauri, L., & Lujan-Mora, S.
- Affiliations
- Universidad de Las Américas (EC)
- Venue
- IEEE Access
- Published
- 2019
- Volume
- 7
- Pages
- 125634-125648
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W2972056002
- Cited by
- 78
- References
- 40
- Citations by year
- 2026: 5 · 2025: 5 · 2024: 12 · 2023: 14 · 2022: 17 · 2021: 13 · 2020: 12
- FWCI
- 30.74
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- IEEE Access (publishedVersion, cc-by)
- LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- RUA, Repositorio Institucional de la Universidad de Alicante (Universidad de Alicante) (submittedVersion, cc-by)
- Greater South Information System
- Funders
- Universidad de Las Américas, Ecuador (FGE.PAV.18.10)
- Topics
- Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Health Sciences › Health Professions › Occupational Therapy › Assistive Technology in Communication and Mobility
- Keywords
- web accessibility, low vision, heuristic evaluation, access barriers, university websites, Webometrics ranking
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A Heuristic Method to Evaluate Web Accessibility for Users With Low Vision. https://app.cochat.ai/papers/10.1109/access.2019.2939068
- The paper
- Acosta-Vargas, P., Antonio Salvador-Ullauri, L., & Lujan-Mora, S. (2019). A Heuristic Method to Evaluate Web Accessibility for Users With Low Vision. IEEE Access. https://doi.org/10.1109/access.2019.2939068
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.