Comparing Six Free Accessibility Evaluation Tools
Open accessAbstract
The web is continuously growing in content and functionality having an important impact on society.An objective of the Digital Agenda for Europe is to make the web technologies available for all.Recently, a European directive has been issued that requires the accessibility of public websites by September 2020.Given the low accessibility of public websites, systematic evaluation and monitoring strategies are needed.Covering the huge number of public websites with reasonable effort is only possible in a tool-based approach which makes the selection of a suitable tool a critical issue.The goal of this paper is to compare six accessibility evaluation tools as regards the coverage of accessibility-related issues, identification, reporting, and documenting of accessibility errors.The comparison is illustrated with six case studies of Romanian municipal websites.The results show large differences as regards the way of reporting and the number of errors which suggest that only one tool is not enough.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Previous research has found significant differences between results obtained with various accessibility evaluation tools and suggests using more than one tool to increase confidence.
“Previous research has found evidence that there are significant differences between the results obtained with various accessibility evaluation tools and suggests using more than one tool to increase confidence in results, e.g. Padure and Pribeanu [33].”
From Does the law matter? An empirical study on the accessibility of Finnish higher education institutions’ web pages (2024) · cites this paper as Padure and Pribeanu [33]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
No references could be resolved yet.
Research works printed but not linked: 13
- Akgul, Y., & Vatansever, K. "Web ac- cessibility evaluation of government websites for people with disabilities in Turkey". Journal of Advanced Manage- ment Science, 4(3), 2016, 201-210.
- Alshamari, M. "Accessibility Evaluation of Arabic E-Commerce Web Sites Using Automated Tools". Journal of Software Engineering and Applications, 9, 2016, 439-451.
- Brajnik, G. "Comparing accessibility evaluation tools: a method for tool effec- tiveness". Universal Access in the Infor- mation Society, 3(3-4), 2004, 252-263.
- EU (2016) Directive of the European Par- liament and of the Council of 26 October 2016 on the accessibility of the websites and mobile applications of public sector bodies. Official Journal of the European Union, L327.
- Gay, G., Qi Li, C. Open, interactive, cus- tomizable, web accessibility checking. In: Proceedings of the 7th international cross-disciplinary conference on web ac- cessibility. ACM, New York, p 2, 2010.
- Karaim, N. A., & Inal, Y. (2019). "Usa- bility and accessibility evaluation of Lib- yan government websites". Universal Access in the Information Society, 18(1), 207-216.
- Paternò, F., & Schiavone, A. G. (2015). "The role of tool support in public poli- cies and accessibility". interac- tions, 22(3), 60-63.
- Padure M, Pribeanu, C. "Exploring dif- ferences between five accessibility eval- uation tools". In: Moldoveanu A, Dix A (Eds.) Proceedings of RoCHI 2019, Bu- charest, Romania, 17-18 October 2019, 87-90.
- Pribeanu, C. "Large-scale accessibility evaluation of Romanian websites". Inter- national Journal of User-System Interac- tion, 12(2), 83-98.
- Silva, C., Eler, M., Fraser, G. "A survey on the tool support for the automatic evaluation of mobile accessibility." In Proceedings of the 8th International Conference on Software Development and Technologies for Enhancing Acces- sibility and Fighting Info-exclusion, 286- 293. ACM, 2018.
- Vigo, M., Brown, J., Conway, V. "Benchmarking web accessibility evalu- ation tools: measuring the harm of sole reliance on automated tests". Proceedings W4A 2013, Article 1, 2013.
- Yesilada, Y., Harper, S. Web Accessibil- ity -A Foundation for Research, 2 nd Edi- tion, Springer, 2019.
- Marian PĂDURE received the PhD degree in Psychology from the Babeș-Bolyai University from Cluj-Napoca in 2011. Currently he is research assistant at Special Education Department in Babes-Bolyai University, coordinator at The Center for Counseling in Assistive Technologies for Students with Visual Impairment and Re- sponsible at the Office for students with disabilities. His research interests include assistive technologies and accessibility, CAI, e-learning, learning styles, disability in- clusion. He is author/co-author of 3 books and over 100 journal and conference pa- pers/presentations.
Cited by
Sorted by citations
Accessibility challenges of e-commerce websites
Acosta-Vargas, P., Salvador-Acosta, B., Salvador-Ullauri, L., & Jadán-Guerrero, J. (2022) · PeerJ Computer Science · doi:10.7717/peerj-cs.891 · Cited by 32 · see citing works
Laamanen, M., Ladonlahti, T., Puupponen, H., & Kärkkäinen, T. (2024) · Universal Access in the Information Society · doi:10.1007/s10209-022-00931-6 · Cited by 29 · see citing works
“Previous research has found evidence that there are significant differences between the results obtained with various accessibility evaluation tools and suggests using more than one tool to increase confidence in results, e.g. Padure and Pribeanu [33].”
- Authors
- PADURE, M., & PRIBEANU, C.
- Affiliations
- Academia Oamenilor de Știință din România (RO)
- ORCID
- PRIBEANU, Costin
- Venue
- Informatica Economica
- Published
- 2020-03-30
- Volume
- 24
- Issue
- 1/2020
- Pages
- 15-25
- ISSN
- 1453-1305 (ISSN-L)
- 1842-8088
- OpenAlex
- W3016167905
- Cited by
- 32
- References
- 0
- Citations by year
- 2026: 2 · 2024: 3 · 2023: 8 · 2022: 13 · 2021: 5 · 2020: 1
- FWCI
- 12.14
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- Informatica Economica (publishedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Social Sciences › Political Science and International Relations › E-Government and Public Services
- Keywords
- web accessibility, accessibility evaluation tools, tool comparison, public websites, Digital Agenda for Europe, accessibility errors, error reporting, municipal websites
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
