CoChat
Sign Up
Cited by 84Digital Accessibility for DisabilitiesUsability and User Interface DesignWeb visibility and informetrics

Accessibility of university websites worldwide: a systematic literature review

Open access
Campoverde-Molina, Milton, Luján-Mora, Sergio, Valverde, Llorenç·2023·Universal Access in the Information Society
DOI 10.1007/s10209-021-00825-zPMC PMC8259087
Open this paper in CoChat — free accountOpen PDF

Abstract

The identity and institutional image of universities are presented to the world through their websites. On their websites, universities publish their academic offerings, their mission, their vision, their academic objectives, their achievements, their regulations, their news and all their university work. Hence, the importance of university websites is accessible. The accessibility of university websites has been evaluated several times in the past, but there is no work that has summarized all the evaluations performed to provide a general overview of the situation. Therefore, in this research we have performed a systematic literature review (SLR) to consolidate, analyze, synthesize and interpret the accessibility results of university websites published in 42 papers that have been selected for this study. The methodology used in this SLR was that proposed in Kitchenham's guidelines, which includes three stages: planning the review, conducting the review and reporting the review. The results present the analysis and synthesis of the evaluations of 9,140 universities in 67 countries. Of these, 38,416 web pages, 91,421 YouTube videos and 28,395 PDF documents were evaluated. Manual methods, methods with automatic tools and the combination of both methods were used for the evaluation. Most websites were evaluated using the ISO/IEC 40500:2012 and Section 508 standards. The accessibility guidelines most commonly violated in the evaluations were: adaptable, compatible, distinguishable, input assistance, keyboard accessible, navigable, predictable, readable and text alternatives. In conclusion, the university websites, YouTube videos and PDF documents analyzed in the 42 papers present important accessibility problems. The main contribution of this SLR is the consolidation of the results of the 42 studies selected to determine the findings and trends in the accessibility of university websites around the world.

Integrity signals

Registry

Crossref · 10.1007/s10209-021-00825-z

Cites

64 of 64 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

10 of 56 cited sources checked · 6 supported · 0 flagged · accuracy not yet measured

  • Findingown data

    WCAG 2.1 was used in papers [66, 72, 73, 76] to evaluate university websites.

    “WCAG 2.1 has been used in papers [66, 72, 73, 76] to evaluate the accessibility of university websites.”
    Quote in the paperSubstantial claim2 of 4 sources checked
    • cites [66] Ismail, A., Kuppusamy, K.: Web accessibility investigation and identification of major issues of higher education websites with statistical measures: a case study of college websites. J. King Saud Univ. Comput. Inf. Sci. (2019) source
    • cites [72] Acosta-Vargas, P., González, M., Luján-Mora, S.: Dataset for evaluating the accessibility of the websites of selected Latin American universities. Data in Brief 28, 105013 (2020) source
    • cites [73] Barricelli, B.R., Casiraghi, E., Dattolo, A., Rizzi, A.: 15 years of stanca act: are italian public universities websites accessible? Univ. Access Inf. Soc. 20, 185–200 (2021) source
    • cites [76] Acosta, T., Acosta-Vargas, P., Zambrano-Miranda, J., Luján-Mora, S.: Web Accessibility Evaluation of Videos Published on YouTube by Worldwide Top-Ranking Universities. IEEE Access 8, 110994–111011 (2020) source

      The Background (p. 3) says this study uses WCAG 2.1 criteria to evaluate prerecorded videos and video players associated with universities. It distinguishes that work from studies evaluating university websites without considering videos (Background, pp. 3–4). Thus, the source supports the use of WCAG 2.1, but not the claim that this paper evaluated the accessibility of university websites as a whole. (LLM, conf=0.88)

      Experimental — accuracy not yet measured
  • Findingown data

    Papers [47, 48, 54, 76] do not state which web pages were evaluated.

    “The papers [47, 48, 54, 76] do not refer in their content to the web pages that have been evaluated in the universities.”
    Quote in the paperNot a substantial claim2 of 4 sources checked
    • cites [47] Roig-Vila, R., Ferrández, S., Ferri-Miralles, I.: Assessment of web content accessibility levels in spanish official online education environments. Int. Educ. Stud. 7(6), 31–45 (2014) source
    • cites [48] Shawar, B.A.: Evaluating web accessibility of educational websites. Int. J. Emerg. Technol. Learn. 10(4), 4–10 (2015) source
    • cites [54] Hassouna, M., Sahari, N., Ismail, A.: University website accessibility for totally blind users. J. Inf. Commun. Technol. 16(1), 63–80 (2017) source

      The Introduction (page unavailable) says the paper evaluates Palestinian university websites, but the passages retrieved do not identify which pages of those websites were evaluated. Searches for page selection, homepages, and specific page types found no decisive passage; those results do not establish that the full paper omits this information. (LLM, conf=0.68)

      Experimental — accuracy not yet measured
    • cites [76] Acosta, T., Acosta-Vargas, P., Zambrano-Miranda, J., Luján-Mora, S.: Web Accessibility Evaluation of Videos Published on YouTube by Worldwide Top-Ranking Universities. IEEE Access 8, 110994–111011 (2020) source
  • Findingcitation

    Before this review, no study had brought together all prior assessments of university website accessibility into a general overview.

    “there is no work that has summarized all the evaluations performed to provide a general overview of the situation.”
    • Also stated in: § Introduction · plain
      “Before carrying out this SLR, we made sure that no other similar research study exists [11].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [11] García-Holgado, A., García-Peñalvo, F.J.: Mapping the systematic literature studies about software ecosystems. In: Sixth International Conference on Technological Ecosystems for Enhancing Multiculturality, pp. 910–918 (2018) source
  • Findingown data

    The review selected 42 papers for analysis.

    “42 papers that have been selected for this study”
    • Also stated in: § Conducting the systematic literature review · plain
      “leaving 42 papers selected for this SLR”
    • Also stated in: § Results · plain
      “Table 7 shows the 42 papers yielded by the search and selection process.”
    Quote in the paperSubstantial claim
  • Findingown data

    The ISO/IEC 40500:2012 and Section 508 standards were the most common benchmarks used to assess the websites.

    “Most websites were evaluated using the ISO/IEC 40500:2012 and Section 508 standards.”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim. 48
  • May combine two claims. 32
  • The quote only partly states this claim. 2

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Paper [76] evaluated 91,421 YouTube videos.

    “A total of 91,421 YouTube videos have been evaluated in the paper [76]”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [76] Acosta, T., Acosta-Vargas, P., Zambrano-Miranda, J., Luján-Mora, S.: Web Accessibility Evaluation of Videos Published on YouTube by Worldwide Top-Ranking Universities. IEEE Access 8, 110994–111011 (2020) source

  • May combine two claims.

    Premisecitation

    A 2019 study assessed search quality across PubMed, Google Scholar, and 26 other academic databases and concluded that Google Scholar should not be used as a primary search resource.

    “In a study conducted in 2019 [32], the search quality of PubMed, Google Scholar and other 26 academic search databases was evaluated; the results showed that Google Scholar is not suitable as a primary search resource.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [32] Gusenbauer, M., Haddaway, N.R.: Which Academic Search Systems are Suitable for Systematic Reviews or Meta-Analyses? Evaluating Retrieval Qualities of Google Scholar, PubMed and 26 other Resources. Research Synthesis Methods pp. 1–52 (2019) source

  • Probably not a substantial claim: may not pass S3 (load-bearing). The quote only partly states this claim.

    Findingcitation

    Paper [76] used manual evaluation of YouTube videos with the authors' expert knowledge.

    “One paper [76] presented the manual evaluation of the YouTube videos published on university websites with the expert knowledge of the authors of the paper on web accessibility.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [76] Acosta, T., Acosta-Vargas, P., Zambrano-Miranda, J., Luján-Mora, S.: Web Accessibility Evaluation of Videos Published on YouTube by Worldwide Top-Ranking Universities. IEEE Access 8, 110994–111011 (2020) source

      The Discussion (p. 14) says the authors manually evaluated videos on universities’ official YouTube channels, and the Methods (p. 5) describes manual evaluation as work by experts. The paper does not establish that the evaluated videos were published on university websites, rather than on those YouTube channels, or explicitly describe the evaluation as drawing on the authors’ own expert knowledge. (LLM, conf=0.86)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingcitation

    Paper [68] used heuristic review with UX Check Tool and manual evaluation with two web accessibility experts.

    “One paper [68] presented a heuristic review of university websites using the UX Check Tool and a manual evaluation with two experts in web accessibility”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [68] Acosta-Vargas, P., Antonio Salvador-Ullauri, L., Luján-Mora, S.: A Heuristic Method to Evaluate Web Accessibility for Users With Low Vision. IEEE Access 7, 125634–125648 (2019) source

  • May combine two claims.

    Findingcitation

    Paper [54] used automatic evaluation with CynthiaSays and manual evaluation with 16 blind users.

    “One paper [54] presented an automatic evaluation using CynthiaSays and a manual evaluation with 16 blind users of the websites of the analyzed universities”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [54] Hassouna, M., Sahari, N., Ismail, A.: University website accessibility for totally blind users. J. Inf. Commun. Technol. 16(1), 63–80 (2017) source

      The Phase One Results report automatic evaluation with CynthiaSays, but it assessed a prototype, not the university websites. The Phase Two Results report feedback from 16 blind users on that prototype; the manual evaluation in Phase One was performed by experts, not those users. The citation therefore combines real elements of the study but misstates what was evaluated and by whom. (LLM, conf=0.89)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Findingown data

    Only papers [45, 76] have dealt with evaluating other resources on web pages; the rest focused only on web pages.

    “only papers [45, 76] have dealt with the evaluation of other resources that exist on the web pages, and the rest have only focused on the web pages.”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites [45] Thompson, T., Comden, D., Ferguson, S., Burgstahler, S., Moore, E.: Seeking predictors of web accessibility in U.S. higher education institutions. Inf. Technol. Disab. 13(1), 1–18 (2013)

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [76] Acosta, T., Acosta-Vargas, P., Zambrano-Miranda, J., Luján-Mora, S.: Web Accessibility Evaluation of Videos Published on YouTube by Worldwide Top-Ranking Universities. IEEE Access 8, 110994–111011 (2020) source

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    The evaluations used manual methods, automatic tools, and a combination of both.

    “Manual methods, methods with automatic tools and the combination of both methods were used for the evaluation.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The review analyzed 28,395 PDF documents.

    “38,416 web pages, 91,421 YouTube videos and 28,395 PDF documents were evaluated.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    For WCAG 2.1, selection used sampling in 3 papers and random selection in 1 paper.

    “WCAG 2.1. The selection of university websites for web accessibility evaluation was made using the following methods: sampling (3 papers), randomly selected (1 paper).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The search identified two prior literature reviews.

    “The search with these strings yielded two literature reviews that are summarized and analyzed below”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    For WCAG 2.0, selection used sampling in 21 papers, random selection in 3 papers, and all universities in 1 paper.

    “WCAG 2.0. The methods for selecting university websites for web accessibility evaluation are as follows: sampling (21 papers), randomly selected (3 papers), all universities (1 paper)”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The evaluation covered 5 continents and 68 countries.

    “the scope of the evaluation is 5 continents and 68 countries”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Findingown data

    From the 219 papers retrieved, 177 were excluded, and 42 remained for the review.

    “In summary, of the 219 papers found, 177 were discarded, leaving 42 papers selected for this SLR.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    For WCAG 1.0, university website selection used sampling in 11 papers, random selection in 1 paper, and all universities in 1 paper.

    “WCAG 1.0. The selection of university websites for web accessibility evaluation was made using the following methods: sampling (11 papers), randomly selected (1 paper), all universities (1 paper)”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Several papers scored 0.71 but were included in the review.

    “After the calculation, it was possible to see that several items scored 0.71, but were included in the SLR.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Paper [45] evaluated 28,395 PDF documents.

    “28,395 PDF documents in the paper [45]”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [45] Thompson, T., Comden, D., Ferguson, S., Burgstahler, S., Moore, E.: Seeking predictors of web accessibility in U.S. higher education institutions. Inf. Technol. Disab. 13(1), 1–18 (2013)

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • The quote only partly states this claim.

    Findingown data

    WCAG 1.0 was used in 5 papers.

    “WCAG 1.0 is used for the evaluation of university websites in 5 papers”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Paper [45] analyzed 3,251 universities, the highest number among the selected papers.

    “The paper with the highest number of universities evaluated is [45], which analyzes 3,251 universities.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [45] Thompson, T., Comden, D., Ferguson, S., Burgstahler, S., Moore, E.: Seeking predictors of web accessibility in U.S. higher education institutions. Inf. Technol. Disab. 13(1), 1–18 (2013)

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Findingown data

    The database search returned 104 Scopus papers, 99 Web of Science papers, and 16 IEEE Xplore papers, for 219 in total.

    “In the Scopus scientific database 104 papers were found, in Web of Science 99 papers and in the IEEE Xplore 16 papers. Of the 219 papers,”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Findingnone given

    Many earlier studies have assessed university website accessibility against WCAG, accessibility laws, and standards.

    “The evaluation of the accessibility of university websites has been carried out in numerous research studies to determine their compliance with WCAG, accessibility laws and standards.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Eighteen papers were published in journals indexed in SJR and JCR.

    “18 papers in SJR and JCR”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Of the 42 selected papers, 11 were conducted in the USA.

    “Of the 42 selected papers, 11 were conducted in the USA.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The highest percentage of errors was Priority 1 with 40%, Priority 2 with 35%, and Priority 3 with 25%.

    “The highest percentage of errors is in Priority 1 with 40 %, Priority 2 with 35 % and Priority 3 with 25 %.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Paper [42] used manual evaluation with Internet Explorer 7.0, Web Accessibility Toolbar, JAWS, and the authors' expert knowledge.

    “One paper [42] presented a manual evaluation of university websites using the Internet Explorer 7.0, Web Accessibility Toolbar (WAT), JAWS and the expert knowledge of the authors of the document”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [42] Thompson, T., Burgstahler, S., Moore, E.J.: Web accessibility: a longitudinal study of college and university home pages in the northwestern United States. Disab. Rehabilit. Assist. Technol. 5(2), 108–114 (2010) source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The analyzed papers were published in 27 journals.

    “The papers analyzed in this SLR have been published in 27 journals.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    For WCAG 2.0, errors by principle were Operable 36%, Perceivable 32%, Understandable 24%, and Robust 8%.

    “The percentage of errors by principle was Operable with 36 %, Perceivable with 32 %, Understandable with 24 % and Robust with 8 %.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    For WCAG 2.1, errors by principle were Perceivable 56%, Operable 28%, Understandable 13%, and Robust 3%.

    “The percentage of errors by principle was: Perceivable with 56 %, Operable with 28 %, Understandable with 13 % and Robust with 3 %.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The evaluation methods used were manual evaluation, automatic tools, and a combination of both.

    “The evaluation methods used in the selected papers are: manual evaluation, evaluation with automatic tools and the combination of both methods.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    Some selected papers do not detail the errors found in the evaluation.

    “some papers do not detail the errors found in the evaluation”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingcitation

    Paper [37] used automatic evaluation with Bobby and manual evaluation with JAWS, Internet Explorer, and two experts.

    “One paper [37] presented an automatic evaluation using Bobby and a manual evaluation using JAWS, Internet Explorer and the help of two experts from university websites”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [37] Thompson, T., Burgstahler, S., Comden, D.: Research on web accessibility in higher education. Inf. Technol. Disab. 9(2), 1–16 (2003)

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    PDF documents and YouTube videos published on university websites were also evaluated.

    “In addition, PDF documents and YouTube videos published on university websites have been evaluated.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Twenty papers were published in SJR-indexed journals.

    “20 papers in SJR”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The web pages evaluated on university websites included homepage, search page, lists of colleges/departments/degree programs, campus directory, admissions homepage, staff/students, course listings, employment homepage, academic calendar, job listings, campus map, and library homepage.

    “the web pages evaluated on the university websites are: university homepage, search page, list of university colleges, departments, and/or degree programs, campus directory of faculty, admissions homepage, staff and/or students, course listings, employment homepage, academic calendar, job listings, campus map and library homepage.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    For WCAG 1.0, the most common errors by checkpoint were lack of text alternatives (1.1), missing language (4.3), tables used for layout or complex structure (5.5), and poor navigation (12.1, 13.1).

    “The most common errors found by checkpoint in the selected papers are: lack of text alternatives (1.1), missing language (4.3), tables used for layout or with complex structure (5.5) and poor navigation (12.1, 13.1).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    AChecker was used in 15 papers, the most among automatic tools.

    “AChecker (15 papers)”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Four papers were not indexed in SJR and JCR.

    “4 papers are not indexed in SJR and JCR”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The 42 analyzed papers evaluate 9,140 universities distributed in 5 continents and 68 countries.

    “The 42 papers analyzed evaluate 9,140 universities distributed in 5 continents and 68 countries.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The CRPD treats access to information and communication, including the Web, as a fundamental human right.

    “In the Convention on the Rights of Persons with Disabilities (CRPD) of the United Nations [1], access to information and communication, including the Web, is defined as a basic human right.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [1] United Nations: Convention on the Rights of Persons with Disabilities - Articles (2006). http://cort.as/-GlqK

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    WCAG 1.0 contains 3 priorities, 14 guidelines, 65 checkpoints, and 3 conformance levels.

    “WCAG 1.0 includes 3 priorities, 14 guidelines, 65 checkpoints and 3 conformance levels, as shown in Fig. 1.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [14] Chisholm, W., Vanderheiden, G., Jacobs, I.: Web Content Accessibility Guidelines 1.0 (1999). https://www.w3.org/TR/WAI-WEBCONTENT/

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationreasoning

    The total number of papers using ISO/IEC 40500:2012 is 29, making it the most used web accessibility standard in the evaluations.

    “the total number of papers that use ISO/IEC 40500:2012 is 29 and this is the web accessibility standard most used in the evaluations.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: WCAG 2.1 incorporates all principles, guidelines, and success criteria of WCAG 2.0.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    WCAG 2.0 was declared the international standard ISO/IEC 40500:2012.

    “It should be noted that WCAG 2.0 was declared as the international standard ISO/IEC 40500:2012 [5, 6].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [5] International Organization for Standardization: Information technology — W3C Web Content Accessibility Guidelines (WCAG) 2.0 (2012). https://www.iso.org/standard/58625.html

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [6] Shawn Lawton Henry: WCAG 2.0 is now also ISO/IEC 40500! (2012). https://www.w3.org/blog/2012/10/wcag-20-is-now-also-isoiec-405/

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The 2011 WHO World Disability Report estimated that over a billion people, roughly 15% of the global population, have some form of disability.

    “more than a billion people are estimated to live with some form of disability, or about 15 % of the world’s population (based on 2010 global population estimates).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [8] World Health Organization: Summary World Report on Disability (2011). https://cutt.ly/ZrDBPje

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Under Article 21 of the CRPD, governments should urge private providers of public services, including online, to offer information and services in formats that are accessible and usable by people with disabilities.

    “In the Article 21—Freedom of expression and opinion, and access to information, it is stated that governments should urge “private entities that provide services to the general public, including through the Internet, to provide information and services in accessible and usable formats for persons with disabilities” [1].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [1] United Nations: Convention on the Rights of Persons with Disabilities - Articles (2006). http://cort.as/-GlqK

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    WCAG 2.1, issued in June 2018, extends WCAG 2.0 with 17 additional success criteria and one new guideline.

    “The WCAG 2.1 [7], published in June 2018, includes the WCAG 2.0 principles, guidelines, success criteria and conformance levels, plus 17 new success criteria and a new guideline.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [7] Kirkpatrick, A., O connor, J., Campbell, A., Cooper, M.: Web Content Accessibility Guidelines (WCAG) 2.1 (2018). https://www.w3.org/TR/WCAG21/

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    The review defined seven quality assessment questions, each worth 1 point, for a maximum score of 7.

    “Seven questions have been defined to measure the quality of each paper. Each question has a score of 1; therefore, the maximum total score is 7.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The W3C/WAI created WCAG-EM as a procedure for assessing websites against WCAG 2.0.

    “The procedure for evaluating websites using WCAG 2.0 developed by the W3C/WAI is the Website Accessibility Conformance Evaluation Methodology (WCAG-EM) [18].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [18] Velleman, E., Abou-Zahra, S.: Website Accessibility Conformance Evaluation Methodology (WCAG-EM) 1.0 (2014). https://www.w3.org/TR/WCAG-EM/

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Study design and conductown data

    The review excluded papers whose normalized quality score fell below 0.70.

    “Papers with a normalized score of less than 0.70 were excluded from the SLR.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    This review used Kitchenham's guidelines, comprising planning, conducting, and reporting phases.

    “This research adopts the Kitchenham’s guidelines, which include the following steps: “Planning Review, Conducting Review and Reporting Review” [24].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [24] Kitchenham, B.: Procedures for performing systematic reviews. Keele, UK, Keele Univ. 33, 1–33 (2004)

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    According to WebAIM, automated tools cannot establish whether a site is accessible or compliant, so human testing is always required.

    “However, WebAIM [21] states that “no automated evaluation tool can tell you if your site is accessible, or even compliant. Human testing is always necessary because accessibility is about the human experience.””
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [21] WebAIM: Accessibility Evaluation Tools (2019). https://webaim.org/articles/tools/

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    WCAG 2.0, issued in December 2008, has 61 success criteria grouped under 12 guidelines and 4 principles.

    “The WCAG 2.0 [15], published in December 2008, includes 61 success criteria that are organized under 12 guidelines and 4 principles: Perceptible (P), Operable (O), Understandable (U) and Robust (R).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [15] Caldwell, B., Cooper, M., Reid, L.G., Vanderheiden, G.: Web Content Accessibility Guidelines (WCAG) 2.0 (2008). https://www.w3.org/TR/WCAG20/

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationreasoning

    Millions of people with disabilities are currently studying at universities.

    “which allows to affirm that millions of people with disabilities are studying at universities right now.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Because of the SARS-CoV-2 pandemic, schools, colleges, institutes, and universities had to shut their facilities and move to online education.

    “Schools, colleges, institutes and universities have been forced to close their facilities, adapting them to online education due to the SARS-COV-2 pandemic [9].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [9] UNESCO: As a new academic year begins, UNESCO warns that only one third of students will return to school (2020). https://n9.cl/l5ma

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The W3C released WCAG 1.0 in May 1999 as a way to address web accessibility problems.

    “WCAG 1.0 [14] was published in May 1999 by the W3C, with the aim of providing solutions to web accessibility problems.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [14] Chisholm, W., Vanderheiden, G., Jacobs, I.: Web Content Accessibility Guidelines 1.0 (1999). https://www.w3.org/TR/WAI-WEBCONTENT/

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Assessing web accessibility objectively demands more than merely running automatic tools.

    “An objective evaluation of web accessibility requires more than the simple use of automatic tools for the evaluation of websites [23].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [23] Brewer, J.: Using Combined Expertise to Evaluate Web Accessibility (2019). https://www.w3.org/WAI/test-evaluate/combined-expertise/

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    A 2019 systematic review of web accessibility evaluation methods reported that automatic tools, expert evaluation, and user testing are the techniques used most often.

    “In an SLR carried out in 2019 [19], on web accessibility evaluation methods, it was determined that the main methods used are: “1) automatic tools, 2) evaluation by experts and 3) user tests are the most widely used techniques according to the literature.””
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [19] Nuñez, A., Moquillaza, A., Paz, F.: Web Accessibility Evaluation Methods: A Systematic Review. In: Marcus A., Wang W. (eds) Design, User Experience, and Usability. Practice and Case Studies. HCII 2019, vol. 11586, pp. 226–237. Cham (2019) source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    A website that complies with WCAG 2.1 also complies with WCAG 2.0.

    “Therefore, when a website is compliant with WCAG 2.1, the website is also compliant with WCAG 2.0.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    When students with and without disabilities share regular classrooms, universities incur duties such as making their web platforms and educational resources accessible.

    “The admission of people with and without disabilities in regular classrooms generates obligations that university institutions must comply with, for example, the accessibility of their web platforms and their educational resources.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The CRPD's Article 24 mandates that all people be included and participate in education.

    “Article 24 Education of the CRPD [1] requires the inclusion and participation of all persons in the educational environment.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [1] United Nations: Convention on the Rights of Persons with Disabilities - Articles (2006). http://cort.as/-GlqK

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation

    Bobby, launched in 1996, is one of the first web accessibility evaluation tools and lost continuity in 2005.

    “Bobby, launched in 1996 [79], is one of the first tools for web accessibility evaluation, which had several versions one more advanced than another and lost its continuity in 2005.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [79] Hoffmann, J.: An Early History of Web Accessibility (2019). https://thehistoryoftheweb.com/accessibility-tools/

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    All authors were involved in the whole SLR process.

    “all the authors were involved in the whole SLR process to give more strength to their results”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    It is difficult to compare results from different studies because no baseline exists and each study applies different methods and tools.

    “it is quite difficult to compare the results from different studies because there does not exist a baseline and each study applies different methods and tools.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation

    There seems to be a lack of knowledge or interest in web accessibility laws, because many countries already have policies and laws.

    “there seems to be a lack of knowledge or interest in the laws that should be applied, because in many countries policies and laws on web accessibility already exist [78].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [78] Mueller, M.J., Jolly, R., Eggert, E.: Web Accessibility Laws & Policies (2018). https://www.w3.org/WAI/policies/

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    The SLR defined customized search strings with key terms and applied them in data extraction from scientific databases.

    “this SLR starts by defining customized search strings with the key terms of this research. These search strings were applied in the extraction of data from each of the scientific databases.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    University websites do not appear to be created with web accessibility in mind.

    “university websites do not appear to be created with web accessibility in mind”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    WCAG has been evolving from WCAG 1.0 to 2.0 to 2.1 to draft 2.2 and first public working draft 3.0, each seeking wider scope.

    “WCAG has been evolving from WCAG 1.0 [14], WCAG 2.0 [15], WCAG 2.1 [7], to draft WCAG 2.2 [16] and first public working draft WCAG 3.0 [81], and each of them seeks a wider scope.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [14] Chisholm, W., Vanderheiden, G., Jacobs, I.: Web Content Accessibility Guidelines 1.0 (1999). https://www.w3.org/TR/WAI-WEBCONTENT/

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [15] Caldwell, B., Cooper, M., Reid, L.G., Vanderheiden, G.: Web Content Accessibility Guidelines (WCAG) 2.0 (2008). https://www.w3.org/TR/WCAG20/

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [7] Kirkpatrick, A., O connor, J., Campbell, A., Cooper, M.: Web Content Accessibility Guidelines (WCAG) 2.1 (2018). https://www.w3.org/TR/WCAG21/

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    The SLR applied inclusion, exclusion, and quality criteria in selecting papers.

    “we have applied inclusion, exclusion and quality criteria in the selection of the papers”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    The SLR does not include papers not published in journals, so grey literature is not considered.

    “this SLR does not include papers not published in journals such as conferences, editorials, prefaces, discussion, comments, tutorial summaries, workshop summaries and panels. Therefore, “grey” literature is not considered in this SLR.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    The SLR authors contrasted these errors with WCAG success criteria under their knowledge.

    “the authors of this SLR contrasted these errors with the WCAG success criteria under their knowledge”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The W3C lists 154 tools for automatic web accessibility evaluation.

    “the W3C lists 154 tools [20] for the evaluation of web accessibility in an automatic way”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [20] Eggert, E., Abou-Zahra, S.: Web Accessibility Evaluation Tools List (2016). https://www.w3.org/WAI/ER/tools/

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

69 of 69

  • The paper relies on this without support.

    University websites are important because they present universities' identity, image, and academic information to the world.

    “The identity and institutional image of universities are presented to the world through their websites. On their websites, universities publish their academic offerings, their mission, their vision, their academic objectives, their achievements, their regulations, their news and all their university work.”

  • The paper relies on this without support.

    The results of the 42 selected studies can be meaningfully consolidated despite differences in their evaluation methods, standards, and contexts.

    “consolidate, analyze, synthesize and interpret the accessibility results of university websites published in 42 papers”
    “The main contribution of this SLR is the consolidation of the results of the 42 studies selected to determine the findings and trends in the accessibility of university websites around the world.”

  • The paper relies on this without support.

    The 42 selected papers are representative of university website accessibility worldwide.

    “The results present the analysis and synthesis of the evaluations of 9,140 universities in 67 countries.”
    “Accessibility of university websites worldwide: a systematic literature review”

  • The paper relies on this without support.

    WCAG 2.1 incorporates all principles, guidelines, and success criteria of WCAG 2.0.

    “Taking into account that WCAG 2.1 adopts all the principles, guidelines and success criteria of WCAG 2.0”
4 of 4

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Incorrect percentage for studies spanning multiple continents

    Minor

    Three of the 42 selected papers represent 7.14%, not 6.12%.

    Reported 6.12 · recomputed 7.142857

  • Worldwide country count changes from 67 to 68

    Minor

    The abstract reports evaluations in 67 countries, while the results report 68 countries.

2 of 2

Cites

Flagged first, then by citations

Research works printed but not linked: 3

  1. Thompson, T., Burgstahler, S., Comden, D.: Research on web accessibility in higher education. Inf. Technol. Disab. 9(2), 1–16 (2003)
  2. Sodhar, I.N., Bhanbhro, H., Amur, Z.H.: Evaluation of web accessibility of engineering university websites of Pakistan through online tools. Int. J. Comp. Sci. Netw. Sec. 19(12), 85–90 (2019)
  3. Universal Access in the Information Society: Home (2020). https://www.springer.com/journal/10209

Cited by

Sorted by citations

Related

Authors
Campoverde-Molina, M., Luján-Mora, S., & Valverde, L.
Affiliations
Universitat de les Illes Balears (ES)
ORCID
Venue
Universal Access in the Information Society
Publisher
Springer Science and Business Media LLC
Published
2021-07-06
Volume
22
Issue
1
Pages
133-168
ISSN
  • 1615-5289 (ISSN-L)
  • 1615-5297
DOI
10.1007/s10209-021-00825-z
PMC
PMC8259087
OpenAlex
W3182871611
Cited by
84
References
64
Citations by year
2026: 19 · 2025: 23 · 2024: 21 · 2023: 12 · 2022: 8 · 2021: 1
FWCI
3.28
Type
journal article
Language
en
Open access
Licence
Locations
  • Universal Access in the Information Society (publishedVersion)
  • PubMed Central (publishedVersion)
  • PubMed (publishedVersion)
  • Repositorio Institucional de la Universidad de Alicante (Universidad de Alicante) (submittedVersion)
Funders
  • Erasmus+ (609785-EPP-1-2019-1-ES-EPPKA2-CBHE-JP)
  • Ministerio de Ciencia, Innovación y Universidades (PID2019-111196RB-I00)
Topics
  • Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
  • Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
  • Physical Sciences › Computer Science › Information Systems › Web visibility and informetrics
Keywords
web accessibility, university websites, higher education institutions, accessibility evaluation, Section 508, YouTube videos, PDF documents, keyboard accessibility, text alternatives

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-09-29.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-09-29.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review