CoChat
Sign Up
Cited by 1Digital Accessibility for DisabilitiesSoftware Engineering ResearchMobile and Web Applications

A Multi-Agent Framework for Accessible Web Code Generation

Open access
Aljedaani, Wajdi, Shaik, Mohammad Aftab, Aljohani, Ahmed·2026·Proceedings of the 23rd International Web for All Conference
DOI 10.1145/3800424.3800453
Open this paper in CoChat — free account

Abstract

Large Language Models (LLMs) have become widely used for automated code generation. However, their ability to produce accessibility-compliant web interfaces remains limited and underexplored. We present A11yAgent, a multi-agent framework for accessibility website generation that decomposes website creation into coordinated stages that refine user prompts, generate semantically structured code, automatically render and validate executability, evaluate accessibility, and iteratively repair detected violations. This design integrates accessibility constraints directly into the generation process, eliminating the need for manual deployment, external testing, and post-hoc remediation. To evaluate effectiveness, we conducted a comparative study with 20 professional front-end and full-stack developers. Participants generated websites using standard LLM-based workflows, while the same prompts were processed through A11yAgent. Accessibility compliance was assessed using automated tools. Results reveal consistent structural and semantic accessibility failures across state-of-the-art LLMs, whereas A11yAgent produces highly accessible outputs. Our results reveal current LLM accessibility blind spots and demonstrate how our multi-agent framework effectively bridges these gaps to ensure robust WCAG compliance.

Verified 2026-10-25 · 3 references not checked yet

AI snapshot

Written from the paper’s full text.

TL;DR

A11yAgent is a multi-agent framework that embeds accessibility specification, code generation, rendering/validation, automated WCAG evaluation, and iterative repair into one closed-loop pipeline. In a comparative study with 20 professional developers, it reduced detected accessibility violations to 86 versus 304–1027 for single-LLM baselines, though contrast violations remained the dominant residual issue.

Key findings

  • Across 20 websites generated from participant-written prompts, single-LLM baselines produced many more violations: Claude n=1027, GPT n=992, Qwen n=304, while A11yAgent reached n=86. This is reported as a 71.7% reduction versus Qwen and 91.6% versus Claude.
  • A11yAgent achieved zero violations in 34 of 38 measured error types, eliminating landmark, heading, form-labeling, ARIA, navigation, and keyboard-access violations that appeared in baselines.
  • Residual A11yAgent violations were mostly contrast-related: 79 of its 86 remaining violations were contrast-related. Under WCAG 1.4.3, A11yAgent had n=32 violations versus GPT n=46, Qwen n=65, and Claude n=700; under WCAG 1.4.6, A11yAgent had n=47 versus GPT n=658, Qwen n=89, and Claude n=122.
  • Iteration analysis found 30% of generated websites reached full compliance within one to two iterations, while 50% required three repair cycles. Timing results showed 12 of 20 websites generated and fully verified in under 10 seconds; 4 took 30–60 seconds and 3 took more than 60 seconds.

Methods

  • Comparative study with 20 professional front-end/full-stack developers, recruited by convenience sampling; 27 initially participated, with exclusions for less than three years’ experience or backend-only work. Participants wrote prompts for websites in a domain of their choice and generated sites using a selected state-of-the-art LLM (GPT, Qwen, or Claude).
  • The same prompts were processed by A11yAgent, a multi-agent pipeline with Specification Agent, Code Generation Agent, Syntax Repair, Accessibility Detection, Code Repair Agent, and Iteration/Verification. It uses GPT-5.2 for specification and Qwen3-Coder-480B for code generation/repair, with termination when Critical=0 and Serious=0 or after max iterations i≤5.
  • Accessibility was assessed with aXe and WAVE, mapped to WCAG 2.2 guidelines and conformance levels; reports from both tools were consolidated, duplicates removed, and final unique violation counts recorded.

Limitations

  • The paper states that automated tools (aXe and WAVE) cannot capture all accessibility barriers, particularly cognitive accessibility and real user interaction; it also notes the fixed iteration cap i≤5 as a conservative safety threshold.
  • External validity limitations stated: 20 professional developers recruited via convenience sampling, artifacts limited to static HTML/CSS/JavaScript websites, possible influence of selected baseline models and prompting strategies over time, and a controlled single-pass baseline that does not fully reflect iterative professional development.
  • Evident from the text: A11yAgent’s results came from its internal evaluation loop, while baseline websites underwent external auditing with WAVE and aXe; contrast remained the primary unresolved challenge, and full elimination of enhanced-contrast violations was not always achieved within the iteration limit.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1145/3800424.3800453

Cites

78 of 81 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

34 of 54 cited sources checked · 8 supported · 1 flagged · accuracy not yet measured

  • Premisecitation
    § 1 Introduction · p. 1

    The state of web accessibility remains critically poor.

    “Despite the enforcement of legal frameworks such as the Americans with Disabilities Act (ADA) [72] and deep ethical obligations to foster inclusivity, the state of web accessibility remains critically poor [39, 69].”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites [39] [39] Abid Ismail and KS Kuppusamy. 2022. Web accessibility investigation and identification of major issues of higher education websites with statistical measures: A case study of college websites. Journal of King Saud University-Computer and Information Sciences 34, 3 (2022), 901-911. - source

      Section 5.1 (Discussions; page unavailable) reports thousands of accessibility problems across 44 college websites, supporting a poor assessment of those sites. The claim describes web accessibility generally, whereas this study’s findings are limited to the college websites it evaluated. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
    • cites [69] [69] Mohan Sunkara, Akshay Kolgar Nayak, Sandeep Kalari, Satwik Ram Kodandaram, Sampath Jayarathna, Hae-Na Lee, and Vikas Ashok. 2024. Assessing the Accessibility and Usability of Web Archives for Blind Users. In International Conference on Theory and Practice of Digital Libraries . Springer, 203-221. - source
  • Premisecitation
    § 3 A11yAgent · p. 5

    Prior work indicates that GPT-series models are effective at inferring implicit accessibility requirements from underspecified prompts.

    “Prior work further indicates that GPT-series models are effective at inferring implicit accessibility requirements from underspecified prompts, making them suitable for initializing accessibility-aware design constraints [68].”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [68] [68] Hyunjae Suh, Mahan Tafreshipour, Sam Malek, and Iftekhar Ahmed. 2025. Human or LLM? A Comparative Study on Accessible Code Generation Capability. arXiv preprint arXiv:2503.15885 (2025). - source

      Section 3.4 (p. 6) says naive code-generation prompts did not mention accessibility, and Section 4.2 (pp. 8–9) reports that GPT-4o’s naive-generated code had fewer measured accessibility issues than human-written code. That supports accessibility performance without explicit instructions, but the paper does not establish that GPT-4o inferred implicit requirements from underspecified prompts. (LLM, conf=0.85)

      Experimental — accuracy not yet measured
  • Premisecitation
    § 3 A11yAgent · p. 5

    The repair procedure follows the reflect-revise paradigm, where each repair step is driven by the accessibility tool flags reported in the A11y Report.

    “In principle, our repair procedure follows the reflectrevise paradigm [65, 67], where each repair step is driven by the accessibility tool flags reported in the A11y Report , which guide targeted code modifications toward WCAG compliance.”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites [65] [65] Quan Shi, Michael Tang, Karthik Narasimhan, and Shunyu Yao. 2024. Can Language Models Solve Olympiad Programming? arXiv preprint arXiv:2404.10952 (2024). - source
    • cites [67] [67] Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language Agents with Verbal Reinforcement Learning. arXiv:2303.11366 [cs.AI] - source
  • Premisecitation
    § 3 A11yAgent · p. 3

    A key challenge in prompt-based website generation is that user-provided prompts often lack essential functional or structural requirements of a complete website.

    “A key challenge in prompt-based website generation is that user-provided prompts often lack essential functional or structural requirements of a complete website [75, 78].”
    Quote in the paperSubstantial claim2 of 2 sources checked
    • cites [75] [75] Chenyang Yang, Yike Shi, Qianou Ma, Michael Xieyang Liu, Christian Kästner, and Tongshuang Wu. 2025. What Prompts Don't Say: Understanding and Managing Underspecification in LLM Prompts. arXiv preprint arXiv:2505.13360 (2025). - source

      The Introduction (page unavailable) defines prompt underspecification as omitting essential requirements, and Section 4.2 (page unavailable) applies requirement-aware optimization to a website-generation task. But the paper does not establish that user-provided website prompts often omit functional or structural requirements needed for a complete website; its website example concerns a front-end development system prompt. (LLM, conf=0.83)

      Experimental — accuracy not yet measured
    • cites [78] [78] Jingming Zhuo, Songyang Zhang, Xinyu Fang, Haodong Duan, Dahua Lin, and Kai Chen. 2024. ProSA: Assessing and understanding the prompt sensitivity of LLMs. arXiv preprint arXiv:2410.12405 (2024). source

      The Introduction (page unavailable) discusses how variations in prompt wording affect LLM outputs, and §2.1 defines task requirements and different expressions of them. Neither addresses website generation or says that user prompts often omit essential functional or structural requirements; searches for website generation and incomplete requirements found no such passage. (LLM, conf=0.76)

      Experimental — accuracy not yet measured
  • Study design and conductcitation
    § 3 A11yAgent · p. 5

    The CRA is strictly bound by the UISpec to prevent LLMs from hallucinating new content during code modification.

    “Furthermore, to prevent a common failure mode where LLMs hallucinate new content during code modification [50]-the agent is strictly bound by the UISpec .”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [50] [50] Fang Liu, Yang Liu, Lin Shi, Houkun Huang, Ruifeng Wang, Zhen Yang, Li Zhang, Zhongqi Li, and Yuchi Ma. 2024. Exploring and evaluating hallucinations in llm-powered code generation. arXiv preprint arXiv:2404.00971 (2024). - source

      The Introduction describes LLM-generated code that conflicts with user requirements or context, supporting the cited concern about hallucination. It does not specifically establish hallucinating new content during code modification. The UISpec constraint is the citing authors’ own proposed response, not a claim attributed to this source. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
  • May combine two claims. 45
  • Classified as probably not a substantial claim. 42
  • The quote only partly states this claim. 1

  • May combine two claims.

    Premisecitation
    § 3 A11yAgent · p. 5

    Specialized coding models outperform general-purpose LLMs in syntax accuracy and debugging reliability on complex programming tasks.

    “Recent benchmarks show that specialized coding models outperform general-purpose LLMs in syntax accuracy and debugging reliability on complex programming tasks [45, 52].”
    Quote in the paperSubstantial claim2 of 2 sources checked
    • cites [45] [45] Satyadhar Joshi. 2025. Open-Source vs. Commercial Coding Assistants: A 2025 Comparison of DeepSeek R1, Qwen 2.5 and Claude 3.7. (2025). - source

      The Coding Performance Comparison section reports that Qwen3-Coder outperformed many proprietary alternatives on SWE-bench, supporting a narrower claim about coding performance. The paper does not establish that specialized coding models generally outperform general-purpose LLMs in syntax accuracy or debugging reliability on complex tasks; its Benchmark Evaluation section describes bug-fixing benchmarks but gives no such comparative finding. Page numbers are unavailable in our copy. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
    • cites [52] [52] Chenyang Lyu, Lecheng Yan, Rui Xing, Wenxi Li, Younes Samih, Tianbo Ji, and Longyue Wang. 2024. Large language models as code executors: An exploratory study. arXiv preprint arXiv:2410.06667 (2024). - source

      The Main Results section reports the opposite model comparison for the task studied: general-purpose o1-Preview outperformed the evaluated coding models in code-execution accuracy. The paper evaluates execution of supplied snippets, not syntax accuracy or debugging reliability, so it cannot establish those parts of the claim. (LLM, conf=0.86)

      Checked by a reasoning model — may be wrong.

      Experimental — accuracy not yet measured

  • Classified as probably not a substantial claim.

    Premisecitation
    § 1 Introduction · p. 1

    LLMs can identify accessibility violations that automated checkers often miss.

    “while López-Gil and Pereira [51] found that LLMs can identify accessibility violations that automated checkers often miss.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [51] [51] Juan-Miguel López-Gil and Juanan Pereira. 2025. Turning manual web accessibility success criteria into automatic: an LLM-based approach. Universal Access in the Information Society 24, 1 (2025), 837-852. - source

  • May combine two claims.

    Premisecitation
    § 1 Introduction · p. 1

    LLMs are trained on large-scale web code corpora that frequently violate WCAG standards, leading models to reproduce inaccessible patterns in their outputs.

    “LLMs are trained on large-scale web code corpora that frequently violate WCAG standards [20, 25]-leading models to reproduce inaccessible patterns in their outputs [57].”
    Quote in the paperSubstantial claim1 of 3 sources checked
    • cites [20] [20] BenCaldwell, Michael Cooper, Loretta Guarino Reid, Gregg Vanderheiden, Wendy Chisholm, John Slatin, and Jason White. 2008. Web content accessibility guidelines (WCAG) 2.0. WWWConsortium (W3C) 290, 1-34 (2008), 5-12. -

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [25] [25] Matjaž Debevc, Tilen Škraba, Bogdan Cerovac, Ines Kožuh, and Nataša Rajh. 2023. Monitoring website accessibility: Evaluating current approaches and a proposal for improvements. Journal of accessibility and design for all 13, 2 (2023), 162-187. -

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [57] [57] Achraf Othman, Amira Dhouib, and Aljazi Nasser Al Jabor. 2023. Fostering websites accessibility: A case study on the use of the Large Language Models ChatGPT for automatic remediation. In Proceedings of the 16th international conference on pervasive technologies related to assistive environments . 707-713. - source

      The citation [57] is attached to the claim that LLMs reproduce inaccessible patterns in their outputs. The Results and Analysis section says ChatGPT’s performance depends on training-data quality, and §4.2 says some accessibility issues required manual remediation. Neither passage says the model reproduced inaccessible code patterns or links its outputs to non-WCAG-compliant training code; searches for those points found no supporting passage. (LLM, conf=0.73)

      Experimental — accuracy not yet measured

  • Classified as probably not a substantial claim.

    Premisecitation
    § 1 Introduction · p. 1

    LLMs have achieved strong performance in generating source code from natural language descriptions.

    “In parallel, LLMs have achieved strong performance in generating source code from natural language descriptions [16, 22, 48, 76, 77].”
    Quote in the paperSubstantial claim3 of 5 sources checked
    • cites [16] [16] Rohan Bavishi, Caroline Lemieux, Roy Fox, Koushik Sen, and Ion Stoica. 2019. AutoPandas: neural-backed generators for program synthesis. Proceedings of the ACM on Programming Languages 3, OOPSLA (2019), 1-27. - source

      The Evaluation (§4.3) reports that AutoPandas synthesized solutions for 17 of 26 real-world benchmarks, supporting the narrower point that a neural-backed system can generate programs. The Introduction and Conclusion specify that it synthesizes pandas programs from input-output examples, not natural-language descriptions; the paper does not establish performance by LLMs. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
    • cites [22] [22] Mark Chen. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 (2021). - source
    • cites [48] [48] Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, et al. 2022. Competition-level code generation with alphacode. Science 378, 6624 (2022), 1092-1097. -

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [76] [76] Ting Zhang, Bowen Xu, Ferdian Thung, Stefanus Agus Haryono, David Lo, and Lingxiao Jiang. 2020. Sentiment analysis for software engineering: How far can pre-trained transformer models go?. In 2020 IEEE International Conference on Software Maintenance and Evolution (ICSME) . IEEE, 70-80. - source
    • cites [77] [77] Li Zhong, Zilong Wang, and Jingbo Shang. 2024. Debug like a human: A large language model debugger via verifying runtime execution step-by-step. arXiv preprint arXiv:2402.16906 (2024). - source

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductcitation
    § 3 A11yAgent · p. 3

    These tools were selected due to their widespread use in accessibility research and practice and for their established coverage of WCAG success criteria.

    “These tools were selected due to their widespread use in accessibility research and practice and for their established coverage of WCAG success criteria [40, 53, 62].”
    Quote in the paperSubstantial claim2 of 3 sources checked
    • cites [40] [40] Rita Ismailova and Yavuz Inal. 2022. Comparison of online accessibility evaluation tools: an analysis of tool effectiveness. IEEe Access 10 (2022), 58233-58239. - source

      The Study Design section says the six tools were used to assess WCAG 2.0 compliance, and the Introduction describes accessibility tools used in earlier research. But the Selection of Tools section gives different reasons for choosing them—single-page reports, non-commercial availability, and on-page results. It does not establish widespread use of the selected tools or WCAG success-criteria coverage as reasons for their selection. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
    • cites [53] [53] Sehi L'Yi, Harrison G Zhang, Andrew P Mar, Thomas C Smits, Lawrence Weru, Sofía Rojas, Alexander Lex, and Nils Gehlenborg. 2025. A comprehensive evaluation of life sciences data resources reveals significant accessibility barriers. Scientific Reports 15, 1 (2025), 23676. -

      Not checked in the free run.

    • cites [62] [62] Jonathan Robert Pool. 2023. Accessibility metatesting: comparing nine testing tools. In Proceedings of the 20th International Web for All Conference . 1-4. - source

      The Introduction and Related Work describe accessibility tools used in practice and prior research, and §3 says the nine tools seek to measure conformance to industry standards. But §3 identifies their selection criteria as maintenance, low cost, and availability as REST APIs or NPM packages—not widespread use or established coverage of WCAG success criteria. No page numbers are available in our copy. (LLM, conf=0.84)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Premisecitation
    § 3 A11yAgent · p. 5

    GPT-5.2 has demonstrated strong performance in multi-step reasoning and constraint formulation.

    “We therefore use GPT-5.2 7 , which has demonstrated strong performance in multi-step reasoning and constraint formulation [33].”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [33] [33] Arian Hosseini, Alessandro Sordoni, Daniel Toyama, Aaron Courville, and Rishabh Agarwal. 2024. Not all llm reasoners are created equal. arXiv preprint arXiv:2410.01748 (2024). - source

      The Experiments & Results section lists the models evaluated, including GPT-4o and GPT-4o mini, but not GPT-5.2. Searches for “GPT-5.2” and “constraint formulation” found no passage assessing that model’s performance in either multi-step reasoning or constraint formulation. The paper’s findings about other models do not establish the claim about GPT-5.2. (LLM, conf=0.76)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Study design and conductcitation
    § 4.1 Method · p. 7

    WAVE and aXe were selected because they are widely used in accessibility research and integrated into A11yAgent's internal evaluation pipeline, ensuring consistent measurement across all conditions.

    “These tools were selected because they are widely used in accessibility research [15, 17, 19, 21] and are integrated into A11yAgent's internal evaluation pipeline, ensuring consistent measurement across all conditions.”
    Quote in the paperSubstantial claim4 of 4 sources checked
    • cites [15] [15] Abdullah Alsaeedi. 2020. Comparing web accessibility evaluation tools and evaluating the accessibility of webpages: proposed frameworks. Information 11, 1 (2020), 40. - source

      The Conclusions describe WAVE as a well-known accessibility tool, and the Introduction gives an example of its use in accessibility research. The cited paper compares WAVE with SiteImprove, not aXe, so it supports the research-use rationale for WAVE but not for both named tools. The A11yAgent pipeline statement follows the citations and is not treated as attributed to this source. (LLM, conf=0.88)

      Experimental — accuracy not yet measured
    • cites [17] [17] Pooja Naresh Bhatia and Sam Malek. 2024. A historical review of Web Accessibility using WAVE. In Proceedings of the 5th ACM/IEEE Workshop on Gender Equality, Diversity, and Inclusion in Software Engineering . 55-62. - source

      Section 2.3 describes WAVE as a popular automated accessibility checker, and Section 3.3.1 uses it in the paper’s accessibility research. The cited paper does not establish that aXe is widely used; the statement about A11yAgent’s internal pipeline appears after the citations and is not attributed to this source. (LLM, conf=0.86)

      Experimental — accuracy not yet measured
    • cites [19] [19] Andreas Burkard, Gottfried Zimmermann, and Bettina Schwarzer. 2021. Monitoring systems for checking websites on accessibility. Frontiers in Computer Science 3 (2021), 628770. - source

      The Prior Work passage (Web Content Accessibility Guidelines 2.1; page unavailable) identifies two earlier accessibility-tool studies that evaluated WAVE. The Introduction and axe Monitor passages (pages unavailable) show that this paper evaluated WAVE and axe Expert, but they do not establish that axe was widely used in accessibility research. The citation is attached to the research-use rationale, not the statement about A11yAgent’s internal pipeline. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
    • cites [21] [21] Jennifer Campbell-Meier and Diane Velasquez. 2025. Measuring digital accessibility in New Zealand public libraries: A protocol-based web evaluation using the WAVE Tool. Journal of the Australian Library and Information Association 74, 3 (2025), 373-393. - source

  • May combine two claims.

    Premisecitation
    § 5 Discussion · p. 8

    Standard LLMs attempt to infer missing requirements when prompts are underspecified, which leads to inconsistent compliance.

    “Standard LLMs attempt to infer missing requirements when prompts are underspecified, which leads to inconsistent compliance [42, 75].”
    Quote in the paperSubstantial claim2 of 2 sources checked
    • cites [42] [42] Xue Jiang, Yihong Dong, Lecheng Wang, Zheng Fang, Qiwei Shang, Ge Li, Zhi Jin, and Wenpin Jiao. 2024. Self-planning code generation with large language models. ACM Transactions on Software Engineering and Methodology 33, 7 (2024), 1-30. - source

      The Introduction says that directly generating code satisfying complex intent is challenging for LLMs, and §6 (Threats to Validity) describes their sensitivity to prompts. Neither passage says that LLMs infer missing requirements from underspecified prompts or that this causes inconsistent compliance; searches for ambiguous or incomplete specifications did not establish that link. Page numbers are unavailable in our copy. (LLM, conf=0.76)

      Experimental — accuracy not yet measured
    • cites [75] [75] Chenyang Yang, Yike Shi, Qianou Ma, Michael Xieyang Liu, Christian Kästner, and Tongshuang Wu. 2025. What Prompts Don't Say: Understanding and Managing Underspecification in LLM Prompts. arXiv preprint arXiv:2505.13360 (2025). - source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 5 Discussion · p. 8

    Contrast adjustments account for a disproportionate share of maintenance effort in human-led accessibility remediation.

    “This behavior highlights challenges similar to those reported in human-led accessibility remediation, where contrast adjustments account for a disproportionate share of maintenance effort [20, 51].”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites [20] [20] BenCaldwell, Michael Cooper, Loretta Guarino Reid, Gregg Vanderheiden, Wendy Chisholm, John Slatin, and Jason White. 2008. Web content accessibility guidelines (WCAG) 2.0. WWWConsortium (W3C) 290, 1-34 (2008), 5-12. -

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [51] [51] Juan-Miguel López-Gil and Juanan Pereira. 2025. Turning manual web accessibility success criteria into automatic: an LLM-based approach. Universal Access in the Information Society 24, 1 (2025), 837-852. - source

      The Related work section discusses color-contrast false positives and the need for human verification, but does not compare the maintenance effort required for contrast adjustments with other remediation tasks. Searches for contrast remediation and maintenance workload found no passage addressing the claimed disproportionate share. (LLM, conf=0.76)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation
    § 5 Discussion · p. 8

    Accessibility must be operationalized as a system-level constraint rather than a stylistic preference.

    “and supports recent arguments that accessibility must be operationalized as a system-level constraint rather than a stylistic preference [58].”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [58] [58] Zsuzsanna B Palmer and Sushil K Oswal. 2025. Constructing websites with generative AI tools: The accessibility of their workflows and products for users with disabilities. Journal of Business and Technical Communication 39, 1 (2025), 93-114. - source

  • Classified as probably not a substantial claim.

    Premisecitation
    § 6 Threats to Validity · p. 11

    Prior iterative self-revision work suggests that a small number of rounds, such as three, is often sufficient for repair.

    “Prior iterative self-revision work suggests that a small number of rounds (e.g., three) is often sufficient for repair [24]”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [24] [24] Arup Datta, Ahmed Aljohani, and Hyunsook Do. 2025. Secure Code Generation at Scale with Reflexion. arXiv preprint arXiv:2511.03898 (2025). - source

  • May combine two claims.

    Interpretationcitation
    § 5 Discussion · p. 8

    Claude and GPT-5 generated hundreds of contrast violations, indicating a lack of inherent visual grounding for accessibility-critical design decisions.

    “Claude and GPT-5 generated hundreds of contrast violations, indicating a lack of inherent visual grounding for accessibility-critical design decisions [51, 75].”
    Quote in the paperSubstantial claim2 of 2 sources checked
    • cites [51] [51] Juan-Miguel López-Gil and Juanan Pereira. 2025. Turning manual web accessibility success criteria into automatic: an LLM-based approach. Universal Access in the Information Society 24, 1 (2025), 837-852. - source

      The Methods (section 3.2; page unavailable) list GPT-3.5, GPT-4, Claude, and Bard, but not GPT-5. The Discussion (section 5; page unavailable) describes tests of alternative text, link purpose, and language tags, not generated designs or counts of contrast violations. A search for hundreds of contrast violations and visual grounding found no passage addressing that claim. (LLM, conf=0.76)

      Experimental — accuracy not yet measured
    • cites [75] [75] Chenyang Yang, Yike Shi, Qianou Ma, Michael Xieyang Liu, Christian Kästner, and Tongshuang Wu. 2025. What Prompts Don't Say: Understanding and Managing Underspecification in LLM Prompts. arXiv preprint arXiv:2505.13360 (2025). - source

      The paper’s A.3 Process of requirements curation discusses testing GPT-4o and smaller models on prompt requirements; its front-end-development discussion concerns prompt optimization, not visual accessibility testing. Searches for Claude, GPT-5, contrast violations, and visual grounding found no passage reporting the claimed hundreds of violations or drawing that conclusion. (LLM, conf=0.76)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Premisecitation
    § 5 Discussion · p. 8

    Accessibility is not an inherent capability of current LLMs but a specialized constraint that is frequently overridden by training biases toward common, and often inaccessible, web design patterns.

    “This finding aligns with recent evidence that accessibility is not an inherent capability of current LLMs but a specialized constraint that is frequently overridden by training biases toward common, and often inaccessible, web design patterns [43, 51, 58].”
    Quote in the paperSubstantial claim2 of 3 sources checked
    • cites [43] [43] Malik Asif Jilani, Arslan Sheikh, Faiz Ali Shah, and Syed Muhammad Saqlain. 2026. Usability evaluation of academic library websites: a systematic literature review. Information Discovery and Delivery 54, 1 (2026), 50-70. - source
    • cites [51] [51] Juan-Miguel López-Gil and Juanan Pereira. 2025. Turning manual web accessibility success criteria into automatic: an LLM-based approach. Universal Access in the Information Society 24, 1 (2025), 837-852. - source

      The Abstract and §6 (Limitations; page unavailable) discuss LLM-based scripts for detecting accessibility errors and the need to choose prompts for each task. They do not assess whether accessibility is an inherent LLM capability or whether training biases cause LLMs to favor inaccessible web design patterns. Searches for those claims found no supporting passage. (LLM, conf=0.75)

      Experimental — accuracy not yet measured
    • cites [58] [58] Zsuzsanna B Palmer and Sushil K Oswal. 2025. Constructing websites with generative AI tools: The accessibility of their workflows and products for users with disabilities. Journal of Business and Technical Communication 39, 1 (2025), 93-114. - source

      In “Progress or Regression,” the paper reports that two of three AI website builders produced sites with no or only nominal screen-reader accessibility, and links some problems to insufficient disability- and accessibility-related training data. “To Sum Up” likewise identifies that data gap as a major culprit. The paper does not establish that accessibility is frequently overridden by training biases toward common web-design patterns, and its findings concern the tested builder tools rather than current LLMs generally. (LLM, conf=0.86)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Findingown data
    § 1 Introduction · p. 2

    A11yAgent substantially reduces accessibility violations while preserving functional correctness and code quality.

    “demonstrating that A11yAgent substantially reduces accessibility violations while preserving functional correctness and code quality.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 4.2 Results · p. 7

    Incorrect heading nesting occurred n=24 times in GPT, n=12 times in Qwen, and n=16 times in Claude.

    “Incorrect heading nesting occurred 𝑛 = 24 times in GPT, 𝑛 = 12 times in Qwen, and 𝑛 = 16 times in Claude.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 4.2 Results · p. 7

    Missing label text for text inputs appeared n=7 times in GPT, n=15 times in Qwen, and n=27 times in Claude.

    “Missing label text for text inputs appeared 𝑛 = 7 times in GPT, 𝑛 = 15 times in Qwen, and 𝑛 = 27 times in Claude”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 4.2 Results · p. 7

    Empty label text (SC 3.3.2) occurred n=59 times in GPT, n=28 times in Qwen, and n=63 times in Claude.

    “Empty label text (SC 3.3.2) occurred 𝑛 = 59 times in GPT, 𝑛 = 28 times in Qwen, and 𝑛 = 63 times in Claude.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 4.2 Results · p. 7

    A11yAgent eliminated all heading-related violations, producing consistently navigable heading hierarchies across all evaluated websites.

    “A11yAgent eliminated all headingrelated violations, producing consistently navigable heading hierarchies across all evaluated websites.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 4.2 Results · p. 7

    Under WCAG 1.3.1, baseline models repeatedly failed to produce valid landmark hierarchies, proper heading structure, and correctly associated form labels.

    “Under WCAG 1.3.1 (Info and Relationships), baseline models repeatedly failed to produce valid landmark hierarchies, proper heading structure, and correctly associated form labels.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 4.2 Results · p. 7

    Missing associated labels for text inputs occurred n=3 times in GPT, n=15 times in Qwen, and n=29 times in Claude.

    “while missing associated labels for text inputs occurred 𝑛 = 3 times in GPT, 𝑛 = 15 times in Qwen, and 𝑛 = 29 times in Claude.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 5 Discussion · p. 8

    Baseline models repeatedly generated content not contained within landmarks, incorrect heading hierarchies, text inputs without labels, missing associated labels, empty label elements, select fields without accessible names, and unlabeled textareas.

    “Baseline models repeatedly generated content not contained within landmarks, incorrect heading hierarchies, text inputs without labels, missing associated labels, empty label elements, select fields without accessible names, and unlabeled textareas.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 5 Discussion · p. 9

    More complex interfaces required additional processing: 4 websites took 30-60 seconds, and 3 took more than 60 seconds to reach full compliance.

    “More complex interfaces required additional processing: 4 websites took 30-60 seconds, and 3 took more than 60 seconds to reach full compliance.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 5 Discussion · p. 8

    The system reduced the affected elements by 60%, narrowing the violation count from 5 to 2.

    “the system reduced the affected elements by 60%, narrowing the violation count from 5 to 2.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 4.2 Results · p. 7

    Under WCAG 1.4.6, GPT produced n=658 violations, Qwen n=89, and Claude n=122.

    “Under WCAG 1.4.6 (Contrast - Enhanced), GPT produced 𝑛 = 658 violations, Qwen 𝑛 = 89, and Claude 𝑛 = 122.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Findingnone given
    § 5.1 Implications for Theory & Practice · p. 10

    The A11yAgent tool clearly displays the reasoning behind each decision, the identified WCAG violations, and the resulting code repairs, so it does not operate as a closed black box system.

    “The tool clearly displays the reasoning behind each decision, the identified WCAG violations, and the resulting code repairs, ensuring it does not operate as a closed”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 4.2 Results · p. 7

    Under WCAG 1.4.3, GPT generated n=46 violations, Qwen n=65, and Claude n=700.

    “Under WCAG 1.4.3 (Contrast - Minimum), GPT generated 𝑛 = 46 violations, Qwen 𝑛 = 65, and Claude 𝑛 = 700.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation
    § Abstract · p. 1

    LLMs' ability to produce accessibility-compliant web interfaces remains limited and underexplored.

    “However, their ability to produce accessibility-compliant web interfaces remains limited and underexplored.”
    • Also stated in: § 1 Introduction · p. 1 · plain
      “However, existing code generation research has largely prioritized functional correctness-ensuring code compiles and meets task requirements [44, 46, 66]-while giving limited attention to the accessibility of generated user interfaces [10].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [10] [10] Wajdi Aljedaani, Abdulrahman Habib, Ahmed Aljohani, Marcelo Eler, and Yunhe Feng. 2024. Does chatgpt generate accessible code? investigating accessibility challenges in llm-generated source code. In Proceedings of the 21st International Web for All Conference . 165-176. - source

  • May combine two claims.

    Premisecitation
    § 1 Introduction · p. 1

    Manual accessibility evaluation and remediation are time-consuming, resource-intensive, and often fail to produce fully accessible websites.

    “Moreover, manual accessibility evaluation and remediation are time-consuming, resource-intensive, and often fail to produce fully accessible websites [59].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [59] [59] Rohan Patel, Pedro Breton, Catherine M Baker, Yasmine N El-Glaly, and Kristen Shinohara. 2020. Why software is not accessible: Technology professionals' perspectives and challenges. In Extended abstracts of the 2020 CHI conference on human factors in computing systems . 1-9. - source

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given
    § 1 Introduction · p. 1

    Accessibility compliance requires coordinated handling of structure, semantics, interaction, and visual design.

    “This limitation is especially pronounced for accessibility, where compliance requires coordinated handling of structure, semantics, interaction, and visual design.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given
    § 4.1 Method · p. 7

    Each reported violation was mapped to the associated WCAG 2.2 guidelines and testable success criteria.

    “We mapped each reported violation to the associated WCAG 2.2 guidelines and testable success criteria.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning
    § 4.1 Method · p. 7

    The same prompts were provided to A11yAgent to generate corresponding websites, ensuring a consistent comparison setting between single-agent LLM outputs and the multi-agent framework.

    “The same prompts were then provided to A11yAgent to generate corresponding websites, ensuring a consistent comparison setting between single-agent LLM outputs and our multi-agent framework.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning
    § 4.1 Method · p. 6

    Participants were required to have at least three years of experience in web development, specializing in front-end or full-stack development, to ensure sufficient knowledge of web development practices, content creation, and common web elements.

    “First, participants were required to have at least three years of experience in web development, specializing in front-end or full-stack development. This criterion ensured that participants had sufficient knowledge of web development practices, content creation, and common web elements.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning
    § 4.1 Method · p. 7

    To ensure data integrity, reports from both tools were consolidated, duplicate entries removed, and the final unique violation count recorded for each generated interface.

    “To ensure data integrity, we consolidated the reports from both tools, removed duplicate entries, and recorded the final unique violation count for each generated interface.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data
    § 4.2 Results · p. 7

    Semantic correctness does not reliably emerge from current LLM generation, but can be enforced through explicit structural specification prior to code synthesis.

    “This confirms that semantic correctness does not reliably emerge from current LLM generation, but can be enforced through explicit structural specification prior to code synthesis.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationcitation
    § 3 A11yAgent · p. 5

    Deploying Qwen3-Coder at the execution layer improves compilability and reduces syntactic errors, while GPT-5.2 remains responsible for high-level specification and coordination.

    “Deploying Qwen3-Coder at the execution layer improves compilability and reduces syntactic errors, while GPT-5.2 remains responsible for high-level specification and coordination.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationnone given
    § 3 A11yAgent · p. 5

    The CRA's primary objective is to eliminate WCAG violations without altering the website's visual identity or functional logic.

    “Its primary objective is to eliminate WCAG violations without altering the website's visual identity or functional logic.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning
    § 3 A11yAgent · p. 5

    The system tracks violation patterns across iterations; when identical violations persist across consecutive iterations, the framework identifies the repair process as stalled and records this condition.

    “To avoid unproductive repair cycles, the system tracks violation patterns across iterations. When identical violations persist across consecutive iterations, the framework identifies the repair process as stalled and records this condition.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). The quote only partly states this claim.

    Study design and conductnone given
    § 3 A11yAgent · p. 5

    The post-repair verification enforces ten critical structural requirements including metadata, semantic structure, interactive elements, and media content.

    “This verification step acts as a final quality gate, ensuring adherence to ten critical structural requirements:”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Study design and conductreasoning
    § 3 A11yAgent · p. 4

    Using WAVE alongside aXe reduces reliance on a single rule engine and increases confidence in detected violations.

    “Using WAVE alongside aXe reduces reliance on a single rule engine and increases confidence in detected violations.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning
    § 4.1 Method · p. 7

    A11yAgent does not finalize a website until aXe and WAVE report zero violations, or the maximum iteration limit is reached.

    “Because the agent is built upon a closedloop architecture, it does not finalize a website until aXe and WAVE report zero violations, or the maximum iteration limit is reached.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given
    § 3 A11yAgent · p. 5

    The iteration and verification stage continues until either the complete elimination of all Critical and Serious violations, or the exhaustion of a predefined safety threshold (i ≤ 5).

    “This cycle continues until a strict termination condition is met: either the complete elimination of all Critical and Serious violations, or the exhaustion of a predefined safety threshold ( 𝑖 ≤ 5).”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning
    § 3 A11yAgent · p. 5

    This mechanism prevents infinite looping and supports stable, reproducible evaluation.

    “This mechanism prevents infinite looping and supports stable, reproducible evaluation.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductreasoning
    § 4.1 Method · p. 6

    Specialized training in web accessibility or WCAG standards was not an inclusion criterion, intentionally to preserve the ecological validity of the study.

    “Crucially, specialized training in web accessibility or WCAG standards was not an inclusion criterion. This was an intentional design choice to preserve the ecological validity of the study.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning
    § 3 A11yAgent · p. 3

    The Syntax Repair stage ensures that generated code is executable before accessibility evaluation.

    “The Syntax Repair stage (SR) ensures that generated code is executable before accessibility evaluation.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisereasoning
    § 2.2 Large Language Models in Accessibility. · p. 2

    There is currently no framework that integrates strict accessibility checks directly into the creation process to ensure the code is accessible from the very beginning.

    “There is currently no framework that integrates strict accessibility checks directly into the creation process to ensure the code is accessible from the very beginning.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning
    § 4.1 Method · p. 6

    Participants were instructed to write prompts that included common web elements to ensure a meaningful accessibility evaluation.

    “Prompts were required to include common web elements, such as navigation menus, headers, footers, images, videos, and forms, to ensure a meaningful accessibility evaluation.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning
    § 3 A11yAgent · p. 3

    The output of the Syntax Repair stage is a fully functional web page ready for automated accessibility assessment.

    “The output of this stage is a fully functional web page ready for automated accessibility assessment.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given
    § 2.2 Large Language Models in Accessibility. · p. 2

    Current approaches rely on post-hoc repair; fixing code only after it has been written and errors are found.

    “Ultimately, current approaches rely on post-hoc repair; fixing code only after it has been written and errors are found.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § 3 A11yAgent · p. 5

    This design enables fully automated generation and verification of accessibility-aware websites, while explicitly reporting cases where accessibility requirements could not be fully satisfied.

    “This design enables fully automated generation and verification of accessibility-aware websites, while explicitly reporting cases where accessibility requirements could not be fully satisfied.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning
    § 3 A11yAgent · p. 5

    The CRA is explicitly restricted from adding new functionality, sections, or content that does not exist in the original prompt, ensuring that the repair process focuses exclusively on accessibility compliance.

    “The CRA is explicitly restricted from adding new functionality, sections, or content that does not exist in the original prompt, ensuring that the repair process focuses exclusively on accessibility compliance.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductcitation
    § 3 A11yAgent · p. 3

    A11yAgent performs automated testing using two complementary evaluation tools: aXe and WAVE.

    “A11yAgent performs automated testing using two complementary evaluation tools: aXe 1 and WAVE 2 .”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning
    § 4.1 Method · p. 6

    After screening, the final participant pool consisted of 20 developers.

    “After screening, three participants were excluded due to having less than three years of experience, and four were excluded because they worked only in backend development. As a result, the final participant pool consisted of 20 developers.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Study design and conductnone given
    § 3 A11yAgent · p. 4

    These enhanced checks improve the detection of interaction and perception barriers that are often under-reported by standard automated tools.

    “These enhanced checks improve the detection of interaction and perception barriers that are often under-reported by standard automated tools.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given
    § 4.1 Method · p. 6

    Participants needed to be familiar with LLM-based tools and have practical experience using LLMs for development tasks.

    “Second, participants needed to be familiar with LLM-based tools and have practical experience using LLMs for development tasks.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning
    § 3 A11yAgent · p. 5

    The interface's visibility enables it to function as a pedagogical model, allowing developers to learn and internalize accessibility standards during the generation process.

    “Importantly, this visibility enables the interface to function as a pedagogical model. Developers can actively learn and internalize accessibility standards during the generation process by observing how agents directly map specific WCAG violations to corresponding code repairs.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data
    § 4.2 Results · p. 8

    Residual errors indicate strong stability at the foundational Level A tier, particularly compared with baseline models, in which structural omissions were widespread.

    “These residual errors indicate strong stability at the foundational Level A tier, particularly compared with baseline models, in which structural omissions were widespread.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning
    § 5 Discussion · p. 10

    A11yAgent represents a shift from probabilistic accessibility to verifiable accessibility.

    “In this sense, A11yAgent represents a shift from probabilistic accessibility to verifiable accessibility-an essential step toward reliable, automated, and inclusive web development.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given
    § 5 Discussion · p. 8

    Accessibility checks are enforced at multiple stages-during initial accessibility detection, throughout iteration cycles, and again at final verification-to ensure no error slips through.

    “We also enforce accessibility checks at multiple stages-during initial accessibility detection, throughout the iteration cycles, and again at final verification-to ensure no error slips through.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning
    § 5.1 Implications for Theory & Practice · p. 10

    The A11yAgent framework can serve as an embedded learning tool.

    “The A11yAgent framework can serve as an embedded learning tool.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationown data
    § 5.1 Implications for Theory & Practice · p. 10

    The success of A11yAgent's pipeline suggests a paradigm shift toward proactive enforcement of accessibility.

    “The success of A11yAgent's pipeline suggests a paradigm shift toward proactive enforcement.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Digital accessibility has been a reactive process: developers build a product, auditors identify violations, and teams attempt to fix them later.

  • May combine two claims.

    Interpretationown data
    § 5 Discussion · p. 9

    Because the initial generation already incorporates structural constraints, the repair agents receive significantly fewer errors to fix, minimizing the duration of the iterative correction loop.

    “Because the initial generation already incorporates structural constraints, the repair agents receive significantly fewer errors to fix (e.g., the maximum number of violations received on the first iteration amounted to 5), thereby minimizing the duration of the iterative correction loop.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning
    § 6 Threats to Validity · p. 11

    Participants were required to generate realistic, developer-authored prompts and to compare outputs under identical conditions.

    “we required participants to generate realistic, developerauthored prompts and to compare outputs under identical conditions”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data
    § 4.2 Results · p. 7

    While multi-agent repair substantially reduced contrast failures compared to baselines, color-on-color logic remains the dominant residual challenge for the multi-agent approach.

    “Thus, while multi-agent repair substantially reduced contrast failures compared to baselines, color-on-color logic remains the dominant residual challenge for our multi-agent approach.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductreasoning
    § 6 Threats to Validity · p. 11

    The study used two complementary accessibility tools, consolidated overlapping results, and applied deterministic post-repair validation rules to enforce critical semantic requirements such as landmarks, labels, and heading structure.

    “we used two complementary tools, consolidated overlapping results, and applied deterministic post-repair validation rules to enforce critical semantic requirements (e.g., landmarks, labels, and heading structure).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data
    § 5 Discussion · p. 8

    Single-pass LLMs are capable of generating functional code but consistently fail to produce accessible code.

    “Our empirical study results highlight that, while single-pass LLMs are capable of generating functional code, they consistently fail to produce accessible code.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisecitation
    § 5 Discussion · p. 8

    Adequate contrast is a mathematical property requiring explicit measurement rather than subjective judgment.

    “Accessibility research has long established that adequate contrast is a mathematical property requiring explicit measurement rather than subjective judgment [20].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [20] [20] BenCaldwell, Michael Cooper, Loretta Guarino Reid, Gregg Vanderheiden, Wendy Chisholm, John Slatin, and Jason White. 2008. Web content accessibility guidelines (WCAG) 2.0. WWWConsortium (W3C) 290, 1-34 (2008), 5-12. -

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductreasoning
    § 6 Threats to Validity · p. 11

    The number of repair rounds is capped at i ≤ 5 as a conservative safety threshold that bounds computational cost while accommodating harder cases where multiple interdependent issues must be resolved sequentially.

    “We cap the number of repair rounds at 𝑖 ≤ 5 as a conservative safety threshold that bounds computational cost while accommodating harder cases where multiple interdependent issues must be resolved sequentially.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning
    § 5.1 Implications for Theory & Practice · p. 10

    Transparent systems can help developers learn and apply accessibility best practices naturally within their regular workflow.

    “transparent systems can help developers learn and apply accessibility best practices naturally within their regular workflow.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning
    § 5 Discussion · p. 10

    Accessible code generation requires architectures that explicitly encode accessibility constraints, measure compliance using independent tools, and iteratively correct failures.

    “Instead, accessible code generation requires architectures that explicitly encode accessibility constraints, measure compliance using independent tools, and iteratively correct failures.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning
    § 6 Threats to Validity · p. 11

    The results may not generalize to large-scale frameworks or mobile applications.

    “acknowledging that the results may not generalize to large-scale frameworks or mobile applications.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given
    § 6 Threats to Validity · p. 11

    The study involved 20 professional developers recruited via convenience sampling.

    “The study involved 20 professional developers recruited via convenience sampling”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data
    § 4.2 Results · p. 8

    Closed-loop multi-agent remediation effectively resolves most fundamental accessibility barriers, while visual contrast remains the primary unresolved challenge for fully automated WCAG-compliant web generation.

    “Overall, these results show that closed-loop multi-agent remediation effectively resolves most fundamental accessibility barriers, while visual contrast remains the primary unresolved challenge for fully automated WCAG-compliant web generation.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductreasoning
    § 6 Threats to Validity · p. 11

    Multiple state-of-the-art LLMs were evaluated using the same prompts and identical evaluation criteria, ensuring that observed differences are attributable to the framework rather than experimental variance.

    “we evaluated multiple state-of-the-art LLMs using the same prompts and identical evaluation criteria, ensuring that observed differences are attributable to the framework rather than experimental variance.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductreasoning
    § 6 Threats to Validity · p. 11

    The use of a controlled single-pass generation baseline enables isolation of framework effects, though it does not fully reflect the iterative practices typical of professional development.

    “the use of a controlled single-pass generation baseline enables isolation of framework effects, though it does not fully reflect the iterative practices typical of professional development”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given
    § 6 Threats to Validity · p. 11

    The findings may be influenced by the selected baseline models and prompting strategies, which may evolve over time.

    “our findings may be influenced by the selected baseline models and prompting strategies, which may evolve over time.”
    Quote in the paperNot a substantial claim
75 of 75

  • The paper relies on this without support.

    The majority of generalist web developers lack deep accessibility expertise, so the participant pool accurately reflects the real-world target audience for whom A11yAgent is designed.

    “Because the majority of generalist web developers lack deep accessibility expertise, this participant pool accurately reflects the real-world target audience for whom A11yAgent is explicitly designed to assist.”

  • The paper relies on this without support.

    A11yAgent's internal evaluation loop using aXe and WAVE yields violation counts comparable to external auditing of baseline websites.

    “Notably, while the baseline websites underwent external auditing (WAVE and aXe), A11yAgent's results were derived from its internal evaluation loop.”

  • The paper relies on this without support.

    Participant-written prompts are realistic developer-authored prompts.

    “This process produced realistic developer-authored prompts.”

  • The paper relies on this without support.

    The selected LLMs (GPT, Qwen, Claude) are state-of-the-art and representative baselines for single-agent LLM-based web code generation.

    “Participants were then given the autonomy to select an LLM from a predefined list of state-of-the-art models to generate the requested website.”

  • The paper relies on this without support.

    Digital accessibility has been a reactive process: developers build a product, auditors identify violations, and teams attempt to fix them later.

    “Digital accessibility has been a reactive process: developers build a product, auditors identify violations, and teams attempt to fix them later.”

  • The paper relies on this without support.

    Currently, creating WCAG-compliant interfaces requires deep expertise in complex standards such as ARIA roles, creating a bottleneck for many developers.

    “Currently, creating WCAG-compliant interfaces requires deep expertise in complex standards (e.g., ARIA roles), creating a bottleneck for many developers.”
6 of 6

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Nested interactive controls attributed to the wrong model

    Minor

    The results text attributes three nested-interactive-control violations to Claude, but Table 1 assigns three to GPT and zero to Claude.

  • Focus-order and focus-visible footnotes are reversed

    Minor

    The sentence associates focus order with footnote 3 and visible focus indicators with footnote 4, but the linked pages are respectively titled focus-visible and focus-order.

2 of 2

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. Matjaž Debevc Tilen Škraba Bogdan Cerovac Ines Kožuh and Nataša Rajh. 2023. Monitoring website accessibility: Evaluating current approaches and a proposal for improvements. Journal of accessibility and design for all 13 2 (2023) 162–187.

Cited by

Sorted by citations

Authors
Aljedaani, W., Shaik, M. A., & Aljohani, A.
Affiliations
  • Rochester Institute of Technology (US)
  • Wade Trim Group Inc., Pittsburgh, USA
  • Saudi Electronic University (SA)
ORCID
Venue
Proceedings of the 23rd International Web for All Conference
Publisher
ACM
Published
2026-04-13
Pages
147-159
DOI
10.1145/3800424.3800453
OpenAlex
W7163149535
Semantic Scholar
350ba74a686541b28340b6ad27eeca2f263df75e
Cited by
1
References
48
Citations by year
2026: 1
FWCI
8.85
Type
conference paper
Open access
Licence
Locations
International Web for All Conference (W4A) (publishedVersion, cc-by)
Topics
  • Social Sciences › Human Factors and Ergonomics › Digital Accessibility for Disabilities
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
  • Physical Sciences › Computer Science › Information Systems › Mobile and Web Applications
Keywords
web accessibility, large language models, automatic code generation, multi-agent framework, WCAG compliance, accessibility violations, front-end web development, prompt refinement, accessibility evaluation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-25.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-25.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review