CoChat
Sign Up
Cited by 19Multimodal Machine Learning ApplicationsSoftware Engineering ResearchMobile and Web Applications

WebCode2M: A Real-World Dataset for Code Generation from Webpage Designs

Open access
Gui, Yi, Li, Zhen, Wan, Yao, Shi, Yemin, Zhang, Hongyu, Chen, Bohua, Su, Yi, Chen, Dongping +5 more·2025·Proceedings of the ACM on Web Conference 2025
DOI 10.1145/3696410.3714889
Open this paper in CoChat — free accountOpen PDF

Abstract

Automatically generating webpage code from webpage designs can significantly reduce the workload of front-end developers, and recent Multimodal Large Language Models (MLLMs) have shown promising potential in this area. However, our investigation reveals that most existing MLLMs are constrained by the absence of high-quality, large-scale, real-world datasets, resulting in inadequate performance in automated webpage code generation. To fill this gap, this paper introduces WebCode2M, a new dataset comprising 2.56 million instances, each containing a design image along with the corresponding webpage code and layout details. Sourced from real-world web resources, WebCode2M offers a rich and valuable dataset for webpage code generation across a variety of applications. The dataset quality is ensured by a scoring model that filters out instances with aesthetic deficiencies or other incomplete elements. To validate the effectiveness of WebCode2M, we introduce a baseline model based on the Vision Transformer (ViT), named WebCoder, and establish a benchmark for fair comparison. Additionally, we introduce a new metric, TreeBLEU, to measure the structural hierarchy recall. The benchmarking results demonstrate that our dataset significantly improves the ability of MLLMs to generate code from webpage designs, confirming its effectiveness and usability for future applications in front-end design tools. Finally, we highlight several practical challenges introduced by our dataset, calling for further research. The code and dataset are publicly available at our project homepage: https://webcode2m.github.io.

AI snapshot

Written from the paper’s full text.

TL;DR

WebCode2M is a 2.56-million-instance real-world dataset pairing webpage design images with HTML/CSS code and layout trees, assembled from Common Crawl through purification, rendering, neural scoring, and DOM extraction. Fine-tuning Pix2Struct-1.3B as WebCoder on this data improves structural and visual code-generation metrics over specialized baselines and most general MLLMs, though GPT-4o remains stronger on visual similarity.

Key findings

  • The dataset contains 2,563,905 training instances plus three 256-sample test splits by code length; average training samples have 31,216 tokens, 158 tags, and DOM depth 13, compared with WebSight’s 647 tokens, 19 tags, and depth 5.
  • A ResNet-50 neural scorer trained on 10,000 manually scored screenshots reached 75% exact-score accuracy and about 90% binary high/low accuracy; filtering removed 52.5% of the raw data scoring below 2.
  • WebCoder-1.3B, fine-tuned on WebCode2M, outperformed WebSight VLM-7B, Design2Code-18B, and a WebSight-trained Pix2Struct baseline across all metrics on the test subsets; its TreeBLEU was 0.35, 0.22, and 0.15 on Short, Mid, and Long, versus roughly 0.03–0.06 for the specialized baselines.
  • Among general MLLMs, GPT-4o performed best overall, with visual scores of 0.85, 0.81, and 0.82 and CLIP scores of 0.77, 0.77, and 0.74 across Short, Mid, and Long; most models degraded as target code length increased.

Methods

  • Data came from the first segment of Common Crawl’s CC-MAIN-2023-50 version, with about 0.5 billion webpages randomly sampled; HTML/CSS was purified by length filtering, removing comments, meta/script tags, hidden elements, and ineffective styles, then merged into a single style block.
  • Screenshots were rendered with Playwright, filtered by the neural scorer, and paired with layout trees containing bounding boxes, size, location, and hierarchy; the final dataset was de-duplicated by screenshot hash and manually inspected for test candidates.
  • WebCoder was produced by fully fine-tuning Pix2Struct-1.3B on WebCode2M, using up to 1,024 image patches, sequence lengths of 2,048 then 1,024 tokens, batch size 1, and 4 NVIDIA A800 80GB GPUs; evaluation used CLIP similarity, Visual Score, and the proposed TreeBLEU.

Limitations

  • The paper states that the neural scorer is subjective and only about 90% accurate, so some low-quality data may remain, and manually screening millions of entries was impractical.
  • The paper states that models show significant variance across metrics, indicating unstable generation and a need for more robust frameworks and better evaluation metrics.
  • The paper states that web-crawled data may retain a minimal amount of inappropriate content despite NSFW and profanity filtering; evident from the text, the screenshots are rendered from cleaned code rather than captured from live pages, and each test subset contains only 256 examples.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1145/3696410.3714889

Cites

34 of 34 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

16 of 24 cited sources checked · 5 supported · 1 flagged · accuracy not yet measured

  • Study design and conductreasoning

    To ensure a fair comparison, all baselines employ the one-pass generation strategy.

    “Therefore, to ensure a fair comparison, all baselines will employ the one-pass generation strategy.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [45] 45. Chenglei Si, Yanzhe Zhang, Zhengyuan Yang, Ruibo Liu, and Diyi Yang. 2024. Design2Code: How Far Are We From Automating Front-End Engineering? CoRR abs/2403.03163 (2024). source

      The paper’s Self-Revision Prompting section describes a second generation step: the model receives its initial code and rendered webpage, then revises the code. Table 1 also evaluates self-revision alongside direct and text-augmented prompting. Thus, the paper does not say that all its baselines use one-pass generation; the citing sentence’s future-tense choice about its own baselines is not established by this source. (LLM, conf=0.85)

      Checked by a reasoning model — may be wrong.

      Experimental — accuracy not yet measured
  • Premisecitation

    Modern MLLMs, such as GPT-4V, exhibit a notable decline in performance when confronted with real-world webpage designs that feature complex structures and a larger variety of unique HTML tags.

    “Modern MLLMs, such as GPT-4V, also exhibit a notable decline in performance when confronted with real-world webpage designs that feature complex structures and a larger variety of unique HTML tags [45].”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [45] 45. Chenglei Si, Yanzhe Zhang, Zhengyuan Yang, Ruibo Liu, and Diyi Yang. 2024. Design2Code: How Far Are We From Automating Front-End Engineering? CoRR abs/2403.03163 (2024). source

      Section 4.5 reports a significant performance drop on hard real-world webpages for GPT-4o, and Appendix Table 7 identifies the total number of HTML tags as a strong difficulty indicator for GPT-4V. The copy does not establish that a larger variety of unique tags causes the decline, and the explicitly reported drop is for GPT-4o rather than GPT-4V. (LLM, conf=0.86)

      Experimental — accuracy not yet measured
  • Premisecitation

    A significant disparity exists between WebSight samples and real-world data.

    “However, a significant disparity exists between these samples and real-world data [45].”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [45] 45. Chenglei Si, Yanzhe Zhang, Zhengyuan Yang, Ruibo Liu, and Diyi Yang. 2024. Design2Code: How Far Are We From Automating Front-End Engineering? CoRR abs/2403.03163 (2024). source
  • Premisecitation

    WebSight is a dataset comprising approximately 0.8 million synthesized samples generated by LLMs.

    “WebSight [27] is another dataset comprising approximately 0.8 million synthesized samples generated by LLMs.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [27] 27. Hugo Laurençon, Léo Tronchon, and Victor Sanh. 2024. Unlocking the conver- sion of Web Screenshots into HTML Code with the WebSight Dataset. CoRR abs/2403.09029 (2024). arXiv:2403.09029 source

      The Related work passage says the earlier WebSight-v0.1 release contained 823K synthetic screenshot–HTML pairs, and the dataset-construction passages describe using LLMs to generate the HTML. But the paper describes the current WebSight-v0.2 dataset as containing 2 million pairs; the citing sentence omits the v0.1 qualifier needed for its 0.8 million figure. (LLM, conf=0.88)

      Experimental — accuracy not yet measured
  • Findingown data

    WebCoder, fine-tuned from the smaller Pix2Struct-1.3B, outperforms both Design2Code-18B and WebSight VLM-8B across all evaluation metrics, including CLIP-based visual similarity, low-level appearance accuracy, and TreeBLEU.

    “Compared to the two fine-tuned baselines, Design2Code-18B [45] and WebSight VLM-8B [27], our model, fine-tuned from the smaller Pix2Struct-1.3B,outperforms both across all evaluation metrics, including CLIP-based visual similarity [35], low-level appearance accuracy [45], and our proposed TreeBLEU to measure the structural hierarchy recall.”
    Quote in the paperSubstantial claim3 of 3 sources checked
    • cites [45] 45. Chenglei Si, Yanzhe Zhang, Zhengyuan Yang, Ruibo Liu, and Diyi Yang. 2024. Design2Code: How Far Are We From Automating Front-End Engineering? CoRR abs/2403.03163 (2024). source
    • cites [27] 27. Hugo Laurençon, Léo Tronchon, and Victor Sanh. 2024. Unlocking the conver- sion of Web Screenshots into HTML Code with the WebSight Dataset. CoRR abs/2403.09029 (2024). arXiv:2403.09029 source
    • cites [35] 35. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. In Proceedings of the 38th International Con- ference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event (Proceedings of Machine Learning Research, Vol. 139). PMLR, 8748-8763. source

      The cited paper’s Table 14 (p. 38) reports results for T5 variants, while §3.7 (p. 40) discusses language-task benchmarks. Searches for WebCoder, Pix2Struct, the named baselines, and the visual and TreeBLEU metrics found no passage addressing the claimed comparison. (LLM, conf=0.78)

      Experimental — accuracy not yet measured
  • May combine two claims. 34
  • Classified as probably not a substantial claim. 15
  • The quote only partly states this claim. 2

  • May combine two claims.

    Premisecitation

    Design2Code contains only 484 real-world samples, intended solely for testing and insufficient for effective fine-tuning.

    “For instance, Design2Code [45] contains only 484 real-world samples, intended solely for testing and insufficient for effective fine-tuning.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [45] 45. Chenglei Si, Yanzhe Zhang, Zhengyuan Yang, Ruibo Liu, and Diyi Yang. 2024. Design2Code: How Far Are We From Automating Front-End Engineering? CoRR abs/2403.03163 (2024). source

      Section 2.1 (Test Set Curation; page unavailable) says the authors curated 484 real-world webpages for their benchmark. Section E (Finetuning details; page unavailable) says they fine-tuned on WebSight instead, but does not establish that 484 samples are insufficient for effective fine-tuning. The claim of insufficiency is therefore unsupported by this source. (LLM, conf=0.88)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Premisecitation

    Existing datasets are either too small to provide meaningful generalization or consist of synthetic data that does not fully capture the complexity and variability of real-world webpage designs.

    “However, this approach faces a significant limitation since existing datasets are either too small to provide meaningful generalization [21,45] or consist of synthetic data that does not fully capture the complexity and variability of real-world webpage designs [27,54].”
    Quote in the paperSubstantial claim3 of 4 sources checked
    • cites [21] 21. Hongcheng Guo, Wei Zhang, Junhao Chen, Yaonan Gu, Jian Yang, Junjia Du, Binyuan Hui, Tianyu Liu, Jianxin Ma, Chang Zhou, and Zhoujun Li. 2024. IW- Bench: Evaluating Large Multimodal Models for Converting Image-to-Web. CoRR abs/2409.18980 (2024). source
    • cites [45] 45. Chenglei Si, Yanzhe Zhang, Zhengyuan Yang, Ruibo Liu, and Diyi Yang. 2024. Design2Code: How Far Are We From Automating Front-End Engineering? CoRR abs/2403.03163 (2024). source
    • cites [27] 27. Hugo Laurençon, Léo Tronchon, and Victor Sanh. 2024. Unlocking the conver- sion of Web Screenshots into HTML Code with the WebSight Dataset. CoRR abs/2403.09029 (2024). arXiv:2403.09029 source

      The paper describes WebSight as a synthetic dataset, and §5.1 reports that a model trained on it struggles with complex layouts and designs substantially different from its training data. This supports a limitation of this synthetic dataset, but the paper does not establish that synthetic datasets generally fail to capture the complexity and variability of real-world webpage designs. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
    • cites [54] 54. Sukmin Yun, Haokun Lin, Rusiru Thushara, Mohammad Qazim Bhat, Yongxin Wang, Zutao Jiang, Mingkai Deng, Jinhong Wang, Tianhua Tao, Junbo Li, Haonan Li, Preslav Nakov, Timothy Baldwin, Zhengzhong Liu, Eric P. Xing, Xiaodan Liang, and Zhiqiang Shen. 2024. Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMs. CoRR abs/2406.20098 (2024). arXiv:2406.20098 source

      For the synthetic-data clause cited to this source, Table 1 (Section 3, page not specified) identifies several existing datasets as synthetic and shows that they have substantially shorter code and fewer tags than the real-world Design2Code dataset. Section 3.1 also says WebSight covers a narrower spectrum of complexity than Web2Code. These comparisons support a concern about complexity, but the paper does not establish the broader claim that synthetic datasets fail to capture the variability of real-world webpage designs. (LLM, conf=0.82)

      Experimental — accuracy not yet measured

  • Classified as probably not a substantial claim.

    Premisecitation

    The complexity of code generation increases with the growth in the total number of HTML tags, the diversity of unique tags, and the depth of the DOM tree.

    “as highlighted in a recent study [45], the complexity of code generation increases with the growth in the total number of HyperText Markup Language (HTML) tags, the diversity of unique tags, and the depth of the Document Object Model (DOM) tree.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [45] 45. Chenglei Si, Yanzhe Zhang, Zhengyuan Yang, Ruibo Liu, and Diyi Yang. 2024. Design2Code: How Far Are We From Automating Front-End Engineering? CoRR abs/2403.03163 (2024). source

      Section 4.5 and the Table 7 caption say that webpages with more HTML tags tend to be harder for the model to generate. Section 2.2 treats DOM-tree depth as a measure of complexity and records the number of unique tags, but the cited passages do not establish that generation difficulty increases as either of those measures grows. (LLM, conf=0.85)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Findingcitation

    Webpage code from online websites is often lengthy and includes redundant elements, which significantly impairs the model's ability to accurately learn the correlation between webpage code and screenshots.

    “Our investigation reveals that webpage code from online websites is often lengthy and includes redundant elements, which significantly impairs the model's ability to accurately learn the correlation between webpage code and screenshots [27].”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [27] 27. Hugo Laurençon, Léo Tronchon, and Victor Sanh. 2024. Unlocking the conver- sion of Web Screenshots into HTML Code with the WebSight Dataset. CoRR abs/2403.09029 (2024). arXiv:2403.09029 source

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductcitation

    For the candidate test dataset, duplicates were further removed using CLIP similarity and a manual inspection was conducted on each entry.

    “For the candidate test dataset, we further remove duplicates using CLIP [35] similarity and conduct a manual inspection on each entry.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [35] 35. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. In Proceedings of the 38th International Con- ference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event (Proceedings of Machine Learning Research, Vol. 139). PMLR, 8748-8763. source

      The citation is attached to “CLIP similarity,” but the searched passages contain no mention of CLIP or image-similarity deduplication. Section 2.2 (p. 6) describes deduplicating C4 text by repeated three-sentence spans, not the citing authors’ candidate test dataset; searches for CLIP similarity and manual inspection found no supporting passage. (LLM, conf=0.78)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Findingown data

    WebSight samples average 647 tokens, 19 tags, and a DOM depth of 5, whereas real-world samples can involve up to 50 times more tokens, six times as many tags, and double the DOM depth.

    “Specifically, WebSight samples average 647 tokens, 19 tags, and a DOM depth of 5, whereas our study reveals that real-world samples can involve up to 50 times more tokens, six times as many tags, and double the DOM depth (see Figure 2).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    On average, WebCode2M samples contain 31,216 tokens, 158 tags, and a DOM depth of 13.

    “On average, the samples contain 31,216 tokens, 158 tags, and a DOM depth of 13.”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data

    WebCoder surpasses both specialized and general-purpose models on TreeBLEU.

    “For Tree-BLEU-a metric measuring the recall of 1-height subtrees in the target DOM tree-our approach surpasses both specialized and general-purpose models, indicating that our model better reflects real-world node types and substructures.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    GPT-4o achieves higher similarity in CLIP and visual appearance than WebCoder but has a lower substructure recall rate.

    “The only exception is GPT-4o, which achieves higher similarity in CLIP and visual appearance but has a lower substructure recall rate.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    GPT-4V excels in capturing text and color from images when generating webpage code but struggles with capturing the hierarchy of UI elements.

    “Our empirical study of GPT-4V reveals that, while it excels in capturing text and color from images when generating webpage code, it struggles with capturing the hierarchy of UI elements.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    GPT-4o's performance remains highly stable as the complexity increases, without showing significant degradation; as the dataset transitions from WebCode2M-Short to WebCode2M-Mid and then to WebCode2M-Long, the visual score changes from 0.85 to 0.81, and then to 0.82.

    “Moreover, its performance remains highly stable as the complexity increases, without showing significant degradation.For instance, as the dataset transitions from WebCode2M-Short to WebCode2M-Mid and then to WebCode2M-Long, the visual score changes from 0.85 to 0.81, and then to 0.82.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Findingnone given

    Around one-third of the test cases fail due to the recitation error of Gemini.

    “Around one-third of the test cases fail due to the”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Most general-purpose MLLMs struggle with webpage code generation; among these models, only GPT-4V matches the performance of WebCoder trained on WebCode2M, while GPT-4o significantly outperforms all other models, and all remaining general-purpose models generally underperform compared to specialized models.

    “(3) Most general-purpose MLLMs struggle with webpage code generation.Among these models, only GPT-4V matches the performance of our model trained on WebCode2M, while GPT-4o significantly outperforms all other models.All remaining general-purpose models generally underperform compared to specialized models, with consistently low scores across all metrics.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The pix2code dataset comprises basic block elements and text-based UI elements, suitable for both Android and iOS UIs; WebSight consists of structurally simple webpages; WebCode2M closely mirrors typical real webpages, featuring a variety of layout structures and rich elements such as images.

    “The pix2code dataset comprises basic block elements and text-based UI elements, suitable for both Android and iOS UIs.In contrast, WebSight consists of structurally simple webpages.Our dataset, on the other hand, closely mirrors typical real webpages, featuring a variety of layout structures and rich elements such as images.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingnone given

    WebCoder is a baseline model based on the ViT, introduced to establish a benchmark for fair comparison.

    “We introduce a baseline model based on the ViT, named WebCoder, and establish a benchmark for fair comparison.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Using the scorer, samples with scores less than two were removed, which accounted for 52.5% of the entire raw dataset.

    “Using this scorer, we remove samples with scores less than two, which accounted for 52.5% of the entire raw dataset (as shown in the outer circle of Figure 4).”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Manually screening all the data is impractical.

  • May combine two claims.

    Findingown data

    In the LLaVA family, several models show a significant improvement across all metrics as model parameters increase, with LLaVA-v1.5-7B and LLaVA-onevision-7B achieving the best performance, while LLaVA-onevision-0.5B performs poorly across all metrics.

    “(2) Model size matters.In LLaVA family, several models show a significant improvement across all metrics as model parameters increase, with LLaVA-v1.5-7Band LLaVA-onevision-7B achieving the best performance, while LLaVA-onevision-0.5Bperforms poorly across all metrics, indicating that MLLMs require more parameters to achieve better results in webpage generation tasks.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    English constitutes approximately 50% of the corpus, with other languages such as Russian, German, Spanish, and French each representing nearly 5% of the total corpus.

    “English constitutes approximately 50% of the corpus, with other languages such as Russian, German, Spanish, and French each representing nearly 5% of the total corpus.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    WebCode2M includes both a comprehensive training dataset and a high-quality test dataset, making it much larger in scale.

    “This dataset includes both a comprehensive training dataset and a high-quality test dataset, making it much larger in scale.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    WebCode2M is significantly larger than pix2code, Design2Code, and WebSight, comprising 2.56 million samples.

    “Compared to pix2code and De-sign2Code, which contain only a few thousand or fewer data samples, and WebSight, which includes 0.8 million samples, our dataset is significantly larger, comprising 2.56 million samples.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    On the two visual similarity metrics (visual score and CLIP similarity), WebCoder exceeds most general-purpose models and either matches or outperforms GPT-4V.

    “Additionally, on the two visual similarity metrics-visual score and CLIP similarity-our model exceeds most general-purpose models and either matches or outperforms GPT-4V.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    As the dataset transitions from WebCode2M-Short to WebCode2M-Mid and finally to WebCode2M-Long, the highest TreeBLEU score for specialized models drops from 0.35 to 0.15, the highest CLIP similarity decreases from 0.73 to 0.69, and the highest Visual Score declines from 0.78 to 0.65.

    “For example, as the dataset transitions from WebCode2M-Short to WebCode2M-Mid and finally to WebCode2M-Long, the highest TreeBLEU score for specialized models drops from 0.35 to 0.15, the highest CLIP similarity decreases from 0.73 to 0.69, and the highest Visual Score declines from 0.78 to 0.65.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Generating lengthy code is challenging; almost all metrics for nearly all models drop significantly as the target code length increases.

    “(1) Generating lengthy code is challenging.Almost all metrics for nearly all models drop significantly as the target code length increases.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Findingown data

    The code base, dataset, and new benchmark model are open-sourced and freely available to the research and developer communities.

    “We open-source the code base, the dataset, and the new benchmark model, making them freely available to the research and developer communities, for further innovation in automating front-end engineering.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingnone given

    TreeBLEU is a novel metric to measure the structural hierarchy recall.

    “Additionally, we introduce a novel metric, TreeBLEU, to measure the structural hierarchy recall.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The score distribution of the manually labeled subset reveals that 80% of the data fell within the low-quality category, scoring between 0 and 1, while only 20% of the entries, exhibiting scores of 2 or higher, demonstrate a commendable level of structural integrity and aesthetic appeal.

    “The statistics reveal that 80% of the data fell within the low-quality category, scoring between 0 and 1.Conversely, only 20% of the entries, exhibiting scores of 2 or higher, demonstrate a commendable level of structural integrity and aesthetic appeal.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Our dataset contains a significantly greater number and diversity of HTML tags compared to the pix2code and WebSight datasets.

    “From Table2, it is apparent to see that our dataset contains a significantly greater number and diversity of HTML tags and exhibits a more intricate DOM tree structure compared to the pix2code and WebSight datasets.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingnone given

    Compared to Design2Code, WebCode2M significantly reduces the average code length to about one-tenth of its original size, while maintaining the diversity and quantity of HTML tags, thereby preserving the high quality of the dataset.

    “Notably, compared to Design2Code, our WebCode2M significantly reduces the average code length to about one-tenth of its original size, while maintaining the diversity and quantity of HTML tags, thereby preserving the high quality of the dataset.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Study design and conductnone given

    The annotation strategy of dividing six annotators into two groups to perform consensus annotation minimizes the influence of subjective factors on the scoring results.

    “We then divide them into two groups to perform consensus annotation, where annotators within each group evaluate the same data.This annotation strategy minimizes the influence of subjective factors on the scoring results.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    The model's generation capability is not sufficiently stable and performs poorly on some test cases.

    “This indicates that the model's generation capability is not sufficiently stable and performs poorly on some test cases”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The first bad words list contains dirty, naughty, obscene, and otherwise inappropriate words in multiple languages, while the second serves as a supplementary list with additional sensitive or stop words.

    “The first list contains dirty, naughty, obscene, and otherwise inappropriate words in multiple languages (e.g.,”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [2] 2. 2012. The List-of-Dirty-Naughty-Obscene-and-Otherwise-Bad-Words project on Github. https://github.com/LDNOOBW/List-of-Dirty-Naughty- Obscene-and-Otherwise-Bad-Words/

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [3] 3. 2016. The sensitive-stop-words project on Github. https://github.com/fwwdn/ sensitive-stop-words/

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    The NSFW detector predicts two scores, normal and NSFW, with lower NSFW scores indicating a lower probability of harmful content.

    “It predicts two scores:”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [1] 1. 2010. The nsfw_image_detection model on HuggingFace. https://huggingface. co/Falconsai/nsfw_image_detection/

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Study design and conductown data

    A consistency analysis of all the data from the annotators shows that although certain differences exist among annotators within the same group, the overall trend remains similar.

    “We also conduct a consistency analysis of all the data from the annotators (see Figure 3).Although certain differences exist among annotators within the same group, the overall trend remains similar.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductown data

    The scoring model is trained on a manually annotated subset of 10,000 entries, curated by six annotators using consensus-based annotation, achieving a validation accuracy of 90% in distinguishing high-quality instances from low-quality ones.

    “This scoring model is trained on a manually annotated subset of 10,000 entries, curated by six annotators using consensus-based annotation, achieving a validation accuracy of 90% in distinguishing high-quality instances from low-quality ones.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    The layout data, represented by bounding boxes of HTML elements, includes key information such as size, location, and hierarchy of page components, and will aid the model in learning to generate the webpage DOM tree structure more accurately.

    “The layout, represented by the bounding boxes (BBox) of HTML elements, includes key information such as the size, location, and hierarchy of page components.This additional layout data will aid the model in learning to generate the webpage DOM tree structure more accurately.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductown data

    The ResNet-50 scorer achieves 75% accuracy on the test subset of the manually scored dataset and nearly 90% accuracy in binary classification, determining whether the score is greater than or equal to two.

    “Utilizing the rated data, we train a ResNet-50 [22] model to serve as a scorer, predicting the score of input screenshots.This scorer achieves 75% accuracy on the test subset of the manually scored dataset and nearly 90% accuracy in binary classification, determining whether the score is greater than or equal to two.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [22] 22. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In Proceedings of 2016 IEEE Conference on Com- puter Vision and Pattern Recognition, CVPR 2016, Las Vegas, NV, USA, June 27-30, 2016. IEEE Computer Society, 770-778. source

  • Classified as probably not a substantial claim.

    Study design and conductown data

    Manual review and documentation of filtering results for the first chunk under different thresholds was performed to evaluate filter effectiveness.

    “To evaluate the effectiveness of the filters, we manually review and document the filtering results for the first chunk under different thresholds.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Study design and conductnone given

    The Visual Score used is from the first version of their paper, which may differ from the latest version, particularly with respect to the sub-indicator text color.

    “We use the Visual Score from the first version of their paper, which may differ from the latest version, particularly with respect to the sub-indicator text color.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductreasoning

    Additional filtering steps-an image filter on screenshots and a profanity filter on web text-were applied, and only samples that passed both filters were retained.

    “Specifically, we apply an image filter to the screenshots and a profanity filter to the web text.Only samples that passed both filters are retained.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Pix2Struct-1.3B is notable for its robustness to extreme aspect ratios and ability to adapt dynamically to changes in sequence length and resolution.

    “Pix2Struct-1.3B is notable for its robustness to extreme aspect ratios and ability to adapt dynamically to changes in sequence length and resolution.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [28] 28. Kenton Lee, Mandar Joshi, Iulia Raluca Turc, Hexiang Hu, Fangyu Liu, Julian Mar- tin Eisenschlos, Urvashi Khandelwal, Peter Shaw, Ming-Wei Chang, and Kristina Toutanova. 2023. Pix2Struct: Screenshot Parsing as Pretraining for Visual Lan- guage Understanding. In Proceedings of International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA (Proceedings of Machine Learning Research, Vol. 202). PMLR, 18893-18912.

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    The WebCode2M dataset may contain a minimal amount of inappropriate content, such as violent material, despite extensive filtering efforts, because it is derived from web-crawled data.

    “since our dataset is derived from web-crawled data, it may contain a minimal amount of inappropriate content, such as violent material, despite extensive filtering efforts.”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Interpretationreasoning

    Some low-quality data may remain in the final dataset due to the neural scorer's subjectivity and imperfect accuracy.

    “As a result, some low-quality data may remain in the final dataset.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    Subtree matching between the generated HTML DOM tree and the reference can be straightforwardly performed through string comparison because their subtree collections are represented as sets of strings.

    “Given that the subtree collections of the generated HTML DOM tree and the reference are represented as sets of strings, subtree matching can be straightforwardly performed through string comparison.”
    Quote in the paperSubstantial claim
43 of 43

  • The paper relies on this without support.

    Large-scale human-designed screenshots are hard to collect manually.

    “As large-scale human-designed screenshots are hard to collect manually, we opt to reversely generate screenshot image from a curated open-source web dataset via rendering the webpage code.”

  • The paper relies on this without support.

    Manually screening all the data is impractical.

    “Given the large volume of our dataset, manually screening all the data is impractical.”

  • The paper relies on this without support.

    All existing webpage code generation models fail to accurately reproduce the image elements in the designs.

    “All existing webpage code generation models fail to accurately reproduce the image elements in the designs, severely hindering their practical application.”

  • The paper relies on this without support.

    Normal web pages may occasionally contain a small number of inappropriate words, while those with excessive amounts of inappropriate content, such as adult websites, tend to have significantly higher word counts.

    “While normal web pages may occasionally contain a small number of inappropriate words, those with excessive amounts of inappropriate content, such as adult websites, tend to have significantly higher word counts.”

  • The paper relies on this without support.

    The original HTML DOM tree contains many element attributes that are not necessary for calculating the similarity of substructures.

    “The original HTML DOM tree contains many element attributes that are not necessary for calculating the similarity of substructures.”
5 of 5

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • WebCode2M averages are attributed to the wrong dataset

    Major

    The stated averages of 31,216 tokens, 158 tags, and DOM depth 13 belong to Design2Code in Table 2. Table 2 gives WebCode2M averages of 5,366 tokens, 184 tags, and DOM depth 15.

  • GPT-4o also trails Gemini on TreeBLEU

    Minor

    Table 4 gives Gemini higher TreeBLEU scores than GPT-4o on all three test subsets, contradicting the claim that GPT-4o leads every other general-purpose model on every metric.

  • WebSight baseline has two parameter sizes

    Minor

    The text calls the fine-tuned WebSight baseline an 8B model, but its row in the specialized-model results calls it a 7B model.

3 of 3

Cites

Flagged first, then by citations

Research works printed but not linked: 11

  1. 2024. The Claude 3 Model Family: Opus Sonnet Haiku. https://api.semanticscholar. org/CorpusID:268232499
  2. Zhangqian Bi, Yao Wan, Zheng Wang, Hongyu Zhang, Batu Guan, Fangxin Lu, Zili Zhang, Yulei Sui, Hai Jin, and Xuanhua Shi. 2024. Iterative Refinement of Project-Level Code Context for Precise Code Generation with Compiler Feedback. In Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11--16, 2024. Association for Computational Linguistics, 2336--2353.
  3. Dongping Chen, Ruoxi Chen, Shilin Zhang, Yaochen Wang, Yinuo Liu, Huichi Zhou, Qihui Zhang, Yao Wan, Pan Zhou, and Lichao Sun. 2024. MLLM-as-a- Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21--27, 2024. OpenReview.net.
  4. Daniel Fried, Armen Aghajanyan, Jessy Lin, Sida Wang, Eric Wallace, Freda Shi, Ruiqi Zhong, Scott Yih, Luke Zettlemoyer, and Mike Lewis. 2023. InCoder: A Generative Model for Code Infilling and Synthesis. In Proceedings of The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1--5, 2023. OpenReview.net.
  5. Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising Diffusion Probabilistic Models. In Advances in Neural Information Processing Systems, December 6--12, 2020, virtual.
  6. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-Rank Adaptation of Large Language Models. In Proceedings of The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25--29, 2022. OpenReview.net.
  7. Kenton Lee, Mandar Joshi, Iulia Raluca Turc, Hexiang Hu, Fangyu Liu, Julian Martin Eisenschlos, Urvashi Khandelwal, Peter Shaw, Ming-Wei Chang, and Kristina Toutanova. 2023. Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding. In Proceedings of International Conference on Machine Learning, ICML 2023, 23--29 July 2023, Honolulu, Hawaii, USA (Proceedings of Machine Learning Research, Vol. 202). PMLR, 18893--18912.
  8. Raymond Li Loubna Ben Allal Yangtian Zi Niklas Muennighoff Denis Kocetkov Chenghao Mou Marc Marone Christopher Akiki Jia Li Jenny Chim Qian Liu Evgenii Zheltonozhskii Terry Yue Zhuo ThomasWang Olivier Dehaene Mishig Davaadorj Joel Lamy-Poirier João Monteiro Oleh Shliazhko Nicolas Gontier Nicholas Meade Armel Zebaze Ming-Ho Yee Logesh Kumar Umapathi Jian Zhu Benjamin Lipkin Muhtasham Oblokulov Zhiruo Wang Rudra Murthy V Jason T. Stillerman Siva Sankalp Patel Dmitry Abulkhanov Marco Zocca Manan Dey Zhihan Zhang Nour Fahmy Urvashi Bhattacharyya Wenhao Yu Swayam Singh Sasha Luccioni Paulo Villegas Maxim Kunakov Fedor Zhdanov Manuel Romero Tony Lee Nadav Timor Jennifer Ding Claire Schlesinger Hailey Schoelkopf Jan Ebert Tri Dao Mayank Mishra Alex Gu Jennifer Robinson Carolyn Jane Anderson Brendan Dolan-Gavitt Danish Contractor Siva Reddy Daniel Fried Dzmitry Bahdanau Yacine Jernite Carlos Muñoz Ferrandis Sean Hughes ThomasWolf Arjun Guha Leandro vonWerra and Harm de Vries. 2023. StarCoder: may the source be with you! Trans. Mach. Learn. Res. 2023 (2023).
  9. Shih-Yang Liu, Chien-Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang-Ting Cheng, and Min-Hung Chen. 2024. DoRA: Weight- Decomposed Low-Rank Adaptation. In Proceedings of Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21--27, 2024. OpenReview.net.
  10. Shaoqing Ren, Kaiming He, Ross B. Girshick, and Jian Sun. 2015. Faster RCNN: Towards Real-Time Object Detection with Region Proposal Networks. In Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7--12, 2015, Montreal, Quebec, Canada. 91--99.
  11. Replit. 2023. Meet Replit AI, your partner in code. https://replit.com/ai/.

Cited by

Sorted by citations

Related

Authors
Gui, Y., Li, Z., Wan, Y., Shi, Y., Zhang, H., Chen, B., Su, Y., Chen, D., Wu, S., Zhou, X., Jiang, W., Jin, H., & Zhang, X.
Affiliations
  • Huazhong University of Science and Technology (CN)
  • Peking University (CN)
  • Chongqing University (CN)
  • Hubei University of Automotive Technology (CN)
  • Rabbitpre AI, Shenzhen, China
  • University of Notre Dame (US)
ORCID
Venue
Proceedings of the ACM on Web Conference 2025
Publisher
ACM
Published
2025-04-22
Pages
1834-1845
DOI
10.1145/3696410.3714889
OpenAlex
W4409671919
Semantic Scholar
13811c0afa6388653e8504fe4e136a9dc4d68f96
Cited by
17
References
19
Citations by year
2026: 12 · 2025: 5
FWCI
12.69
Type
conference paper
Language
en
Open access
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations
  • International World Wide Web Conference (WWW) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
Major Program (JD) of Hubei Province (2023BAA024)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
  • Physical Sciences › Computer Science › Information Systems › Mobile and Web Applications
Keywords
front-end development, multimodal large language models, layout information, dataset filtering, vision transformer

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-11.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-11.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review