The DOI matches its DataCite record.
On AI-Inspired UI-Design
Open accessAbstract
Graphical User Interface (or simply UI) is a primary mean of interaction between users and their devices. In this paper, we discuss three complementary Artificial Intelligence (AI) approaches for triggering the creativity of app designers and inspiring them create better and more diverse UI designs. First, designers can prompt a Large Language Model (LLM) to directly generate and adjust UIs. Second, a Vision-Language Model (VLM) enables designers to effectively search a large screenshot dataset, e.g. from apps published in app stores. Third, a Diffusion Model (DM) can be trained to specifically generate UIs as inspirational images. We present an AI-inspired design process and discuss the implications and limitations of the approaches.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Using LLMs for code generation shifts the developer's focus from ideation to parsing and debugging unfamiliar code.
“Using LLMs for code generation shifts the developer's focus from ideation [Wei et al. 2024] to parsing and debugging unfamiliar code.”
From How Do Programming Students Use Generative AI? (2025) · cites this paper as Wei et al. 2024
The UI Code Generation section describes LLM-generated HTML, and the Conclusion identifies UI design as a process involving ideation. But neither passage says that code generation shifts developers’ focus from ideation to parsing and debugging unfamiliar code; the UI Code Adjustment section discusses fixing generated HTML through additional prompts instead. The cited paper supports the code-generation and ideation context, not the claimed shift. (LLM, conf=0.82)
Experimental — accuracy not yet measuredOnce learners decide to turn to a ChatGPT-like assistant for help, especially if they already have prior experience with generative AI, they predominantly seek code solutions rather than support in creatively working through the problem.
“Our findings suggest that once learners decide turn to a ChatGPT-like assistant for help, especially if they already have prior experience with generative AI, they predominantly seek code solutions rather than support in creatively working through the problem [Wei et al. 2024].”
From How Do Programming Students Use Generative AI? (2025) · cites this paper as Wei et al. 2024
The UI Code Generation section describes generating HTML, while Boosting Creativity with AI and the design-process discussion concern designers and practitioner teams. Our copy does not address learners’ help-seeking choices, whether prior generative-AI experience affects them, or whether learners predominantly seek code solutions rather than creative support. Searches for learners, prior experience, and code-solution preferences found no passage on those points; the cited passages have no page numbers. (LLM, conf=0.75)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
No references could be resolved yet.
Research works printed but not linked: 13
- Qiuyuan Chen, Chunyang Chen, Safwat Hassan, Zhengchang Xing, Xin Xia, and Ahmed E. Has- san. "How Should I Improve the UI of My App?: A Study of User Reviews of Popular Apps in the Google Play". In: ACM Transactions on Soft- ware Engineering and Methodology 30.3 (2021), pp. 1-37. DOI: 10.1145/3447808. · DOI
- Ali Ebrahimi Pourasad and Walid Maalej. "Does GenAI Make Usability Testing Obsolete?" In: Proceedings of The 47th IEEE/ACM Interna- tional Conference on Software Engineering ICSE 2025. 2025 (to appear). URL: https://arxiv.org/ abs/2411.00634. · arXiv
- Yen Dieu Pham, Davide Fucci, and Walid Maalej. "A first implementation of a design thinking work- shop during a mobile app development course project". In: Proceedings of the 2nd Interna- tional Workshop on Software Engineering Ed- ucation for Millennials. SEEM '18. Gothenburg, Sweden: Association for Computing Machinery, 2018, pp. 56-63. ISBN: 9781450357500. DOI: 10. 1145/3194779.3194785. · DOI
- Jialiang Wei, Anne-Lise Courbis, Thomas Lam- bolais, Binbin Xu, Pierre Louis Bernard, Gérard Dray, and Walid Maalej. "GUing: A Mobile GUI Search Engine using a Vision-Language Model". In: ACM Transactions on Software Engineering and Methodology (TOSEM) (2024). DOI: 10 . 1145/3702993. · DOI
- Kristian Kolthoff, Christian Bartelt, and Simone Paolo Ponzetto. "Data-driven prototyping via natural-language-based GUI retrieval". In: Auto- mated Software Engineering 30.1 (2023), p. 34. DOI: 10.1007/s10515-023-00377-x. · DOI
- Jialiang Wei, Anne-Lise Courbis, Thomas Lam- bolais, Binbin Xu, Pierre Louis Bernard, and Gérard Dray. "Boosting GUI Prototyping with Diffusion Models". In: 2023 IEEE 31st Inter- national Requirements Engineering Conference (RE). 2023, pp. 275-280. DOI: 10 . 1109 / RE57278.2023.00035. · DOI
- Sidong Feng, Mingyue Yuan, Jieshan Chen, Zhenchang Xing, and Chunyang Chen. Design- ing with Language: Wireframing UI Design Intent with Generative Large Language Models. 2023. URL: http://arxiv.org/abs/2312.07755. · arXiv
- Jialiang Wei, Anne-Lise Courbis, Thomas Lam- bolais, Binbin Xu, Pierre Louis Bernard, Gérard Dray, and Walid Maalej. "Getting Inspiration for Feature Elicitation: App Store-vs. LLM-based Approach". In: Proceedings of 39th IEEE/ACM International Conference on Automated Software Engineering (ASE 2024). ACM, 2024. ISBN: 979- 8-4007-1248-7/24/10. DOI: 10 . 1145 / 3691620 . 3695591. · DOI
- Wayne Xin Zhao et al. A Survey of Large Lan- guage Models. 2023. URL: http://arxiv.org/abs/ 2303.18223.
- Alec Radford et al. "Learning Transferable Visual Models From Natural Language Supervision". In: Proceedings of the 38th International Con- ference on Machine Learning. Ed. by Marina Meila and Tong Zhang. Vol. 139. PMLR, 2021, pp. 8748-8763. URL: http://arxiv.org/abs/2103. 00020.
- Biplab Deka et al. "Rico: A mobile app dataset for building data-driven design applications". In: UIST 2017 -Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology. 2017, pp. 845-854. DOI: 10 . 1145/3126594.3126651. · DOI · Paper page
- Sen Chen, Lingling Fan, Chunyang Chen, and Yang Liu. "Automatically Distilling Storyboard With Rich Features for Android Apps". In: IEEE Transactions on Software Engineering 49.2 (2023), pp. 667-683. DOI: 10.1109/TSE.2022. 3159548. · DOI
- Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjorn Ommer. "High- Resolution Image Synthesis with Latent Diffu- sion Models". In: Proceedings of the IEEE Com- puter Society Conference on Computer Vision and Pattern Recognition. Vol. 2022-June. 2022, pp. 10674-10685. DOI: 10 . 1109 / CVPR52688 . 2022.01042. · DOI · Paper page
Cited by
Sorted by citations
How Do Programming Students Use Generative AI?
Rahe, C., & Maalej, W. (2025) · Proceedings of the ACM on Software Engineering · doi:10.1145/3715762 · Cited by 45
Self-citation
Metadata
- Authors
- Wei, J., Courbis, A.-L., Lambolais, T., Dray, G., & Maalej, W.
- Affiliations
- Université de Montpellier (FR)
- Universität Hamburg (DE)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2024
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2406.13631
- OpenAlex
- W4399913022
- Cited by
- 1
- References
- 0
- Citations by year
- 2025: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Digital Media and Visual Art
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Keywords
- graphical user interface design, large language models, vision-language models, diffusion models, App Store, design creativity, AI-generated images
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
