pix2code: Generating Code from a Graphical User Interface Screenshot
The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
Transforming a graphical user interface screenshot created by a designer into computer code is a typical task conducted by a developer in order to build customized software, websites, and mobile applications. In this paper, we show that deep learning methods can be leveraged to train a model end-to-end to automatically reverse engineer user interfaces and generate code from a single input image with over 77% of accuracy for three different platforms (i.e. iOS, Android and web-based technologies).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · cited 2×
“n. Therefore, pre-training the language model to learn vectorial representations would allow the relationships between tokens in the DSL to be inferred (i.e. learning word embeddings such as word2vec [13]) and as a result alleviate semantical error in the generated code. Furthermore, one-hot encoding does not scale to…”
BackgroundMethodShow, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248
Show and tell: A neural image caption generator
Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298935 · Cited by 6,318
Deep visual-semantic alignments for generating image descriptions
Karpathy, A., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298932 · Cited by 4,902
StackGAN: Text to Photo-Realistic Image Synthesis with Stacked Generative Adversarial Networks
Zhang, H., Xu, T., Li, H., Zhang, S., Wang, X., Huang, X., & Metaxas, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.629 · Cited by 3,021 · cited 1×
“Generative Adversarial Networks (GANs) [5] have shown to be extremely powerful at generating images and sequences [2, 20].”
BackgroundGenerating Sequences With Recurrent Neural Networks
Graves, A. (2013) · arXiv · doi:10.48550/arxiv.1308.0850 · Cited by 2,894
Recurrent Neural Network Regularization
Zaremba, W., Sutskever, I., & Vinyals, O. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.2329 · Cited by 2,123
Research works printed but not linked: 4
- Matej Balog Alexander L Gaunt Marc Brockschmidt Sebastian Nowozin and Daniel Tarlow. 2016. DeepCoder: Learning to Write Programs. arXiv preprint arXiv:1611.01989 (2016). Matej Balog Alexander L Gaunt Marc Brockschmidt Sebastian Nowozin and Daniel Tarlow. 2016. DeepCoder: Learning to Write Programs. arXiv preprint arXiv:1611.01989 (2016). · arXiv
- Gaunt Alexander L, 2016, Terpret: A probabilistic programming language for program induction. arXiv preprint arXiv:1608.04428 · arXiv
- Ian Goodfellow Jean Pouget-Abadie Mehdi Mirza Bing Xu David Warde-Farley Sherjil Ozair Aaron Courville and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in neural information processing systems. 2672--2680. Ian Goodfellow Jean Pouget-Abadie Mehdi Mirza Bing Xu David Warde-Farley Sherjil Ozair Aaron Courville and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in neural information processing systems. 2672--2680.
- 10.5555/2627435.2670313 · doi:10.5555/2627435.2670313
Cited by
Items 101–200 of 239
Automated Mapping of Adaptive App GUIs from Phones to TVs
Hu, H., Dong, R., Grundy, J., Nguyen, T. M., Liu, H., & Chen, C. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3631968 · Cited by 12
Exemplar based experience transfer
Maheshwari, P., Bansal, N., Dwivedi, S., Kumar, R., Manerikar, P., & Srinivasan, B. V. (2019) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3301275.3302300 · Cited by 7
ImageCLEF 2019: Multimedia Retrieval in Lifelogging, Medical, Nature, and Security Applications
Ionescu, B. E., Müller, H., Péteri, R., Dang‐Nguyen, D., Piras, L., Riegler, M., Tran, M., Lux, M., Gurrin, C. G., Cid, Y. D., et al. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-15719-7_40 · Cited by 8
How artificial intelligence can improve web development and testing
Stocco, A. (2019) · International Conference on Art, Science, and Engineering of Programming (<Programming>) · doi:10.1145/3328433.3328447 · Cited by 7
Feng, Z., Fang, J., Cai, B., & Zhang, Y. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86365-4_5 · Cited by 14
UI2HTML: utilizing LLM agents with chain of thought to convert UI into HTML code
Yuan, D., Yang, G., & Zhang, T. (2025) · Automated Software Engineering · doi:10.1007/s10515-025-00509-5 · Cited by 11
Alamer, S. A., Ilyas, Q. M., Ahmad, M., & Irfan, D. (2021) · International Journal of Cloud Applications and Computing · doi:10.4018/ijcac.2022010110 · Cited by 9
Towards a Deep Learning Approach for Automatic GUI Layout Generation
Yao, X., Yap, M. H., & Zhang, Y. (2022) · Lecture notes in networks and systems · doi:10.1007/978-981-19-0604-6_2 · Cited by 6
Wang, D., & Zheng, Y. (2022) · IEEE Access · doi:10.1109/access.2022.3218912 · Cited by 8
Variable Name Recovery in Decompiled Binary Code using Constrained Masked Language ModelingPreprint
Banerjee, P., Pal, K. K., Wang, F., & Baral, C. R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.12801 · Cited by 26
What Did My Users Experience? Discovering Visual Stimuli on Graphical User Interfaces of the Web
Menges, R., Staab, S., Schaefer, C., Walber, T., & Kumar, C. (2025) · ACM Transactions on the Web · doi:10.1145/3715881 · Cited by 9
Calò, T., & Russis, L. D. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3734190 · Cited by 6
Owl Eye: An AI-Driven Visual Testing Tool
Gamal, A., Emad, R., Mohamed, T., Mohamed, O., Hamdy, A., & Ali, S. H. (2023) · Novel Intelligent and Leading Emerging Sciences Conference (NILES) · doi:10.1109/niles59815.2023.10296575 · Cited by 8
Samir, M., Elsayed, A., & Marie, M. I. (2024) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2024.0150369 · Cited by 9
Understanding mobile GUI: From pixel-words to screen-sentences
Fu, J., Zhang, X., Wang, Y., Zeng, W., & Zheng, N. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128200 · Cited by 28
A Rapid Combined Model for Automatic Generating Web UI Codes
Zhang, W., Luan, S., & Tian, L. (2022) · Wireless Communications and Mobile Computing · doi:10.1155/2022/4415479 · Cited by 5
Automatic Front-end Code Generation from image Via Multi-Head Attention
Zhang, Z., Ding, Y., & Huang, C. (2023) · International Conference on Computer Engineering and Application (ICCEA) · doi:10.1109/iccea58433.2023.10135462 · Cited by 5
Multi-Plane Program Induction with 3D Box PriorsPreprint
Li, Y., Mao, J., Zhang, X., Freeman, W. T., Tenenbaum, J. B., Snavely, N., & Wu, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.10007 · Cited by 13
LaTCoder: Converting Webpage Design to Code with Layout-as-Thought
Gui, Y., Li, Z., Zhang, Z., Wang, G., Lv, T., Jiang, G., Liu, Y., Chen, D., Wan, Y., Zhang, H., et al. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3737016 · Cited by 28
A Deep Learning Based Object Detection System for User Interface Code Generation
Aşıroğlu, B., Senan, S., Görgel, P., Isenkul, M. E., Ensarı, T., Sezen, A., & Dağtekin, M. (2022) · International Congress on Human-Computer Interaction, Optimization and Robotic Applications (ICHORA) · doi:10.1109/hora55278.2022.9799941 · Cited by 4
Transforming Ideas into Code: A Novel Approach to Design-to-Development Workflow
Raptis, G. E., & Katsini, C. (2023) · International Conference on Computer and Applications (ICCA) · doi:10.1109/icca59364.2023.10401412 · Cited by 4
Cognification of Program Synthesis—A Systematic Feature-Oriented Analysis and Future Direction
Subahi, A. F. (2020) · Computers · doi:10.3390/computers9020027 · Cited by 4
GUI visual aspect migration: a framework agnostic solution
Verhaeghe, B., Anquetil, N., Etien, A., Ducasse, S., Seriai, A., & Derras, M. (2021) · Automated Software Engineering · doi:10.1007/s10515-021-00284-z · Cited by 5
DeepCov: Coverage Guided Deep Learning Framework Fuzzing
Wu, J., Li, S., Li, J., Luo, L., Yu, H., & Sun, G. (2022) · IEEE International Conference on Data Science in Cyberspace · doi:10.1109/dsc55868.2022.00061 · Cited by 6
Zafar, H., Khan, S. U. R., Mashkoor, A., & Nisa, H. U. (2024) · Frontiers in Computer Science · doi:10.3389/fcomp.2024.1397805 · Cited by 8
Inferring Questions from Programming Screenshots
Ahmed, F., Tan, X., Adewole, F., Datta, S., & Nayebi, M. (2025) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr66628.2025.00111 · Cited by 5
An interpretable metric of visual aesthetics for GUI design
Wang, C., Miniukovich, A., & Ren, X. (2024) · Behaviour and Information Technology · doi:10.1080/0144929x.2024.2325030 · Cited by 6
Beyond Manual Modeling: Automating GUI Model Generation Using Design Documents
Cao, S., Chen, R., Pan, M., Yang, W., & Li, X. (2024) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3691620.3695032 · Cited by 5
Optimizing Web Interfaces with AI-Generated CSS Through Machine Learning and Deep Learning
Bhutia, R., Dave, S. A. K., Mallela, I. R., Mall, R., Das, A., & Joshi, A. (2024) · International Conference on System Modeling & Advancement in Research Trends (SMART) · doi:10.1109/smart63812.2024.10882500 · Cited by 3
JustSpeak: Automated, User-Configurable, Interactive Agents for Speech Tutoring
Zhang, X., Miyaki, T., & Rekimoto, J. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3459744 · Cited by 3
Win GUI Crawler: A tool prototype for desktop GUI image and metadata collection
Savić, M., Mäntylä, M., & Claes, M. (2022) · IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) · doi:10.1109/icstw55395.2022.00046 · Cited by 4
Source Code Generation-based on NLP and Ontology
Alokla, A., Gad, W. K., Aref, M. M., & Salem, A.-B. M. (2022) · International journal of intelligent computing and information sciences/International Journal of Intelligent Computing and Information Sciences · doi:10.21608/ijicis.2022.117905.1160 · Cited by 3
pic2code: Generating HTML Code from Handwritten Picture
Arefin, M., Hossen, K. M., & Khan, R. (2023) · International Conference on Computer and Information Technology (ICCIT) · doi:10.1109/iccit60459.2023.10441098 · Cited by 5
An Exploratory Study of ML Sketches and Visual Code Assistants
Gomes, L., Hellendoorn, V. J., Aldrich, J., & Abreu, R. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00124 · Cited by 6
UX-Painter: Fostering UX Improvement in an Agile Setting
Gardey, J. C., Grigera, J., Rossi, G. H., & Garrido, A. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-25648-6_4 · Cited by 3
Literature Survey On Automatic Code Generation Techniques
(2018) · i-manager s Journal on Computer Science · doi:10.26634/jcom.6.2.15005 · Cited by 3
Automated Web Design And Code Generation Using Deep Learning
Et.al, V. S. (2021) · Türk bilgisayar ve matematik eğitimi dergisi · doi:10.17762/turcomat.v12i6.1401 · Cited by 3
Paper2Wire – A Case Study of User-Centred Development of Machine Learning Tools for UX Designers
Buschek, D., Anlauff, C., & Lachner, F. (2021) · i-com · doi:10.1515/icom-2021-0002 · Cited by 11
Berari, R., Tauteanu, A., Fichou, D., Brie, P., Dogariu, M., Stefan, L., Constantin, M. G., & Ionescu, B. E. (2021) · CLEF (Working Notes) · Cited by 3
ImageCLEF 2020: Multimedia Retrieval in Lifelogging, Medical, Nature, and Internet Applications
Ionescu, B. E., Müller, H., Péteri, R., Dang‐Nguyen, D., Zhou, L., Piras, L., Riegler, M. A., Halvorsen, P., Tran, M., Lux, M., et al. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-45442-5_69 · Cited by 5
Deep Learning Anomaly Detection for Drone-based Surveillance
Hamdi, S. (2021) · theses.fr (ABES) · doi:10.70675/3fa63c77z4e70z4395zb38fz9d8bfa4499e3 · Cited by 4
WebCoder: Multimodal Approach for Automated Web Service UI Code Generation
Su, H., Yu, L., & Yang, C. (2025) · IEEE International Conference on Web Services (ICWS) · doi:10.1109/icws67624.2025.00087 · Cited by 2
Irawati, D. A., Bölükbaşı, E., & Riener, A. (2025) · International Conference on Mobile and Ubiquitous Multimedia (MUM) · doi:10.1145/3771882.3771913 · Cited by 3
AI-Powered UI Generation: Evaluating Interactive Design Paradigms with Vision-Language Models
Davariya, S. (2025) · IEEE India Council International Subsections Conference (INDISCON) · doi:10.1109/indiscon66021.2025.11252136 · Cited by 2
Tholander, J., & Jönsson, M. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791847 · Cited by 6
NL2pSQL: Generating Pseudo-SQL Queries from Under-Specified Natural Language Questions
Chen, F., Hwang, S.-W., Choo, J., Ha, J.-W., & Kim, S. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1262 · Cited by 5
Widget detection on screenshots using computer vision and machine learning algorithms
Radzikowski, K., Chęciński, K., Forc, M., Lepak, Ł., Jablonski, M., Kuśmirek, W., Twardowski, B., Wawrzyński, P., & Nowak, R. M. (2019) · Photonics Applications in Astronomy, Communications, Industry, and High-Energy Physics Experiments (WILGA) · doi:10.1117/12.2536406 · Cited by 2
Research on Recognition Method of Interface Elements Based on Machine Learning
Cheng, J., Li, J., & Chen, J. (2021) · International Conference on Intelligent Computing, Automation and Applications (ICAA) · doi:10.1109/icaa53760.2021.00049 · Cited by 2
UI Layers Group Detector: Grouping UI Layers via Text Fusion and Box Attention
Xiao, S., Zhou, T., Chen, Y., Zhang, D., Chen, L., Sun, L., & Yue, S. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20500-2_25 · Cited by 2
GUI Element Detection via YOLOV8: A Deep Learning Approach for Widget Identification
Jose, S., Samuel, P., & Idicula, S. M. (2024) · International Conference on Trends in Engineering Systems and Technologies (ICTEST) · doi:10.1109/ictest60614.2024.10576164 · Cited by 5
Low-Code Flutter Application Development Solution
Jazi, A. N., Alfonso, I., & Cabot, J. (2024) · ACM/IEEE International Conference on Model Driven Engineering Languages and Systems (MODELS) · doi:10.1145/3652620.3688330 · Cited by 2
Ahmed, A., Azab, S. S., Moussa, S. M., & Abdelhamid, Y. (2025) · Neural Computing and Applications · doi:10.1007/s00521-025-11565-z · Cited by 2
Fine-Tuning a Pre-trained CAE for Deep One Class Anomaly Detection in Video Footage
Hamdi, S., Snoussi, H., & Abid, M. (2021) · Communications in computer and information science · doi:10.1007/978-3-030-71804-6_1 · Cited by 2
Ionescu, B. E., Müller, H., Péteri, R., Abacha, A. B., Demner‐Fushman, D., Hasan, S. A., Sarrouti, M., Pelka, O., Friedrich, C. M., de Herrera, A. G. S., et al. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-72240-1_72 · Cited by 3
Dynamic Web with Automatic Code Generation Using Deep Learning
Sharma, P., Chaudhary, V., Malhotra, N., Gupta, N., & Mittal, M. (2020) · Advances in intelligent systems and computing · doi:10.1007/978-981-15-1286-5_61 · Cited by 2
Research on Machine Learning Program Generation Algorithm Based on AORBCO
Wang, S., Gao, W., & Wang, S. (2024) · International Journal of Advanced Network Monitoring and Controls · doi:10.2478/ijanmc-2024-0013 · Cited by 2
Pereira, P. F. F., Rodrigues, F., & Ferreira, C. A. (2019) · Iberian Conference on Information Systems and Technologies (CISTI) · doi:10.23919/cisti.2019.8760681 · Cited by 2
CodeeGAN: Code Generation via Adversarial Training
Deng, Y., Fu, C., & Li, Y. (2019) · Communications in computer and information science · doi:10.1007/978-981-15-1304-6_2 · Cited by 2
Regenerating a Graphical User Interface Using Deep Learning
Cheddi, F., Tahiri, A., & Serrhini, M. (2021) · Advances in intelligent systems and computing · doi:10.1007/978-981-16-5559-3_28 · Cited by 2
Unsupervised Program Synthesis for Images using Tree-Structured LSTM.Preprint
Zhou, C., Li, C., & Póczos, B. (2020) · arXiv (Cornell University) · Cited by 3
Text2App: A Framework for Creating Android Apps from Text DescriptionsPreprint
Hasan, M., Mehrab, K. S., Ahmad, W. U., & Shahriyar, R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.08301 · Cited by 10
Web Components Template Generation from Web Screenshot
Anunphop, P., & Chongstitvatana, P. (2021) · Advances in Information Technology · doi:10.1145/3468784.3468787 · Cited by 1
What’s Behind the Screen? Unveiling UI Hierarchies in Process-Related UI Logs
Martínez-Rojas, A., Rodríguez-Ruíz, A., Enríquez, J. G., & Ramirez, A. J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-70396-6_15 · Cited by 5
On-Device Interaction Mining MHCI024
Arsan, D., Guo, C., Wellyanto, M. R., Ji, E. R., Talton, J. O., & Kumar, R. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3743726 · Cited by 1
From Mock-Ups to IFML-Like GUI Models: Using Large Language Models in Web Engineering
Nirumand, A., & Cabot, J. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-97207-2_20 · Cited by 1
Dicu, M., & Chira, C. (2025) · Procedia Computer Science · doi:10.1016/j.procs.2025.09.422 · Cited by 1
UI Component Detection in Graphical Interfaces: A YOLO-Based Comparative Analysis
Dicu, M. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-12478-4_9 · Cited by 1
GUIMigrator: Semantics-Preserving Transpilation from Android XML to Compose and SwiftUI
橘高, 義., Hu, X., Yang, X., & Xia, X. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3808215 · Cited by 7
Code Generation from Images Using Neural Networks
Nikam, C., Keshervani, R., Shah, S., & Aghav, J. V. (2021) · Algorithms for intelligent systems · doi:10.1007/978-981-16-3802-2_12 · Cited by 1
Using Deep Learning to Support the User Interface Design of Mobile Applications with App Inventor
Baulé, D. D. S., von Wangenheim, C. G., von Wangenheim, A., Hauck, J. C. R., & Júnior, E. C. V. (2021) · Brazilian Symposium on Human Factors in Computing Systems (IHC) · doi:10.1145/3472301.3484340 · Cited by 3
Construction of Semantic Model for GUI of Mobile Applications Using Deep Learning
Liu, Q., Zhang, T., Gao, J. Z., Liu, S., & Cheng, J. (2022) · IEEE International Conference on Artificial Intelligence Testing (AITest) · doi:10.1109/aitest55621.2022.00010 · Cited by 1
Creating Dynamic Prototypes from Web Page Sketches
Calò, T., & Russis, L. D. (2022) · ACM SIGPLAN International Workshop on Programming Abstractions and Interactive Notations, Tools, and Environments (PAINT) · doi:10.1145/3563836.3568724 · Cited by 1
A model for automatic generating reusable code from multiple GUIs
Araújo, C. A. G. (2022) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3551349.3559563 · Cited by 1
Towards Automated Embedded Systems Programming
Yusuf, I. N. B. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse-companion58688.2023.00061 · Cited by 1
Automatic Generation of Frontend Code from Design Interface
He, X., Wang, J., Liu, Z., Xie, L., Wang, H., & Le, X. (2023) · International Conference on Information Science and Technology · doi:10.1109/icist59754.2023.10367169 · Cited by 1
Zhao, D., Xing, Z., Lu, Q., Xu, X., & Zhu, L. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00144 · Cited by 2
Deep neural networks based user interface detection for mobile applications using symbol marker
Park, J., Yun, Y.-S., Eun, S., Cha, S., So, S.-S., & Jung, J. (2018) · ACM Conference on Research in Adaptive and Convergent Systems (RACS) · doi:10.1145/3264746.3264808 · Cited by 1
SvgAI – Training Methods Analysis of Artificial Intelligent Agent to use SVG Editor
Dang, A. H., & Kameyama, W. (2019) · International Conference on Advanced Communication Technology · doi:10.23919/icact.2019.8702041 · Cited by 1
Edge Detection Method for the Graphic User Interface of Complex Information System
Song, Y., Xue, C., Wang, X., & Zhang, P. (2019) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-27928-8_66 · Cited by 1
Visual Compiler: Towards Translating Digital UI Design Draft to Front-End Code Automatically
Pan, J., Chen, X., Chen, T., Tang, B., Yang, J., Chen, Y., Lin, Y., Xiao, C., & Meng, J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-50017-7_28 · Cited by 2
A Process for Automatic Generation of Medical Mobile Applications using Voice Recognition
Avelino, J. F., Alor‐Hernández, G., Paredes‐Valverde, M. A., Rodríguez-Mazahua, L., & Abud-Figueroa, M. A. (2020) · Research in Computing Science · Cited by 1
Automated Visual Testing of Application User Interfaces Using Static Analysis of Screenshots
Packevičius, Š., Rudžionienė, G., & Bareiša, E. (2021) · International Journal of Software Engineering and Knowledge Engineering · doi:10.1142/s0218194021500042 · Cited by 1
SAIP: A Prototype Tool for Semi-Automatic User Interface Prototyping.
Chen, Y., Cao, S., Ma, G., & Li, T. (2021) · REFSQ Workshops · Cited by 1
NeurTEx: A Neural Framework for Template Extraction from Flat Images
Aggarwal, V., Vaddamanu, P., Guda, B. P. R., Srinivasan, B. V., Chhaya, N., & Vinay, V. (2022) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3491101.3519718 · Cited by 1
IW-Bench: Evaluating Large Multimodal Models for Converting Image-to-Web
Guo, H., Zhang, W., Chen, J., Gu, Y., Yang, J., Du, J., Cao, S., Hui, B., Liu, T., Ma, J., et al. (2024) · arXiv · doi:10.48550/arxiv.2409.18980
“Recently, the task of converting images into web code (Laurençon et al., 2024; Patil et al., 2020; Beltramelli, 2017) has garnered significant attention due to its impressive performance.”
SvgAI — Training artificial intelligent agent to use SVG editor
Dang, A. H., & Kameyama, W. (2018) · International Conference on Advanced Communication Technology · doi:10.23919/icact.2018.8323672
Building, growing and sustaining ML communities
Andrews, M., & Witteveen, S. (2018)
Learning to Infer User Interface Attributes from ImagesPreprint
Schlattner, P., Bielik, P., & Vechev, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.13243 · Cited by 1
Hierarchical segmentation of graphical interfaces for Document Object Model reconstruction
Perțicaș, C. F., Baba, M. S., Davoudi, H., & Florian, R. V. (2018) · International Joint Conference on Artificial Intelligence
User-Interface-Generierung aus Handschriften im Design-Sprint-Prozess
Barenkamp, M. (2020) · Informatik-Spektrum · doi:10.1007/s00287-020-01265-4
Towards learning visual semantics
Cai, H., Pant, S. R., & Li, W. (2020) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3368089.3417040 · Cited by 2
An Attention Mechanism Extension of Automatic HTML Generation from Web Page Design Images
Chinen, H., Ohki, H., Gyohten, K., & Takami, T. (2020) · IEEJ Transactions on Electronics Information and Systems · doi:10.1541/ieejeiss.140.1393
Deep Learning Object Detection Models in Robotic Process Automation
Vuorimaa, W. (2020) · Aaltodoc (Aalto University) · Cited by 1
Pointer Program Synthesis as Non-deterministic Planning
Lu, X., & Yu, B. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-77474-5_9
Uma abordagem para avaliação automatizada de acessibilidade de Rich Internet Applications
Antonelli, H. L. (2022) · Digital Library of Theses and Dissertations (Universidade de São Paulo) · doi:10.11606/t.55.2022.tde-17032022-101437
An Automatic GUI Generation Method Based on Generative Adversarial Network
Yao, X., Yap, M. H., & Zhang, Y. (2022) · Lecture notes in networks and systems · doi:10.1007/978-981-19-2394-4_59
From Sketch to Simulation in Virtual Laboratory Using Deep Learning
Cheddi, F., Serrhini, M., & Tahiri, A. (2022) · Lecture notes in networks and systems · doi:10.1007/978-3-031-01942-5_22
Implementing Machine Learning-Based Simulation in Physics Virtual Laboratory
Cheddi, F. (2023) · Lecture notes in networks and systems · doi:10.1007/978-3-031-26852-6_27
The Future of Web Development and Emerging Technologies
Singh, V., Jain, S., & Bang, C. B. (2023) · International Journal of Advanced Research in Science Communication and Technology · doi:10.48175/ijarsct-10713
Evaluating an Artificial Intelligence Approach for Converting Sketches to UI Layouts
Dang, S. (2023) · Carleton University's Institutional Repository (MacOdrum Library, Carleton University) · doi:10.22215/etd/2023-15460 · Cited by 2
Metadata
- Authors
- Beltramelli, T.
- Affiliations
- Uizard Technologies, Copenhagen, Denmark
- Corresponding author
- Beltramelli, Tony
- Venue
- Proceedings of the ACM SIGCHI Symposium on Engineering Interactive Computing Systems
- Publisher
- ACM
- Published
- 2018-06-19
- Pages
- 1-6
- OpenAlex
- W2962724148
- Cited by
- 237
- References
- 15
- Citations by year
- 2026: 22 · 2025: 32 · 2024: 25 · 2023: 25 · 2022: 30 · 2021: 38 · 2020: 30 · 2019: 23 · 2018: 12
- FWCI
- 20.21
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM SIGCHI Symposium on Engineering Interactive Computing Systems (EICS) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Information Systems › Mobile and Web Applications
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Keywords
- deep learning, end-to-end learning, iOS, Android, web application, software development automation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.