The DOI matches its Crossref record; 14 of the 15 references it cites were found in a registry or an index.
pix2code: Generating Code from a Graphical User Interface Screenshot
Abstract
Transforming a graphical user interface screenshot created by a designer into computer code is a typical task conducted by a developer in order to build customized software, websites, and mobile applications. In this paper, we show that deep learning methods can be leveraged to train a model end-to-end to automatically reverse engineer user interfaces and generate code from a single input image with over 77% of accuracy for three different platforms (i.e. iOS, Android and web-based technologies).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · cited 2×
“n. Therefore, pre-training the language model to learn vectorial representations would allow the relationships between tokens in the DSL to be inferred (i.e. learning word embeddings such as word2vec [13]) and as a result alleviate semantical error in the generated code. Furthermore, one-hot encoding does not scale to…”
BackgroundMethodShow, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248
Show and tell: A neural image caption generator
Vinyals, O., Toshev, A., Bengio, S., & Erhan, D. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298935 · Cited by 6,318
Deep visual-semantic alignments for generating image descriptions
Karpathy, A., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298932 · Cited by 4,902
StackGAN: Text to Photo-Realistic Image Synthesis with Stacked Generative Adversarial Networks
Zhang, H., Xu, T., Li, H., Zhang, S., Wang, X., Huang, X., & Metaxas, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.629 · Cited by 3,021
Recurrent Neural Network Regularization
Zaremba, W., Sutskever, I., & Vinyals, O. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.2329 · Cited by 2,123
Towards Diverse and Natural Image Descriptions via a Conditional GAN
Dai, B., Fidler, S., Urtasun, R., & Lin, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.323 · Cited by 477
Research works printed but not linked: 5
- Matej Balog Alexander L Gaunt Marc Brockschmidt Sebastian Nowozin and Daniel Tarlow. 2016. DeepCoder: Learning to Write Programs. arXiv preprint arXiv:1611.01989 (2016). Matej Balog Alexander L Gaunt Marc Brockschmidt Sebastian Nowozin and Daniel Tarlow. 2016. DeepCoder: Learning to Write Programs. arXiv preprint arXiv:1611.01989 (2016). · arXiv
- Gaunt Alexander L, 2016, Terpret: A probabilistic programming language for program induction. arXiv preprint arXiv:1608.04428 · arXiv
- Ian Goodfellow Jean Pouget-Abadie Mehdi Mirza Bing Xu David Warde-Farley Sherjil Ozair Aaron Courville and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in neural information processing systems. 2672--2680. Ian Goodfellow Jean Pouget-Abadie Mehdi Mirza Bing Xu David Warde-Farley Sherjil Ozair Aaron Courville and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in neural information processing systems. 2672--2680.
- Alex Graves. 2013. Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850 (2013). Alex Graves. 2013. Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850 (2013). · arXiv
- 10.5555/2627435.2670313 · doi:10.5555/2627435.2670313
Cited by
Chen, C., Su, T., Meng, G., Xing, Z., & Liu, Y. (2018) · Proceedings of the 40th International Conference on Software Engineering · doi:10.1145/3180155.3180240 · Cited by 188
Stige, Å., Zamani, E. D., Mikalef, P., & Zhu, Y. (2024) · Information Technology & People · doi:10.1108/itp-07-2022-0519 · Cited by 167
In-IDE Code Generation from Natural Language: Promise and Challenges
Xu, F. F., Vasilescu, B., & Neubig, G. (2022) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3487569 · Cited by 154
“In specific domains such as user interface design and prototyping, recent advances in deep learning models also enable developers to sketch the user interface visually and then automatically generates user interface code with the sketch [13], or using existing screenshots [84].”
Screen Recognition: Creating Accessibility Metadata for Mobile Applications from Pixels
Zhang, X., de Greef, L., Swearngin, A., White, S., Murray, K. I., Yu, L., Shan, Q., Nichols, J. W., Wu, J., Fleizach, C., et al. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445186 · Cited by 232
Object detection for graphical user interface: old fashioned or deep learning or a combination?
Chen, J., Xie, M., Xing, Z., Chen, C., Xu, X., Zhu, L., & Li, G. (2020) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3368089.3409691 · Cited by 163
StoryDroid: Automated Generation of Storyboard for Android Apps
Chen, S., Fan, L., Chen, C., Su, T., Li, W., Liu, Y., & Xu, L. (2019) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2019.00070 · Cited by 137
VINS: Visual Search for Mobile User Interface Design
Bunian, S., Li, K., Jemmali, C., Harteveld, C., Fu, Y., & El‐Nasr, M. S. (2021) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445762 · Cited by 139
Opportunities and Challenges in Code Search Tools
Liu, C., Xia, X., Lo, D., Gao, C., Yang, X., & Grundy, J. (2021) · ACM Computing Surveys · doi:10.1145/3480027 · Cited by 117
A Survey of Machine Learning for Big Code and Naturalness
Allamanis, M., Barr, E. T., Devanbu, P., & Sutton, C. (2019) · ACM Computing Surveys · doi:10.1145/3212695 · Cited by 150
LayoutGAN: Synthesizing Graphic Layouts With Vector-Wireframe Adversarial Networks
Li, J., Yang, J., Hertzmann, A., Zhang, J., & Xu, T. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2963663 · Cited by 303
Wireframe-based UI Design Search through Image Autoencoder
Chen, J., Chen, C., Xing, Z., Xia, X., Zhu, L., Grundy, J., & Wang, J. (2020) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3391613 · Cited by 123
Auto-Generation of Smart Contracts from Domain-Specific Ontologies and Semantic Rules
Choudhury, O., Rudolph, N., Sylla, I., Fairoza, N., & Das, A. K. (2018) · IEEE International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData) · doi:10.1109/cybermatics_2018.2018.00183 · Cited by 80
Translating video recordings of mobile app usages into replayable scenarios
Bernal-Cárdenas, C., Cooper, N., Moran, K., Chaparro, O., Marcus, A., & Poshyvanyk, D. (2020) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3377811.3380328 · Cited by 75
Screen Parsing: Towards Reverse Engineering of UI Models from Screenshots
Wu, J., Zhang, X., Nichols, J., & Bigham, J. P. (2021) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3472749.3474763 · Cited by 81
GUIGAN: Learning to Generate GUI Designs Using Generative Adversarial Networks
Zhao, T., Chen, C., Liu, Y., & Zhu, X. (2021) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse43902.2021.00074 · Cited by 91
Automatic HTML Code Generation from Mock-Up Images Using Machine Learning Techniques
Aşıroğlu, B., Mete, B. R., Yıldız, E., Nalçakan, Y., Sezen, A., Dağtekin, M., & Ensarı, T. (2019) · Electric Electronics, Computer Science, Biomedical Engineerings' Meeting (EBBT) · doi:10.1109/ebbt.2019.8741736 · Cited by 56
GUI-Squatting Attack: Automated Generation of Android Phishing Apps
Chen, S., Fan, L., Chen, C., Xue, M., Liu, Y., & Xu, L. (2019) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2019.2956035 · Cited by 66
How Should I Improve the UI of My App? A Study of User Reviews of Popular Apps in the Google Play
Chen, Q., Chen, C., Hassan, S., Xing, Z., Xia, X., & Hassan, A. E. (2021) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3447808 · Cited by 60
AI-based Test Automation: A Grey Literature Analysis
Ricca, F., Marchetto, A., & Stocco, A. (2021) · IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) · doi:10.1109/icstw52544.2021.00051 · Cited by 56
Cooper, N., Bernal-Cárdenas, C., Chaparro, O., Moran, K., & Poshyvanyk, D. (2021) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse43902.2021.00091 · Cited by 81
Automated Cross-Platform GUI Code Generation for Mobile Apps
Chen, S., Fan, L., Su, T., Ma, L., Liu, Y., & Xu, L. (2019) · doi:10.1109/ai4mobile.2019.8672718 · Cited by 43
Mozaffari, M. A., Zhang, X., Cheng, J., & Guo, J. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517511 · Cited by 63
Psychologically-inspired, unsupervised inference of perceptual groups of GUI widgets from GUI images
Xie, M., Xing, Z., Feng, S., Xu, X., Zhu, L., & Chen, C. (2022) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3540250.3549138 · Cited by 35
Learning to Describe Scenes with Programs
Liu, Y., Wu, Z., Ritchie, D., Freeman, W. T., Tenenbaum, J. B., & Wu, J. (2018) · International Conference on Learning Representations · Cited by 34
Reverse-engineering bar charts using neural networks
Zhou, F., Zhao, Y. H., Chen, W., Tan, Y., Yaqi, X., Chen, Y., Liu, C., & Zhao, Y. (2020) · Journal of Visualization · doi:10.1007/s12650-020-00702-6 · Cited by 31
Automatically Distilling Storyboard With Rich Features for Android Apps
Chen, S., Fan, L., Chen, C., & Liu, Y. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3159548 · Cited by 39
Closing the gap between designers and developers in a low code ecosystem
Bexiga, M. G., Garbatov, S., & Seco, J. C. (2020) · ACM/IEEE International Conference on Model Driven Engineering Languages and Systems Companion (MODELS-C) · doi:10.1145/3417990.3420195 · Cited by 29
Learning Structural Similarity of User Interface Layouts Using Graph Networks
Manandhar, D., Ruta, D., & Collomosse, J. P. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58542-6_44 · Cited by 42
Leveraging Large Language Models for End-User Website Generation
Calò, T., & Russis, L. D. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-34433-6_4 · Cited by 31
Automatic Graphics Program Generation Using Attention-Based Hierarchical Decoder
Zhu, Z., Xue, Z., & Yuan, Z. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-20876-9_12 · Cited by 26
image2emmet: Automatic code generation from web user interface image
Xu, Y., Bo, L., Sun, X., Li, B., Jiang, J., & Zhou, W. (2021) · Journal of Software Evolution and Process · doi:10.1002/smr.2369 · Cited by 27
Auto-Icon+ : An Automated End-to-End Code Generation Tool for Icon Designs in UI Development
Feng, S., Jiang, M., Zhou, T., Zhen, Y., & Chen, C. (2022) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3531065 · Cited by 29
Misty: UI Prototyping Through Interactive Conceptual Blending
Lu, Y., Leung, A., Swearngin, A., Nichols, J. W., & Barik, T. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713924 · Cited by 42
Transformers in source code generation: A comprehensive survey
Ghaemi, H., Alizadehsani, Z., Shahraki, A., & Corchado, J. M. (2024) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2024.103193 · Cited by 31
Sketch2Code: Transformation of Sketches to UI in Real-time Using Deep Neural Network
Jain, V., Agrawal, P., Banga, S., Kapoor, R., & Gulyani, S. (2019) · arXiv · doi:10.48550/arxiv.1910.08930 · Cited by 20
Divide-and-Conquer: Generating UI Code from Screenshots
Wan, Y., Wang, C., Dong, Y., Wang, W., Li, S., Huo, Y., & Lyu, M. R.-T. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3729364 · Cited by 79
A systematic literature review on automatic website generation
Kaluarachchi, T., & Wickramasinghe, M. (2023) · Journal of Computer Languages · doi:10.1016/j.cola.2023.101202 · Cited by 19
Deep learning‐based prototyping of android GUI from hand‐drawn mockups
Abdelhamid, A. A., Alotaibi, S. R., & Mousa, A. (2020) · IET Software · doi:10.1049/iet-sen.2019.0378 · Cited by 18
Code generation from a graphical user interface via attention-based encoder–decoder model
Chen, W.-Y., Podstreleny, P., Cheng, W.-H., Chen, Y.-Y., & Hua, K. (2021) · Multimedia Systems · doi:10.1007/s00530-021-00804-7 · Cited by 23
A Tree-Based Structure-Aware Transformer Decoder for Image-To-Markup Generation
Zhong, S., Song, S., Li, G., & Chan, S.-H. G. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548424 · Cited by 18
Aguiar, A., Restivo, A., Correia, F. F., Ferreira, H. S., & Dias, J. P. (2019) · International Conference on Art, Science, and Engineering of Programming (<Programming>) · doi:10.1145/3328433.3328456 · Cited by 18
A novel syntax-aware automatic graphics code generation with attention-based deep neural network
Pang, X., Zhou, Y., Li, P., Lin, W., Wu, W., & Wang, J. Z. (2020) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2020.102636 · Cited by 20
Kolthoff, K. (2019) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2019.00148 · Cited by 22
WebCode2M: A Real-World Dataset for Code Generation from Webpage Designs
Gui, Y., Li, Z., Wan, Y., Shi, Y., Zhang, H., Chen, B., Su, Y., Chen, D., Wu, S., Zhou, X., et al. (2025) · Proceedings of the ACM on Web Conference 2025 · doi:10.1145/3696410.3714889 · Cited by 17
Code Shaping: Iterative Code Editing with Free-form AI-Interpreted Sketching
Yen, R., Zhao, J., & Vogel, D. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713822 · Cited by 29
Detection of GUI Elements on Sketch Images Using Object Detector Based on Deep Neural Networks
Yun, Y.-S., Jung, J., Eun, S., So, S.-S., & Heo, J. (2018) · Lecture notes in electrical engineering · doi:10.1007/978-981-13-0311-1_16 · Cited by 17
YOLOv5-MGC: GUI Element Identification for Mobile Applications Based on Improved YOLOv5
Cheng, J., Tan, D., Zhang, T., Wei, A., & Chen, J. (2022) · Mobile Information Systems · doi:10.1155/2022/8900734 · Cited by 22
Paper2Wire: a case study of user-centred development of machine learning tools for UX designers
Buschek, D., Anlauff, C., & Lachner, F. (2020) · Mensch und Computer (MuC) · doi:10.1145/3404983.3405506 · Cited by 22
Ionescu, B. E., Müller, H., Péteri, R., Abacha, A. B., Sarrouti, M., Demner‐Fushman, D., Hasan, S. A., Kozlovski, S., Liauchuk, V., Cid, Y. D., et al. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-85251-1_23 · Cited by 18
Review Sharing via Deep Semi-Supervised Code Clone Detection
Guo, C., Yang, H., Huang, D., Zhang, J., Dong, N., Xu, J., & Zhu, J. (2020) · IEEE Access · doi:10.1109/access.2020.2966532 · Cited by 23
Translating Video Recordings of Complex Mobile App UI Gestures into Replayable Scenarios
Bernal-Cárdenas, C., Cooper, N., Havranek, M., Moran, K., Chaparro, O., Poshyvanyk, D., & Marcus, A. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3192279 · Cited by 17
Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineering
Si, C., Zhang, Y., Li, R., Yang, Z., Liu, R., & Yang, D. (2025) · Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) · doi:10.18653/v1/2025.naacl-long.199 · Cited by 17
“Beltramelli (2018) made a notable attempt back in 2017 with CNN and RNN models on a narrow set of simplistic UI designs.”
Ionescu, B. E., Müller, H., Péteri, R., Abacha, A. B., Datla, V. V., Hasan, S. A., Demner‐Fushman, D., Kozlovski, S., Liauchuk, V., Cid, Y. D., et al. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58219-7_22 · Cited by 15
Modeling Visual Containment for Web Page Layout Optimization
Kikuchi, K., Otani, M., Yamaguchi, K., & Simo‐Serra, E. (2021) · Computer Graphics Forum · doi:10.1111/cgf.14399 · Cited by 23
Sketch2Vis: Generating Data Visualizations from Hand-drawn Sketches with Deep Learning
Teng, Z., Fu, Q., White, J., & Schmidt, D. C. (2021) · International Conference on Machine Learning and Applications · doi:10.1109/icmla52953.2021.00141 · Cited by 15
UIC opilot : Automating UI Synthesis via Hierarchical Code Generation from Webpage Designs
Gui, Y., Wan, Y., Li, Z., Zhang, Z., Chen, D., Zhang, H., Su, Y., Chen, B., Zhou, X., Jiang, W., et al. (2025) · International World Wide Web Conference (WWW) · doi:10.1145/3696410.3714891 · Cited by 62
Tao, N., Ventresque, A., Nallur, V., & Saber, T. (2024) · Algorithms · doi:10.3390/a17070287 · Cited by 25
Improving pix2code based Bi-directional LSTM
Liu, Y., Hu, Q., & Shu, K. (2018) · IEEE International Conference on Automation, Electronics and Electrical Engineering (AUTEEE) · doi:10.1109/auteee.2018.8720784 · Cited by 21
Chen, Y.-C., Lee, Y.-J., Kao, K.-C., Tsai, J., Liang, E.-C., Chiu, W.-C., Shih, F., & Chang, Y.-J. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580689 · Cited by 22
Animation2API: API Recommendation for the Implementation of Android UI Animations
Wang, Y., Liu, H., Gao, S., & Tang, X. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3294971 · Cited by 16
ILuvUI: Instruction-tuned LangUage-Vision modeling of UIs from Machine Conversations
Jiang, Y., Schoop, E., Swearngin, A., & Nichols, J. W. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712129 · Cited by 39
Zheng, S., Hu, Z., & Ma, Y. (2019) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3289600.3290610 · Cited by 13
Program-Guided Image Manipulators
Zhang, X., Mao, J., Li, Y., Freeman, W. T., Tenenbaum, J. B., & Wu, J. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00413 · Cited by 25
MLLM-Based UI2Code Automation Guided by UI Layout Information
Wu, F., Gao, C., Li, S., Wen, X.-C., & Liao, Q. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3728925 · Cited by 27
A First Look at Dark Mode in Real-world Android Apps
Ma, S., Chen, C., Khalajzadeh, H., & Grundy, J. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3604607 · Cited by 12
Unified modeling language code generation from diagram images using multimodal large language models
Bates, A., Vavricka, R., Carleton, S., Shao, R., & Pan, C. (2025) · Machine Learning with Applications · doi:10.1016/j.mlwa.2025.100660 · Cited by 51
Does GenAI Make Usability Testing Obsolete?
Pourasad, A. E., & Maalej, W. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00138 · Cited by 33
Web pages from mockup design based on convolutional neural network and class activation mapping
Cizotto, A. A. J., de Souza, R. C. T., Mariani, V. C., & Coelho, L. D. S. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-15108-3 · Cited by 23
Challenges of automatically evaluating rich internet applications accessibility
Antonelli, H. L., Sensiate, L., Watanabe, W. M., & Fortes, R. P. D. M. (2019) · ACM International Conference on Design of Communication (SIGDOC) · doi:10.1145/3328020.3353950 · Cited by 12
Practical, Automated Scenario-Based Mobile App Testing
Yu, S., Fang, C., Du, M., Ding, Z., Chen, Z., & Su, Z. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3414672 · Cited by 11
Modern Integrated Development Environment (IDEs)
Alizadehsani, Z., Gomez, E. G., Ghaemi, H., Rodríguez, S., Jordán, J., Fernández, A., & Pérez-Lancho, B. (2021) · Lecture notes in networks and systems · doi:10.1007/978-3-030-78901-5_24 · Cited by 18
A novel code generator for graphical user interfaces
Cai, B., Luo, J., & Feng, Z. (2023) · Scientific Reports · doi:10.1038/s41598-023-46500-6 · Cited by 18
Exploring the Role of AI in Web Design and Development: A Voyage through Automated Code Generation
Muthazhagu, V. H., & Surendiran, B. (2024) · International Conference on Intelligent and Innovative Technologies in Computing, Electrical and Electronics (IITCEE) · doi:10.1109/iitcee59897.2024.10467409 · Cited by 10
Wimmer, C., Untertrifaller, A., & Grechenig, T. (2020) · Australian Conference on Human-Computer Interaction (OzCHI) · doi:10.1145/3441000.3441015 · Cited by 10
Icon2Code: Recommending code implementations for Android GUI components
Zhao, Y., Li, L., Sun, X., Liu, P., & Grundy, J. (2021) · Information and Software Technology · doi:10.1016/j.infsof.2021.106619 · Cited by 10
Tao, N., Ventresque, A., & Saber, T. (2022) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec55065.2022.9870312 · Cited by 10
Generative Colorization of Structured Mobile Web Pages
Kikuchi, K., Inoue, N., Otani, M., Simo‐Serra, E., & Yamaguchi, K. (2023) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv56688.2023.00364 · Cited by 11
History-Driven Fuzzing for Deep Learning Libraries
Harzevili, N. S., Mohajer, M. M., Wei, M., Pham, H. V., & Wang, S. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3688838 · Cited by 20
Guo, C., Huang, D., Dong, N., Ye, Q., Xu, J., Fan, Y., Yang, H., & Xu, Y. (2019) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner.2019.8668037 · Cited by 16
deepKnit: Learning-based Generation of Machine Knitting Code
Scheidt, F., Ou, J., Ishii, H., & Meisen, T. (2020) · Procedia Manufacturing · doi:10.1016/j.promfg.2020.10.068 · Cited by 12
UI layers merger: merging UI layers via visual learning and boundary prior
Chen, Y., Zhen, Y., Shi, C., Li, J., Chen, L., Li, Z., Sun, L., Zhou, T., & Chang, Y. (2023) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.2200099 · Cited by 9
EGFE: End-to-end Grouping of Fragmented Elements in UI Designs with Multimodal Learning
Chen, L., Chen, Y., Xiao, S., Song, Y., Sun, L., Zhen, Y., Zhou, T., & Chang, Y. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3623313 · Cited by 11
Tao, N., Ventresque, A., & Saber, T. (2023) · IEEE Latin American Conference on Computational Intelligence (LA-CCI) · doi:10.1109/la-cci58595.2023.10409384 · Cited by 14
Semantic GUI Scene Learning and Video Alignment for Detecting Duplicate Video-based Bug Reports
Yan, Y., Cooper, N., Chaparro, O., Moran, K., & Poshyvanyk, D. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3639163 · Cited by 21
GUI Widget Detection and Intent Generation via Image Understanding
Zhu, P., Li, Y., Li, T., Yang, W. M., & Xu, Y. (2021) · IEEE Access · doi:10.1109/access.2021.3131753 · Cited by 11
Transforming Hand Drawn Wireframes into Front-End Code with Deep Learning
Riaz, S., Ali, A., Band, S. S., & Mosavi, A. H. (2022) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2022.024819 · Cited by 12
Visual Recognition Of Graphical User Interface Components Using Deep Learning Technique
Rahmadi, A. A., & Sudaryanto, A. (2020) · Jurnal Ilmu Komputer dan Informasi · doi:10.21609/jiki.v13i1.845 · Cited by 10
Source-Code Generation Using Deep Learning: A Survey
Ahmed, A., Azab, S. S., & Abdelhamid, Y. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-49011-8_37 · Cited by 17
Tao, N., Ventresque, A., & Saber, T. (2023) · IEEE Latin American Conference on Computational Intelligence (LA-CCI) · doi:10.1109/la-cci58595.2023.10409432 · Cited by 8
Converting Web Pages Mockups to HTML using Machine Learning
Bouças, T., & Esteves, A. (2020) · International Conference on Web Information Systems and Technologies · doi:10.5220/0010116302170224 · Cited by 8
Style-Aware Sketch-to-Code Conversion for the Web
Calò, T., & Russis, L. D. (2022) · ACM SIGCHI Symposium on Engineering Interactive Computing Systems (EICS) · doi:10.1145/3531706.3536462 · Cited by 9
Push-button synthesis of watch companions for Android apps
Li, C., Jiang, Y., & Xu, C. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510003.3510056 · Cited by 12
Perspective Plane Program Induction From a Single Image
Li, Y., Mao, J., Zhang, X., Freeman, W. T., Tenenbaum, J. B., & Wu, J. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00449 · Cited by 12
Kolthoff, K., Bartelt, C., & Ponzetto, S. P. (2020) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3324884.3415289 · Cited by 8
Xiao, J., Wan, Y., Huo, Y., Wang, Z., Xu, X., Wang, W., Xu, Z., Wang, Y., & Lyu, M. R. (2024) · arXiv · doi:10.48550/arxiv.2411.03292 · Cited by 7
“Pix2code [8] combines CNNs and LSTM to generate a domain-specific language (DSL) from GUI images.”
Designing and Developing User Interfaces with AI: Advancing Tools, Workflows, and Practices
Lu, Y., Jiang, Y., Knearem, T., Kliman-Silver, C. E., Lutteroth, C., Nichols, J. W., & Stuerzlinger, W. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3706736 · Cited by 11
Automatically Generating Websites from Hand-drawn Mockups
Ferreira, J., Restivo, A., & Ferreira, H. S. (2021) · International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP) · doi:10.5220/0010193600480058 · Cited by 7
Speed Control Turbin Pada Pembangkit Listrik Micro Hydro Pintu Air Bendungan
Sitanggang, D. P. (2022) · Jurnal Sains Informatika Terapan · doi:10.62357/jsit.v1i3.107 · Cited by 7
Rani, L. N. (2022) · Jurnal Sains Informatika Terapan · doi:10.62357/jsit.v1i2.59 · Cited by 7
Automatic Code Generation From Low Fidelity Graphical User Interface Sketches Using Deep Learning
Adefris, B. B., Habtie, A. B., & Taye, Y. G. (2022) · International Conference on Information and Communication Technology for Development for Africa (ICT4DA) · doi:10.1109/ict4da56482.2022.9971204 · Cited by 7
Metadata
- Authors
- Beltramelli, T.
- Affiliations
- Uizard Technologies, Copenhagen, Denmark
- Corresponding author
- Beltramelli, Tony
- Venue
- Proceedings of the ACM SIGCHI Symposium on Engineering Interactive Computing Systems
- Publisher
- ACM
- Published
- 2018-06-19
- Pages
- 1-6
- OpenAlex
- W2962724148
- Cited by
- 237
- References
- 15
- Citations by year
- 2026: 22 · 2025: 32 · 2024: 25 · 2023: 25 · 2022: 30 · 2021: 38 · 2020: 30 · 2019: 23 · 2018: 12
- FWCI
- 20.21
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM SIGCHI Symposium on Engineering Interactive Computing Systems (EICS) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Information Systems › Mobile and Web Applications
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Keywords
- deep learning, end-to-end learning, iOS, Android, web application, software development automation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.