Rico: A Mobile App Dataset for Building Data-Driven Design Applications
Open accessAbstract
Data-driven models help mobile app designers understand best practices and trends, and can be used to make predictions about design performance and support the creation of adaptive UIs. This paper presents Rico, the largest repository of mobile app designs to date, created to support five classes of data-driven applications: design search, UI layout generation, UI code generation, user interaction modeling, and user perception prediction. To create Rico, we built a system that combines crowdsourcing and automation to scalably mine design and interaction data from Android apps at runtime. The Rico dataset contains design data from more than 9.7k Android apps spanning 27 categories. It exposes visual, textual, structural, and interactive design properties of more than 72k unique UI screens. To demonstrate the kinds of applications that Rico enables, we present results from training an autoencoder for UI layout similarity, which supports query- by-example search over UIs.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The UICrit dataset contains 1,000 mobile UI screenshots with task descriptions sampled from Rico.
“We used the UICrit dataset [16], which contains 1,000 mobile UI screenshots with task descriptions sampled from Rico [15].”
From Criticmate: Stagewise Human–AI Co-Critique in Single-Screen UI Evaluation (2026) · cites this paper as [15]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Reverse Engineering Mobile Application User Interfaces with REMAUI (T)
Nguyen, T. A., & Csallner, C. (2015) · 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2015.32 · Cited by 200 · cited 1× · see citing works
“As a result, prior work has studied how different code components are used to implement UIs in popular apps [33] and developed models to automatically generate code from UI designs [29].”
Background
Flagged first, then by citations
Research works printed but not linked: 35
- Alharbi, K., and Yeh, T. Collect, decompile, extract, stats, and diff: Mining design pattern changes in Android apps. In Proc. MobileHCI (2015).
- Amini, S. Analyzing Mobile App Privacy Using Computation and Crowdsourcing. PhD thesis, Carnegie Mellon University, 2014.
- Azim, T., and Neamtiu, I. Targeted and depth-first exploration for systematic testing of android apps. In ACM SIGPLAN Notices (2013).
- Bell, S., and Bala, K. Learning visual similarity for product design with convolutional neural networks. ACM TOG (2015).
- Bengio, Y. Learning deep architectures for ai. Foundations and Trends in Machine Learning 2, 1 (2009).
- Bhoraskar, R., Han, S., Jeon, J., Azim, T., Chen, S., Jung, J., Nath, S., Wang, R., and Wetherall, D. Brahmastra: Driving apps to test the security of third-party components. In Proc. SEC (2014).
- Brandt, J., Dontcheva, M., Weskamp, M., and Klemmer, S. R. Example-centric programming: integrating web search into the development environment. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM (2010), 513-522.
- Deka, B., Huang, Z., and Kumar, R. ERICA: Interaction mining mobile apps. In Proc. UIST (2016).
- Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In Proc. CVPR (2009).
- Eckert, C., and Stacey, M. Sources of inspiration: A language of design. Design Studies 21, 5 (2000), 523-538.
- Eckert, C., Stacey, M., and Earl, C. References to past designs. Studying Designers 5 (2005), 3-21.
- Frank, M., Dong, B., Felt, A. P., and Song, D. Mining permission request patterns from android and facebook applications. In Proc. ICDM (2012).
- Fu, B., Lin, J., Li, L., Faloutsos, C., Hong, J., and Sadeh, N. Why people hate your app: Making sense of user feedback in a mobile app store. In Proc. KDD (2013).
- Funkhouser, T., Min, P., Kazhdan, M., Chen, J., Halderman, A., Dobkin, D., and Jacobs, D. A search engine for 3D models. ACM TOG (2003).
- Koch, J., and Oulasvirta, A. Computational layout perception using gestalt laws. In Proc. CHI (Extended Abstracts) (2016).
- Kumar, R., Satyanarayan, A., Torres, C., Lim, M., Ahmad, S., Klemmer, S. R., and Talton, J. O. Webzeitgeist: Design mining the web. In Proc. CHI (2013).
- Lee, K., Flinn, J., Giuli, T., Noble, B., and Peplin, C. Amc: Verifying user interface properties for vehicular applications. In Proc. Mobisys (2013).
- Leuthold, S., Schmutz, P., Bargas-Avila, J. A., Tuch, A. N., and Opwis, K. Vertical versus dynamic menus on the world wide web: Eye tracking study measuring the influence of menu design and task complexity on user performance and subjective preference. Computers in human behavior 27, 1 (2011), 459-472.
- McAuley, J., Targett, C., Shi, Q., and Van Den Hengel, A. Image-based recommendations on styles and substitutes. In Proc SIGIR, ACM (2015), 43-52.
- Miller, S. R., and Bailey, B. P. Searching for inspiration: An in-depth look at designers example finding practices. In ASME 2014 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (2014).
- Miniukovich, A., and De Angeli, A. Visual impressions of mobile app interfaces. In Proc. Nordic CHI (2014).
- Miniukovich, A., and De Angeli, A. Computation of interface aesthetics. In Proc. CHI (2015).
- Miniukovich, A., and De Angeli, A. Visual diversity and user interface quality. In Proc. British HCI (2015).
- Miniukovich, A., and De Angeli, A. Pick me!: Getting noticed on google play. In Proc. CHI (2016).
- Nair, V., and Hinton, G. E. Rectified linear units improve restricted boltzmann machines. In Proc. ICML (2010), 807-814.
- ODonovan, P., Agarwala, A., and Hertzmann, A. Learning layouts for single-page graphic designs. IEEE TVCG (2014).
- Reinecke, K., Yeh, T., Miratrix, L., Mardiko, R., Zhao, Y., Liu, J., and Gajos, K. Z. Predicting users' first impressions of website aesthetics with a quantification of perceived visual complexity and colorfulness. In Proc. CHI (2013).
- Ritchie, D., Kejriwal, A. A., and Klemmer, S. R. d. tour: Style-based exploration of design example galleries. In Proc. UIST (2011).
- Sahami Shirazi, A., Henze, N., Schmidt, A., Goldberg, R., Schmidt, B., and Schmauder, H. Insights into layout patterns of mobile user interfaces by an automatic analysis of Android apps. In Proc. EICS (2013).
- Szydlowski, M., Egele, M., Kruegel, C., and Vigna, G. Challenges for dynamic analysis of iOS applications. In Open Problems in Network Security. Springer, 2012, 65-77. Session: Crowd UIST 2017, Oct. 22-25, 2017, Québec City, Canada
- Talton, J., Yang, L., Kumar, R., Lim, M., Goodman, N., and Měch, R. Learning design patterns with bayesian grammar induction. In Proc. UIST, ACM (2012).
- Tractinsky, N., Inbar, O., Tsimhoni, O., and Seder, T. Slow down, you move too fast: Examining animation aesthetics to promote eco-driving. In Proc. AutoUI, ACM (2011), 193-202.
- van der Geest, T., and Loorbach, N. Testing the visual consistency of web sites. Technical communication 52, 1 (2005), 27-36.
- Viennot, N., Garcia, E., and Nieh, J. A measurement study of google play. In ACM SIGMETRICS Performance Evaluation Review, vol. 42, ACM (2014), 221-233.
- Yi, L., Guibas, L., Hertzmann, A., Kim, V. G., Su, H., and Yumer, E. Learning hierarchical shape segmentation and labeling from online repositories. In Proc. SIGGRAPH (2017).
Cited by
Sorted by citations
Examining Image-Based Button Labeling for Accessibility in Android Apps through Large-Scale Analysis
Ross, A. S., Zhang, X., Fogarty, J., & Wobbrock, J. O. (2018) · Proceedings of the 20th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3234695.3236364 · Cited by 90 · see citing works
Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMs
You, K., Zhang, H., Schoop, E., Weers, F., Swearngin, A., Nichols, J., Yang, Y., & Gan, Z. (2024) · arXiv · doi:10.48550/arxiv.2404.05719 · Cited by 40 · see citing works
Self-citationMultiBench: Multiscale Benchmarks for Multimodal Representation Learning
Liang, P. P., Lyu, Y., Fan, X., Wu, Z., Cheng, Y., Wu, J., Chen, L., Wu, P., Lee, M. A., Zhu, Y., et al. (2021) · arXiv · doi:10.48550/arxiv.2107.07502 · Cited by 85 · see citing works
“We use the E NRICO (Enhanced Rico) dataset [40, 93] of Android app screens (consisting of an image as well as a set of apps and their locations) categorized by their design motifs and collected for data-driven design applications such as design search, user interface (UI) layout generation, UI code…”
MethodKapoor, R., Butala, Y. P., Russak, M., Koh, J. Y., Kamble, K., AlShikh, W., & Salakhutdinov, R. (2025) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-73113-6_10 · Cited by 12 · see citing works
UISGPT: Automated Mobile UI Design Smell Detection with Large Language Models
Yang, B., & Li, S. (2024) · Electronics · doi:10.3390/electronics13163127 · Cited by 10 · see citing works
Criticmate: Stagewise Human–AI Co-Critique in Single-Screen UI Evaluation
Ko, J., Choi, J., Morales, C., Kim, D., & Ko, M. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790929 · Cited by 1 · see citing works
Vibe Design: Human-in-the-loop Agentic Framework for UI Design with Large Language Models
Hwang, J., & Kang, J. (2026) · doi:10.2139/ssrn.6297816 · Cited by 1 · see citing works
- Authors
- Deka, B., Huang, Z., Franzen, C., Hibschman, J., Afergan, D., Li, Y., Nichols, J., & Kumar, R.
- Affiliations
- University of Illinois Urbana-Champaign (US)
- Northwestern University (US)
- Google (United States) (US)
- Venue
- Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology
- Publisher
- ACM
- Published
- 2017-10-20
- Pages
- 845-854
- OpenAlex
- W2765874585
- Cited by
- 482
- References
- 1
- Citations by year
- 2026: 54 · 2025: 73 · 2024: 75 · 2023: 81 · 2022: 52 · 2021: 65 · 2020: 52 · 2019: 18 · 2018: 11 · 2017: 1
- FWCI
- 16.95
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- ACM Symposium on User Interface Software and Technology (UIST) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Physical Sciences › Computer Science › Information Systems › Mobile and Web Applications
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Keywords
- mobile application design, Android application, user interaction modeling, design search, autoencoder, crowdsourcing, adaptive user interfaces
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
