Testing the Limits: Unusual Text Inputs Generation for Mobile App Crash Detection with Large Language Model
The DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
Mobile applications have become a ubiquitous part of our daily life, providing users with access to various services and utilities. Text input, as an important interaction channel between users and applications, plays an important role in core functionality such as search queries, authentication, messaging, etc. However, certain special text (e.g., -18 for Font Size) can cause the app to crash, and generating diversified unusual inputs for fully testing the app is highly demanded. Nevertheless, this is also challenging due to the combination of explosion dilemma, high context sensitivity, and complex constraint relations. This paper proposes InputBlaster which leverages the LLM to automatically generate unusual text inputs for mobile app crash detection. It formulates the unusual inputs generation problem as a task of producing a set of test generators, each of which can yield a batch of unusual text inputs under the same mutation rule. In detail, InputBlaster leverages LLM to produce the test generators together with the mutation rules serving as the reasoning chain, and utilizes the in-context learning schema to demonstrate the LLM with examples for boosting the performance. InputBlaster is evaluated on 36 text input widgets with cash bugs involving 31 popular Android apps, and results show that it achieves 78% bug detection rate, with 136% higher than the best baseline. Besides, we integrate it with the automated GUI testing tool and detect 37 unseen crashes in real-world apps.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“InputBlaster [260] generates unusual text input that could cause mobile app crash.”
From Retrieval-Augmented Generation for AI-Generated Content: A Survey (2024) · cites this paper as [260]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
Rico: A Mobile App Dataset for Building Data-Driven Design Applications
Deka, B., Huang, Z., Franzen, C., Hibschman, J., Afergan, D., Li, Y., Nichols, J., & Kumar, R. (2017) · Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3126594.3126651 · Cited by 482
Reinforcement Learning with Augmented Data
Laskin, M., Lee, K., Stooke, A., Pinto, L., Abbeel, P., & Srinivas, A. (2020) · arXiv · doi:10.48550/arxiv.2004.14990 · Cited by 299
Gender and Representation Bias in GPT-3 Generated Stories
Lucy, L., & Bamman, D. (2021) · Proceedings of the Third Workshop on Narrative Understanding · doi:10.18653/v1/2021.nuse-1.5 · Cited by 287
Large Language Models are Few-shot Testers: Exploring LLM-based General Bug Reproduction
Kang, S., Yoon, J., & Yoo, S. (2023) · 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) · doi:10.1109/icse48619.2023.00194 · Cited by 204
Retrieval-Based Prompt Selection for Code-Related Few-Shot Learning
Nashid, N., Sintaha, M., & Mesbah, A. (2023) · 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) · doi:10.1109/icse48619.2023.00205 · Cited by 178
An extensive study on pre-trained models for program understanding and generation
Zeng, Z., Tan, H., Zhang, H., Li, J., Zhang, Y., & Zhang, L. (2022) · Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis · doi:10.1145/3533767.3534390 · Cited by 153
What Can Transformers Learn In-Context? A Case Study of Simple Function Classes
Garg, S., Tsipras, D., Liang, P., & Valiant, G. (2022) · arXiv · doi:10.48550/arxiv.2208.01066 · Cited by 110
Fuzzing deep-learning libraries via automated relational API inference
Deng, Y., Yang, C., Wei, A., & Zhang, L. (2022) · Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3540250.3549085 · Cited by 86
Research works printed but not linked: 57
- Toufique Ahmed and Premkumar Devanbu. 2022. Few-shot training LLMs for project-specific code-summarization. ASE (2022).
- Nadia Alshahwan and Mark Harman. 2011. Automated web application testing using search based software engineering. In ASE. IEEE, 3--12.
- 10.1145/2393596.2393666 · DOI · Paper page
- 10.1109/APSEC.2016.036 · doi:10.1109/apsec.2016.036
- 10.1002/spe.2564 · doi:10.1002/spe.2564
- 10.1145/2509136.2509549 · DOI
- Tom Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared D Kaplan Prafulla Dhariwal Arvind Neelakantan Pranav Shyam Girish Sastry Amanda Askell et al. 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020) 1877--1901.
- Tom Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared D Kaplan Prafulla Dhariwal Arvind Neelakantan Pranav Shyam Girish Sastry Amanda Askell et al. 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020) 1877--1901.
- 10.1145/3387903.3389308 · DOI
- 10.1016/j.cose.2018.02.002 · DOI
- 10.1145/3498707 · DOI
- 10.1007/978-3-030-59152-6_18 · doi:10.1007/978-3-030-59152-6_18
- Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, and Geoffrey E Hinton. 2020. Big self-supervised models are strong semi-supervised learners. Advances in neural information processing systems 33 (2020), 22243--22255.
- Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2022. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311 (2022). · arXiv
- 10.1007/978-3-030-30806-3_8 · doi:10.1007/978-3-030-30806-3_8
- Yinlin Deng, Chunqiu Steven Xia, Chenyuan Yang, Shizhuo Dylan Zhang, Shujing Yang, and Lingming Zhang. 2023. Large Language Models are Edge-Case Fuzzers: Testing Deep Learning Libraries via FuzzGPT. ISSTA (2023).
- 10.1145/3377811.3380402 · DOI
- 10.1109/ISSRE.2013.6698888 · DOI
- 10.1109/ICSE.2019.00042 · doi:10.1109/icse.2019.00042
- 10.1109/SP40000.2020.00071 · DOI
- Lukav Holik, Petr Jank, Anthony W Lin, Philipp Rmmer, and Tom Vojnar. 2017. String constraints with concatenation and transducers solved efficiently. Proceedings of the ACM on Programming Languages 2, POPL (2017), 1--32.
- 10.1109/ASE.2019.00044 · DOI · Paper page
- Nan Jiang, Kevin Liu, Thibaud Lutellier, and Lin Tan. 2023. Impact of Code Language Models on Automated Program Repair. ICSE (2023).
- 10.1145/2377656.2377662 · DOI
- Sebastian Krings, Joshua Schmidt, Patrick Skowronek, Jannik Dunkelau, and Dierk Ehmke. 2020. Towards constraint logic programming over strings for test data generation. In Declarative Programming and Knowledge Management: Conference on Declarative Programming, DECLARE 2019, Unifying INAP, WLP, and WFLP, Cottbus, Germany, September 9--12, 2019, Revised Selected Papers 22. Springer, 139--159.
- Caroline Lemieux, Jeevana Priya Inala, Shuvendu K Lahiri, and Siddhartha Sen. 2023. CODAMOSA: Escaping Coverage Plateaus in Test Generation with Pretrained Large Language Models. In ICSE.
- 10.1007/978-3-319-03077-7_2 · DOI
- Yuanchun Li, Ziyue Yang, Yao Guo, and Xiangqun Chen. 2017. DroidBot: A Lightweight UI-Guided Test Input Generator for Android (ICSE-C '17).
- Yuanchun Li, Ziyue Yang, Yao Guo, and Xiangqun Chen. 2017. Droidbot: a lightweight ui-guided test input generator for android. In 2017 IEEE/ACM 39th International Conference on Software Engineering Companion (ICSE-C). IEEE, 23--26.
- Yuanchun Li, Ziyue Yang, Yao Guo, and Xiangqun Chen. 2019. Humanoid: a deep learning-based approach to automated black-box Android app testing. In ASE. IEEE, 1070--1073.
- 10.1109/TR.2018.2834476 · DOI · Paper page
- 10.1007/978-3-319-08867-9_43 · doi:10.1007/978-3-319-08867-9_43
- 10.1109/ICSE.2017.65 · DOI · Paper page
- Zhe Liu, Chunyang Chen, Junjie Wang, Xing Che, Yuekai Huang, Jun Hu, and Qing Wang. 2022. Fill in the Blank: Context-aware Automated Text Input Generation for Mobile GUI Testing. arXiv preprint arXiv:2212.04732 (2022). · arXiv
- 10.1145/3324884.3416547 · DOI · Paper page
- 10.1109/TSE.2022.3150876 · DOI · Paper page
- 10.1109/ICSE48619.2023.00168 · DOI
- 10.1145/3510454.3516848 · DOI
- 10.1145/3491102.3501903 · DOI
- 10.1145/2491411.2491450 · DOI · Paper page
- 10.1145/2931037.2931054 · DOI · Paper page
- Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient Estimation of Word Representations in Vector Space. Computer Science (2013).
- Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022. Rethinking the Role of Demonstrations: What Makes In-Context Learning Work? arXiv preprint arXiv:2202.12837 (2022). · arXiv
- 10.1145/3395363.3397354 · DOI · Paper page
- Gabriel Poesia, Oleksandr Polozov, Vu Le, Ashish Tiwari, Gustavo Soares, Christopher Meek, and Sumit Gulwani. 2022. Synchromesh: Reliable code generation from pre-trained language models. ICLR (2022).
- 10.1109/ICSTW.2017.21 · doi:10.1109/icstw.2017.21
- 10.1145/2435349.2435379 · DOI · Paper page
- 10.1109/32.799955 · DOI · Paper page
- 10.1007/s40593-022-00300-7 · DOI · Paper page
- 10.1145/3106237.3106298 · DOI · Paper page
- Nezih Sunman, Yiğit Soydan, and Hasan Sözer. 2022. Automated web application testing driven by pre-recorded test cases. Journal of Systems and Software (2022), 111441.
- 10.1145/2660267.2660372 · DOI
- 10.1007/978-3-319-63390-9_21 · DOI
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems (2017).
- Jue Wang Yanyan Jiang Chang Xu Chun Cao Xiaoxing Ma and Jian Lu. 2020. Combodroid: generating high-quality test inputs for android apps via use case combinations. In ICSE. 469--480. · DOI
- 10.1609/aaai.v36i3.20215 · DOI · Paper page
- Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. 2022. Opt: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068 (2022). · arXiv
Cited by
Software Testing With Large Language Models: Survey, Landscape, and Vision
Wang, J., Huang, Y., Chen, C., Liu, Z., Wang, S., & Wang, Q. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3368208 · Cited by 573
“Deng et al. [64] use LLM to extract key information related to the test scenario from a traffic rule, and represent the extracted information in a test scenario schema, then synthesize the corresponding scenario scripts to construct the test scenario.”
Self-citationRetrieval-Augmented Generation for AI-Generated Content: A Survey
Zhao, P., Zhang, H., Yu, Q., Wang, Z., Geng, Y., Fu, F., Yang, L., Zhang, W., Jiang, J., & Cui, B. (2024) · arXiv · doi:10.48550/arxiv.2402.19473 · Cited by 240
“InputBlaster [293] aims to generate unusual text input that could cause mobile app crash.”
GUIPilot: A Consistency-Based Mobile GUI Testing Approach for Detecting Application-Specific Bugs
Liu, R., Teoh, X., Lin, Y., Chen, G., Ren, R., Poshyvanyk, D., & Dong, J. S. (2025) · Proceedings of the ACM on Software Engineering · doi:10.1145/3728909 · Cited by 12
Metadata
- Authors
- Liu, Z., Chen, C., Wang, J., Chen, M., Wu, B., Tian, Z., Huang, Y., Hu, J., & Wang, Q.
- Affiliations
- University of Chinese Academy of Sciences, Beijing, China
- Technical University of Munich, Munich, Germany
- Institute of Software, Chinese Academy of Sciences, Beijing, China
- Pennsylvania State University, State College, Pennsylvania, USA
- ORCID
- Venue
- Proceedings of the IEEE/ACM 46th International Conference on Software Engineering
- Publisher
- ACM
- Published
- 2024-04-12
- Pages
- 1-12
- Cited by
- 58
- References
- 9
- Funders
- National Natural Science Foundation of China (No.62232016, No.62072442, No.62272445)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.