Fully automated functional fuzzing of Android apps for detecting non-crashing logic bugs
Open accessThe DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Abstract
Android apps are GUI-based event-driven software and have become ubiquitous in recent years. Obviously, functional correctness is critical for an app’s success. However, in addition to crash bugs, non-crashing functional bugs (in short as “non-crashing bugs” in this work) like inadvertent function failures, silent user data lost and incorrect display information are prevalent, even in popular, well-tested apps. These non-crashing functional bugs are usually caused by program logic errors and manifest themselves on the graphic user interfaces (GUIs). In practice, such bugs pose significant challenges in effectively detecting them because (1) current practices heavily rely on expensive, small-scale manual validation ( the lack of automation ); and (2) modern fully automated testing has been limited to crash bugs ( the lack of test oracles ). This paper fills this gap by introducing independent view fuzzing , a novel, fully automated approach for detecting non-crashing functional bugs in Android apps. Inspired by metamorphic testing, our key insight is to leverage the commonly-held independent view property of Android apps to manufacture property-preserving mutant tests from a set of seed tests that validate certain app properties. The mutated tests help exercise the tested apps under additional, adverse conditions. Any property violations indicate likely functional bugs for further manual confirmation. We have realized our approach as an automated, end-to-end functional fuzzing tool, Genie. Given an app, (1) Genie automatically detects non-crashing bugs without requiring human-provided tests and oracles (thus fully automated ); and (2) the detected non-crashing bugs are diverse (thus general and not limited to specific functional properties ), which set Genie apart from prior work. We have evaluated Genie on 12 real-world Android apps and successfully uncovered 34 previously unknown non-crashing bugs in their latest releases — all have been confirmed, and 22 have already been fixed. Most of the detected bugs are nontrivial and have escaped developer (and user) testing for at least one year and affected many app releases, thus clearly demonstrating Genie’s effectiveness. According to our analysis, Genie achieves a reasonable true positive rate of 40.9%, while these 34 non-crashing bugs could not be detected by prior fully automated GUI testing tools (as our evaluation confirms). Thus, our work complements and enhances existing manual testing and fully automated testing for crash bugs.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The oracle problem in software testing: A survey
Barr, E. T., Harman, M., McMinn, P., Shahbaz, M., & Yoo, S. (2015) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2014.2372785 · Cited by 1,104
A survey on tree edit distance and related problems
Bille, P. (2005) · Theoretical Computer Science · doi:10.1016/j.tcs.2004.12.030 · Cited by 751 · cited 1×
“In practice, to efficiently compute Δ ( ℓ 𝑖 𝑗 ) , we adapt the classic tree edit distance algorithms [Bille 2005; Pawlik and Augsten 2016; Zhang and Shasha 1989] to compute the minimal edit operations between the two ordered GUI trees 𝑇 𝑖 and 𝑇 𝑗 (corresponding to ℓ 𝑖 and ℓ 𝑗 ).”
Compiler validation via equivalence modulo inputs
Le, V., Afshari, M., & Su, Z. (2014) · Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation · doi:10.1145/2594291.2594334 · Cited by 388 · cited 1×
“It has been widely used in testing many other specification-less software systems, e.g. , compilers [Le et al. 2014; Sun et al. 2016], debuggers [Tolksdorf et al. 2019], and machine learning-based systems [Tian et al. 2018; Xie et al. 2011].”
Self-citation
Cited by
Free lunch for testing: fuzzing deep-learning libraries from open source
Wei, A., Deng, Y., Yang, C., & Zhang, L. (2022) · Proceedings of the 44th International Conference on Software Engineering · doi:10.1145/3510003.3510041 · Cited by 106
“We leverage metamorphic relations [23, 59] to detect performance bugs with FreeFuzz.”
MethodFuzzing deep-learning libraries via automated relational API inference
Deng, Y., Yang, C., Wei, A., & Zhang, L. (2022) · Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3540250.3549085 · Cited by 86
“It is commonly used in prior work on fuzzing or testing [22, 61, 80].”
MethodAutomated infrastructure as code program testing
Sokolowski, D., Spielmann, D., & Salvaneschi, G. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3393070 · Cited by 20
GUIPilot: A Consistency-Based Mobile GUI Testing Approach for Detecting Application-Specific Bugs
Liu, R., Teoh, X., Lin, Y., Chen, G., Ren, R., Poshyvanyk, D., & Dong, J. S. (2025) · Proceedings of the ACM on Software Engineering · doi:10.1145/3728909 · Cited by 12
Metadata
- Authors
- Su, T., Yan, Y., Wang, J., Sun, J., Xiong, Y., Pu, G., Wang, K., & Su, Z.
- Venue
- Proceedings of the ACM on Programming Languages
- Published
- 2021-10-15
- Volume
- 5
- Issue
- OOPSLA
- Pages
- 1-31
- ISSN
- 2475-1421 (ISSN-L)
- OpenAlex
- W3206846523
- Cited by
- 60
- References
- 3
- FWCI
- 13.96
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Fully automated functional fuzzing of Android apps for detecting non-crashing logic bugs. https://app.cochat.ai/papers/10.1145/3485533
- The paper
- Su, T., Yan, Y., Wang, J., Sun, J., Xiong, Y., Pu, G., Wang, K., & Su, Z. (2021). Fully automated functional fuzzing of Android apps for detecting non-crashing logic bugs. Proceedings of the ACM on Programming Languages. https://doi.org/10.1145/3485533
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.