The DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
Sapienz: multi-objective automated testing for Android applications
Open accessAbstract
We introduce Sapienz, an approach to Android testing that uses multi-objective search-based testing to automatically explore and optimise test sequences, minimising length, while simultaneously maximising coverage and fault revelation. Sapienz combines random fuzzing, systematic and search-based exploration, exploiting seeding and multi-level instrumentation. Sapienz significantly outperforms (with large effect size) both the state-of-the-art technique Dynodroid and the widely-used tool, Android Monkey, in 7/10 experiments for coverage, 7/10 for fault detection and 10/10 for fault-revealing sequence length. When applied to the top 1,000 Google Play apps, Sapienz found 558 unique, previously unknown crashes. So far we have managed to make contact with the developers of 27 crashing apps. Of these, 14 have confirmed that the crashes are caused by real faults. Of those 14, six already have developer-confirmed fixes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A fast and elitist multiobjective genetic algorithm: NSGA-II
Deb, K., Pratap, A., Agarwal, S., & Meyarivan, T. (2002) · IEEE Transactions on Evolutionary Computation · doi:10.1109/4235.996017 · Cited by 50,053
A Comparison of Web and Mail Survey Response Rates
Kaplowitz, M. D., Hadlock, T. D., & Levine, R. (2004) · Public Opinion Quarterly · doi:10.1093/poq/nfh006 · Cited by 1,661 · cited 1×
“For such a ‘cold call’ outreach activity, 58 from 175 emails is relatively high [35,47].”
BackgroundDEAP: evolutionary algorithms made easy
Fortin, F.-A., Rainville, F.-M. D., Gardner, M.-A., Parizeau, M., & Gagné, C. (2012) · Cited by 1,642
A generalized Kruskal-Wallis test for comparing K samples subject to unequal patterns of censorship
BRESLOW, N. (1970) · Biometrika · doi:10.1093/biomet/57.3.579 · Cited by 1,411
The oracle problem in software testing: A survey
Barr, E. T., Harman, M., McMinn, P., Shahbaz, M., & Yoo, S. (2015) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2014.2372785 · Cited by 1,104
Self-citationArcuri, A., & Briand, L. (2011) · Proceedings of the 33rd International Conference on Software Engineering · doi:10.1145/1985793.1985795 · Cited by 968
A Critique and Improvement of the CL Common Language Effect Size Statistics of McGraw and Wong
Vargha, A., & Delaney, H. D. (2000) · Journal of Educational and Behavioral Statistics · doi:10.3102/10769986025002101 · Cited by 938
Guide to the Software Engineering Body of Knowledge - SWEBOK
Abran, A., Bourque, P., Dupuis, R., & Moore, J. W. (2001) · Cited by 934
Search-based software engineering
Harman, M., & Jones, B. F. (2001) · Information and Software Technology · doi:10.1016/s0950-5849(01)00189-6 · Cited by 914
Self-citationA Critique and Improvement of the "CL" Common Language Effect Size Statistics of McGraw and Wong
Vargha, A., Delaney, H. D., & Vargha, A. (2000) · Journal of Educational and Behavioral Statistics · doi:10.2307/1165329 · Cited by 828
Search-based software engineering: Trends, techniques and applications
Harman, M., Mansouri, S. A., & Zhang, Y. (2012) · ACM Computing Surveys · doi:10.1145/2379776.2379787 · Cited by 777
Self-citationDynodroid: an input generation system for Android apps
Machiry, A., Tahiliani, R., & Naik, M. (2013) · Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering · doi:10.1145/2491411.2491450 · Cited by 701
The Current State and Future of Search Based Software Engineering
Harman, M. (2007) · Future of Software Engineering (FOSE '07) · doi:10.1109/fose.2007.29 · Cited by 631
Self-citationFraser, G., & Arcuri, A. (2013) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2012.14 · Cited by 586 · cited 1×
“SBSE representation: Sapienz performs the whole test suite evolution [13, 33] thus each individual corresponds to a test suite.”
BackgroundUsing GUI ripping for automated testing of Android applications
Amalfitano, D., Fasolino, A. R., Tramontana, P., De Carmine, S., & Memon, A. M. (2012) · Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering · doi:10.1145/2351676.2351717 · Cited by 514
Targeted and depth-first exploration for systematic testing of android apps
Azim, T., & Neamtiu, I. (2013) · Conference on Object-Oriented Programming Systems, Languages, and Applications · doi:10.1145/2509136.2509549 · Cited by 482
Automated Test Input Generation for Android: Are We There Yet? (E)
Choudhary, S. R., Gorla, A., & Orso, A. (2015) · 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2015.89 · Cited by 463
Guide to the Software Engineering Body of Knowledge (SWEBOK(R)): Version 3.0
Bourque, P., & Fairley, R. E. (2014) · IEEE Computer Society Press eBooks · Cited by 456
Automated concolic testing of smartphone apps
Anand, S., Naik, M., Harrold, M. J., & Yang, H. (2012) · Proceedings of the ACM SIGSOFT 20th International Symposium on the Foundations of Software Engineering · doi:10.1145/2393596.2393666 · Cited by 401
Real Challenges in Mobile App Development
Joorabchi, M. E., Mesbah, A., & Kruchten, P. (2013) · 2013 ACM / IEEE International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem.2013.9 · Cited by 375
Techniques for improving response rates in OM survey research
Frohlich, M. T. (2002) · Journal of Operations Management · doi:10.1016/s0272-6963(02)00003-7 · Cited by 345
PUMA: programmable UI-automation for large-scale dynamic analysis of mobile apps
Hao, S., Liu, B., Nath, S., Halfond, W. G. J., & Govindan, R. (2014) · Proceedings of the 12th annual international conference on Mobile systems, applications, and services · doi:10.1145/2594368.2594390 · Cited by 340
Automated parameter optimization of classification techniques for defect prediction models
Tantithamthavorn, C., McIntosh, S., Hassan, A. E., & Matsumoto, K. (2016) · Proceedings of the 38th International Conference on Software Engineering · doi:10.1145/2884781.2884857 · Cited by 331
Guided GUI testing of android apps with minimal restart and approximate learning
Choi, W., Necula, G., & Sen, K. (2013) · Proceedings of the 2013 ACM SIGPLAN international conference on Object oriented programming systems languages & applications · doi:10.1145/2509136.2509552 · Cited by 315
A Grey-Box Approach for Automated GUI-Model Generation of Mobile Applications
Yang, W., Prasad, M. R., & Xie, T. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-37057-1_19 · Cited by 313
MobiGUITAR: Automated Model-Based Testing of Mobile Apps
Amalfitano, D., Fasolino, A. R., Tramontana, P., Ta, B. D., & Memon, A. M. (2015) · IEEE Software · doi:10.1109/ms.2014.55 · Cited by 278
Search Based Software Engineering: Techniques, Taxonomy, Tutorial
Harman, M., McMinn, P., de Souza, J. T., & Yoo, S. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-25231-0_1 · Cited by 245
Self-citationAutomated testing with targeted event sequence generation
Jensen, C. S., Prasad, M. R., & Møller, A. (2013) · Proceedings of the 2013 International Symposium on Software Testing and Analysis · doi:10.1145/2483760.2483777 · Cited by 200
Lessons Learned in Software Testing
Kaner, C., Bach, J., & Pettichord, B. (2001) · Cited by 197
Automatically Discovering, Reporting and Reproducing Android Application Crashes
Moran, K., Linares‐Vásquez, M., Bernal-Cardenas, C., Vendome, C., & Poshyvanyk, D. (2016) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst.2016.34 · Cited by 181
Mutation-driven generation of unit tests and oracles
Fraser, G., & Zeller, A. (2010) · International Symposium on Software Testing and Analysis · doi:10.1145/1831708.1831728 · Cited by 180
Efficient parallel genetic algorithms: theory and practice
Cantú-Paz, E., & Goldberg, D. E. (2000) · Computer Methods in Applied Mechanics and Engineering · doi:10.1016/s0045-7825(99)00385-0 · Cited by 177
Understanding the Test Automation Culture of App Developers
Kochhar, P. S., Thung, F., Nagappan, N., Zimmermann, T., & Lo, D. (2015) · 2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst.2015.7102609 · Cited by 159
Reducing combinatorics in GUI testing of android applications
Mirzaei, N., Garcia, J., Bagheri, H., Sadeghi, A., & Malek, S. (2016) · Proceedings of the 38th International Conference on Software Engineering · doi:10.1145/2884781.2884853 · Cited by 158
Searching for better configurations: a rigorous approach to clone evaluation
Wang, T., Harman, M., Jia, Y., & Krinke, J. (2013) · Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering · doi:10.1145/2491411.2491420 · Cited by 147
Self-citationTesting android apps through symbolic execution
Mirzaei, N., Malek, S., Păsăreanu, C. S., Esfahani, N., & Mahmood, R. (2012) · ACM SIGSOFT Software Engineering Notes · doi:10.1145/2382756.2382798 · Cited by 143
Efficient unit test case minimization
Leitner, A., Oriol, M., Zeller, A., Ciupa, I., & Meyer, B. (2007) · doi:10.1145/1321631.1321698 · Cited by 121
The Seed is Strong: Seeding Strategies in Search-Based Software Testing
Fraser, G., & Arcuri, A. (2012) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst.2012.92 · Cited by 120
The Role of the Tester's Knowledge in Exploratory Software Testing
Itkonen, J., Mäntylä, M., & Lassenius, C. (2012) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2012.55 · Cited by 119
Automated web application testing using search based software engineering
Alshahwan, N., & Harman, M. (2011) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2011.6100082 · Cited by 116
Self-citationSystematic execution of Android test suites in adverse conditions
Adamsen, C. Q., Mezzetti, G., & Møller, A. (2015) · International Symposium on Software Testing and Analysis · doi:10.1145/2771783.2771786 · Cited by 111
On the Accuracy, Efficiency, and Reusability of Automated Test Oracles for Android Devices
Lin, Y., Rojas, J. F., Chu, E. T.-H., et al. (2014) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2014.2331982 · Cited by 55
Automatically generating realistic test input from web services
Bozkurt, M., & Harman, M. (2011) · IEEE International Symposium on Service-Oriented System Engineering (SOSE) · doi:10.1109/sose.2011.6139088 · Cited by 50
Self-citationArcuri, A. (2011) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2011.44 · Cited by 40
Yoo, S., Harman, M., & Ur, S. (2013) · Empirical Software Engineering · doi:10.1007/s10664-013-9247-y · Cited by 31
Self-citationConcept Location with Genetic Algorithms: A Comparison of Four Distributed Architectures
Asadi, F., Antoniol, G., & Guéhéneuc, Y. (2010) · Symposium on Search Based Software Engineering · doi:10.1109/ssbse.2010.26 · Cited by 29
Dynamically Testing GUIs Using Ant Colony Optimization (T)
Carino, S., & Andrews, J. H. (2015) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2015.70 · Cited by 25
Cited by
A Survey of App Store Analysis for Software Engineering
Martin, W., Sarro, F., Jia, Y., Zhang, Y., & Harman, M. (2017) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2016.2630689 · Cited by 459
“[151] introduced Sapienz, anAndroid testing tool that the authors applied to the top 1,000Google Play apps, revealing 558 previously undetected crashing test causes.”
MethodSelf-citationGuided, stochastic model-based GUI testing of Android apps
Su, T., Meng, G., Chen, Y., Wu, K., Yang, W., Yao, Y., Pu, G., Liu, Y., & Su, Z. (2017) · Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering · doi:10.1145/3106237.3106298 · Cited by 371
Zhu, X., Wen, S., Camtepe, S., & Xiang, Y. (2022) · ACM Computing Surveys · doi:10.1145/3512345 · Cited by 334
CodaMosa: Escaping Coverage Plateaus in Test Generation with Pre-trained Large Language Models
Lemieux, C., Inala, J. P., Lahiri, S. K., & Sen, S. (2023) · 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) · doi:10.1109/icse48619.2023.00085 · Cited by 306
Metadata
- Authors
- Mao, K., Harman, M., & Jia, Y.
- Venue
- Proceedings of the 25th International Symposium on Software Testing and Analysis
- Publisher
- ACM
- Published
- 2016-07-18
- Pages
- 94-105
- OpenAlex
- W2463553622
- Cited by
- 541
- References
- 47
- FWCI
- 43.08
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.