From Start-ups to Scale-ups: Opportunities and Open Problems for Static and Dynamic Program Analysis
Open accessThe DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Abstract
This paper describes some of the challenges and opportunities when deploying static and dynamic analysis at scale, drawing on the authors' experience with the Infer and Sapienz Technologies at Facebook, each of which started life as a research-led start-up that was subsequently deployed at scale, impacting billions of people worldwide. The paper identifies open problems that have yet to receive significant attention from the scientific community, yet which have potential for profound real world impact, formulating these as research questions that, we believe, are ripe for exploration and that would make excellent topics for research projects. Note: This paper accompanies the authors' joint keynote at the 18th IEEE International Working Conference on Source Code Analysis and Manipulation, September 23rd-24th, 2018 - Madrid, Spain.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The oracle problem in software testing: A survey
Barr, E. T., Harman, M., McMinn, P., Shahbaz, M., & Yoo, S. (2015) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2014.2372785 · Cited by 1,104
Self-citationAbstract Interpretation Frameworks
COUSOT, P., & COUSOT, R. (1992) · Journal of Logic and Computation · doi:10.1093/logcom/2.4.511 · Cited by 710
Applied Machine Learning at Facebook: A Datacenter Infrastructure Perspective
Hazelwood, K., Bird, S., Brooks, D., Chintala, S., Diril, U., Dzhulgakov, D., Fawzy, M., Jia, B., Jia, Y., Kalro, A., et al. (2018) · 2018 IEEE International Symposium on High Performance Computer Architecture (HPCA) · doi:10.1109/hpca.2018.00059 · Cited by 590
A Survey of App Store Analysis for Software Engineering
Martin, W., Sarro, F., Jia, Y., Zhang, Y., & Harman, M. (2017) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2016.2630689 · Cited by 459 · cited 2×
“Many systems, such as those typically deployed through app stores, for example [94], have ample mechanisms for users to report bugs, and most practicing developers are typically overwhelmed by such lists; their most pressing problem is not simply finding more bugs; their problem is more nuanced.”
BackgroundSelf-citationDisruption and recovery of computing tasks: field study, analysis, and directions
Iqbal, S. T., & Horvitz, E. (2007) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1240624.1240730 · Cited by 300
Development and Deployment at Facebook
Feitelson, D. G., Frachtenberg, E., & Beck, K. L. (2013) · IEEE Internet Computing · doi:10.1109/mic.2013.25 · Cited by 243
Cited by
Machine Learning Testing: Survey, Landscapes and Horizons
Zhang, J. M., Harman, M., Ma, L., & Liu, Y. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2019.2962027 · Cited by 897
Self-citationLarge language models for software engineering: survey and open problems
Fan, A., Gokkaya, B., Harman, M., Lyubarskiy, M., Sengupta, S., Yoo, S., & Zhang, J. M. (2023) · 2023 IEEE/ACM International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE) · doi:10.1109/icse-fose59343.2023.00008 · Cited by 650
“Test flakiness is one of the most pressing and impactful problems that inhibit test effectiveness in industry [125].”
BackgroundSelf-citationLarge Language Models for Software Engineering: Survey and Open Problems
Fan, A., Gokkaya, B., Harman, M., Lyubarskiy, M., Sengupta, S., Yoo, S., & Zhang, J. M. (2023) · arXiv · doi:10.48550/arxiv.2310.03533 · Cited by 523
Self-citationScaling static analyses at Facebook
Distefano, D., Fähndrich, M., Logozzo, F., & O'Hearn, P. W. (2019) · Communications of the ACM · doi:10.1145/3338112 · Cited by 212
Self-citationA large-scale longitudinal study of flaky tests
Lam, W., Winter, S., Wei, A., Xie, T., Marinov, D., & Bell, J. (2020) · Proceedings of the ACM on Programming Languages · doi:10.1145/3428270 · Cited by 64
“For example, for the static bug-detection tool Infer [2020] at Facebook, Harman and O’Hearn [2018] found that bugs reported to developers at “post land” (after code has been merged into the SUT) had a close to 0% fix rate, yet bugs reported at “diff time” (when the developer submits the code change) had a fix rate of…”
Method
Metadata
- Authors
- Harman, M., & O'Hearn, P.
- Venue
- 2018 IEEE 18th International Working Conference on Source Code Analysis and Manipulation (SCAM)
- Publisher
- IEEE
- Published
- 2018-09
- Pages
- 1-23
- OpenAlex
- W2901387825
- Cited by
- 151
- References
- 6
- FWCI
- 15.15
- Type
- conference paper
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.