An empirical comparison of compiler testing techniques
The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Abstract
Compilers, as one of the most important infrastructure of today's digital world, are expected to be trustworthy. Different testing techniques are developed for testing compilers automatically. However, it is unknown so far how these testing techniques compared to each other in terms of testing effectiveness: how many bugs a testing technique can find within a time limit.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The oracle problem in software testing: A survey
Barr, E. T., Harman, M., McMinn, P., Shahbaz, M., & Yoo, S. (2015) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2014.2372785 · Cited by 1,104
Finding and understanding bugs in C compilers
Yang, X., Chen, Y., Eide, E., & Regehr, J. (2011) · Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation · doi:10.1145/1993498.1993532 · Cited by 825 · cited 18×
“To compare the effectiveness of the three compiler testing techniques, we use CSmith [29] to generate a set of test programs and use this set of test programs as the inputs to the compilers.”
BackgroundMethodInfluentialCompiler validation via equivalence modulo inputs
Le, V., Afshari, M., & Su, Z. (2014) · Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation · doi:10.1145/2594291.2594334 · Cited by 388 · cited 10×
“The number of variants is recommended by their paper [13].”
BackgroundMethodInfluential
Cited by
An empirical study on TensorFlow program bugs
Zhang, Y., Chen, Y., Cheung, S.-C., Xiong, Y., & Zhang, L. (2018) · Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis · doi:10.1145/3213846.3213866 · Cited by 304
Self-citationDeep learning library testing via effective model generation
Wang, Z., Yan, M., Chen, J., Liu, S., & Zhang, D. (2020) · Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3368089.3409761 · Cited by 167
“In traditional software systems, the test-oracle problem has been well studied and there are some ways of relieving the test-oracle problem [13, 17, 19, 22, 37, 49].”
MethodSelf-citationPerfection Not Required? Human-AI Partnerships in Code Translation
Weisz, J. D., Muller, M., Houde, S., Richards, J., Ross, S. I., Martinez, F., Agarwal, M., & Talamadupula, K. (2021) · 26th International Conference on Intelligent User Interfaces · doi:10.1145/3397481.3450656 · Cited by 129
Automated conformance testing for JavaScript engines via deep compiler fuzzing
Ye, G., Tang, Z., Tan, S. H., Huang, S., Fang, D., Sun, X., Bian, L., Wang, H., & Wang, Z. (2021) · Proceedings of the 42nd ACM SIGPLAN International Conference on Programming Language Design and Implementation · doi:10.1145/3453483.3454054 · Cited by 77
Bayesian Data Analysis in Empirical Software Engineering Research
Furia, C. A., Feldt, R., & Torkar, R. (2019) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2019.2935974 · Cited by 42
Metadata
- Authors
- Chen, J., Hu, W., Hao, D., Xiong, Y., Zhang, H., Zhang, L., & Xie, B.
- Venue
- Proceedings of the 38th International Conference on Software Engineering
- Publisher
- ACM
- Published
- 2016-05-14
- Pages
- 180-190
- ISSN
- 0270-5257 (ISSN-L)
- OpenAlex
- W2370472429
- Cited by
- 107
- References
- 3
- FWCI
- 7.43
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.