CoChat

A Survey of Compiler Testing

Chen, Junjie, Patra, Jibesh, Pradel, Michael, Xiong, Yingfei, Zhang, Hongyu, Hao, Dan, Zhang, Lu · 2021 · ACM Computing Surveys
Cited by 215

The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.

Abstract

Virtually any software running on a computer has been processed by a compiler or a compiler-like tool. Because compilers are such a crucial piece of infrastructure for building software, their correctness is of paramount importance. To validate and increase the correctness of compilers, significant research efforts have been devoted to testing compilers. This survey article provides a comprehensive summary of the current state-of-the-art of research on compiler testing. The survey covers different aspects of the compiler testing problem, including how to construct test programs, what test oracles to use for determining whether a compiler behaves correctly, how to execute compiler tests efficiently, and how to help compiler developers take action on bugs discovered by compiler testing. Moreover, we survey work that empirically studies the strengths and weaknesses of current compiler testing research and practice. Based on the discussion of existing work, we outline several open challenges that remain to be addressed in future work.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3363562

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

  • A Survey on Software Fault Localization

    Wong, W. E., Gao, R., Li, Y., Abreu, R., & Wotawa, F. (2016) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2016.2521368 · Cited by 1,110 · cited 1×

    “Christi et al. [33] investigate the impact of delta debugging for test input reduction (including test program reduction) on spectrum-based bug localization that localizes bugs based on statistics of coverage status of each program element during failing and passing executions [1, 121].”
    Background
  • Finding and understanding bugs in C compilers

    Yang, X., Chen, Y., Eide, E., & Regehr, J. (2011) · Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation · doi:10.1145/1993498.1993532 · Cited by 825 · cited 10×

    “In test program generation, to generate only valid programs without undefined behaviors, tools like Csmith [123] rely on static analysis to approximate the boundary of a valid program, and thus cannot generate all valid programs.”
    BackgroundMethodInfluential
  • Compiler validation via equivalence modulo inputs

    Le, V., Afshari, M., & Su, Z. (2014) · Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation · doi:10.1145/2594291.2594334 · Cited by 388 · cited 10×

    “Even though considerable efforts have been made to improve the quality of compilers, similar to all other software, compilers still contain bugs [72, 103, 123].”
    BackgroundMethodInfluential

Cited by

See all 215

Metadata

Authors
Chen, J., Patra, J., Pradel, M., Xiong, Y., Zhang, H., Hao, D., & Zhang, L.
Venue
ACM Computing Surveys
Published
2020-02-06
Volume
53
Issue
1
Pages
1-36
ISSN
  • 0360-0300 (ISSN-L)
  • 1557-7341
OpenAlex
W3007855180
Cited by
215
References
3
FWCI
41.15
Type
journal article
Language
en
Open access
closed

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.