Finding and understanding bugs in C compilers
The DOI matches its Crossref record; 22 of the 23 references it cites were found in a registry or an index.
Abstract
Compilers should be correct. To improve the quality of C compilers, we created Csmith, a randomized test-case generation tool, and spent three years using it to find compiler bugs. During this period we reported more than 325 previously unknown bugs to compiler developers. Every compiler we tested was found to crash and also to silently generate wrong code when presented with valid input. In this paper we present our compiler-testing tool and the results of our bug-hunting study. Our first contribution is to advance the state of the art in compiler testing. Unlike previous tools, Csmith generates programs that cover a large subset of C while avoiding the undefined and unspecified behaviors that would destroy its ability to automatically find wrong-code bugs. Our second contribution is a collection of qualitative and quantitative results about the bugs we have found in open-source C compilers.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This paper demonstrates that modern titanic LLMs can be leveraged to directly perform both generation-based and mutation-based fuzzing studied for decades, while being fully automated, generalizable, and applicable to domains challenging for traditional approaches (such as DL systems).”
From Large Language Models Are Zero-Shot Fuzzers: Fuzzing Deep-Learning Libraries via Large Language Models (2023) · cites this paper as [80]
Citations
Cites
International Organization for Standardization
(2011) · SpringerReference · doi:10.1007/springerreference_16847 · Cited by 12,534
Klein, G., Elphinstone, K., Heiser, G., Andronick, J., Cock, D., Derrin, P., Elkaduwe, D., Engelhardt, K., Kolanski, R., Norrish, M., et al. (2009) · Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles · doi:10.1145/1629575.1629596 · Cited by 1,633 · cited 1×
“As measured by the responses to our bug reports, the defects discovered by Csmith are important.”
MethodAn empirical study of the reliability of UNIX utilities
Miller, B. P., Fredriksen, L., & So, B. (1990) · Communications of the ACM · doi:10.1145/96267.96279 · Cited by 1,281
Formal verification of a realistic compiler
Leroy, X. (2009) · Communications of the ACM · doi:10.1145/1538788.1538814 · Cited by 1,187 · cited 1×
“Wolfe [30], talk ing about independent software vendors (ISVs) says: An ISV with a complex code can work around correctness, turn off the optimizer in one or two .les, and usually they have to do that for any of the compilers they use (emphasis ours).”
BackgroundEncyclopedia of Software Engineering
(2002) · Wiley eBooks · doi:10.1002/0471028959 · Cited by 1,161
Simplifying and isolating failure-inducing input
Zeller, A., & Hildebrandt, R. (2002) · IEEE Transactions on Software Engineering · doi:10.1109/32.988498 · Cited by 1,079
An experimental evaluation of the assumption of independence in multiversion programming
Knight, J. C., & Leveson, N. G. (1986) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1986.6312924 · Cited by 774 · cited 3×
“In summary, despite the fact that Knight and Leveson [13] found a substantial number of correlated errors in an experiment on Nversion programming, Csmith has yielded no evidence of correlated failures among unrelated C compilers.”
BackgroundInfluential(1998) · Cited by 469
Grammar-based whitebox fuzzing
Godefroid, P., Kiezun, A., & Levin, M. Y. (2008) · Proceedings of the 29th ACM SIGPLAN Conference on Programming Language Design and Implementation · doi:10.1145/1375581.1375607 · Cited by 462 · cited 1×
“[8] showed a way to solve this problem by integrating a constraint solver with a grammar for the language being generated.”
BackgroundHDD: hierarchical delta debugging
Misherghi, G., & Su, Z. (2006) · Proceedings of the 28th international conference on Software engineering · doi:10.1145/1134285.1134307 · Cited by 274 · cited 1×
“Delta debugging [31] automates test-case reduction, but all existing variants that are intended for reducing C programs—such as hierarchical delta debugging [18] and Wilkerson’s implementation [29]—introduce undefined behavior.”
Background
Research works printed but not linked: 7
- F. Bellard . TCC: Tiny C compiler, ver. 0.9.25 , May 2009 . http://bellard.org/tcc/. F. Bellard. TCC: Tiny C compiler, ver. 0.9.25, May 2009. http://bellard.org/tcc/.
- C. L. Biffle . Undefined behavior in Google NaCl , Jan. 2010 . http://code.google.com/p/nativeclient/issues/detail?id=245. C. L. Biffle. Undefined behavior in Google NaCl, Jan. 2010. http://code.google.com/p/nativeclient/issues/detail?id=245.
- 10.1002/spe.v37:14 · doi:10.1002/spe.v37:14
- B. Turner . Random Program Generator , Jan. 2007 . http://sites.google.com/site/brturn2/randomcprogramgenerator. B. Turner. Random Program Generator, Jan. 2007. http://sites.google.com/site/brturn2/randomcprogramgenerator.
- 10.5555/1060289.1060313 · doi:10.5555/1060289.1060313
- D. S. Wilkerson . Delta ver . 2006 .08.03, Aug. 2006. http://delta.tigris.org/. D. S. Wilkerson. Delta ver. 2006.08.03, Aug. 2006. http://delta.tigris.org/.
- C. Zhao Automated test program generation for an industrial optimizing compiler . In Proc. ICSE Workshop on Automation of Software Test , pages 36 -- 43 , May 2009 . C. Zhao et al. Automated test program generation for an industrial optimizing compiler. In Proc. ICSE Workshop on Automation of Software Test, pages 36--43, May 2009.
Cited by
Items 601–700 of 833
Procedural Abstraction on RISC-V Using TP Register
Li, Z., & Liu, B. (2024) · IEEE Information Technology, Networking, Electronic and Automation Control Conference (ITNEC) · doi:10.1109/itnec60942.2024.10733182 · Cited by 1
“This enables more complex functions for some chips with small memory or storage [25]. b) Check Csmith and Mibench : The paper uses Mibench and Csmith to validate the correctness and estimate the effect of the optimization [26].”
MethodLiftFuzz: Validating Binary Lifters through Context-aware Fuzzing with GPT
Zhou, Y., Yang, F., Song, Z., Zhang, K., Chen, J., & Zhang, K. (2024) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3658644.3670276 · Cited by 1
“…from real-world scenarios where source code may not be accessible, such as in IoT firmware, malware, legacy code, or Commercial-Off-The-Shelf (COTS) software, but also since compilers themselves are not exempt from bugs or errors [49, 55] and toolchains can be contaminated, e.g. , Xcode Ghost [54].”
Detecting Compiler Error Recovery Defects via Program Mutation Exploration
Tang, Y., Zhang, J., Li, H., Huang, Z., & Jiang, H. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3510912 · Cited by 2
“Although there exist many other test program generators, such as Csmith [11], Epiphron [15], they mainly target generating C programs which are not suitable for our purpose.”
The Ethical Compiler: Addressing the Is-Ought Gap in Compilation (Invited Talk)
Bowman, W. J. (2025) · ACM SIGPLAN Workshop on Partial Evaluation and Program Manipulation (PEPM) · doi:10.1145/3704253.3706135 · Cited by 1
“Compilers should be correct [36].”
Insights From Bugs in FPGA High-Level Synthesis Tools: An Empirical Study of Bambu Bugs
Wang, Z., Jiang, H., Li, X., Guo, S., Zhao, X. G., & Zhang, Y. (2025) · IEEE Transactions on Reliability · doi:10.1109/tr.2025.3547739 · Cited by 1
Compiler Fuzzing in Continuous Integration: A Case Study on Dafny
Boonriong, K., Zetzsche, S., & Donaldson, A. F. (2025) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst62969.2025.10988954 · Cited by 2
“This is a widely-used oracle in compiler testing, having been used to test compilers for languages such as C [48], OpenCL [34], Java [13], Verilog [26] and Rust [41].”
De-duplicating Silent Compiler Bugs via Deep Semantic Representation
Chen, J., Fan, X., Yang, C., Liu, S., & Sun, J. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3729375 · Cited by 3
Static Program Reduction via Type-Directed Slicing
Nguyen, L. N. D., Islam, T., Wang, T., Lenz, S., & Kellogg, M. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3728968 · Cited by 4
Evaluating Disassembly Errors With Only Binaries
Wijayadi, L. A., Jiang, Y., Yap, R. H. C., Liang, Z., & Liu, Z. (2025) · ACM Asia Conference on Computer and Communications Security (ASIA CCS) · doi:10.1145/3708821.3733884 · Cited by 1
“History shows that compilers do have bugs, e.g., fuzzers such as Csmith [25] have successfully found compiler bugs.”
Is This the Same Code? A Comprehensive Study of Decompilation Techniques for WebAssembly Binaries
Wu, W.-C., Yan, Y., Egilsson, H. D., Park, D., Chan, S., Hauser, C., & Wang, W. (2025) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-031-94448-2_6 · Cited by 1
Boosting Program Reduction with the Missing Piece of Syntax-Guided Transformations
Xu, Z., Tian, Y., Zhang, M., & Sun, C. P. (2025) · Proceedings of the ACM on Programming Languages · doi:10.1145/3763053 · Cited by 2
Synchronized Behavior Checking: A Method for Finding Missed Compiler Optimizations
Zhang, Y., Wang, Y., Wang, L., & Wang, K. (2025) · Proceedings of the ACM on Programming Languages · doi:10.1145/3763105 · Cited by 1
Validating Soundness and Completeness in Pattern-Match Coverage Analyzers
Moser, C. F., Sotiropoulos, T., Zhang, C., & Su, Z. (2025) · Proceedings of the ACM on Programming Languages · doi:10.1145/3763171 · Cited by 1
“Recent advancements in random program generation offer promising solutions for testing programming language implementations and program analysis tools [Chaliasos et al. 2022; Even-Mendoza et al. 2023; Frank et al. 2024; Pałka et al. 2011; Yang et al. 2011].”
Exploiting Booster Pass Chain for Compiler Phase Ordering
Chen, Y., Chen, H., Yao, Y., Yu, P., Xu, F., & Ma, X. (2025) · Asia-Pacific Symposium on Internetware · doi:10.1145/3755881.3755899 · Cited by 1
Self-citationLLM4FP: LLM-Based Program Generation for Triggering Floating-Point Inconsistencies Across Compilers
Wang, Y., & Rubio-González, C. (2025) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1145/3731599.3767362 · Cited by 4
Model-Based Testing of an Intermediate Verifier Using Executable Operational Semantics
Losavio, L., Paganoni, M., & Furia, C. A. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-10794-7_20 · Cited by 1
Midtgaard, J. (2025) · Workshop Dedicated to Olivier Danvy on the Occasion of His 64th Birthday · doi:10.1145/3759427.3760378 · Cited by 2
Isolating Compiler Faults via Multiple Pairs of Adversarial Compilation Configurations
Li, Q., Yang, Y., Sun, M., Wu, J., Shi, Q., & Zhou, Y. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3786786 · Cited by 2
“Compiler testing identifies the presence of bugs [49], while fault localization (the focus of our work) aims to pinpoint the specific source files or components that are responsible for these bugs [10].”
Brack: A Verified Compiler for Scheme via CakeML
Lasnier, P. Y., Yallop, J., & Myreen, M. O. (2026) · ACM SIGPLAN International Conference on Certified Programs and Proofs (CPP) · doi:10.1145/3779031.3779098 · Cited by 1
Unmasking the Type Inference Capabilities of LLMs for Java Code Snippets
Dong, Y., Xu, Z., Tian, Y., & Sun, C. P. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3790099 · Cited by 4
“Future work may also explore leveraging more expressive program generators [16, 43, 44, 76] to create code snippets that exercise more advanced type-system features.”
Finding Bugs in WebAssembly Interface Type Binding Generators
Stanley, E. C., & Eide, E. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00025 · Cited by 1
“Random semantically valid C programs are generated using Csmith [53] and passed as input to rust-bindgen.”
Self-citationFinding Bugs in MLIR Compiler Infrastructure via Lowering Space Exploration
Liang, J., Huang, S., & Su, T. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00059 · Cited by 1
“For example, Csmith [33] randomly generates C programs and detects miscompilations by comparing their execution results across GCC and LLVM. HirGen [32] generates diverse computational graphs to test whether the compiled results produced by the TVM framework yield consistent execution outputs…”
WEST: Specification-Based Test Generation for WebAssembly
Youn, D., Shin, W., & Ryu, S. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00119 · Cited by 1
“Tools such as Csmith [49], LangFuzz [50], and Grammar-based Whitebox Fuzzing [51] have explored the use of grammars and semantic constraints together to generate test inputs for complex languages like C and JavaScript.”
Execution-Aware Program Reduction for WebAssembly via Record and Replay
Baek, D., Lehmann, D., Titzer, B. L., Ryu, S., & Pradel, M. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00073 · Cited by 1
“…Fast-Reduce in the following ways: (1) Fast-Reduce relies on dynamic analysis specifically built to integrate with the test case generator CSmith [40], whereas RR-Reduce employs a general-purpose record and replay tool; (2) Fast-Reduce selects an arbitrary final effect of the function to inline,…”
Yang, Y., Zhou, C., Wang, R., Zhang, W., Shen, Y., Shi, X., Ma, T., Gao, C., Wang, Z., Fu, Y., et al. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00293 · Cited by 1
“…testing has been widely employed to identify semantic bugs in various systems, including SSL/TLS implementations [7]– [10], C compilers [11], JVM implementations [12], web application firewalls [13], security policies for APIs [14], antivirus software [15], and file systems [16], among…”
TeTRIS: General-purpose Fuzzing for Translation Bugs in Source-to-Source Code Transpilers
Arafat, Y., & Nagy, S. (2025) · Annual Computer Security Applications Conference · doi:10.1109/acsac67867.2025.00025 · Cited by 1
“We compare T E TRIS to four state-of-the-art fuzzers: general-purpose AFL++ [16], Polyglot [9], and AFL-Compiler-Fuzzer [20]; as well as compiler fuzzer CSmith [58].”
InfluentialUnderstanding Query Optimization Bugs in Graph Database Systems
Chen, Y., & Yu, Z. (2026) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3779212.3790244 · Cited by 1
“Given that optimization is the buggiest component in compiler implementations [40, 60], it is reasonable to conjecture that query optimization implementation also suffers from the correctness issue.”
Self-citationDeveloping, testing, building and maintaining an Eclipse project based on EMF and Xtext
Bettini, L. (2026) · Automated Software Engineering · doi:10.1007/s10515-026-00613-0 · Cited by 2
Selective Feature-Sensitive Coverage for Conformance Testing of Programming Languages
Lee, K., Kim, S., Park, J., & Ryu, S. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3808231 · Cited by 1
“For example, Csmith and its successors generate random C programs to expose miscompilation bugs in compilers [30, 57].”
Bounded Exhaustive Random Program Generation for Testing Solidity Compilers
Ma, H., Donaldson, A. F., Shen, Q., Tian, Y., Chen, J., & Cheung, S.-C. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3744916.3773237 · Cited by 1
CombCT: Compiler Testing via Combinatorial Testing
Luo, C. W., Cui, S., Yan, J., Chen, J., Suo, C., Wu, W., Liu, C., & Hu, C. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3744916.3764548 · Cited by 1
“Rather than writing test programs manually, existing approaches fall into two categories: generating complete programs from scratch [3, 4, 15, 31, 58], and generating test programs through program mutation [9, 11, 29, 52, 63].”
InfluentialSystematically Cover SQL Syntactic Structures via 𝑘-Sequence
Zhou, H., Zheng, Y., Gao, Y., Song, J., Xie, X., Yang, R., Cui, Z., Dou, W., & Wei, J. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3832289 · Cited by 1
Grammar-based program generation based on model finding
Soeken, M., & Drechsler, R. (2013) · International Design and Test Workshop (IDT) · doi:10.1109/idt.2013.6727084 · Cited by 1
“Many approaches for grammar-based program generation have been proposed in the past, e.g. [5]–[10].”
BackgroundLocalizing Faults Using Verification Technique
Dutta, S. (2022) · Innovations in Software Engineering Conference (ISEC) · doi:10.1145/3511430.3511445 · Cited by 1
“Also, the wrong safety check bug of the LLVM compiler as reported in [30] is seeded in the transformed program and the resulting program is shown in Fig.”
MethodLi, Y., Nie, Y., & Kuang, X. (2022) · IEEE International Conference on Data Science in Cyberspace · doi:10.1109/dsc55868.2022.00057 · Cited by 2
“Generation-based methods [1, 2, 3] usually generate test cases from scratch based on a given grammar template.”
BackgroundHyperPUT: generating synthetic faulty programs to challenge bug-finding tools
Felici, R., Pozzi, L., & Furia, C. A. (2024) · Empirical Software Engineering · doi:10.1007/s10664-023-10430-8 · Cited by 1
“The idea of constructing programs to be used as test inputs (PUTs: programs under test) has been successfully used for other purposes, such as to detect semantic compiler bugs that result in incorrect compilation (Yang et al. 2011).”
MethodFormal Verification of Just-in-Time Compilation
Barrière, A. (2025) · ACM eBooks · doi:10.1145/3705593 · Cited by 1
Spectrum-based rule- and item-level localization of faults in context-free grammars
Raselimo, M., & Fischer, B. (2024) · Journal of Systems and Software · doi:10.1016/j.jss.2024.112067 · Cited by 3
Grammar-based test suite construction using coverage-directed algorithms over LR-graphs
Rossouw, C., & Fischer, B. (2024) · Journal of Systems and Software · doi:10.1016/j.jss.2024.112068 · Cited by 1
Lincroft, G., Cho, M., Hough, K., Bazzaz, M., & Bell, J. (2024) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3643991.3644908 · Cited by 2
Towards Cross-Build Differential Testing
Dietrich, J. B., White, T., Terragni, V., & Hassanshahi, B. (2025) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst62969.2025.10989044 · Cited by 1
LegoFuzz: Interleaving Large Language Models for Compiler Testing
Ni, Y. (2025) · ACM SIGPLAN International Conference on Systems, Programming, Languages, and Applications: Software for Humanity (SPLASH) · doi:10.1145/3758316.3763251 · Cited by 1
Velev, M. N., Zhang, C., Gao, P., & Groce, A. D. (2015) · International Symposium on Quality Electronic Design (ISQED) · doi:10.1109/isqed.2015.7085461 · Cited by 1
“Swarm testing [9] is a testing technique that improves the effectiveness of random testing [2][3][11][17][21] by changing the probability distribution of test generation randomly with each generated test.”
MethodProgramming a Smalltalk VM in Coq
Shingarov, B. (2017) · International Workshop on Smalltalk Technology · doi:10.1145/3139903.3139905 · Cited by 1
“For example, to aim understanding of C compiler bugs, Yang et al. built a test bench generating large sets of test vectors; their tool discovered 325 previously unknown cases of miscom-pilation in GCC and LLVM [29].”
MethodLearning to Restrict Test Range for Compiler Test
Zhu, J., Wang, L., Gu, Y., & Lin, X. (2019) · IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) · doi:10.1109/icstw.2019.00064 · Cited by 1
“It detected 325 bugs in 3 years [1].”
BackgroundSynthesis of Benchmarks for the C Programming Language by Mining Software Repositories
Guimarães, B. C. F., Magalhães, J. W. D. S., da Silva, A. F., & Pereira, F. M. Q. (2019) · doi:10.1145/3355378.3355380 · Cited by 1
“Em 2011, pes-quisadores da Universidade de Utah lançaram CSmith [23], até a presente data, o gerador de programas aleatórios mais popular entre usuários da linguagem C. Nenhum dos trabalhos acima citados tinha como foco trei-nar técnicas de auto-adaptação.”
BackgroundSource-free Machine-checked Validation of Native Code in Coq
Hamlen, K. W., Fisher, D., & Lundquist, G. R. (2019) · doi:10.1145/3338502.3359759 · Cited by 1
A change-centric approach to compile configurable systems with #ifdefs
Braz, L., Gheyi, R., Mongiovi, M., Ribeiro, M., Medeiros, F., & Teixeira, L. (2016) · ACM SIGPLAN Notices · doi:10.1145/3093335.2993250 · Cited by 1
Practical Machine-Checked Formalization of Change Impact Analysis
Palmskog, K., Çelik, A., & Gligoric, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-45237-7_9 · Cited by 2
“When target languages lack fully formal semantics, as is the case for OCaml, extraction cannot be fully trusted, but empirical evaluations are nevertheless encouraging [24,58].”
BackgroundTemporal Property-Based Testing of a Timed C Compiler using Time-Flow Graph Semantics
Natarajan, S., & Broman, D. (2020) · Forum on specification and Design Languages · doi:10.1109/fdl50818.2020.9232935 · Cited by 1
“Compiler testing tools are widely used when testing compilers and such tools have found many bugs in frequently used compilers [12].”
MethodInfluentialIdentification of Misleading Location Information in Compiler Diagnoses
Wang, M., Ji, W., Jing, D., & Liu, H. (2020) · Asia-Pacific Software Engineering Conference (APSEC) · doi:10.1109/apsec51365.2020.00056 · Cited by 2
“CSmith proposed by Yang et al. [13] is able to generate C programs with more complex control flow and more complex data structures.”
BackgroundExperience report: a do-it-yourself high-assurance compiler
Pike, L., Wegmann, N., Niller, S., & Goodloe, A. E. (2012) · ACM SIGPLAN Notices · doi:10.1145/2398856.2364553 · Cited by 3
Removing Uninteresting Bytes in Software Fuzzing
Hussain, A., & Alipour, M. A. (2022) · IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) · doi:10.1109/icstw55395.2022.00058 · Cited by 1
“Wasteful fuzzer performance is also contributed by the generation of meaningless test seeds, which has been addressed by grammar-based fuzzers like Peach [18], LangFuzz [19], Sulley [20], CSmith [21], and Jsfunfuzz [22].”
BackgroundMisse-Chanabier, P., Polito, G., Bouraqadi, N., Ducasse, S., Fabresse, L., & Tesone, P. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-08129-3_7 · Cited by 2
On the naturalness of fuzzer-generated code
Kambhamettu, R. H., Billos, J., Oluwaseun-Apo, T., Gafford, B., Padhye, R., & Hellendoorn, V. J. (2022) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3524842.3527972 · Cited by 3
“We conduct this study on C programs, using Csmith [19] and C-Reduce [16] as a model generation-based compiler fuzzer and test-case reducer respectively.”
BackgroundMethodBackend Bug Finder — a platform for effective compiler fuzzing
Stepanov, D., & Itsykson, V. (2022) · Информационно-управляющие системы · doi:10.31799/1684-8853-2022-6-31-40 · Cited by 1
Compiling with dependent types
Bowman, W. J. (2018) · Digital Repository Service (Northeastern University) · doi:10.17760/d20316239 · Cited by 2
Proceedings of the 2nd International Fuzzing Workshop
(2023) · doi:10.1145/3605157 · Cited by 1
A logical study of program equivalence
Jaber, G. (2014) · theses.fr (ABES) · doi:10.70675/4714cfc0z7540z46edz8647z654a35f388b1 · Cited by 1
Modélisation et Vérification Formelles de Systèmes de Contrôle de Trains
Xie, Y. (2019) · theses.fr (ABES) · doi:10.70675/4731b3aezfbdaz4b44z8a23z8453dbb0b410 · Cited by 1
“In (Yang et al. 2011), a group used several years to find the bugs in the existing and widely used C compilers (e.”
MethodADDFuzzer: A New Fuzzing Framework of Android Device Drivers
Cui, B., Ni, Y., & Fu, Y. (2015) · International Conference on Broadband and Wireless Computing, Communication and Applications (BWCCA) · doi:10.1109/bwcca.2015.57
Formal Replay of Translation Validation for Highly Optimised C
Sewell, T. A. L. (2018) · EPiC series in computing · doi:10.29007/dw2m
“In their study on compiler testing, Yang et al reported 325 previously unknown defects in 11 different C compilers [8].”
BackgroundAUSPICE-R: Automatic Safety-Property Proofs for Realistic Features in Machine Code
Tan, J., Tay, H. J., Gandhi, R., & Narasimhan, P. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-47958-3_3
“Reasoning about machine code provides a foundational approach for verification, as machine code proofs are not affected by miscompilation bugs that may cause safety problems [26], as compared to proofs about source-code.”
BackgroundCompiler Testing Approach based on Generalized Equivalent Substitution
Wu, X., Zhang, P., Huang, Q., Ma, Q., Shao, G., & Li, P. (2017) · International Conference on Machinery, Electronics and Control Simulation (MECS) · doi:10.2991/mecs-17.2017.91
“At present, the three widely used compiler testing techniques include Randomized Differential Testing (RDT) ([8]), a variant of RDT—Different Optimization Levels (DOL) ([9]), and Equivalence Modulo Inputs (EMI) ([6]).”
MethodAutomatic testing of interactive JavaScript debuggers
Lehmann, D. (2017) · ACM SIGPLAN International Conference on Systems, Programming, Languages, and Applications: Software for Humanity (SPLASH) · doi:10.1145/3135932.3135945
“Automatic testing is yet another way to find bugs in compilers [6, 9], interpreters [4], and refactoring engines [3] and has seen widespread use and great success in recent years.”
BackgroundMocov: Model Based Fuzzing Through Coverage Guided Technology
Chen, C., Chen, Z., Hao, Y., & Cui, B. (2017) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-3-319-69811-3_37
Strong Security Guarantees: From Alloy to Coq (Research Poster)
Souaf, S., & Loulergue, F. (2018) · International Conference on High Performance Computing & Simulation (HPCS) · doi:10.1109/hpcs.2018.00167
“The authors of a study on compiler testing [7] were indeed unable to find any bug in formally proved component of the CompCert compiler, while it found several “release blocking” bugs in mainstream compilers such as GCC or clang.”
BackgroundVerified Programs for Frequent Itemset Mining
Loulergue, F., & Whitney, C. D. (2018) · IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI) · doi:10.1109/smartworld.2018.00262
“For example, a study about compiler testing [8] did not find any bug in the formally verified (using Coq) parts of the CompCert compiler [9], while several “release blocking” bugs were found in released versions of GCC and Clang.”
BackgroundHobbs, C. (2015) · Auerbach Publications eBooks · doi:10.1201/b18965-26
An abstract memory functor for verified C static analyzers
Blazy, S., Laporte, V., & Pichardie, D. (2016) · ACM SIGPLAN Notices · doi:10.1145/3022670.2951937
Securing software systems via fuzz testing and verification
Chen, H. (2019) · DR-NTU (Nanyang Technological University) · doi:10.32657/10356/136764
“6) Skyfire [21], Radamsa [137], Csmith [139] can be used to generate external seeds which will be imported, or integrated as structure-aware mutators conducted by mutation manager.”
BackgroundStatic analyzer debugging and quality assurance approaches
MENSHIKOV, M. A. (2020) · Proceedings of the Institute for System Programming of RAS · doi:10.15514/ispras-2020-32(3)-3
Semantic-based vulnerability analysis of software
Li, Y. (2019) · DR-NTU (Nanyang Technological University) · doi:10.32657/10356/142914
Feedback Driven Grammar-Based Fuzzing
Asryan, S. (2018) · Mathematical Problems of Computer Science · doi:10.51408/1963-0022
“Currently there are several methods and instruments [1-6, 8-11] that use whitebox fuzzing or grammar-based data generation approaches to address the challenge.”
BackgroundMethodTSDolly: A program generator for TypeScript
Britto, G. A., Teixeira, L., & Gheyi, R. (2021) · Brazilian Symposium on Programming Languages (SBLP) · doi:10.1145/3475061.3475079
“Some of the proposed programgeneration approaches are basedon languagegrammars [3, 20], and do not aim to generate programs that successfully compile or that satisfy other arbitrary complex constraints.”
BackgroundMethodStatic Worst-Case Analyses and Their Validation Techniques for Safety-Critical Systems
Wägemann, P. (2022) · Springer eBooks · doi:10.1007/978-3-030-83128-8_11 · Cited by 1
Fast PokeEMU: Scaling Generated Instruction Tests Using Aggregation and State Chaining
Yan, Q., & McCamant, S. (2018) · ACM SIGPLAN Notices · doi:10.1145/3296975.3186417
Hobbs, C. (2019) · CRC Press eBooks · doi:10.1201/9780429323010-20
Hobbs, C. (2015) · Auerbach Publications eBooks · doi:10.1201/b18965-27
Testing a PL/I Compiler Using Precomputation-based Program Generation
Postema, J., Fabry, J., Barthol, Y., & Oprescu, A. (2022) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst53961.2022.00042
Geração Automática de Benchmarks para Compilação Preditiva
Kind, C., Coelho, J., Kind, B. C., & Pereira, F. M. Q. (2022) · doi:10.1145/3561320.3561323 · Cited by 1
“Vários trabalhos buscam expandir a metodologia utilizada por CSmith [4, 22, 23].”
BackgroundInfluentialPrincipled Composition of Function Variants for Dynamic Software Diversity and Program Protection
Priamo, G., D’Elia, D. C., & Querzoni, L. (2022) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3551349.3559553
“We use 500 Csmith [28] programs as seeds for the pipeline and repeatedly tested them in different assortments.”
BackgroundMethodOptimizing WinAFL for Image Parsing Engine Vulnerability Discovery in PDF Readers
Zhang, Y., & Du, Z. (2023) · Frontiers in Computing and Intelligent Systems · doi:10.54097/fcis.v3i1.6028
Detecting Interpreter Bugs via Filling Function Calls in Skeletal Program Enumeration
Xia, X., & Feng, Y. (2023) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre59848.2023.00066
Verifying Compiler Optimisations
Hayes, I. J., Utting, M., & Webb, B. J. (2023) · Lecture notes in computer science · doi:10.1007/978-981-99-7584-6_1
Localizing Faults Using Verification Technique
Dutta, S. (2022) · Journal of Systems and Software · doi:10.1016/j.jss.2023.111897
Quantitative Verification and Synthesis of Resilient Networks
Schou, M. K. (2023) · VBN Forskningsportal (Aalborg Universitet) · doi:10.54337/aau588616986
Scope-based Compiler Differential Testing
Qu, R., Huang, J., Zhang, L., Qiao, T., & Zhang, J. (2023) · IEEE International Conference on Software Quality, Reliability and Security (QRS) · doi:10.1109/qrs60937.2023.00043
“For example, the test program randomly generated by Csmith [5] based on a given C grammar can easily pass the compiler’s parsing stage.”
MethodInfluentialTesting concolic execution through consistency checks
Coppa, E., & Izzillo, A. (2024) · Journal of Systems and Software · doi:10.1016/j.jss.2024.112001
Scalable Machine Learning Methods and Datasets for Software Vulnerability Detection
McLaughlin, C. (2024) · Queensland University of Technology · doi:10.5204/thesis.eprints.251655 · Cited by 1
Inconsistencies in TeX-Produced Documents
Tan, J., & Rigger, M. (2024) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3650212.3680370
“It has been used to find bugs in various domains, such as document processors [52], software libraries [28, 37], program analysers [48, 69], database systems [55, 72, 82], runtime engines [27, 40, 64], and most notably compilers [33, 54, 74, 80].”
Baasantogtokh, D., Yoon, Y., Batzorig, M., Sahlabadi, M., & Yim, K. (2024) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-3-031-72322-3_4
MG+: Towards Efficient Context Inconsistency Detection by Minimized Link Generation
Chen, C., Wang, H., Zhang, L., Xu, C., & Yu, P. (2024) · Software Testing Verification and Reliability · doi:10.1002/stvr.1899
“The oracle problem focuses on how we could identify bugs when we could not derive expected outputs for complex software systems, for example, compilers [58–61], language virtual machines [62, 63], neural networks [64, 65] and SMT solvers [66].”
Sanusi, B. A., Ogunshile, E. K. A., Aydın, M. E., & Olabiyisi, S. O. (2024) · International Conference on Software Engineering and Development (ICSED) · doi:10.1145/3686614.3686622
“These compiler defects can give rise to incorrect binary code that is generated from the correct source code [27].”
An efficient propositional system for Abductive Logic Programming
Gavanelli, M., Julián-Iranzo, P., & Sáenz-Pérez, F. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10928-7
“This technique has been successfully applied in various domains, including compilers (McKeeman 1998; Yang et al. 2011; Le et al. 2014), virtual machines (Chen et al. 2016), network protocols (Chen and Su 2015), and more (Petsios et al. 2017).”
Efficient Code Causes Inefficiency in Compiler Optimizations
Chen, H. (2024) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3691620.3695369
Modeling Register Pairs in CompCert
Loitzl, A., & Zuleger, F. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-76554-4_8
Metadata
- Authors
- Yang, X., Chen, Y., Eide, E., & Regehr, J.
- Affiliations
- University of Utah (US)
- Venue
- Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation
- Publisher
- ACM
- Published
- 2011-06-04
- Pages
- 283-294
- ISSN
- 1531-7102 (ISSN-L)
- OpenAlex
- W4238083723
- Cited by
- 825
- References
- 23
- Citations by year
- 2026: 71 · 2025: 115 · 2024: 93 · 2023: 88 · 2022: 59 · 2021: 50 · 2020: 58 · 2019: 45 · 2018: 50 · 2017: 40 · 2016: 42 · 2015: 45 · 2014: 24 · 2013: 23 · 2012: 19
- FWCI
- 15.95
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Software › Logic, programming, and type systems
- Keywords
- compiler bugs, C compiler, compiler testing, undefined behavior, bug reports
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.