An Empirical Study of Bugs in WebAssembly Compilers
Abstract
WebAssembly is the newest programming language for the Web. It defines a portable bytecode format for use as a compilation target for programs developed in high-level languages such as C, C++, and Rust. As a result, WebAssembly binaries are generally created by WebAssembly compilers rather than being written manually. To port native code to the Web, WebAssembly compilers need to address the differences between the source and target languages and dissimilarities in their execution environments. A deep understanding of the bugs in WebAssembly compilers can help compiler developers determine where to focus development and testing efforts. In this paper, we conduct two empirical studies to understand the characteristics of the bugs found in WebAssembly compilers. First, we perform a qualitative analysis of bugs in Emscripten, the most widely-used WebAssembly compiler. We investigate 146 bug reports in Emscripten related to the unique challenges WebAssembly compilers encounter compared with traditional compilers. Second, we provide a quantitative analysis of 1,054 bugs in three open-source WebAssembly compilers, AssemblyScript, Emscripten, and Rustc/Wasm-Bindgen. We analyze these bugs along three dimensions: lifecycle, impact, and sizes of bug-inducing inputs and bug fixes. These studies deepen our understanding of WebAssembly compiler bugs. We hope that the findings of our study will shed light on opportunities to design practical tools for testing and debugging WebAssembly compilers.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Traditional benchmarking faces challenges like benchmark manipulation, where systems are engineered to perform exceptionally well on specific benchmarks without delivering proportional real-world performance.
“However, traditional benchmarking faces challenges like benchmark manipulation, where systems are engineered to perform exceptionally well on specific benchmarks without delivering proportional real-world performance [20].”
From Inadequacies of Large Language Model Benchmarks in the Era of Generative Artificial Intelligence (2026) · cites this paper as [20]
Cites
Simplifying and isolating failure-inducing input
Zeller, A., & Hildebrandt, R. (2002) · IEEE Transactions on Software Engineering · doi:10.1109/32.988498 · Cited by 1,084 · see citing works
Lu, S., Park, S., Seo, E., et al. (2008) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/1346281.1346323 · Cited by 898 · see citing works
Bringing the web up to speed with WebAssembly
(2017) · doi:10.1145/3062341.3062363 · Cited by 539 · see citing works
An empirical study on TensorFlow program bugs
Zhang, Y., Chen, Y., Cheung, S.-C., Xiong, Y., & Zhang, L. (2018) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3213846.3213866 · Cited by 311 · see citing works
Understanding and detecting real-world performance bugs
Jin, G., Song, L., Shi, X., Scherpelz, J., & Lu, S. (2012) · ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI) · doi:10.1145/2254064.2254075 · Cited by 305 · see citing works
Li, Z., Tan, L., Wang, X., et al. (2006) · doi:10.1145/1181309.1181314 · Cited by 293 · see citing works
The secret life of bugs: Going past the errors and omissions in software repositories
Aranda, J., & Venolia, G. D. (2009) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2009.5070530 · Cited by 259 · see citing works
An empirical study on configuration errors in commercial and open source systems
Yin, Z., Ma, X., Zheng, J., et al. (2011) · doi:10.1145/2043556.2043572 · Cited by 254 · see citing works
Yin, Z., Yuan, D., Zhou, Y., Pasupathy, S., & Bairavasundaram, L. N. (2011) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/2025113.2025121 · Cited by 249 · see citing works
Bug characteristics in open source software
Tan, L., Liu, C., Zhen-min, L., Wang, X., Zhou, Y., & Zhai, C. (2013) · Empirical Software Engineering · doi:10.1007/s10664-013-9258-8 · Cited by 241 · see citing works
Characterization of linux kernel behavior under errors
Gu, W., Kalbarczyk, Z., Ravishankar, R., et al. (2004) · IEEE/IFIP International Conference on Dependable Systems and Networks (DSN) · doi:10.1109/dsn.2003.1209956 · Cited by 198 · see citing works
An Empirical Study of Bugs in Machine Learning Systems
Thung, F., Wang, S., Lo, D., & Jiang, L. (2012) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre.2012.22 · Cited by 149 · see citing works
Toward understanding compiler bugs in GCC and LLVM
Sun, C. P., Le, V., Zhang, Q., & Su, Z. (2016) · International Symposium on Software Testing and Analysis · doi:10.1145/2931037.2931074 · Cited by 127 · see citing works
Mining input grammars from dynamic taints
Höschele, M., & Zeller, A. (2016) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/2970276.2970321 · Cited by 126 · see citing works
An empirical study of bugs in test code
Vahabzadeh, A., Fard, A. M., & Mesbah, A. (2015) · IEEE International Conference on Software Maintenance and Evolution (ICSME) · doi:10.1109/icsm.2015.7332456 · Cited by 113 · see citing works
Outguard: Detecting In-Browser Covert Cryptocurrency Mining in the Wild
Kharraz, A., Ma, Z., Murley, P., et al. (2019) · doi:10.1145/3308558.3313665 · Cited by 95 · see citing works
An Empirical Study of Client-Side JavaScript Bugs
Ocariza, F. S., Bajaj, K., Pattabiraman, K., et al. (2013) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem.2013.18 · Cited by 94 · see citing works
An Empirical Study of Real-World WebAssembly Binaries
Hilbig, A., Lehmann, D., & Pradel, M. (2021) · doi:10.1145/3442381.3450138 · Cited by 93 · see citing works
Lehmann, D., & Pradel, M. (2019) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3297858.3304068 · Cited by 83 · see citing works
An empirical study of operating systems errors
Chou, A., Yang, J., Chelf, B., et al. (2001) · ACM SIGOPS Operating Systems Review · doi:10.1145/502059.502042 · Cited by 77 · see citing works
An empirical study of reported bugs in server software with implications for automated bug diagnosis
Sahoo, S. K., Criswell, J., & Adve, V. (2010) · doi:10.1145/1806799.1806870 · Cited by 74 · see citing works
An empirical study of optimization bugs in GCC and LLVM
Zhou, Z., Ren, Z., Gao, G., & Jiang, H. (2020) · Journal of Systems and Software · doi:10.1016/j.jss.2020.110884 · Cited by 62 · see citing works
New Kid on the Web: A Study on the Prevalence of WebAssembly in the Wild
Musch, M., Wressnegger, C., Johns, M., et al. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-22038-9_2 · Cited by 48 · see citing works
Correlations between bugginess and time-based commit characteristics
Eyolfson, J., Tan, L., & Lam, P. (2013) · Empirical Software Engineering · doi:10.1007/s10664-013-9245-0 · Cited by 46 · see citing works
Automatically reducing tree-structured test inputs
Herfert, S., Patra, J., & Pradel, M. (2017) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2017.8115697 · Cited by 44 · see citing works
Synthesizing qualitative research in software engineering
Huang, X., Zhang, H., Zhou, X., Babar, M. A., & Yang, S. (2018) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3180155.3180235 · Cited by 43 · see citing works
Everything Old is New Again: Binary Security of WebAssembly
Lehmann, D., Kinder, J., & Pradel, M. (2020) · USENIX Security Symposium · Cited by 43 · see citing works
Disselkoen, C., Renner, J., Watt, C., et al. (2019) · doi:10.1145/3337167.3337171 · Cited by 33 · see citing works
Romano, A., Zheng, Y., & Wang, W. (2020) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3324884.3416580 · Cited by 30 · see citing works
Self-citationTaintAssembly: Taint-Based Information Flow Control Tracking for WebAssemblyPreprint
Fu, W., Lin, R., & Inge, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.01050 · Cited by 20 · see citing works
Swivel: Hardening WebAssembly against SpectrePreprint
Narayan, S., Disselkoen, C., Moghimi, D., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.12730 · Cited by 16 · see citing works
Taint Tracking for WebAssemblyPreprint
Szanto, A., Tamm, T., & Pagnoni, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.08349 · Cited by 12 · see citing works
Romano, A., & Wang, W. (2020) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3377812.3382155 · Cited by 9 · see citing works
Self-citationRomano, A., & Wang, W. (2020) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3324884.3415293 · Cited by 8 · see citing works
Self-citationWATT : A novel web-based toolkit to generate WebAssembly-based libraries and applications
Jeong, H., Jeong, J., Park, S., et al. (2018) · International Conference on Consumer Electronics · doi:10.1109/icce.2018.8326230 · Cited by 7 · see citing works
Automatic performance testing using input-sensitive profiling
Luo, Q. (2016) · ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE) · doi:10.1145/2950290.2983975 · Cited by 3 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Inadequacies of Large Language Model Benchmarks in the Era of Generative Artificial Intelligence
McIntosh, T. R., Susnjak, T., Arachchilage, N., Liu, T., Xu, D., Watters, P., & Halgamuge, M. N. (2026) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2025.3569516 · Cited by 104 · see citing works
“For example, hardware manufacturers might optimize compilers or system settings to inflate benchmark scores, leading to misleading conclusions [21].”
Background
- Authors
- Romano, A., Liu, X., Kwon, Y., & Wang, W.
- Affiliations
- University at Buffalo, State University of New York (US)
- University of Virginia (US)
- Venue
- 2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE)
- Publisher
- IEEE
- Published
- 2021-11
- Pages
- 42-54
- OpenAlex
- W4206578767
- Cited by
- 49
- References
- 36
- Citations by year
- 2026: 7 · 2025: 17 · 2024: 11 · 2023: 11 · 2022: 3
- FWCI
- 4.70
- Type
- conference paper
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE/ACM International Conference on Automated Software Engineering (ASE) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Artificial Intelligence › Logic, programming, and type systems
- Keywords
- WebAssembly, compiler bugs, Emscripten, bug reports, compiler testing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
