Stochastic superoptimization
The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
We formulate the loop-free binary superoptimization task as a stochastic search problem. The competing constraints of transformation correctness and performance improvement are encoded as terms in a cost function, and a Markov Chain Monte Carlo sampler is used to rapidly explore the space of all possible programs to find one that is an optimization of a given target program. Although our method sacrifices completeness, the scope of programs we are able to consider, and the resulting quality of the programs that we produce, far exceed those of existing superoptimizers. Beginning from binaries compiled by llvm -O0 for 64-bit x86, our prototype implementation, STOKE, is able to produce programs which either match or outperform the code produced by gcc -O3, icc -O3, and in some cases, expert handwritten assembly.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Equation of State Calculations by Fast Computing Machines
Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H., & Teller, E. (1953) · The Journal of Chemical Physics · doi:10.1063/1.1699114 · Cited by 37,192
Monte Carlo sampling methods using Markov chains and their applications.
Hastings, W. K. (1970) · Biometrika · doi:10.1093/biomet/57.1.97 · Cited by 15,195
An introduction to MCMC for machine learning.
Andrieu, C., de Freitas, N., Doucet, A., & Jordan, M. I. (2003) · Machine Learning · doi:10.1023/a:1020281327116 · Cited by 2,451
Pin: building customized program analysis tools with dynamic instrumentation
Luk, C.-K., Cohn, R., Muth, R., Patil, H., Klauser, A., Lowney, G., Wallace, S., Reddi, V. J., & Hazelwood, K. (2005) · Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation · doi:10.1145/1065010.1065034 · Cited by 2,310
An updated set of basic linear algebra subprograms (BLAS)
(2002) · ACM Transactions on Mathematical Software · doi:10.1145/567806.567807 · Cited by 834
Combinatorial sketching for finite programs
Solar-Lezama, A., Tancau, L., Bodik, R., Seshia, S., & Saraswat, V. (2006) · Proceedings of the 12th international conference on Architectural support for programming languages and operating systems · doi:10.1145/1168857.1168907 · Cited by 635
Veach, E., & Guibas, L. J. (1997) · Proceedings of the 24th annual conference on Computer graphics and interactive techniques - SIGGRAPH '97 · doi:10.1145/258734.258775 · Cited by 511
The Markov chain Monte Carlo revolution
Diaconis, P. (2008) · Bulletin of the American Mathematical Society · doi:10.1090/s0273-0979-08-01238-x · Cited by 295
Extracting protein alignment models from the sequence database
Neuwald, A. F., Liu, J. S., Lipman, D. J., & Lawrence, C. E. (1997) · Nucleic Acids Research · doi:10.1093/nar/25.9.1665 · Cited by 234
Synthesis of loop-free programs
Gulwani, S., Jha, S., Tiwari, A., & Venkatesan, R. (2011) · Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation · doi:10.1145/1993498.1993506 · Cited by 200
Equality saturation: a new approach to optimization
Tate, R., Stepp, M., Tatlock, Z., & Lerner, S. (2009) · Proceedings of the 36th annual ACM SIGPLAN-SIGACT symposium on Principles of programming languages · doi:10.1145/1480881.1480915 · Cited by 193
Automatic generation of peephole superoptimizers
Bansal, S., & Aiken, A. (2006) · Proceedings of the 12th international conference on Architectural support for programming languages and operating systems · doi:10.1145/1168857.1168906 · Cited by 161
Self-citationSuperoptimizer: a look at the smallest program
Massalin, H. (1987) · Proceedings of the second international conference on Architectual support for programming languages and operating systems · doi:10.1145/36206.36194 · Cited by 154
Denali: a goal-directed superoptimizer
Joshi, R., Nelson, G., & Randall, K. (2002) · Proceedings of the ACM SIGPLAN 2002 conference on Programming language design and implementation · doi:10.1145/512529.512566 · Cited by 151
Sampling plausible solutions to multi-body constraint problems
Chenney, S., & Forsyth, D. A. (2000) · Proceedings of the 27th annual conference on Computer graphics and interactive techniques - SIGGRAPH '00 · doi:10.1145/344779.344882 · Cited by 107
Research works printed but not linked: 5
- Cadar C., 2008, OSDI, 209
- Ganesh V., 2007, CAV, 519
- Gilks W. R., 1999, Chapman and Hall/CRC
- Liang P., 2010, ICML, 639
- Warren H. S., 2002, Inc.
Cited by
Mathematical discoveries from program search with large language models
Romera-Paredes, B., Barekatain, M., Novikov, A., Balog, M., Kumar, M. P., Dupont, E., Ruiz, F. J. R., Ellenberg, J. S., Wang, P., Fawzi, O., et al. (2024) · Nature · doi:10.1038/s41586-023-06924-6 · Cited by 544
Ansor: Generating High-Performance Tensor Programs for Deep LearningPreprint
Zheng, L., Jia, C., Sun, M., Zhao, W., Yu, C. H., Haj-Ali, A., Wang, Y., Yang, J., Zhuo, D., Sen, K., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.06762 · Cited by 484
CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis
Nijkamp, E., Pang, B., Hayashi, H., Tu, L., Wang, H., Zhou, Y., Savarese, S., & Xiong, C. (2022) · arXiv · doi:10.48550/arxiv.2203.13474 · Cited by 330
Learning to Perform Local Rewriting for Combinatorial OptimizationPreprint
Chen, X., & Tian, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.00337 · Cited by 322
FlashMeta: a framework for inductive program synthesis
Polozov, O., & Gulwani, S. (2015) · Proceedings of the 2015 ACM SIGPLAN International Conference on Object-Oriented Programming, Systems, Languages, and Applications · doi:10.1145/2814270.2814310 · Cited by 225
Metadata
- Authors
- Schkufza, E., Sharma, R., & Aiken, A.
- Affiliations
- Stanford University (US)
- Venue
- ACM SIGARCH Computer Architecture News
- Published
- 2013-03-16
- Volume
- 41
- Issue
- 1
- Pages
- 305-316
- ISSN
- 0163-5964 (ISSN-L)
- OpenAlex
- W4299968636
- Cited by
- 57
- References
- 15
- Citations by year
- 2026: 5 · 2025: 7 · 2024: 4 · 2023: 8 · 2022: 6 · 2021: 10 · 2020: 5 · 2019: 2 · 2018: 1 · 2017: 3 · 2016: 2 · 2015: 2 · 2014: 2
- FWCI
- 3.75
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- ACM SIGARCH Computer Architecture News (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Physical Sciences › Computer Science › Software › Logic, programming, and type systems
- Physical Sciences › Computer Science › Hardware and Architecture › Embedded Systems Design Techniques
- Keywords
- stochastic search, Markov chain Monte Carlo, program transformation, program optimization, Stokes, x86 assembly
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.