Syntax-guided synthesis

Probably not a substantial claim: may not pass S3 (load-bearing).

The mutation e = ITE(0 ≤ y, y, x) is considered with probability 1/288.

“Thus, the mutation e = ITE(0 ≤ y, y, x) is considered with probability 1/288.”

May combine two claims.

As with other benchmarks, the enumerative solver was always faster, whereas the symbolic solver failed on every instance.

“As with other benchmarks, the enumerative solver was always faster, whereas the symbolic solver failed on every instance.”

Classified as probably not a substantial claim.

We have formalized the problem of syntax-guided synthesis.

“Aimed at formulating the core computational problem common to many recent tools for program synthesis in a canonical and logical manner, we have formalized the problem of syntax-guided synthesis.”

May combine two claims.

In the illustrative example, the correct expression was examined after only four calls to the SMT solver, although about 200 expressions were enumerated by the algorithm.

“In the example we have described, the correct expression was examined after only four calls to the SMT solver, although about 200 expressions were enumerated by the algorithm.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The probability of choosing e = ITE(x ≤ 0, y, x) as the initial candidate is 1/768.

“Thus, the probability of choosing e = ITE(x ≤ 0, y, x) as the initial candidate is 1/768.”

May combine two claims.

For the Hacker's Delight benchmarks (hd) the enumerative solver dominates, followed by the stochastic solver.

“For the Hacker's Delight benchmarks (hd) we see that the enumerative solver dominates, followed by the stochastic solver.”

Probably not a substantial claim: may not pass S3 (load-bearing).

None of the grammars for these problems required the synthesizer to discover the bit-vector constants involved in the efficient implementations.

“It is worth mentioning, however, that none of the grammars for these problems required the synthesizer to discover the bit-vector constants involved in the efficient implementations.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The probability of deciding to mutate the boolean condition x ≤ 0 is 1/6.

“The subexpression to mutate is chosen uniformly at random, and so the probability of deciding to mutate the boolean condition x ≤ 0 is 1/6.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Given a set of concrete examples {(-1, -4), (-1, -3), (-1, -2), (1, 1), (1, 2)}, Score(e) = exp(-2β), and Score(e ) = exp(-3β), and so e becomes the new candidate with probability exp(-β).

“Given a set of concrete examples {(-1, -4), (-1, -3), (-1, -2), (1, 1), (1, 2)}, Score(e) = exp(-2β), and Score(e ) = exp(-3β), and so e becomes the new candidate with probability exp(-β).”

May combine two claims.

The symbolic search was the slowest, failing to terminate on 29 of the 57 benchmarks.

“The symbolic search was the slowest, failing to terminate on 29 of the 57 benchmarks.”

May combine two claims.

The current set of benchmarks is limited to synthesis of loop-free functions with no optimality criterion.

“The current set of benchmarks is limited to synthesis of loop-free functions with no optimality criterion;”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

In the run of the algorithm on the example, although the algorithm enumerated about 200 expressions, only about 80 expressions were stored.

“For instance, in the run of the algorithm on the example, although the algorithm enumerated about 200 expressions, only about 80 expressions were stored.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Our suite includes a range of benchmarks from simple toy problems to nontrivial functions that are difficult to derive by hand.

“Our suite includes a range of benchmarks from simple toy problems to nontrivial functions that are difficult to derive by hand.”

May combine two claims.

The modification to the original algorithm [5] that searches for programs of increasing length provides better running times for most benchmarks in our set, but it can also be more expensive in certain cases.

“This approach provides better running times for most benchmarks in our set, but it can also be more expensive in certain cases.”

• cites [5]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

With one exception, the enumerative solver is the fastest for this class of benchmarks, followed by the stochastic solver.

“With one exception, the enumerative solver is the fastest for this class of benchmarks, followed by the stochastic solver.”

May combine two claims.

The Morton benchmarks, which involve the synthesis of a function to compute Morton numbers, are intended as challenge problems, and could not be completed by any of the synthesizers.

“The Morton benchmarks, which involve the synthesis of a function to compute Morton numbers, are intended as challenge problems, and could not be completed by any of the synthesizers.”

May combine two claims.

Some of the benchmarks can be solved in a few hundredths of a second, whereas others could not be solved by any of our prototype implementations.

“Some of the benchmarks can be solved in a few hundredths of a second, whereas others could not be solved by any of our prototype implementations.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

If e = ITE(x ≤ y, y, x) had been the mutation considered, then Score(e ) = 1, and e would have become the new candidate with probability 1.

“If, on the other hand, e = ITE(x ≤ y, y, x) had been the mutation considered, then Score(e ) = 1, and e would have become the new candidate with probability 1.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

There are 768 integer-valued expressions in the grammar of size six.

“There are 768 integer-valued expressions in the grammar of size six.”

May combine two claims.

Sketch completed all but 11 of the hd benchmarks, and it was able to synthesize array-search up to size 7.

“Sketch completed all but 11 of the hd benchmarks, and it was able to synthesize array-search up to size 7.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Of the 48 boolean conditions in the grammar, y ≤ 0 may be chosen with probability 1/48.

“Of the 48 boolean conditions in the grammar, y ≤ 0 may be chosen with probability 1/48.”

Classified as probably not a substantial claim.

We have transcribed all the hacker's delight and integer benchmarks into the input language of the Sketch synthesis system [24].

“Specifically, we have transcribed all the hacker's delight and integer benchmarks into the input language of the Sketch synthesis system [24].”

• cites [24]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

Syntax guidance is usually a key ingredient in current approaches to synthesis that blend induction and deduction.

“Many current approaches to synthesis blend induction and deduction [22]; syntax guidance is usually a key ingredient in these approaches.”

• cites [22]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S4 (based).

In the presence of let-expressions, the size of an expression directly corresponds to the number of instructions in the corresponding straight-line code.

“In presence of let-expressions, the size directly corresponds to the number of instructions in the corresponding straight-line code, and thus such a metric can be used effectively for applications such as super-optimization.”

Classified as probably not a substantial claim.

Inductive learning is a natural fit for the syntax-guided synthesis problem because the concept class is simply the set L of permissible expressions.

“Inductive learning is thus a natural fit for the syntax-guided synthesis problem introduced in this paper: the concept class is simply the set L of permissible expressions.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

The statement that there exists an expression e in the language generated by a context-free grammar G such that φ[f/e] is valid in a theory T cannot be translated to determining the truth of a formula in the theory T, even with additional quantifiers.

“Finally, it is worth noting that the statement “there exists an expression e in the language generated by a contextfree grammar G such that the formula ϕ[f /e] is valid in a theory T ” cannot be translated to determining the truth of a formula in the theory T , even with additional quantifiers.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

To optimize the computation of the product of two two-by-two matrices, the search space can be limited to implementations that use only 7 multiplication operations, and such a restriction can be expressed only syntactically.

“For example, to optimize the computation of the product of two two-by-two matrices, we can limit the search space to implementations that use only 7 multiplication operations, and such a restriction can be expressed only syntactically.”

Probably not a substantial claim: may not pass S4 (based).

Using the candidate set L to limit the search space for potential implementations has significant computational benefits for solving the syntax-guided synthesis problem.

“First, the user can use the candidate set L to limit the search-space for potential implementations, and this has significant computational benefits for solving the synthesis problem.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Many current approaches to synthesis blend induction and deduction.

“Many current approaches to synthesis blend induction and deduction [22]; syntax guidance is usually a key ingredient in these approaches.”

• cites [22]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

The top-level expression specified by the grammar T := (let [z = U] z + z) is the sum of two identical subexpressions built using arithmetic operators, and such a structure cannot be specified using a standard context-free grammar.

“The top-level expression specified by this grammar is the sum of two identical subexpressions built using arithmetic operators, and such a structure cannot be specified using a standard context-free grammar.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

Counterexample-guided inductive synthesis (CEGIS) is perhaps the most popular approach to inductive synthesis today.

“Counterexample-guided inductive synthesis (CEGIS) [16], [24] shown in Figure 1 is perhaps the most popular approach to inductive synthesis today.”

• cites [16]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [24]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

While let-expressions in a specification can be desugared, the same does not hold when they are used in a grammar.

“While let-expressions in a specification can be desugared, the same does not hold when they are used in a grammar.”

Classified as probably not a substantial claim.

The answer to the syntax-guided synthesis problem need not be unique: there may be two expressions e1 and e2 in the set L of allowed expressions such that both implementations satisfy the correctness specification φ.

“The answer to our synthesis problem need not be unique: there may be two expressions e 1 and e 2 in the set L of allowed expressions such that both implementations satisfy the correctness specification ϕ.”

Probably not a substantial claim: may not pass S4 (based).

Every expression in the set L1 is equivalent to ax+by+c for integer constants a, b, c.

“For example, every expression in the set L 1 is equivalent to ax+by+c, for integer constants a, b, c.”

Probably not a substantial claim: may not pass S2 (checkable).

Multi-modal specifications combining syntactic and semantic constraints have the potential to make programming more intuitive.

“Such forms of multi-modal specifications have the potential to make programming more intuitive.”

May combine two claims.

The overall trend we observe is that the encoding of the problem space into grammar has a significant impact on performance, although the solvers are often good at mitigating the effect of larger search spaces.

“but the overall trend we observe is that the encoding of the problem space into grammar has a significant impact on performance, although the solvers are often good at mitigating the effect of larger search spaces.”

May combine two claims.

Despite these limitations, the symbolic solver was able to solve many of the benchmarks, providing a lower bound on what can be achieved with a straightforward use of off-the-shelf technology.

“Despite these limitations, the symbolic solver was able to solve many of the benchmarks, providing a lower bound on what can be achieved with a straightforward use of off-the-shelf technology.”

May combine two claims.

This experiment is not an entirely fair comparison because, although Sketch uses a specialized constraint solver and carefully tuned encodings, the symbolic solver presented in this paper uses a direct encoding of the problem into sequences of constraints and uses Z3, a widely used off-the-shelf SMT solver which is not as aggressively tuned for synthesis problems.

“This experiment is not an entirely fair comparison because, although Sketch uses a specialized constraint solver and carefully tuned encodings, the symbolic solver presented in this paper uses a direct encoding of the problem into sequences of constraints and uses Z3, a widely used offthe-shelf SMT solver which is not as aggressively tuned for synthesis problems.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Level d5 involves a highly unconstrained grammar, so the synthesizer must discover which operators to use in addition to how to connect them together.

“Level d5, on the other extreme, involves a highly unconstrained grammar, so the synthesizer must discover which operators to use in addition to how to connect them together.”

May combine two claims.

Our algorithm does not attempt to optimize the size of the expression, while the super-optimizer of Schufza et al. [11] does so.

“Our algorithm differs from that of Schufza et al. [11] in three ways: (1) we do not attempt to optimize the size of the expression while the super-optimizer does so;”

• cites [11]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

We have some evidence to suggest that the symbolic solver can discover such constants from the full space of 2^32 possible constants with relatively little additional effort.

“We have some evidence to suggest that the symbolic solver can discover such constants from the full space of 2 32 possible constants with relatively little additional effort.”

Probably not a substantial claim: may not pass S4 (based).

If an answer e exists, then the search at size n = |e| is guaranteed to converge.

“If an answer e exists, then the search at size n = |e| is guaranteed to converge.”

May combine two claims.

We have some evidence that more optimized encodings can make the symbolic approach more competitive, although there are still many problems for which the enumerative approach is more effective.

“We have some evidence that more optimized encodings can make the symbolic approach more competitive, although there are still many problems for which the enumerative approach is more effective.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

In all cases, the complexity of the problems derives from the size of the space of possible functions and not from the complexity of checking whether a candidate solution is correct.

“In all cases, however, the complexity of the problems derives from the size of the space of possible functions and not from the complexity of checking whether a candidate solution is correct.”

May combine two claims.

Nonsymbolic techniques can be effective in exploring spaces of implementations and can surpass symbolic techniques, especially when the problems do not require the synthesizer to derive complex bit-vector constants, which is true for all the bit-vector benchmarks used.

“We can also see that nonsymbolic techniques can be effective in exploring spaces of implementations and can surpass symbolic techniques, especially when the problems do not require the synthesizer to derive complex bit-vector constants, which is true for all the bit-vector benchmarks used.”

Probably not a substantial claim: may not pass S3 (load-bearing).

CEGIS has close connections to algorithmic debugging using counterexamples and counterexample-guided abstraction refinement (CEGAR).

“CEGIS has close connections to algorithmic debugging using counterexamples [19] and counterexample-guided abstraction refinement (CEGAR) [25].”

• cites [19]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [25]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim.

The order of magnitude of the solution times and the relative complexity of the different approaches on different benchmarks can give us an idea of the relative merits of each of the approaches described earlier.

“However, the order of magnitude of the solution times and the relative complexity of the different approaches on different benchmarks can give us an idea of the relative merits of each of the approaches described earlier.”

May combine two claims.

The problems where the enumerative solver succeeded but Sketch failed were the d5 versions of problems 11, 12, 14 and 15, which suggests that the enumerative solver was better at pruning unnecessary instructions from the grammar.

“The problems where the enumerative solver succeeded but Sketch failed were the d5 versions of problems 11, 12, 14 and 15, which suggests that the enumerative solver was better at pruning unnecessary instructions from the grammar.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

For most problems, there are three different levels of grammars numbered d0, d1 and d5; level d0 involves only the instructions necessary for the implementation, so the synthesizer only needs to discover how to connect them together.

“For most problems, there are three different levels of grammars numbered d0, d1 and d5; level d0 involves only the instructions necessary for the implementation, so the synthesizer only needs to discover how to connect them together.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The enumerative learning algorithm [8] adopts a dynamic programming based search strategy that systematically enumerates concepts (expressions) in increasing order of complexity.

“The enumerative learning algorithm [8] adopts a dynamic programming based search strategy that systematically enumerates concepts (expressions) in increasing order of complexity.”

• cites [8]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S4 (based).

Our prototype implementation of the three approaches to solve this problem is the first attempt to compare and contrast existing algorithms on a common set of benchmarks.

“Our prototype implementation of the three approaches to solve this problem is the first attempt to compare and contrast existing algorithms on a common set of benchmarks.”

Classified as probably not a substantial claim.

The verification oracle O_V can be implemented as an SMT solver that checks whether φ[f/e] is valid modulo the theory T.

“O V can be implemented as an SMT solver that checks whether ϕ[f /e] is valid modulo the theory T .”

May combine two claims.

Unlike the hd benchmarks where the different grammars for a given benchmark were strict subsets of each other, in this case the encodings AIG and NAND correspond to different representations of the same space of functions.

“Unlike the hd benchmarks where the different grammars for a given benchmark were strict subsets of each other, in this case the encodings AIG and NAND correspond to different representations of the same space of functions.”

Classified as probably not a substantial claim. May combine two claims.

None of the implementations were engineered with high performance in mind, so the exact solution times are not necessarily representative of the best that can be achieved by a particular approach.

“None of the implementations were engineered with highperformance in mind, so the exact solution times are not necessarily representative of the best that can be achieved by a particular approach.”

May combine two claims.

Although both implementations are immature, these results suggest that it may be easier to derive good pruning rules for the explicit search than an effective fitness function for the stochastic solver.

“so although both implementations are immature, these results suggest that it may be easier to derive good pruning rules for the explicit search than an effective fitness function for the stochastic solver.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The stochastic learning procedure is an adaptation of the algorithm recently used by Schufza et al. [11] for program super-optimization.

“The stochastic learning procedure is an adaptation of the algorithm recently used by Schufza et al. [11] for program super-optimization.”

• cites [11]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The benchmarks provide an initial demonstration of the expressiveness of the base formalism and of the relative merits of the individual solution strategies presented earlier.

“nevertheless, the benchmarks provide an initial demonstration of the expressiveness of the base formalism and of the relative merits of the individual solution strategies presented earlier.”

Classified as probably not a substantial claim. May combine two claims.

Grammars can be conveniently used to express a wide range of constraints, and in particular, to bound the depth and/or the size of the desired expression.

“Grammars can be conveniently used to express a wide range of constraints, and in particular, to bound the depth and/or the size of the desired expression.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

For many of these problems the magic constants come from a handful of values such as 1, 0, or 0xffffffff, so it is unnecessary for the enumerative solver to search the space of 2^32 possible bit-vectors.

“On the other hand, for many of these problems the magic constants come from a handful of values such as 1, 0, or 0xffffffff, so it is unnecessary for the enumerative solver to search the space of 2 32 possible bit-vectors.”

May combine two claims.

Since we do not know the expression size n, we run concurrent searches for different values of n, whereas the super-optimizer can use the size of the input program as an upper bound on program size.

“(3) since we do not know the expression size n, we run concurrent searches for different values of n, whereas the super-optimizer can use the size of the input program as an upper bound on program size.”

• cites [11]Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

The symbolic solver used for these experiments represents one of many possible approaches to encoding the synthesis problem into a series of constraints.

“The symbolic solver used for these experiments represents one of many possible approaches to encoding the synthesis problem into a series of constraints.”

Probably not a substantial claim: may not pass S4 (based).

For many problems, there are different natural ways to encode the space of desired functions into a grammar.

“For many problems, there are different natural ways to encode the space of desired functions into a grammar, so we are interested in observing the effect of these differences in encoding for the different solvers.”