Learning programs by learning from failures

May combine two claims.

Neo can fail to detect inconsistency with the examples, in which case no programs get pruned.

“As it typically uses over-approximate properties, Neo can fail to detect inconsistency with the examples, in which case no programs get pruned.”

• cites Feng et al. 2018 sourceSupported

Probably not a substantial claim: may not pass S3 (load-bearing).

ASP-encoded ILP systems can often learn optimal and recursive programs.

“Systems that implement this approach can often learn optimal and recursive programs and can harness state-of-the-art ASP solvers, but often struggle with scalability, especially in terms of the problem domain size.”

• cites Corapi et al. 2011Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Law et al. 2014 sourceNot checked — no full text of the source
• cites Schüller and Benz 2018 sourcePartly supported

  Section 4 describes searching for an optimal hypothesis, and Section 3.3 discusses configuring the hypothesis space to include a transitive-closure rule. These passages support optimal search and the possibility of recursive rules, but they do not establish that ASP-based ILP systems often learn recursive programs. The paper discusses scalability limitations in Section 7, without specifically identifying problem-domain size as the main limitation. (LLM, conf=0.82)

  Experimental — accuracy not yet measured

May combine two claims.

Compared to Metagol, Popper does not need metarules, so can learn programs with any arity predicates.

“Compared to Metagol, Popper does not need metarules (Cropper and Tourret 2020), so can learn programs with any arity predicates.”

• cites Cropper and Tourret 2020 sourcePartly supported

  Section 2.1 says Metagol-style meta-interpretive learners use metarules, supporting the comparison’s Metagol side. The passages found do not discuss Popper, so they do not establish that Popper needs no metarules or can learn programs with predicates of any arity. Page numbers were not available in our copy. (LLM, conf=0.82)

  Experimental — accuracy not yet measured

Classified as probably not a substantial claim.

Neo synthesises non-recursive programs using SMT encoded properties and a three staged loop.

“Neo (Feng et al. 2018) synthesises non-recursive programs using SMT encoded properties and a three staged loop.”

• cites Feng et al. 2018 sourceSupported

Probably not a substantial claim: may not pass S3 (load-bearing).

ILP systems naturally support lifelong and transfer learning because of their symbolic nature.

“Finally, because of their symbolic nature, ILP systems naturally support lifelong and transfer learning (Cropper 2020), which is considered essential for human-like AI (Lake et al. 2016).”

• cites Cropper 2020Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Lake et al. 2016 sourceSupported

Probably not a substantial claim: may not pass S3 (load-bearing).

Recent meta-level ILP approaches often learn optimal solutions, such as programs with the fewest clauses or literals.

“Recent meta-level ILP approaches often learn optimal solutions, such as programs with the fewest clauses (Muggleton et al. 2015; Cropper and Muggleton 2016; Kaminski et al. 2018) or literals (Corapi et al. 2011; Law et al. 2014).”

• cites Muggleton et al. 2015 sourceSupported
• cites Cropper and Muggleton 2016 sourceNot checked — no full text of the source
• cites Kaminski et al. 2018 sourceSupported

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

A user must either provide a set of metarules, or use a set of metarules restricted to a certain fragment of logic, e.g. dyadic Datalog.

“Ausermusteither(i)provideasetofmetarules, or (ii) use a set of metarules restricted to a certain fragment of logic, e.g. dyadic Datalog (Cropper and Tourret 2020).”

• cites Cropper and Tourret 2020 sourcePartly supported

  The Introduction says suitable metarules are commonly assumed to be supplied as background knowledge, and §3.1.1 studies syntactically restricted fragments. The paper also discusses chained dyadic Datalog, but these passages do not say a user must choose between supplying metarules and using a restricted fragment. (LLM, conf=0.83)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing).

Mode declarations are a popular language bias.

“Mode declarations (Muggleton 1995) are a popular language bias (Blockeel and De Raedt 1998; Srinivasan 2001; Ray 2009; Corapi et al. 2010, 2011; Athakravi et al. 2013; Ahlgren and Yuen 2013; Law et al. 2014).”

• cites Muggleton 1995 sourceNot checked — no full text of the source
• cites Blockeel and De Raedt 1998 sourcePartly supported

  Section 4 (page unavailable) describes TILDE’s user-specified, PROGOL-like rmode specifications, which constrain allowable query refinements through variable modes and types. This supports the use of mode-based declarations as a language bias, but the paper does not establish that they are popular. (LLM, conf=0.78)

  Experimental — accuracy not yet measured
• cites Srinivasan 2001Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

Metarules are another popular syntactic bias used by many ILP approaches, including Metagol and, to an extent, ∂ ILP.

“Metarules (Cropper and Tourret 2020) are another popular syntactic bias used by many ILP approaches (De Raedt and Bruynooghe 1992; Wang et al. 2014; Albarghouthi et al. 2017; Kaminski et al. 2018), including Metagol (Muggleton et al. 2015; Cropper et al. 2020; Cropper and Muggleton 2016) and, to an extent 5 , ∂ ILP (Evans and Grefenstette 2018).”

• cites Cropper and Tourret 2020 sourceSupported
• cites De Raedt and Bruynooghe 1992Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Wang et al. 2014 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

Much recent work in ILP uses ASP to learn Datalog, definite, normal, and answer set programs.

“Much recent work in ILP uses ASP to learn Datalog (Evans et al. 2019), definite (Muggleton et al. 2014; Kaminski et al. 2018; Cropper and Dumancic 2020), normal (Ray 2009; Corapi et al. 2011; Athakravi et al. 2013), and answer set programs (Law et al. 2014).”

• cites Evans et al. 2019 sourceSupported
• cites Muggleton et al. 2014 sourceNot checked — no full text of the source
• cites Kaminski et al. 2018 sourceSupported

Classified as probably not a substantial claim.

Learning recursive programs has long been considered a difficult problem in ILP.

“Learning recursive programs has long been considered a difficult problem in ILP (Muggleton et al. 2012).”

• Also stated in: § 5.3 List transformation problem · p. 38 · plain

  “Learning recursive programs has long been considered a difficult problem in ILP (Muggleton et al. 2012)”

• cites Muggleton et al. 2012 sourceSupported

Probably not a substantial claim: may not pass S3 (load-bearing).

The major problem with metarules is determining which one to use.

“However, the major problem with metarules is determining whichonestouse(CropperandTourret2020).”

• cites Cropper and Tourret 2020 sourceSupported

Probably not a substantial claim: may not pass S3 (load-bearing).

Many ILP systems are not guaranteed to learn optimal solutions, where optimal typically means the smallest program or the program with the minimal description length.

“Many ILP systems (Muggleton 1995; Srinivasan 2001; Blockeel and De Raedt 1998; Ray 2009) are not guaranteed to learn optimal solutions, where optimal typically means the smallest program or the program with the minimal description length.”

• cites Muggleton 1995 sourceNot checked — no full text of the source
• cites Srinivasan 2001Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Blockeel and De Raedt 1998 sourcePartly supported

  Section 4 describes TILDE as selecting the best split at each node using a heuristic, while Section 5 compares the compactness of its theories with those of other ILP systems. This is consistent with the absence of a guarantee that TILDE finds the smallest theory, but the paper does not establish that many ILP systems lack such a guarantee or define optimality as the smallest program or minimum description length. Page numbers are unavailable in our copy. (LLM, conf=0.78)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing).

INSPIRE also uses predefined constraints to remove redundancy from the hypothesis space.

“INSPIRE (Schüller and Benz 2018) also uses predefined constraints to remove redundancy from the hypothesis space (in INSPIRE's case, each hypothesis is a single clause).”

• cites Schüller and Benz 2018 sourcePartly supported

  Section 3.6 (Redundancy elimination module; page unavailable) describes constraints that eliminate redundant rules from INSPIRE’s hypothesis search space. But the parenthetical is not supported: Section 3 (Declarative hypothesis generation; page unavailable) says each generated answer set represents a single rule that may be part of a hypothesis, not that each hypothesis is a single clause. (LLM, conf=0.90)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

ILASP can represent real numbers as strings and delegate the reasoning to Python via Clingo's scripting feature.

“For instance, ILASP (Law et al. 2014) can represent real numbers as strings and delegate the reasoning to Python via Clingo's scripting feature (Gebser et al. 2014).”

• cites Law et al. 2014 sourceNot checked — no full text of the source
• cites Gebser et al. 2014 sourceSupported

Probably not a substantial claim: may not pass S3 (load-bearing).

Many ILP systems allow a user to constrain the hypothesis space though clause constraints.

“Many ILP systems allow a user to constrain the hypothesis space though clause constraints (Muggleton 1995; Srinivasan 2001; Blockeel and De Raedt 1998; Ahlgren and Yuen 2013; Law et al. 2014).”

• cites Muggleton 1995 sourceNot checked — no full text of the source
• cites Srinivasan 2001Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Blockeel and De Raedt 1998 sourcePartly supported

  Section 4 says TILDE users specify which conjunctions may be added to queries, their maximum frequency, and variable modes and types. This supports user-imposed constraints on TILDE’s hypothesis search, but the passage does not establish that many ILP systems offer clause constraints specifically. No page number is available in our copy. (LLM, conf=0.78)

  Experimental — accuracy not yet measured

Classified as probably not a substantial claim.

The idea of multi-shot solving is that state of the solving process for an ASP program can be saved to help solve modifications of that program.

“The idea of multi-shot solving is that state of the solving process for an ASP program can be saved to help solve modifications of that program.”

• cites Gebser et al. 2019 sourceSupported

Classified as probably not a substantial claim.

Metagol performs iterative deepening over the number of clauses allowed in a solution.

“Metagol performs iterative deepening over the number of clauses allowed in a solution (Muggleton et al. 2015).”

• cites Muggleton et al. 2015 sourceSupported

Probably not a substantial claim: may not pass S3 (load-bearing).

Other recent ILP approaches also employ defining an ASP problem where an answer set corresponds to a definite program.

“The idea is to define an ASP problem where an answer set (a model) corresponds to a definite program, an approach also employed by other recent ILP approaches (Corapi et al. 2011; Law et al. 2014; Kaminski et al. 2018; Schüller and Benz 2018).”

• cites Corapi et al. 2011Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Law et al. 2014 sourceNot checked — no full text of the source
• cites Kaminski et al. 2018 sourceSupported

Classified as probably not a substantial claim. May combine two claims.

In the multi-shot cycle, a ground program is given to an ASP solver, yielding an answer set, whose processing leads to a (first-order) extension of the program; only this extension then needs grounding and adding to the running ASP instance, which means that the running solver may maintain learned conflicts.

“The essence of the multi-shot cycle is that a ground program is given to an ASP solver, yielding an answer set, who's processing leads to a (first-order) extension of the program. Only this extension then needs grounding and adding to the running ASP instance, which means that the running solver may, for example, maintain learned conflicts.”

• cites Gebser et al. 2019 sourcePartly supported

  Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Predicate invention helps reduce the size of target programs, which in turn reduces sample complexity and improves predictive accuracy.

“Predicate invention has been shown to help reduce the size of target programs, which in turns reduces sample complexity and improves predictive accuracy (Cropper 2019; Dumancic et al. 2019).”

• cites Cropper 2019 sourceSupported
• cites Dumancic et al. 2019 sourcePartly supported

  Section 4.1 (page unavailable) reports improved predictive performance for models learned from the paper’s latent predicates. But Section 3.5’s reduction concerns candidate clauses pruned during search, not the size of target programs, and the copy does not establish reduced sample complexity or the claimed causal chain. (LLM, conf=0.86)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing).

It is impossible to generate a finite and complete set of metarules for a singleton-free fragment of monadic and dyadic Datalog.

“It is impossible to generate a finite and complete set of metarules for a singleton-free fragment of monadic and dyadic Datalog (Cropper and Tourret 2020).”

• cites Cropper and Tourret 2020 sourcePartly supported

  Section 5.3.1 (page unavailable) says the singleton-free fragment has no derivation reduction for arity at least two, supporting the claim if “complete” means derivationally complete. But the same summary says it has an entailment reduction to a finite set. The unqualified claim therefore overstates what is impossible. (LLM, conf=0.88)

  Experimental — accuracy not yet measured

May combine two claims.

Popper can substantially outperform existing ILP systems both in terms of predictive accuracies and learning times.

“Popper can substantially outperform existing ILP systems both in terms of predictive accuracies and learning times”

May combine two claims.

Popper can outperform existing ILP systems, both in terms of predictive accuracies and learning times.

“Our experimental results on three domains (toy game problems, robot strategies, and list transformations) show that (i) constraints drastically improve learning performance, and (ii) Popper can outperform existing ILP systems, both in terms of predictive accuracies and learning times.”

May combine two claims.

Popper scales well with respect to the optimal solution size, the number of background relations, the domain size, the number of training examples, and the size of the training examples.

“Popper scales well with respect to the optimal solution size, the number of background relations, the domain size, the number of training examples, and the size of the training examples”

May combine two claims.

Popper reasons about lists and infinite domains.

“Popper support types, learning optimal (textually minimal) solutions, learning recursive programs, reasoning about lists and infinite domains, and hypothesis constraints.”

Classified as probably not a substantial claim.

Popper uses hypothesis constraints.

“Popper support types, learning optimal (textually minimal) solutions, learning recursive programs, reasoning about lists and infinite domains, and hypothesis constraints.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

If suitable metarules are known, then Popper can simulate metarules through hypothesis constraints.

“If suitable metarules are known, then, as we show in 'Appendix A', Popper can simulate metarules through hypothesis constraints.”

May combine two claims.

In our first two experiments, the hypothesis spaces contain approximately 10^6 and 10^16 rules.

“By contrast, in our first two experiments (Sect. 5.1), the hypothesis spaces contain approximately 10 6 and 10 16 rules.”

May combine two claims.

Popper does not pre-compute every rule, which allows it to learn rules with many body literals.

“By contrast, Popper does not pre-compute every rule, which allows it to learn rules with many body literals.”

May combine two claims.

The key idea of Popper is that when a hypothesis fails, Popper uses theta-subsumption to translate the hypothesis itself into a set of hypothesis constraints to rule out generalisations and specialisations of it, which does not need a set of pre-computed rules and which is substantially quicker to compute.

“By contrast, the key idea of Popper is that when a hypothesis fails, Popper uses theta-subsumption (Plotkin 1971) to translate the hypothesis itself into a set of hypothesis constraints to rule out generalisations and specialisations of it, which does not need a set of pre-computed rules and which is substantially quicker to compute.”

• cites Plotkin 1971Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Popper learns Prolog programs and cannot currently handle noise.

“ILASP3 learns ASP programs and can handle noise, whereas Popper learns Prolog programs and cannot currently handle noise.”

May combine two claims.

Popper additionally learns definite programs, and thus supports learning programs with infinite domains.

“Moreover, whereas ProSynth synthesises Datalog programs, Popper additionally learns definite programs, and thus supports learning programs with infinite domains.”

Classified as probably not a substantial claim.

In our approach continually builds a set of constraints.

“In other words, in our approach continually builds a set of constraints.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Popper uses the learned constraints to learn definite programs.

“In the next section we introduce Popper, which uses these constraints to learn definite programs.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

When allowed up to ten body literals, Popper performs almost identically.

“When allowed up to ten body literals, Popper performs almost identically.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Finally, Popper generates another hypothesis (h4), tests it against the examples, finds that it does not fail, and returns it.

“Finally, Popper generates another hypothesis ( h4 ), tests it against the examples, finds that it does not fail, and returns it.”

Classified as probably not a substantial claim.

To learn optimal solutions, Popper searches for programs of increasing size.

“To learn optimal solutions (Definition 14), Popper searches for programs of increasing size.”

The quote only partly states this claim.

ILASP would not terminate in the time limit when given at most ten body literals because it pre-computes every rule in the hypothesis space.

“when given this parameter ILASP would not terminate in the time limit because it pre-computes every rule in the hypothesis space.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Popper generates another hypothesis (h3), tests it against the examples, and finds that it fails because it entails the negative example last([e,m,m,a],m) and is therefore too general.

“Popper generates another hypothesis ( h3 ) and tests against the examples and finds that it fails because it entails the negative example last ( [ e , m , m , a ] , m ) and is therefore too general .”

May combine two claims.

Hypothesis constraints allow pruning recursive programs that do not contain a base case clause, left recursive or mutually recursive programs, or programs which contain subsumption redundancy between clauses.

“For instance, hypothesis constraints allow us to prune recursive programs that do not contain a base case clause (Fig. 3), to prune left recursive or mutually recursive programs, or to prune programs which contain subsumption redundancy between clauses.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Popper supports functional and irreflexive constraints, and constraints on recursive programs, such as disallowing left recursion or mutual recursion.

“Popperalsosupports functional and irreflexive constraints, and constraints on recursive programs, such as disallowing left recursion or mutual recursion.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Popper adds the elimination constraint to the meta-level ASP program, which prunes h9 from the hypothesis space.

“Popper adds this constraint to the meta-level ASP program which prunes h9 from the hypothesis space.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Popper supports hypothesis constraints, which are defined over a whole program rather than a single clause.

“Popper additionally supports the more general concept of hypothesis constraints (Definition 6), which are defined over a whole program (a set of clauses) rather than a single clause (also employed in previous work (Athakravi et al. 2014)).”

Probably not a substantial claim: may not pass S3 (load-bearing).

Through constraints, Popper supports the standard notions of recall and input/output arguments of mode declarations.

“Through constraints, Popper also supports the standard notions of recall and input/output 10 arguments of mode declarations (Muggleton1995).”

• cites Muggleton1995Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

For the small dataset, the largest program that Metagol learns is for when n=4, which takes 50 seconds to learn, compared to one second for Popper.

“For the small dataset, the largest program that Metagol learns is for when n = 4, which takes 50 seconds to learn, compared to one second for Popper.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Popper supports clause constraints, which allow a user to prune specific clauses from the hypothesis space.

“As with many ILP systems (Muggleton 1995; Srinivasan 2001; Athakravi et al. 2014; Law et al. 2014; Schüller and Benz 2018), Popper supports clause constraints, which allow a user to prune specific clauses from the hypothesis space.”

Classified as probably not a substantial claim.

The literal not clause(n) ensures that no additional clause is added to the n distinct clauses of the provided hypothesis.

“The literal not clause( n ) ensures that no additional clause is added to the n distinct clauses of the provided hypothesis.”

May combine two claims.

Popper outperforms ILASP2i and ILASP3 on both datasets.

“PopperoutperformsILASP2iandILASP3onbothdatasets.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Popper tests h1 against the examples and finds that it fails because it does not entail any positive example and is therefore too specific.

“Popper then tests h1 against the examples and finds that it fails because it does not entail any positive example and is therefore too specific .”

Probably not a substantial claim: may not pass S3 (load-bearing).

Because h1 does not entail any positive example (is totally incomplete), Popper also generates an elimination constraint.

“In addition, because h1 does not entail any positive example (is totally incomplete), Popper also generates an elimination constraint:”

Probably not a substantial claim: may not pass S3 (load-bearing).

To ensure that variables are distinct, the inequalities V0!=V1, V0!=V2, and V1!=V2 must be imposed.

“To ensure that variables are distinct we need to impose the inequality V0!=V1 and V0!=V2 and V1!=V2 .”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

To add simple types to Popper requires the single constraint shown in Fig. 4.

“For instance, to add simple types to Popper requires the single constraint shown in Fig. 4.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The first clause of h2 specialises both clauses in h1, yet h2 is not a specialisation of h1.

“The first clause of h2 specialises both clauses in h1 , yet h2 is not a specialisation of h1 .”

Probably not a substantial claim: may not pass S3 (load-bearing).

Popper generates the simplest hypothesis h1.

“To start, Popper generates the simplest hypothesis:”

Probably not a substantial claim: may not pass S3 (load-bearing).

Popper adds the generalisation constraint to the meta-level ASP program, which prunes h6 and h7 from the hypothesis space.

“Popper adds this constraint to the meta-level ASP program which prunes h6 and h7 from the hypothesis space.”

May combine two claims.

For the big dataset, the largest program that Metagol learns is for when n=3, which takes 57 seconds to learn, compared to eight seconds for Popper.

“For the big dataset, the largest program that Metagol learns is for when n = 3, which takes 57 seconds to learn, compared to eight seconds for Popper.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Popper implements the LFF approach and works in three separate stages: generate, test, and constrain.

“Popper implements the LFF approach and works in three separate stages: generate, test, and constrain.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Popper adds the specialisation constraint to the meta-level ASP program, which prunes h2 and h5 from the hypothesis space.

“Popper adds this constraint to the meta-level ASP program which prunes h2 and h5 from the hypothesis space.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The encoding allows for V0 = V1 = V2 = 0, where all the variables are A.

“Note that the above encoding allows for V0 = V1 = V2 = 0 , where all the variables are A .”

May combine two claims.

Enumerate takes 159 seconds to find an evens program, whereas Popper takes only four seconds.

“For instance, whereas it takes Enumerate 159 seconds to find an evens program, it takes Popper only four seconds.”

May combine two claims.

Aleph struggles to learn solutions to these problems, with the exceptions of addhead and threesame, which do not need recursion.

“Table 5 also shows that Aleph struggles to learn solutions to these problems. The exceptions are addhead and threesame , which do not need recursion.”

May combine two claims.

ILASP2i initially learns solutions in the given time limit, but struggles as the grid size grows.

“ILASP2i initially learns solutions in the given time limit, but struggles as the grid size grows.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

When ILASP2i learns a solution, it is always an accurate solution.

“Note that when ILASP2i learns a solution, it is always an accurate solution.”

May combine two claims.

Popper scales well in the size of the optimal solution size and the number of background relations.

“Our buttons experiment (Experiment 5.1) showed that Popper scales well in the size of the optimal solution size and the number of background relations.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Popper can trivially learn Datalog programs and test them using ASP.

“Popper can also trivially learn Datalog programs and test them using ASP.”

May combine two claims.

When given the increment relation, Metagol runs into infinite recursion on almost every problem and could not find any solutions.

“when given this relation Metagol runs into infinite recursion 18 on almost every problem and could not find any solutions.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In Sect. 5.3, Popper and Metagol achieved similar performance.

“To do so, we repeat the last experiment from Sect. 5.3, where Popper and Metagol achieved similar performance.”

May combine two claims.

Popper combines ASP and Prolog to support types, learning optimal solutions, learning recursive programs, reasoning about lists and infinite domains, and hypothesis constraints.

“Popper combines ASP and Prolog to support types, learning optimal solutions, learning recursive programs, reasoning about lists and infinite domains, and hypothesis constraints.”

May combine two claims.

Popper outperforms ILASP2i and ILASP3 both in terms of predictive accuracies and learning times.

“Popper outperforms ILASP2i and ILASP3 both in terms of predictive accuracies and learning times.”

The quote only partly states this claim.

Popper outperforms Metagol in all cases when varying the size of the input.

“Popper outperforms Metagol in all cases.”

Classified as probably not a substantial claim.

Popper is an ILP system that learns definite programs.

“We implemented this approach in Popper, an ILP system that learns definite programs.”

May combine two claims.

Given 10,000 examples, Metagol struggles to find a solution in one minute and eventually converges on the default predictive accuracy (50%).

“Given this many examples, Metagol struggles to find a solution in one minute and eventually converges on the default predictive accuracy (50%).”

May combine two claims.

Although Aleph is sometimes faster than Popper, it only learns accurate solutions for addhead and threesame.

“Note that although Aleph is sometimes faster than Popper, it only learns accurate solutions for addhead and threesame”

Probably not a substantial claim: may not pass S3 (load-bearing).

Popper does not support non-observational predicate learning, where examples of the target predicates are not directly given.

“Unlike some ILP systems (Muggleton 1995; Katzouris et al. 2016), Popper does not support non-observational predicate learning (non-OPL) (Muggleton 1995), where examples of the target predicates are not directly given.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

When Metagol learns a solution, it is always an accurate solution.

“Note that when Metagol learns a solution, it is always an accurate solution.”

May combine two claims.

The mean learning times of Popper for examples of length 50 and 500 are both less than a second.

“The mean learning times of Popper for examples of length 50 and 500 are both less than a second.”

May combine two claims.

The mean learning times for Metagol for examples of length 50 and 500 are 20 and 54 seconds respectively.

“The mean learning times for Metagol for examples of length 50 and 500 are 20 and 54 seconds respectively.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Popper does not currently support predicate invention.

“Popper does not currently support predicate invention.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Popper does not currently support noisy examples.

“Popper does not currently support noisy examples.”

May combine two claims.

Compared to ILASP3, Popper does not need to ground a program, so scales better as the domain size grows.

“Compared to ILASP3, Popper does not need to ground a program, so scales better as the domain size grows (Sect. 5.2).”

May combine two claims.

Set covering ILP systems tend to learn overly specific solutions and struggle to learn recursive programs.

“However, these systems tend to learn overly specific solutions and struggle to learn recursive programs (Bratko 1999; Cropper et al. 2020).”

• cites Bratko 1999Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Cropper et al. 2020Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim.

Because hypotheses are logic programs, they can be read by humans.

“Because hypotheses are logic programs, they can be read by humans, crucial for explainable AI and ultra-strong machine learning (Michie 1988).”

• cites Michie 1988Not checked — no full text of the source

Classified as probably not a substantial claim. May combine two claims.

The loop repeats until either the learner finds a hypothesis that entails all the positive and none of the negative examples, or there are no more hypotheses to test.

“This loop repeats until either (i) the learner finds a hypothesis that entails all the positive and none of the negative examples, or (ii) there are no more hypotheses to test.”

Probably not a substantial claim: may not pass S3 (load-bearing).

ILP systems can generalise from small numbers of examples, often a single example.

“ILP systems can generalise from small numbers of examples, often a single example (Lin et al. 2014).”

• cites Lin et al. 2014Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim.

In the generate stage, the learner generates a hypothesis (a logic program) that satisfies a set of hypothesis constraints.

“In the generate stage, the learner generates a hypothesis (a logic program) that satisfies a set of hypothesis constraints (constraints on the syntactic form of hypotheses).”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

ILASP3 is the most similar ILP approach and also employs a generate/test/constrain loop.

“ILASP3(Law2018)isthemostsimilarILPapproachandalsoemploysagenerate/test/constrain loop.”

• cites Law 2018Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim.

In the test stage, the learner tests the hypothesis against training examples.

“In the test stage, the learner tests the hypothesis against training examples.”

Classified as probably not a substantial claim.

The learning from failures (LFF) approach decomposes the ILP problem into three separate stages: generate, test, and constrain.

“In this approach, an ILP system (the learner) decomposes the learning problem into three separate stages: generate , test , and constrain .”

• Also stated in: § 1 Introduction · p. 2 · plain

  “In this approach, the learner (an ILP system) decomposes the ILP problem into three separate stages: generate , test , and constrain .”

• AssumptionRests on an assumption: The fundamental problem in ILP is to efficiently search a large hypothesis space.

Classified as probably not a substantial claim.

ProSynth takes as input a set of candidate Datalog rules and returns a subset of them.

“ProSynth (Raghothaman et al. 2020) takes as input a set of candidate Datalog rules and returns a subset of them.”

• cites Raghothaman et al. 2020Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

ProSynth synthesises Datalog programs.

“Moreover, whereas ProSynth synthesises Datalog programs, Popper additionally learns definite programs, and thus supports learning programs with infinite domains.”

• cites Raghothaman et al. 2020Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

ILP approaches use a language bias to restrict the hypothesis space.

“ILP approaches use a language bias (Nienhuys-Cheng and de Wolf 1997) to restrict the hypothesis space.”

• cites Nienhuys-Cheng and de Wolf 1997 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

This limitation means that ILP systems that use metarules are difficult to use, especially when the BK contains predicate symbols with arity greater than two.

“This limitation means that ILP systems that use metarules are difficult to use, especially when the BK contains predicate symbols with arity greater than two.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The largest number of candidate rules considered in the ProSynth experiments is 1000.

“For instance, the largest number of candidate rules considered in the ProSynth experiments is 1000.”

• cites Raghothaman et al. 2020Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

Without recursion, it is often difficult for an ILP system to generalise from small numbers of examples.

“Without recursion, it is often difficult for an ILP system to generalise from small numbers of examples (Cropper et al. 2015).”

• cites Cropper et al. 2015Not checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

ProSynth learns constraints that disallow certain clause combinations.

“ProSynth learns constraints that disallow certain clause combinations, e.g. to prevent clauses that entail a negative example from occurring together.”

• cites Raghothaman et al. 2020Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

ILASP3 uses a counter-example guided approach and translates an uncovered example e into a constraint that is satisfied if and only if e is covered.

“Another way of viewing ILASP3 is that it uses a counter-example guided (Solar-Lezama et al. 2008) approach and translates an uncovered example e into a constraint that is satisfied if and only if e is covered.”

• cites Solar-Lezama et al. 2008Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Compared with predicate (and mode) declarations, metarules are a much stronger inductive bias because they specify the exact form of clauses in the hypothesis space.

“Compared with predicate (and mode) declarations, metarules are a much stronger inductive bias because they specify the exact form of clauses in the hypothesis space.”

May combine two claims.

These systems tend to learn overly specific solutions and struggle to learn recursive programs.

“However, these systems tend to learn overly specific solutions and struggle to learn recursive programs (Bratko 1999; Cropper et al. 2020).”

• cites Bratko 1999Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Cropper et al. 2020Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

Popper uses ASP to generate definite programs, which allows it to reason about large and infinite problem domains, such as reasoning about lists and real numbers.

“Popper uses ASP to generate definite programs, which allows it to reason about large and infinite problem domains, such as reasoning about lists and real numbers.”

Classified as probably not a substantial claim.

The problem setting is based on the ILP learning from entailment setting.

“Our problem setting is based on the ILP learning from entailment setting (De Raedt 2008).”

• cites De Raedt 2008 sourceNot checked — no full text of the source

May combine two claims.

The hypothesis space is huge for non-trivial inputs, which motivates using learned constraints to prune the hypothesis space.

“As this result shows, the hypothesis space is huge for non-trivial inputs, which motivates using learned constraints to prune the hypothesis space.”

Probably not a substantial claim: may not pass S3 (load-bearing).

If the hypothesis does not cover one of the examples, ILASP3 finds a reason why and then generates constraints to guide subsequent search.

“If the hypothesis does not cover one of the examples, ILASP3 finds a reason why and then generates constraints to guide subsequent search.”

• cites Law 2018Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

Athakravi et al. (2014) introduce domain-dependent constraints, which are constraints on the hypothesis space provided as input by a user.

“Athakravi et al. (2014) introduce domain-dependent constraints , which are constraints on the hypothesis space provided as input by a user.”

• cites Athakravi et al. 2014 sourceNot checked — no full text of the source

Classified as probably not a substantial claim. May combine two claims.

In this approach, the numeric computation is performed when grounding the inputs, so the grounding must be finite.

“However, in this approach, the numeric computation is performed when grounding the inputs, so the grounding must be finite.”

Probably not a substantial claim: may not pass S3 (load-bearing).

ILASP3's constraint loop requires at most | E | iterations, where | E | is the number of ILASP examples, which are partial interpretations.

“ILASP3's constraint loop requires at most | E | iterations, where | E | is the number of ILASP examples, which are partial interpretations.”

• cites Law et al. 2014 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

With each iteration, ILASP3 finds the best hypothesis it can.

“With each iteration, ILASP3 finds the best hypothesis it can.”

• cites Law 2018Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

There may be multiple (sometimes infinite) solutions to an LFF problem.

“There may be multiple (sometimes infinite) solutions.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

ILASP3 learns unstratified ASP programs, including programs with choice rules and weak and hard constraints, and can handle noise.

“ILASP3 learns unstratified ASP programs, including programs with choice rules and weak and hard constraints, and can handle noise.”

• cites Law 2018Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

Hypothesis constraints allow us to prune recursive programs without a base case and subsumption redundant programs.

“Moreover, hypothesis constraints allow us to prune recursive programs without a base case and subsumption redundant programs.”

May combine two claims.

Popper does not pre-compute every clause, which allows it to learn clauses with many body literals.

“By contrast, Popper does not pre-compute every clause, which allows it to learn clauses with many body literals.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Another way of viewing ILASP3 is that it uses a counter-example guided approach and translates an uncovered example e into a constraint that is satisfied if and only if e is covered.

“Another way of viewing ILASP3 is that it uses a counter-example guided (Solar-Lezama et al. 2008) approach and translates an uncovered example e into a constraint that is satisfied if and only if e is covered.”

• cites Solar-Lezama et al. 2008Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

Through hypothesis constraints, Popper can enforce (optional) type, metarule, recall, and functionality restrictions.

“For instance, through hypothesis constraints, Popper can enforce (optional) type, metarule, recall, and functionality restrictions.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Midelfart (1999) extends subsumption to clausal theories.

“Midelfart (1999) extends subsumption to clausal theories:”

• cites Midelfart (1999) sourceNot checked — no full text of the source

Classified as probably not a substantial claim.

Checking subsumption between clauses is decidable.

“checking subsumption between clauses is decidable”

• cites NienhuysCheng and de Wolf 1997Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

A major limitation of most pure ASP-based ILP systems is the intrinsic grounding problem, especially on large domains, such as reasoning about lists or numbers-most ASP implementations do not support lists nor real numbers.

“Therefore, a major limitation of most pure ASP-based ILP systems is the intrinsic grounding problem, especially on large domains, such as reasoning about lists or numbers-most ASP implementations do not support lists nor real numbers.”

May combine two claims.

Popper's constraints do not reason about specific clauses, but instead reason about the syntax of hypotheses using theta-subsumption and are therefore quick to compute.

“Popper's constraints do not reason about specific clauses (because we do not pre-compute the hypothesis space), but instead reason about the syntax of hypotheses using theta-subsumption and are therefore quick to compute.”

Classified as probably not a substantial claim.

Checking entailment between clauses is undecidable.

“whereas checking entailment between clauses is undecidable (Church 1936)”

• cites Church 1936 sourceNot checked — no full text of the source

Classified as probably not a substantial claim. May combine two claims.

The constraints are boolean formulas over the rules in the hypothesis space, an approach that requires a set of precomputed rules.

“The constraints are boolean formulas over the rules in the hypothesis space, an approach that requires a set of precomputed rules.”

• cites Law 2018Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

Popper also allows a user to provide clause constraints.

“Popper also allows a user to provide clause constraints.”

Classified as probably not a substantial claim.

If a clause C1 subsumes a clause C2, then C1 entails C2.

“If a clause C 1 subsumes a clause C 2 then C 1 entails C 2 (Nienhuys-Cheng and de Wolf 1997).”

• cites Nienhuys-Cheng and de Wolf 1997 sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Popper learns Prolog programs, including programs operating over lists and real numbers, but cannot handle noise.

“By contrast, Popper learns Prolog programs, including programs operating over lists and real numbers, but cannot handle noise.”

Classified as probably not a substantial claim. May combine two claims.

Popper also supports such constraints but goes further by learning constraints from failed hypotheses.

“Popper also supports such constraints but goes further by learning constraints from failed hypotheses.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Subsumption is weaker than entailment.

“Subsumption is therefore weaker than entailment.”

• AssumptionRests on an assumption: If C1 entails C2 then it does not necessarily follow that C1 subsumes C2.

Probably not a substantial claim: may not pass S3 (load-bearing).

Deciding subsumption between clauses is NP-complete in general.

“although, in general, deciding subsumption is a NP-complete problem (NienhuysCheng and de Wolf 1997)”

• cites NienhuysCheng and de Wolf 1997Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

Difficulty handling large (or infinite) domains is not specific to ASP.

“Difficulty handling large (or infinite) domains is not specific to ASP.”

• AssumptionRests on an assumption: ∂ ILP uses a neural network to induce programs, but only works on BK formed of a finite set of ground atoms.

Probably not a substantial claim: may not pass S3 (load-bearing).

When a hypothesis fails, Popper translates the hypothesis itself into a set of hypothesis constraints.

“By contrast, when a hypothesis fails, Popper translates the hypothesis itself into a set of hypothesis constraints .”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

ILASP3 is a pure ASP-based ILP system that also employs a constrain loop.

“ILASP3 (Law 2018) is a pure ASP-based ILP system that also employs a constrain loop.”

• cites Law 2018Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

Popper uses constraints to improve efficiency (mostly by removing redundancy), for instance by using variables in order.

“Finally, Popper uses constraints to improve efficiency (mostly by removing redundancy). For instance, Popper uses constraints to use variables in order, which prunes the program p ( B ) :-q ( B ) because we could generate p ( A ) :-q ( A ) .”

Classified as probably not a substantial claim.