ADVENT: LLM-Driven Automatic Predicate Invention for ILP

Yang et al.'s generated predicates correspond only to explicit surface-level patterns; implicit relations requiring cross-example reasoning remain outside their scope.

“However, the generated predicates correspond only to explicit surface-level patterns; implicit relations requiring cross-example reasoning remain outside their scope.”

• cites [14] Yang, 2026 sourceDisputed

  Section 5.1 describes ZENDO as involving binary predicates and complex logical relations, while Section 6.1.2 says the authors’ method maintains robust performance on that task. Section 4.3 says the generated predicate system is supplied with facts to an ILP solver that learns rules from the examples. Although the predicates are designed from text, the paper does not confine the method to surface-level patterns or place cross-example reasoning outside its scope. Page numbers are unavailable in our copy. (LLM, conf=0.84)

  Checked by a reasoning model — may be wrong.

  Experimental — accuracy not yet measured

Popper relies on manually defined mode declarations to control the search.

“While more flexible than template-based methods, Popper still relies on manually defined mode declarations to control the search.”

• cites [3] sourceDisputed

  Section 2.5 (p. 8) explicitly says, “We do not use mode declarations,” and identifies predicate declarations as Popper’s simpler language bias. Section 4.2 (p. 22) says Popper can optionally express some features of mode declarations through hypothesis constraints, but that does not mean it relies on manually defined mode declarations. (LLM, conf=0.95)

  Checked by a reasoning model — may be wrong.

  Experimental — accuracy not yet measured

Existing PI methods produce semantically opaque predicates.

“Existing methods rely on domain expertise and produce semantically opaque predicates, hindering adaptation to unfamiliar domains and cross-task reuse.”

• cites [4] sourceSupported

LLM outputs in formal domains remain prone to errors.

“While LLMs can generate logic programs, their outputs in formal domains remain prone to errors [15].”

• cites [15] sourceSupported

Forgetgol's selective forgetting remains a structural heuristic that does not address the root cause.

“Forgetgol [5] mitigates this through selective forgetting based on usage frequency, but this remains a structural heuristic that does not address the root cause.”

• cites [5] sourcePartly supported

  Section 4.3 (p. 4) describes selective forgetting by restricting which predicate symbols may appear in a hypothesis, and the statistical method estimates relevance from reuse (p. 5). But the paper does not characterize Forgetgol as a heuristic that fails to address the root cause; the Introduction (p. 1) presents forgetting as a way to address the problem of excessive background knowledge. That final assessment is not supported by this source. (LLM, conf=0.84)

  Experimental — accuracy not yet measured

Relying solely on LLM reasoning introduces reliability concerns.

“However, relying solely on LLM reasoning introduces reliability concerns [9].”

• cites [9] sourceSupported

Iteratively refining LLM outputs with formal verification can substantially improve their reasoning quality.

“Recent studies have shown that iteratively refining LLM outputs with formal verification can substantially improve their reasoning quality [10], [11].”

• cites [10] sourcePartly supported

  Section 4.2 (Generation Phase; page unavailable) reports that GPT-4 generated correct Dafny code most successfully when given multiple tries and feedback from the Dafny compiler and verifier. This supports iterative improvement in code-generation correctness, but the paper does not establish a substantial improvement in LLM reasoning quality generally. (LLM, conf=0.88)

  Experimental — accuracy not yet measured
• cites [11] sourceNot checked — no full text of the source

Existing PI methods rely on domain expertise.

“Existing methods rely on domain expertise and produce semantically opaque predicates, hindering adaptation to unfamiliar domains and cross-task reuse.”

• cites [1] Cropper, 2022 sourceSupported
• cites [3] sourceSupported

ILP's compositional structure naturally supports knowledge reusability across tasks: hypotheses learned from earlier tasks can serve as BK for subsequent ones.

“ILP's compositional structure naturally supports this: hypotheses learned from earlier tasks can serve as BK for subsequent ones [1].”

• cites [1] Cropper, 2022 sourceSupported

ILP's performance depends heavily on predefined background knowledge (BK).

“However, ILP's performance depends heavily on predefined background knowledge (BK) [1], a set of predicates that encode known facts and relational structures about the domain, analogous to features in traditional ML.”

• cites [1] Cropper, 2022 sourceSupported

Predicate invention (PI) remains a critical bottleneck in Inductive Logic Programming (ILP).

“Predicate invention (PI), the creation of new predicates to extend the hypothesis space, remains a critical bottleneck in Inductive Logic Programming (ILP).”

• Also stated in: § I. INTRODUCTION · p. 1 · boosted

  “making it one of the most critical open challenges in ILP [1].”

• cites [1] Cropper, 2022 sourceSupported

In ILP, Prolog offers a deductive engine that serves as a natural verifier: by executing candidate predicates on subsets of examples, it precisely reveals whether an invented predicate generalizes as intended.

“In the context of ILP, Prolog [12], the predominant logic programming language used in ILP, offers a deductive engine that serves as a natural verifier: by executing candidate predicates on subsets of examples, it precisely reveals whether an invented predicate generalizes as intended.”

• cites [12] sourcePartly supported

  The Artificial Intelligence section (p. 52) says Prolog is typically used in ILP to represent background knowledge, examples, and induced theories. Section 2.1 (p. 5) describes Prolog’s resolution-based execution of queries. Neither passage says that executing candidate predicates on subsets of examples verifies whether an invented predicate generalizes as intended. (LLM, conf=0.84)

  Experimental — accuracy not yet measured

ILP-only position-free achieves 0/9 success.

“ILP-only position-dependent 6/9 ILP-only position-free 0/9”

ILP-only position-dependent achieves 6/9 success.

“ILP-only position-dependent 6/9 ILP-only position-free 0/9”

The knowledge pool yields gains up to +31 percentage points.

“Experiments on nine poker-hand concepts across seven LLMs show that LLM-driven PI achieves 58% success rate where ILP alone fails entirely, formal verification raises this to 80%, and the knowledge pool yields gains up to +31 percentage points, while producing human-interpretable rules.”

Experiments on nine poker-hand concepts across seven LLMs show that LLM-driven PI achieves 58% success rate where ILP alone fails entirely.

“Experiments on nine poker-hand concepts across seven LLMs show that LLM-driven PI achieves 58% success rate where ILP alone fails entirely, formal verification raises this to 80%, and the knowledge pool yields gains up to +31 percentage points, while producing human-interpretable rules.”

Formal verification raises the success rate to 80%.

“Experiments on nine poker-hand concepts across seven LLMs show that LLM-driven PI achieves 58% success rate where ILP alone fails entirely, formal verification raises this to 80%, and the knowledge pool yields gains up to +31 percentage points, while producing human-interpretable rules.”

ADVENT produces human-interpretable rules.

“Experiments on nine poker-hand concepts across seven LLMs show that LLM-driven PI achieves 58% success rate where ILP alone fails entirely, formal verification raises this to 80%, and the knowledge pool yields gains up to +31 percentage points, while producing human-interpretable rules.”

Preliminary experiments observed syntactic violations of logic programming conventions in LLM-generated predicates.

“In preliminary experiments, we observe two recurring types of errors: syntactic violations of logic programming conventions, and predicates that capture overly specific patterns that fail to generalize.”

The LLM consistently interpreted the transferred data as a Michalski's Trains puzzle with no recognition of the poker domain.

“To verify this transfer, we prompted the LLM to identify the scenario behind the data; the model consistently interpreted it as a Michalski's Trains puzzle with no recognition of the poker domain.”

Preliminary experiments observed predicates that capture overly specific patterns that fail to generalize.

“In preliminary experiments, we observe two recurring types of errors: syntactic violations of logic programming conventions, and predicates that capture overly specific patterns that fail to generalize.”

Formal verification improves success rates for six out of seven models.

“As Table I shows, formal verification improves success rates for six out of seven models.”

Adding formal verification (ADVENT) raises the average success rate to 80% [76-84%].

“Adding formal verification (ADVENT) raises the average to 80% [76-84%], whose confidence interval does not overlap with the no-verification condition, confirming that Prolog deductive verification effectively guides predicate refinement (RQ2).”

DeepSeek's success rate rises from 51% to 87% with non-overlapping confidence intervals.

“The benefit is most pronounced for models with lower baselines: MiMo rises from 31% to 93% and DeepSeek from 51% to 87%, with non-overlapping confidence intervals in both cases.”

Removing the knowledge pool causes drops in six out of seven models.

“Furthermore, removing the knowledge pool reduces the average from 80% to 67% [62-72%], with drops in six out of seven models and non-overlapping confidence intervals, indicating that the LLM's ability to reuse previously invented predicates substantially benefits learning (RQ3).”

The largest gains from the knowledge pool concentrate in compositional concepts (straight flush +31, royal flush +29, full house +23).

“The largest gains concentrate in compositional concepts (straight flush +31, royal flush +29, full house +23), which leverage knowledge from earlier learning.”

Formal verification outperforms LLM self-critique in five of seven models.

“Formal verification outperforms LLM self-critique in five of seven models, with the exception of Gemini-2.5-Pro, where the model's strong self-assessment partially compensates for its weak baseline.”

Removing the knowledge pool reduces the average success rate from 80% to 67% [62-72%].

“Furthermore, removing the knowledge pool reduces the average from 80% to 67% [62-72%], with drops in six out of seven models and non-overlapping confidence intervals, indicating that the LLM's ability to reuse previously invented predicates substantially benefits learning (RQ3).”

In position-dependent mode, ILP alone must enumerate all (5 choose 2) = 10 slot combinations, producing a verbose rule set.

“In positiondependent mode, ILP alone must enumerate all ( 5 2 ) = 10 slot combinations, producing a verbose rule set.”

Even without verification, LLM-driven PI under position-free mode achieves an average success rate of 58% [53-63%].

“In contrast, even without verification, LLMdriven PI under position-free mode achieves an average success rate of 58% [53-63%], demonstrating that LLMs can identify implicit patterns and invent useful predicates that substantially extend ILP's representational capacity (RQ1).”

For flush, the knowledge pool causes a drop of 9 percentage points.

“However, for simpler tasks such as flush ( -9 ), where only a single shared attribute is needed, exposing prior predicates appears to introduce unnecessary complexity.”

The LLM naturally incorporates the inequality constraint when inventing has duplicate rank, yielding the compact rule one pair(A) :- has duplicate rank(A).

“The LLM, by contrast, naturally incorporates it when inventing has duplicate rank , thereby yielding the compact rule: one pair ( A ) :-has duplicate rank ( A ) .”

LLM self-critique achieves 73% [68-78%].

“By comparison, LLM self-critique achieves 73% [6878%], suggesting that grounded execution results provide more reliable refinement signals than LLM self-assessment alone.”

LLM self-critique improves over no verification on average.

“While LLM self-critique also improves over no verification on average, it is less reliable: QWQ-32B drops from 62% to 49%, suggesting that unreliable self-assessment can mislead refinement.”

QWQ-32B's success rate drops from 62% to 49% under LLM self-critique.

“While LLM self-critique also improves over no verification on average, it is less reliable: QWQ-32B drops from 62% to 49%, suggesting that unreliable self-assessment can mislead refinement.”

Models that already generate near-optimal predicates in a single pass (GPT-5-Codex, GLM-5) show marginal gains.

“Models that already generate near-optimal predicates in a single pass (GPT-5-Codex, GLM-5) show marginal gains due to ceiling effects.”

The confidence interval for ADVENT's 80% average does not overlap with the no-verification condition.

“Adding formal verification (ADVENT) raises the average to 80% [76-84%], whose confidence interval does not overlap with the no-verification condition, confirming that Prolog deductive verification effectively guides predicate refinement (RQ2).”

Detecting straight demands cross-object arithmetic operations that ILP's refinement operators cannot construct without pre-defined building blocks.

“Detecting this pattern demands cross-object arithmetic operations that ILP's refinement operators cannot construct without pre-defined building blocks.”

Selective reuse and composition of invented predicates supports cumulative learning with interpretable rules.

“We show that LLMs can interpret invented predicate definitions and assess their relevance to new tasks, enabling selective reuse and composition that supports cumulative learning with interpretable rules.”

Unnamed predicates such as inv 1 and inv 2 have no notion of the patterns these symbols encode, making it hard to assess which prior inventions are relevant to a new task.

“Unnamed predicates such as inv 1 and inv 2 have no notion of the patterns these symbols encode, making it hard to assess which prior inventions are relevant to a new task.”

Without suitable predicates, the system cannot discover rules that explain the target concept well.

“Without suitable predicates, the system cannot discover rules that explain the target concept well.”

LLMs encode broad prior knowledge.

“Trained on large-scale corpora, LLMs encode broad prior knowledge and demonstrate strong capabilities in code generation, abductive reasoning [6], [7], and logic programs synthesis [8]”

The LLM's ability to reuse previously invented predicates substantially benefits learning.

“Furthermore, removing the knowledge pool reduces the average from 80% to 67% [62-72%], with drops in six out of seven models and non-overlapping confidence intervals, indicating that the LLM's ability to reuse previously invented predicates substantially benefits learning (RQ3).”

Cross-task composition arises because the LLM can interpret prior predicate definitions and assess their relevance to the current task.

“Such cross-task composition arises because the LLM can interpret prior predicate definitions and assess their relevance to the current task.”

The benefit of formal verification is most pronounced for models with lower baselines.

“The benefit is most pronounced for models with lower baselines: MiMo rises from 31% to 93% and DeepSeek from 51% to 87%, with non-overlapping confidence intervals in both cases.”

LLMs can identify implicit patterns and invent useful predicates that substantially extend ILP's representational capacity.

“In contrast, even without verification, LLMdriven PI under position-free mode achieves an average success rate of 58% [53-63%], demonstrating that LLMs can identify implicit patterns and invent useful predicates that substantially extend ILP's representational capacity (RQ1).”

As the knowledge pool grows, the approach is bounded by the LLM's context window.

“as the pool grows, this approach is bounded by the LLM's context window, and an increasing number of predicates may dilute the model's attention.”

Prolog deductive verification effectively guides predicate refinement.

“Adding formal verification (ADVENT) raises the average to 80% [76-84%], whose confidence interval does not overlap with the no-verification condition, confirming that Prolog deductive verification effectively guides predicate refinement (RQ2).”

• Also stated in: § C. Effectiveness of Formal Deductive Verification · p. 5 · boosted

  “These results confirm that Prolog deductive verification effectively guides predicate refinement and improves ILP learning performance.”

Prolog deductive verification improves ILP learning performance.

“These results confirm that Prolog deductive verification effectively guides predicate refinement and improves ILP learning performance.”

Grounded execution results provide more reliable refinement signals than LLM self-assessment alone.

“By comparison, LLM self-critique achieves 73% [6878%], suggesting that grounded execution results provide more reliable refinement signals than LLM self-assessment alone.”

The marginal gains for GPT-5-Codex and GLM-5 are due to ceiling effects.

“Models that already generate near-optimal predicates in a single pass (GPT-5-Codex, GLM-5) show marginal gains due to ceiling effects.”

Exposing prior predicates introduces unnecessary complexity for simple concepts such as flush.

“However, for simpler tasks such as flush ( -9 ), where only a single shared attribute is needed, exposing prior predicates appears to introduce unnecessary complexity.”

Unreliable self-assessment can mislead refinement.

“While LLM self-critique also improves over no verification on average, it is less reliable: QWQ-32B drops from 62% to 49%, suggesting that unreliable self-assessment can mislead refinement.”

Gemini-2.5-Pro's strong self-assessment partially compensates for its weak baseline.

“Formal verification outperforms LLM self-critique in five of seven models, with the exception of Gemini-2.5-Pro, where the model's strong self-assessment partially compensates for its weak baseline.”

The gains in compositional concepts leverage knowledge from earlier learning.

“The largest gains concentrate in compositional concepts (straight flush +31, royal flush +29, full house +23), which leverage knowledge from earlier learning.”