Cited by 83Logic, Reasoning, and KnowledgeData Mining Algorithms and ApplicationsAI-based Problem Solving and Planning

The DOI matches its Crossref record; 272 of the 286 references it cites were found in a registry or an index; 85 of the 336 sources its claims cite were checked for support.

Inductive Logic Programming At 30: A New Introduction

Open access
Cropper, Andrew, Dumančić, Sebastijan·2022·Journal of Artificial Intelligence Research

Abstract

Inductive logic programming (ILP) is a form of machine learning. The goal of ILP is to induce a hypothesis (a set of logical rules) that generalises training examples. As ILP turns 30, we provide a new introduction to the field. We introduce the necessary logical notation and the main learning settings; describe the building blocks of an ILP system; compare several systems on several dimensions; describe four systems (Aleph, TILDE, ASPAL, and Metagol); highlight key application areas; and, finally, summarise current limitations and directions for future research.

Overview

AI snapshot

Written from the paper’s full text.

TL;DR

This paper is a new introduction to inductive logic programming (ILP) on its 30th anniversary, covering logical notation, learning settings, system design choices, comparisons, four case studies, applications, and open limitations. It argues ILP remains distinctive for learning interpretable relational and recursive logic programs from few examples, while identifying usability, language bias, predicate invention, lifelong learning, noisy background knowledge, and raw-data perception as key challenges.

Key findings

  • ILP induces hypotheses as logic programs; unlike table-based ML, it represents examples, background knowledge, and hypotheses as logic programs and learns relations, with strengths in data efficiency, recursion, predicate invention, and knowledge transfer.
  • The survey compares systems on learning setting, hypothesis class, background knowledge, language bias, and search method, and on features including noise, optimality, infinite domains, recursion, and predicate invention; it details Aleph, TILDE, ASPAL, and Metagol.
  • Applications discussed include bioinformatics and drug design, the Robot Scientist (reported as the first machine to independently discover new scientific knowledge), ecology, program analysis, data curation and string transformations, learning from trajectories, NLP, physics-informed learning, robotics, and games.
  • Reported figures include: over 100 ILP systems built since 1991 but less than a handful meaningfully usable; Lin et al. learn 17 string transformation tasks over time; Knorf reduces the number of literals in a program by 50% or more; and the basic Metagol implementation is less than 100 lines of Prolog code.

Methods

  • This is a survey/tutorial, not an empirical study. It introduces logic programming notation, formalizes learning from entailment and interpretations, describes ILP system building blocks, compares systems in tables, presents four case studies, surveys applications and related work, and summarizes limitations.
  • It uses illustrative scenarios and examples—such as the happy/lego concept, last-element and quicksort programs, drug design, Bongard problems, kinship, machine repair, and ASPAL’s penguin example—rather than a dataset or experimental evaluation.
  • The main analysis is conceptual synthesis and qualitative comparison; no new experiments or benchmark results are reported.

Limitations

  • The paper states that ILP tools are often hard to use and poorly engineered; language biases are brittle and difficult to choose; predicate invention and abstraction remain difficult; lifelong learning, relevance, noisy background knowledge, probabilistic ILP, explainability, and learning from raw data are unresolved or under-explored.
  • The paper states its scope is mostly definite programs, especially Prolog, and mostly recent developments; it does not detail work combining neural networks with ILP, and its comparison tables exclude many systems such as interactive, theory revision, and probabilistic systems.
  • Evident from the text: as a survey/tutorial, it provides no empirical validation of its comparative claims, and the tables are explicitly described as coarse, high-level overviews that “should not be taken absolutely literally.”

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1613/jair.1.13507

Cites

272 of 272 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

85 of 336 cited sources checked · 55 supported · 1 flagged · accuracy not yet measured

Show all 179 claims

  • Probably not a substantial claim. 274
  • It may state two separate claims as one. 101
  • The quote only partly states this claim. 2
319 of 319
  • The paper relies on this without support.

    Most forms of machine learning learn functions.

    “Moreover, whereas most forms of ML learn functions, ILP learns relations.”
  • The paper relies on this without support.

    Most ML approaches rely on statistical inference.

    “Most ML approaches rely on statistical inference.”
  • The paper relies on this without support.

    ILP relies on logical inference and uses techniques from automated reasoning and knowledge representation.

    “By contrast, ILP relies on logical inference and uses techniques from automated reasoning and knowledge representation .”
  • The paper relies on this without support.

    Prolog is a Turing-complete logic programming language.

    “For instance, Prolog is a Turing-complete logic programming language.”
  • The paper relies on this without support.

    Datalog is a syntactical subset of Prolog that sacrifices features and expressivity (it is not Turing-complete) to gain efficiency and decidability.

    “Datalog is a syntactical subset of Prolog that sacrifices features (such as data structures) and expressivity (it is not Turingcomplete) to gain efficiency and decidability.”
  • The paper relies on this without support.

    LFE and LFI are by far the most popular ILP learning settings.

    “LFE and LFI are by far the most popular learning settings, so we only cover these two.”
  • The paper relies on this without support.

    A definite logic program has only one model (the least Herbrand model), whereas an ASP program can have one, many, or even no models (answer sets).

    “Whereas a definite logic program has only one model (the least Herbrand model), an ASP program can have one, many, or even no models (answer sets).”
  • The paper relies on this without support.

    ASP supports additional language features, such as aggregates and weak and hard constraints.

    “ASP also supports additional language features, such as aggregates and weak and hard constraints.”
  • The paper relies on this without support.

    A key concept in ILP is the generality order over hypotheses (logic programs).

    “A key concept in ILP is the generality order over hypotheses (logic programs).”
  • The paper relies on this without support.

    A hypothesis can be a single clause or multiple clauses.

    “A hypothesis can be a single clause or multiple clauses.”
  • The paper relies on this without support.

    Often, a single clause is insufficient to describe a target concept.

    “Often, a single clause is insufficient to describe a target concept.”
  • The paper relies on this without support.

    The LFE problem in Definition 2 is general.

    “The LFE problem in Definition 2 is general.”
  • The paper relies on this without support.

    ILP systems impose strong restrictions on X, B, and H.

    “ILP systems impose strong restrictions on X , B , and H .”
  • The paper relies on this without support.

    Some ILP systems restrict X to only contain atoms whereas others allow clauses.

    “For instance, some restrict X to only contain atoms whereas others allow clauses.”
  • The paper relies on this without support.

    Some ILP systems restrict H to contain only Datalog clauses.

    “Some restrict H to contain only Datalog clauses.”
  • The paper relies on this without support.

    Training examples are often noisy.

    “However, training examples are often noisy, so it is difficult to find a hypothesis that is both complete and consistent.”
  • The paper relies on this without support.

    Reasoning about full clausal theories is computationally expensive.

    “However, reasoning about full clausal theories is computationally expensive, so most systems learn fragments of clausal logic, usually definite programs.”
  • The paper relies on this without support.

    Many practical applications require non-monotonic reasoning.

    “One motivation for learning normal programs (Section 2.3.1) is that many practical applications require non-monotonic reasoning.”
  • The paper relies on this without support.

    It is often simpler to express a concept with negation as failure (NAF).

    “Moreover, it is often simpler to express a concept with negation as failure (NAF).”
  • The authors state this as an assumption.

    When learning from interpretations, every example is completely specified: every atom in the interpretation has to be true or false, and there is no room for missing values.

    “When learning from interpretations, it is implicitly assumed that every example is completely specified. That is, every atom in the interpretation has to be true or false, and there is no room for missing values.”
  • The paper relies on this without support.

    ILP has traditionally relied on predefined and hand-crafted background knowledge, often designed by domain experts.

    “ILP has traditionally relied on predefined and hand-crafted BK, often designed by domain experts.”
  • The paper relies on this without support.

    Most ILP systems can only use background knowledge provided by a user.

    “This situation is a major problem, as most systems can only use BK provided by a user.”
  • The paper relies on this without support.

    ILP still heavily relies on much human input to solve a problem.

    “However, ILP still heavily relies on much human input to solve a problem.”
  • The paper relies on this without support.

    Addressing the problem of too much background knowledge has been under-researched.

    “Addressing the problem of too much BK has been under-researched.”
  • The authors state this as an assumption.

    The target hypothesis is in both hypothesis spaces.

    “assuming that the target hypothesis is in both spaces.”
  • The paper relies on this without support.

    There is little work determining which metarules to use for a given learning task.

    “Despite their widespread use, there is little work determining which metarules to use for a given learning task.”
  • The paper relies on this without support.

    These approaches assume suitable metarules as input or use metarules without any theoretical guarantees.

    “Instead, these approaches assume suitable metarules as input or use metarules without any theoretical guarantees.”
  • The paper relies on this without support.

    Aleph has good empirical performance.

    “it has good empirical performance”
  • The paper relies on this without support.

    Other ILP systems usually struggle to handle numerical data.

    “Indeed, TILDE is an exception among ILP systems, which usually struggle to handle numerical data.”
  • The paper relies on this without support.

    ASPAL's simplicity has inspired other approaches, notably ILASP.

    “A major advantage of ASPAL is its sheer simplicity, which has inspired other approaches, notably ILASP.”
  • The paper relies on this without support.

    Many natural language processing tasks require an understanding of the syntax and semantics of the language.

    “Many natural language processing tasks require an understanding of the syntax and semantics of the language.”
  • The paper relies on this without support.

    Metagol learns recursive programs.

    “Metagol supports PI, learning recursive programs, and is guaranteed to learn the smallest program.”
  • The paper relies on this without support.

    Metagol is guaranteed to learn the smallest program.

    “Metagol supports PI, learning recursive programs, and is guaranteed to learn the smallest program.”
  • The paper relies on this without support.

    Reaching human-level AI is a desirable goal.

    “Russell (2019) argues that the automatic invention of new high-level concepts is the most important step needed to reach human-level AI.”
    “To reach human-level AI, we would expect a learner to learn thousands or even millions of concepts.”
  • The paper relies on this without support.

    Wider adoption of ILP both inside and outside academia is desirable.

    “For ILP to be more widely adopted both inside and outside of academia, we must develop more standardised, user-friendly, and better-engineered tools.”
  • The paper relies on this without support.

    Improving learning performance is a desirable goal.

    “The central goal of representation learning coincides with the one behind PI: improving learning performance by changing the representation of a problem.”
36 of 36

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Partition relation's example uses a different predicate

    Minor

    The table names the four-argument relation partition, but its example uses pivot instead.

  • Nonrecursive last-element clause reads the head of an empty list

    Minor

    The third clause requires empty(E) and head(E,B) simultaneously. Under the paper's definitions of empty and head, it cannot find the last element of a three-element list as the accompanying description claims.

  • Two supposedly mode-consistent rules violate the declared types

    Minor

    In the second rule, tail makes C and D lists, but equal requires char arguments. In the third, member makes B a list, but the target declaration requires B to be a char.

  • Weak-subsumption claim conflicts with the set-based definition

    Minor

    Under the stated definition, applying the given substitution makes the two q literals identical. Clauses are treated as sets of literals, so the duplicate does not prevent the first clause from subsuming the second.

  • Definite programs are said to have only one model

    Minor

    The earlier definition permits any Herbrand interpretation that satisfies the clauses to be a model; it does not require the model to be least. Thus a definite program can have multiple models even though it has a unique least Herbrand model.

5 of 5

Citations

Cites

Items 201–272 of 272

Flagged first, then by citations

14 of the 286 references couldn’t be matched to a known paper.

Research works printed but not linked: 22

  1. August 28 - September 1, 2017, Proceedings , Vol. 10416 of Lecture Notes in Computer Science , pp. 689-706. Springer.
  2. Bain, M. (1994). Learning logical exceptions in chess . Ph.D. thesis, University of Strathclyde.
  3. Banerji, R. B. (1964). A language for the description of concepts. General Systems , 9 (1), 135-141.
  4. De Raedt, L., & Kersting, K. (2008b). Probabilistic Inductive Logic Programming , p. 1-27. Springer-Verlag, Berlin, Heidelberg.
  5. Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, 8-14 December 2019, Vancouver, BC, Canada , pp. 9165-9174.
  6. Goodacre, J. (1996). Inductive learning of chess rules using Progol . Ph.D. thesis, University of Oxford.
  7. Grobelnik, M. (1992). Markus: an optimized model inference system. In Proceedings of the Logic Approaches to Machine Learning Workshop .
  8. Inoue, K., Doncescu, A., & Nabeshima, H. (2013). Completing causal networks by metalevel abduction. Machine Learning , 91 (2), 239-277.
  9. Law, M. (2018). Inductive learning of answer set programs . Ph.D. thesis, Imperial College London, UK.
  10. Levin, L. A. (1973). Universal sequential search problems. Problemy Peredachi Informatsii , 9 (3), 115-116.
  11. (PAIS 2014) , Vol. 263 of Frontiers in Artificial Intelligence and Applications , pp. 525-530. IOS Press.
  12. Mart´ ınez, D., Ribeiro, T., Inoue, K., Aleny` a, G., & Torras, C. (2015). Learning probabilistic action models from interpretation transitions. In Vos, M. D., Eiter, T., Lierler, Y., & Toni, F. (Eds.), Proceedings of the Technical Communications of the 31st International Conference on Logic Programming (ICLP 2015), Cork, Ireland, August 31 - September 4, 2015 , Vol. 1433 of CEUR Workshop Proceedings . CEUR-WS.org.
  13. McCreath, E., & Sharma, A. (1995). Extraction of meta-knowledge to restrict the hypothesis space for ilp systems. In Eighth Australian Joint Conference on Artificial Intelligence , pp. 75-82.
  14. Mitchell, T. M. (1997). Machine learning . McGraw Hill series in computer science. McGrawHill.
  15. Raghothaman, M., Mendelson, J., Zhao, D., Naik, M., & Scholz, B. (2020). Provenanceguided synthesis of datalog programs. Proc. ACM Program. Lang. , 4 (POPL), 62:162:27.
  16. Palade, V., Goebel, R., Holzinger, A., Verspoor, K., & Wani, M. A. (Eds.), 14th IEEE International Conference on Machine Learning and Applications, ICMLA 2015, Miami, FL, USA, December 9-11, 2015 , pp. 25-31. IEEE.
  17. Sammut, C. (1993). The origins of inductive logic programming: A prehistoric tale. In Proceedings of the 3rd international workshop on inductive logic programming , pp. 127-147. J. Stefan Institute.
  18. Sch¨ uller, P., & Benz, M. (2018). Best-effort inductive logic programming via fine-grained cost-based hypothesis generation - the inspire system at the inductive logic programming competition. Machine Learning. , 107 (7), 1141-1169.
  19. Si, X., Lee, W., Zhang, R., Albarghouthi, A., Koutris, P., & Naik, M. (2018). Syntaxguided synthesis of Datalog programs. In Leavens, G. T., Garcia, A., & Pasareanu, C. S. (Eds.), Proceedings of the 2018 ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ESEC/SIGSOFT FSE 2018, Lake Buena Vista, FL, USA, November 04-09, 2018 , pp. 515-527. ACM.
  20. Srinivasan, A., Muggleton, S., King, R., & Sternberg, M. (1994). Mutagenesis: ILP experiments in a non-determinate biological domain. In Wrobel, S. (Ed.), Proceedings of the Fourth International Inductive Logic Programming Workshop . Gesellschaft fur Mathematik und Datenverarbeitung MBH. GMD-Studien Nr 237.
  21. Srinivasan, A. (2001). The ALEPH manual. Machine Learning at the Computing Laboratory, Oxford University .
  22. Srinivasan, A., Muggleton, S. H., & King, R. D. (1995). Comparing the use of background knowledge by inductive logic programming systems. In Proceedings of the 5th International Workshop on Inductive Logic Programming , pp. 199-230. Department of Computer Science, Katholieke Universiteit Leuven.

Cited by

Sorted by citations

See all 83

Metadata

Authors
Cropper, A., & Dumančić, S.
Affiliations
  • University of Oxford (GB)
  • Delft University of Technology (NL)
Venue
Journal of Artificial Intelligence Research
Published
2022-06-15
Volume
74
Pages
765-850
ISSN
1076-9757 (ISSN-L)
OpenAlex
W3065734471
Cited by
83
References
286
Citations by year
2026: 12 · 2025: 31 · 2024: 12 · 2023: 16 · 2022: 5
FWCI
5.14
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Journal of Artificial Intelligence Research (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • Fonds Wetenschappelijk Onderzoek
  • Vlaamse regering
  • Engineering and Physical Sciences Research Council (EP/V040340/1)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Logic, Reasoning, and Knowledge
  • Physical Sciences › Computer Science › Information Systems › Data Mining Algorithms and Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
Keywords
inductive logic programming, ALEPH, tilde

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-19, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-19 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review