Forgetting to Learn Logic Programs
Open accessAbstract
Most program induction approaches require predefined, often hand-engineered, background knowledge (BK). To overcome this limitation, we explore methods to automatically acquire BK through multi-task learning. In this approach, a learner adds learned programs to its BK so that they can be reused to help learn other programs. To improve learning performance, we explore the idea of forgetting, where a learner can additionally remove programs from its BK. We consider forgetting in an inductive logic programming (ILP) setting. We show that forgetting can significantly reduce both the size of the hypothesis space and the sample complexity of an ILP learner. We introduce Forgetgol, a multi-task ILP learner which supports forgetting. We experimentally compare Forgetgol against approaches that either remember or forget everything. Our experimental results show that Forgetgol outperforms the alternative approaches when learning from over 10,000 tasks.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Forgetgol's selective forgetting remains a structural heuristic that does not address the root cause.
“Forgetgol [5] mitigates this through selective forgetting based on usage frequency, but this remains a structural heuristic that does not address the root cause.”
From ADVENT: LLM-Driven Automatic Predicate Invention for ILP (2026) · cites this paper as [5]
Section 4.3 (p. 4) describes selective forgetting by restricting which predicate symbols may appear in a hypothesis, and the statistical method estimates relevance from reuse (p. 5). But the paper does not characterize Forgetgol as a heuristic that fails to address the root cause; the Introduction (p. 1) presents forgetting as a way to address the problem of excessive background knowledge. That final assessment is not supported by this source. (LLM, conf=0.84)
Experimental — accuracy not yet measuredAs BK grows, the system faces the relevance problem: without a mechanism to judge which prior knowledge applies to a new task, accumulation leads to catastrophic remembering, where irrelevant BK degrades both search efficiency and learning performance.
“However, as BK grows, the system faces the relevance problem [1]: without a mechanism to judge which prior knowledge applies to a new task, accumulation leads to catastrophic remembering [5], where irrelevant BK degrades both search efficiency and learning performance.”
From ADVENT: LLM-Driven Automatic Predicate Invention for ILP (2026) · cites this paper as [5]
Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measuredThe limitations of existing PI methods restrict ILP's ability to learn in new domains and hinder its potential for lifelong learning.
“Together, these limitations restrict ILP's ability to learn in new domains and hinder its potential for lifelong learning [5].”
From ADVENT: LLM-Driven Automatic Predicate Invention for ILP (2026) · cites this paper as [5]
The Introduction (p. 1) says existing approaches struggle to learn across many tasks and that overcoming this limitation is necessary for lifelong learning. It also identifies reliance on predefined background knowledge as a limitation, but does not establish that these limitations restrict ILP's ability to learn in new domains specifically. (LLM, conf=0.82)
Experimental — accuracy not yet measuredForgetgol mitigates catastrophic remembering through selective forgetting based on usage frequency.
“Forgetgol [5] mitigates this through selective forgetting based on usage frequency, but this remains a structural heuristic that does not address the root cause.”
From ADVENT: LLM-Driven Automatic Predicate Invention for ILP (2026) · cites this paper as [5]
Related Work (p. 2) identifies catastrophic remembering as the inability to forget knowledge, and §4.3 (pp. 4–5) describes a statistical forgetting method that estimates relevance from a symbol’s relative reuse. But Forgetgol also uses syntactical forgetting, which is not based on usage frequency; §6.1 (p. 7) says the statistical method did not outperform the remember-everything comparator. The claim therefore overstates frequency-based forgetting as the way Forgetgol mitigates the problem. (LLM, conf=0.86)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Caruana, R. (1997) · Machine Learning · doi:10.1023/a:1007379606734 · Cited by 6,636 · see citing works
Valiant, L. G. (1984) · ACM eBooks · doi:10.1145/3749740.3749746 · Cited by 4,211 · see citing works
Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem
McCloskey, M., & Cohen, N. J. (1989) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(08)60536-8 · Cited by 3,770 · see citing works
Building machines that learn and think like people
Lake, B. M., Ullman, T. D., Tenenbaum, J. B., & Gershman, S. J. (2017) · Behavioral and Brain Sciences · doi:10.1017/s0140525x16001837 · Cited by 2,550 · see citing works
Torrey, L. A., & SHAVLIK, J. W. (2010) · IGI Global eBooks · doi:10.4018/978-1-60566-766-9.ch011 · Cited by 1,887 · see citing works
Learning in the Presence of Concept Drift and Hidden Contexts
Widmer, G., & Kubat, M. (1996) · Machine Learning · doi:10.1023/a:1018046501280 · Cited by 1,857 · see citing works
Learning Logical Definitions from Relations
Quinlan, J. R. (1990) · Machine Learning · doi:10.1023/a:1022699322624 · Cited by 1,644 · see citing works
Muggleton, S. (1995) · New Generation Computing · doi:10.1007/bf03037227 · Cited by 1,411 · see citing works
Blumer, A., Ehrenfeucht, A., Haussler, D., & Warmuth, M. K. (1987) · Information Processing Letters · doi:10.1016/0020-0190(87)90114-1 · Cited by 1,054 · see citing works
A Model of Inductive Bias Learning
Baxter, J. (2000) · Cited by 730 · see citing works
DeepCoder: Learning to Write ProgramsPreprint
Balog, M., Gaunt, A. L., Brockschmidt, M., Nowozin, S., & Tarlow, D. (2016) · International Conference on Learning Representations · doi:10.48550/arxiv.1611.01989 · Cited by 641 · see citing works
Learning Explanatory Rules from Noisy Data
Evans, R., & Grefenstette, E. (2018) · Journal of Artificial Intelligence Research · doi:10.1613/jair.5714 · Cited by 426 · see citing works
Unfold/fold Transformation of Logic Programs
Tamaki, H., & Sato, T. (1984) · Cited by 306 · see citing works
(2013) · Cited by 290 · see citing works
Meta-interpretive learning of higher-order dyadic datalog: predicate invention revisited
Muggleton, S. H., Lin, D., & Tamaddoni-Nezhad, A. (2015) · Machine Learning · doi:10.1007/s10994-014-5471-y · Cited by 184 · see citing works
Automatic Methods of Inductive Inference
Plotkin, G. D. (1972) · ERA · Cited by 170 · see citing works
Inductive Learning of Answer Set Programs
Law, M., Russo, A., & Broda, K. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-11558-0_22 · Cited by 133 · see citing works
Learning to Infer Graphics Programs from Hand-Drawn Images
Ellis, K., Ritchie, D., Solar-Lezama, A., & Tenenbaum, J. B. (2018) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 111 · see citing works
Structured machine learning: the next ten years
Dietterich, T. G., Domingos, P., Getoor, L. C., Muggleton, S., & Tadepalli, P. V. (2008) · Machine Learning · doi:10.1007/s10994-008-5079-1 · Cited by 108 · see citing works
Clause Elimination for SAT and QSAT
Heule, M. J. H., Järvisalo, M., Lonsing, F., Seidl, M., & Biere, A. (2015) · Journal of Artificial Intelligence Research · doi:10.1613/jair.4694 · Cited by 69 · see citing works
Bias reformulation for one-shot function induction
Dianhuan, L., Eyal, D., Kevin, E., Joshua, T., & Stephen, M. (2014) · Frontiers in Artificial Intelligence and Applications · doi:10.3233/978-1-61499-419-0-525 · Cited by 53 · see citing works
Learning to Infer Graphics Programs from Hand-Drawn ImagesPreprint
Ellis, K., Ritchie, D., Solar-Lezama, A., & Tenenbaum, J. B. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.09627 · Cited by 49 · see citing works
Learning Libraries of Subroutines for Neurally–Guided Bayesian Program Induction
Ellis, K., Morales, L., Sablé-Meyer, M., Solar-Lezama, A., & Tenenbaum, J. B. (2018) · neural information processing systems · Cited by 44 · see citing works
Bootstrap Learning via Modular Concept Discovery
Dechter, E., Malmaud, J., Adams, R. P., & Tenenbaum, J. B. (2014) · Cited by 43 · see citing works
Learning efficient logic programs
Cropper, A., & Muggleton, S. (2018) · Machine Learning · doi:10.1007/s10994-018-5712-6 · Cited by 33 · see citing works
Self-citationThe Appropriateness of Predicate Invention as Bias Shift Operation in ILP
Stahl, I. (1995) · Machine Learning · doi:10.1007/bf00993476 · Cited by 32 · see citing works
Incremental Learning of Functional Logic Programs
Ferri, C., Hernández‐Orallo, J., & Ramírez-Quintana, M. J. (2001) · Lecture notes in computer science · doi:10.1007/3-540-44716-4_15 · Cited by 31 · see citing works
Playgol: Learning Programs Through Play
Cropper, A. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/841 · Cited by 30 · see citing works
Self-citationAn empirical study of the use of relevance information in inductive logic programming
Srinivasan, A., King, R. D., & Bain, M. (2003) · doi:10.5555/945365.945387 · Cited by 29 · see citing works
Logical reduction of metarules
Cropper, A., & Tourret, S. (2020) · Machine Learning · doi:10.1007/s10994-019-05834-x · Cited by 29 · see citing works
Self-citationLearning Relational Representations with Auto-encoding Logic Programs
Dumancic, S., Guns, T., Meert, W., & Blockeel, H. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/842 · Cited by 27 · see citing works
Bias reformulation for one-shot function induction
Lin, D., Dechter, E., Ellis, K., Tenenbaum, J. B., & Muggleton, S. (2014) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 24 · see citing works
Intentional Forgetting: An Emerging Field in AI and Beyond
Beierle, C., & Timm, I. J. (2018) · Künstliche Intell. · doi:10.1007/s13218-018-00574-x · Cited by 15 · see citing works
Progressive partial memory learning
Maloof, M. A. (1997) · Cited by 15 · see citing works
Dumančić, S., & Blockeel, H. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/226 · Cited by 14 · see citing works
Knowledge acquisition with forgetting: an incremental and developmental setting
Martínez-Plumed, F., Ferri, C., Hernández-Orallo, J., & Ramírez-Quintana, M. J. (2015) · Adaptive Behavior · doi:10.1177/1059712315608675 · Cited by 14 · see citing works
Learning higher-order logic programs
Cropper, A., Morel, R., & Muggleton, S. (2020) · Machine Learning · doi:10.1007/s10994-019-05862-7 · Cited by 11 · cited 1× · see citing works
“We now show the benefits of forgetting in meta-interpretive (MIL) (Muggleton, Lin, and Tamaddoni-Nezhad 2015; Cropper, Morel, and Muggleton 2019), a powerful form of ILP that supports predicate invention (Stahl 1995) and learning recursive programs.”
BackgroundSelf-citationForgetting and compacting data in concept learning
Sablon, G., & Raedt, L. D. (1995) · Lirias · Cited by 2 · see citing works
Flagged first, then by citations
2 of the 40 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Learning programs by learning from failures
Cropper, A., & Morel, R. (2021) · Machine Learning · doi:10.1007/s10994-020-05934-z · Cited by 86 · see citing works
“Finally, because of their symbolic nature, ILP systems naturally support lifelong and transfer learning (Cropper 2020), which is considered essential for human-like AI (Lake et al.”
BackgroundSelf-citationInductive Logic Programming At 30: A New Introduction
Cropper, A., & Dumančić, S. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13507 · Cited by 83 · see citing works
“Empirically, too much irrelevant BK is detrimental to learning performance (Srinivasan et al., 1995, 2003; Cropper, 2020), this also includes irrelevant language biases (Cropper & Tourret, 2020).”
BackgroundInfluentialSelf-citationADVENT: LLM-Driven Automatic Predicate Invention for ILP
Yu, T., Huang, P.-C., Hsu, C., Ku, C.-T., & Kang, Y. (2026) · 2026 IEEE International Conference on Information Reuse and Integration for Data Science (IRI) · doi:10.1109/iri69576.2026.00025
“However, as BK grows, the system faces the relevance problem [1]: without a mechanism to judge which prior knowledge applies to a new task, accumulation leads to catastrophic remembering [5], where irrelevant BK degrades both search efficiency and learning performance.”
- Authors
- Cropper, A.
- Affiliations
- University of Oxford (GB)
- Corresponding author
- Cropper, Andrew
- ORCID
- Cropper, Andrew
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2020-04-03
- Volume
- 34
- Issue
- 04
- Pages
- 3676-3683
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W2988348802
- Cited by
- 0
- References
- 40
- FWCI
- 0.00
- Type
- conference paper
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- www.aaai.org/ (unspecified)
- Locations
- Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Logic, Reasoning, and Knowledge
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Algorithms
- Keywords
- inductive logic programming, forgetting, multi-task learning, program induction, sample complexity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
