Cited by 25Explainable Artificial Intelligence (XAI)Logic, Reasoning, and KnowledgeAI-based Problem Solving and Planning

A Critical Review of Inductive Logic Programming Techniques for Explainable AI

Open access
Zhang, Zheng, Yilmaz, Levent, Liu, Bo·2024·IEEE Transactions on Neural Networks and Learning Systems

Abstract

Despite recent advances in modern machine learning algorithms, the opaqueness of their underlying mechanisms continues to be an obstacle in adoption. To instill confidence and trust in artificial intelligence (AI) systems, explainable AI (XAI) has emerged as a response to improve modern machine learning algorithms' explainability. Inductive logic programming (ILP), a subfield of symbolic AI, plays a promising role in generating interpretable explanations because of its intuitive logic-driven framework. ILP effectively leverages abductive reasoning to generate explainable first-order clausal theories from examples and background knowledge. However, several challenges in developing methods inspired by ILP need to be addressed for their successful application in practice. For example, the existing ILP systems often have a vast solution space, and the induced solutions are very sensitive to noises and disturbances. This survey paper summarizes the recent advances in ILP and a discussion of statistical relational learning (SRL) and neural-symbolic algorithms, which offer synergistic views to ILP. Following a critical review of the recent advances, we delineate observed challenges and highlight potential avenues of further ILP-motivated research toward developing self-explanatory AI systems.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/tnnls.2023.3246980

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • ILP can get results rivaling deep learning models while being inherently interpretable and explainable.

    “ILP can get results rivaling deep learning models while being inherently interpretable and explainable [19].”

    From Autonomous Task Completion Based on Goal-directed Answer Set Programming (2025) · cites this paper as [19]

    The Introduction says ILP’s symbolic hypotheses are inspectable by humans and provide transparency, supporting the interpretability and explainability part. The Benchmark ILP problem section reports high accuracy for several ILP systems on selected tasks, but compares ILP systems with one another, not with deep-learning models. The claim that ILP can rival deep learning is therefore not established by these passages. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 114 · Cited by 25

Research works printed but not linked: 100

  1. A. Adadi and M. Berrada, "Peeking inside the black-box: a survey on explainable artificial intelligence (xai)," IEEE access, vol. 6, pp. 52138-52160, 2018.
  2. D. Gunning and D. Aha, "Darpa's explainable artificial intelli- gence (xai) program," AI Magazine, vol. 40, no. 2, pp. 44-58, 2019.
  3. J. Furnkranz, T. Kliegr, and H. Paulheim, "On cognitive pref- erences and the plausibility of rule-based models," Machine Learning, pp. 1-46, 2019.
  4. R. Calegari, G. Ciatto, E. Denti, and A. Omicini, "Logic- based technologies for intelligent systems: State of the art and perspectives," Information, vol. 11, no. 3, p. 167, 2020.
  5. U. Schmid, "Inductive programming as approach to compre- hensible machine learning," in Proceedings of the 7th workshop on dynamics of knowledge and belief (DKB-2018) and the 6th workshop KI & Kognition (KIK-2018), co-located with 41st German conference on artificial intelligence, vol. 2194, 2018.
  6. A. Cropper and S. Dumancic, "Learning large logic programs by going beyond entailment," arXiv preprint arXiv:2004.09855, 2020. · arXiv
  7. S. Muggleton, D. Lin, N. Pahlavi, and A. Tamaddoni-Nezhad, "Meta-interpretive learning: application to grammatical infer- ence," Machine learning, vol. 94, no. 1, pp. 25-49, 2014.
  8. L. De Raedt and K. Kersting, "Probabilistic inductive logic programming," in Probabilistic Inductive Logic Programming, pp. 1-27, Springer, 2008.
  9. R. Evans and E. Grefenstette, "Learning explanatory rules from noisy data," Journal of Artificial Intelligence Research, vol. 61, pp. 1-64, 2018.
  10. S. Muggleton, "Inductive logic programming," New generation computing, vol. 8, no. 4, pp. 295-318, 1991.
  11. E. H. Shortliffe and B. G. Buchanan, "A model of inexact reasoning in medicine," Mathematical biosciences, vol. 23, no. 3-4, pp. 351-379, 1975.
  12. M. S. Fox and J. McDermott, "The role of databases in knowledge-based systems," in On Knowledge Base Management Systems, pp. 407-430, Springer, 1986.
  13. S. Slocombe, K. Moore, and M. Zelouf, "Engineering expert system applications," in Annual Conference of the BCS Spe- cialist Group on Expert Systems, 1986.
  14. J. Hayes-Michie, "News from brainware," Pragmatica, vol. 1, pp. 10-11, 1990.
  15. E. Y. Shapiro, "Algorithmic program diagnosis," in Proceedings of the 9th ACM SIGPLAN-SIGACT symposium on Principles of programming languages, pp. 299-308, ACM, 1982.
  16. C. Sammut and R. B. Banerji, "Learning concepts by asking questions," Machine learning: An artificial intelligence ap- proach, vol. 2, pp. 167-192, 1986.
  17. S. Muggleton and W. Buntine, "Machine invention of first- order predicates by inverting resolution," in Machine Learning Proceedings 1988, pp. 339-352, Elsevier, 1988.
  18. L. Muggleton, S.and De Raedt, D. Poole, I. Bratko, P. Flach, K. Inoue, and A. Srinivasan, "ILP turns 20," Machine learning, vol. 86, no. 1, pp. 3-23, 2012.
  19. J.-U. Kietz, "Some lower bounds for the computational com- plexity of inductive logic programming," in European Confer- ence on Machine Learning, pp. 115-123, Springer, 1993.
  20. S.-H. Nienhuys-Cheng and R. De Wolf, Foundations of induc- tive logic programming, vol. 1228. Springer Science & Business Media, 1997.
  21. H.-D. Ebbinghaus, J. Flum, and W. Thomas, Mathematical logic. Springer Science & Business Media, 2013.
  22. S. Muggleton and L. De Raedt, "Inductive logic programming: Theory and methods," The Journal of Logic Programming, vol. 19, pp. 629-679, 1994.
  23. L. De Raedt and M. Bruynooghe, "CLINT: a multi-strategy interactive concept-learner and theory revision system," in Pro- ceedings of the Multi-Strategy Learning Workshop, Virginia, pp. 175-191, 1991.
  24. S. Muggleton, "Inverse entailment and progol," New generation computing, vol. 13, no. 3-4, pp. 245-286, 1995.
  25. N. Lavrač, S. Džeroski, and M. Grobelnik, "Learning nonrecur- sive definitions of relations with LINUS," in European Working Session on Learning, pp. 265-281, Springer, 1991.
  26. P. Idestam-Almquist, "Generalization under implication by using or-introduction," in European Conference on Machine Learning, pp. 56-64, Springer, 1993.
  27. P. Idestam-Almquist, "Learning missing clauses by inverse resolution.," in FGCS, pp. 610-617, 1992.
  28. L. De Raedt, A. Kimmig, and H. Toivonen, "Problog: A probabilistic prolog and its application in link discovery.," in IJCAI, vol. 7, pp. 2462-2467, Hyderabad, 2007.
  29. R. Manhaeve, S. Dumancic, A. Kimmig, T. Demeester, and L. De Raedt, "Deepproblog: Neural probabilistic logic program- ming," Advances in Neural Information Processing Systems, vol. 31, pp. 3749-3759, 2018.
  30. L. De Raedt and S. Džeroski, "First-order jk-clausal theories are PAC-learnable," Artificial Intelligence, vol. 70, no. 1-2, pp. 375- 392, 1994.
  31. G. D. Plotkin, "A note on inductive generalization," Machine intelligence, vol. 5, no. 1, pp. 153-163, 1970.
  32. D. Koller and A. Pfeffer, "Learning probabilities for noisy first- order rules," in IJCAI, pp. 1316-1323, 1997.
  33. L. Getoor, N. Friedman, D. Koller, and B. Taskar, "Learning probabilistic models of link structure," Journal of Machine Learning Research, vol. 3, no. Dec, pp. 679-707, 2002.
  34. B. Taskar, P. Abbeel, and D. Koller, "Discriminative probabilis- tic models for relational data," arXiv preprint arXiv:1301.0604, 2012. · arXiv
  35. O. B. H. Langseth and T. Nielsen, "Structural learning in object oriented domains," in Proceedings of the Fourteenth Florida Artificial Intelligence Research Society Conference, pp. 340- 344, 2001.
  36. J. Neville and D. Jensen, "Dependency networks for relational data," in Fourth IEEE International Conference on Data Mining (ICDM'04), pp. 170-177, IEEE, 2004.
  37. F. Riguzzi, "Learning logic programs with annotated disjunc- tions," in International Conference on Inductive Logic Program- ming, pp. 270-287, Springer, 2004.
  38. D. Poole, "The independent choice logic for modelling multiple agents under uncertainty," Artificial intelligence, vol. 94, no. 1- 2, pp. 7-56, 1997.
  39. T. Sato and Y. Kameya, "PRISM: a language for symbolic- statistical modeling," in IJCAI, vol. 97, pp. 1330-1339, 1997.
  40. S. Muggleton et al., "Stochastic logic programs," Advances in inductive logic programming, vol. 32, pp. 254-264, 1996.
  41. A. Stolcke and S. Omohundro, "Hidden markov model in- duction by bayesian model merging," in Advances in neural information processing systems, pp. 11-18, 1993.
  42. C. R. Anderson, P. Domingos, and D. S. Weld, "Relational markov models and their application to adaptive web naviga- tion," in Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 143- 152, 2002.
  43. K. Kersting, L. De Raedt, and T. Raiko, "Logical hidden markov models," Journal of Artificial Intelligence Research, vol. 25, pp. 425-456, 2006.
  44. J. Cussens, "Parameter estimation in stochastic logic programs," Machine Learning, vol. 44, no. 3, pp. 245-271, 2001.
  45. S. Muggleton, "Learning stochastic logic programs," Electron. Trans. Artif. Intell., vol. 4, no. B, pp. 141-153, 2000.
  46. Y. Kameya, T. Sato, and N.-F. Zhou, "Yet more efficient EM learning for parameterized logic programs by inter-goal sharing," in ECAI, vol. 16, p. 490, Citeseer, 2004.
  47. T. Sato and Y. Kameya, "Parameter learning of logic programs for symbolic-statistical modeling," Journal of Artificial Intelli- gence Research, vol. 15, pp. 391-454, 2001.
  48. L. De Raedt and I. Thon, "Probabilistic rule learning," in Inter- national Conference on Inductive Logic Programming, pp. 47- 58, Springer, 2010.
  49. W. Y. Wang, K. Mazaitis, and W. W. Cohen, "Programming with personalized pagerank: a locally groundable first-order probabilistic logic," in Proceedings of the 22nd ACM interna- tional conference on Information & Knowledge Management, pp. 2129-2138, ACM, 2013.
  50. E. Bellodi and F. Riguzzi, "Learning the structure of probabilis- tic logic programs," in International Conference on Inductive Logic Programming, pp. 61-75, Springer, 2011.
  51. D. Fierens, G. Van den Broeck, J. Renkens, D. Shterionov, B. Gutmann, I. Thon, G. Janssens, and L. De Raedt, "Inference and learning in probabilistic logic programs using weighted boolean formulas," Theory and Practice of Logic Programming, vol. 15, no. 3, pp. 358-401, 2015.
  52. E. Bellodi and F. Riguzzi, "Structure learning of probabilistic logic programs by searching the clause space," Theory and Practice of Logic Programming, vol. 15, no. 2, pp. 169-212, 2015.
  53. A. Cropper and S. Muggleton, "Logical minimisation of meta- rules within meta-interpretive learning," in Inductive Logic Programming, pp. 62-75, Springer, 2015.
  54. R. Morel, A. Cropper, and C.-H. L. Ong, "Typed meta- interpretive learning of logic programs," in European Confer- ence on Logics in Artificial Intelligence, pp. 198-213, Springer, 2019.
  55. J. Devlin, J. Uesato, S. Bhupatiraju, R. Singh, A.-R. Mohamed, and P. Kohli, "Robustfill: Neural program learning under noisy i/o," in Proceedings of the 34th International Conference on Machine Learning-Volume 70, pp. 990-998, JMLR. org, 2017.
  56. M. Bošnjak, T. Rocktäschel, J. Naradowsky, and S. Riedel, "Pro- gramming with a differentiable forth interpreter," in Proceedings of the 34th International Conference on Machine Learning- Volume 70, pp. 547-556, JMLR. org, 2017.
  57. T. Rocktäschel and S. Riedel, "End-to-end differentiable prov- ing," in Advances in Neural Information Processing Systems, pp. 3788-3800, 2017.
  58. A. Payani and F. Fekri, "Inductive logic programming via differentiable deep neural logic networks," arXiv preprint arXiv:1906.03523, 2019. · arXiv
  59. R. A. Rossi, L. K. McDowell, D. W. Aha, and J. Neville, "Transforming graph data for statistical relational learning," Journal of Artificial Intelligence Research, vol. 45, pp. 363- 441, 2012.
  60. D. Koller, "Probabilistic relational models," in International Conference on Inductive Logic Programming, pp. 3-13, Springer, 1999.
  61. J. Chen, S. Muggleton, and J. Santos, "Learning probabilistic logic models from probabilistic examples," Machine learning, vol. 73, no. 1, pp. 55-85, 2008.
  62. M. Jaeger, "Relational bayesian networks," in Proceedings of the Thirteenth conference on Uncertainty in artificial intelligence, pp. 266-273, Morgan Kaufmann Publishers Inc., 1997.
  63. B. Taskar, P. Abbeel, M.-F. Wong, and D. Koller, "Relational markov networks," Introduction to statistical relational learn- ing, pp. 175-200, 2007.
  64. J. Neville and D. Jensen, "Relational dependency networks," Journal of Machine Learning Research, vol. 8, no. Mar, pp. 653-692, 2007.
  65. D. Poole, "Probabilistic horn abduction and bayesian networks," Artificial intelligence, vol. 64, no. 1, pp. 81-129, 1993.
  66. M. Richardson and P. Domingos, "Markov logic networks," Machine learning, vol. 62, no. 1-2, pp. 107-136, 2006.
  67. A. d. Garcez and L. C. Lamb, "Neurosymbolic ai: The 3rd wave," arXiv preprint arXiv:2012.05876, 2020. · arXiv
  68. G. Lample and F. Charton, "Deep learning for symbolic math- ematics," arXiv preprint arXiv:1912.01412, 2019. · arXiv
  69. J. Mao, C. Gan, P. Kohli, J. B. Tenenbaum, and J. Wu, "The neuro-symbolic concept learner: Interpreting scenes, words, and sentences from natural supervision," arXiv preprint arXiv:1904.12584, 2019. · arXiv
  70. P. Minervini, S. Riedel, P. Stenetorp, E. Grefenstette, and T. Rocktäschel, "Learning reasoning strategies in end-to-end differentiable proving," in International Conference on Machine Learning, pp. 6938-6949, PMLR, 2020.
  71. D. Lyu, F. Yang, B. Liu, and S. Gustafson, "Sdrl: interpretable and data-efficient deep reinforcement learning leveraging sym- bolic planning," in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 2970-2977, 2019.
  72. X. Chen, C. Liang, A. W. Yu, D. Song, and D. Zhou, "Com- positional generalization via neural-symbolic stack machines," arXiv preprint arXiv:2008.06662, 2020. · arXiv
  73. Y. Xie, Z. Xu, M. S. Kankanhalli, K. S. Meel, and H. Soh, "Embedding symbolic knowledge into deep networks," arXiv preprint arXiv:1909.01161, 2019. · arXiv
  74. J. Jiang and S. Ahn, "Generative neurosymbolic machines," arXiv preprint arXiv:2010.12152, 2020. · arXiv
  75. K. Teru, E. Denis, and W. Hamilton, "Inductive relation pre- diction by subgraph reasoning," in International Conference on Machine Learning, pp. 9448-9457, PMLR, 2020.
  76. W. W. Cohen, H. Sun, R. A. Hofer, and M. Siegler, "Scalable neural methods for reasoning with a symbolic knowledge base," arXiv preprint arXiv:2002.06115, 2020. · arXiv
  77. X. Chen, C. Liang, A. W. Yu, D. Zhou, D. Song, and Q. V. Le, "Neural symbolic reader: Scalable integration of distributed and symbolic representations for reading comprehension," in International Conference on Learning Representations, 2019.
  78. R. Trivedi, H. Dai, Y. Wang, and L. Song, "Know-evolve: Deep temporal reasoning for dynamic knowledge graphs," in international conference on machine learning, pp. 3462-3471, PMLR, 2017.
  79. M. Allamanis, P. Chanthirasegaran, P. Kohli, and C. Sutton, "Learning continuous semantic representations of symbolic ex- pressions," in International Conference on Machine Learning, pp. 80-88, PMLR, 2017.
  80. B. Dhingra, M. Zaheer, V. Balachandran, G. Neubig, R. Salakhutdinov, and W. W. Cohen, "Differentiable rea- soning over a virtual knowledge base," arXiv preprint arXiv:2002.10640, 2020. · arXiv
  81. L. De Raedt, S. Dumančić, R. Manhaeve, and G. Marra, "From statistical relational to neuro-symbolic artificial intelligence," arXiv preprint arXiv:2003.08316, 2020. · arXiv
  82. J. Xu, Z. Zhang, T. Friedman, Y. Liang, and G. Broeck, "A semantic loss function for deep learning with symbolic knowledge," in International conference on machine learning, pp. 5502-5511, PMLR, 2018.
  83. H. Dong, J. Mao, T. Lin, C. Wang, L. Li, and D. Zhou, "Neural logic machines," arXiv preprint arXiv:1904.11694, 2019. · arXiv
  84. L. Serafini and A. d. Garcez, "Logic tensor networks: Deep learning and logical reasoning from data and knowledge," arXiv preprint arXiv:1606.04422, 2016. · arXiv
  85. G. Marra, M. Diligenti, F. Giannini, M. Gori, and M. Maggini, "Relational neural machines," arXiv preprint arXiv:2002.02193, 2020. · arXiv
  86. L. Weber, P. Minervini, J. Münchmeyer, U. Leser, and T. Rocktäschel, "Nlprolog: Reasoning with weak unification for question answering in natural language," arXiv preprint arXiv:1906.06187, 2019. · arXiv
  87. L. De Raedt and A. Kimmig, "Probabilistic (logic) programming concepts," Machine Learning, vol. 100, no. 1, pp. 5-47, 2015.
  88. S. H. Bach, M. Broecheler, B. Huang, and L. Getoor, "Hinge- loss markov random fields and probabilistic soft logic," arXiv preprint arXiv:1505.04406, 2015. · arXiv
  89. P. Larrañaga and S. Moral, "Probabilistic graphical models in artificial intelligence," Applied soft computing, vol. 11, no. 2, pp. 1511-1528, 2011.
  90. M. Ribera and A. Lapedriza, "Can we do better explanations? a proposal of user-centered explainable ai.," in IUI Workshops, 2019.
  91. L. Yilmaz and B. Liu, "Model credibility revisited: Concepts and considerations for appropriate trust," Journal of Simulation, pp. 1-14, 2020.
  92. N. Díaz-Rodríguez, A. Lamas, J. Sanchez, G. Franchi, I. Don- adello, S. Tabik, D. Filliat, P. Cruz, R. Montes, and F. Herrera, "Explainable neural-symbolic learning (x-nesyl) methodology to fuse deep learning representations with expert knowledge graphs: The monumai cultural heritage use case," Information Fusion, vol. 79, pp. 58-83, 2022.
  93. A. Campagner and F. Cabitza, "Back to the feature: A neural- symbolic perspective on explainable ai," in International Cross- Domain Conference for Machine Learning and Knowledge Extraction, pp. 39-55, Springer, 2020.
  94. A. Bennetot, J.-L. Laurent, R. Chatila, and N. Díaz-Rodríguez, "Towards explainable neural-symbolic visual reasoning," arXiv preprint arXiv:1909.09065, 2019. · arXiv
  95. J.-C. Fernandez, L. Mounier, and C. Pachon, "A model-based approach for robustness testing," in IFIP International Con- ference on Testing of Communicating Systems, pp. 333-348, Springer, 2005.
  96. A. Cropper, S. Dumančić, and S. H. Muggleton, "Turning 30: New ideas in inductive logic programming," arXiv preprint arXiv:2002.11002, 2020. · arXiv
  97. H. Watanabe and S. Muggleton, "Can ILP be applied to large datasets?," in International Conference on Inductive Logic Pro- gramming, pp. 249-256, Springer, 2009.
  98. L. Ai, S. H. Muggleton, C. Hocquette, M. Gromowski, and U. Schmid, "Beneficial and harmful explanatory machine learn- ing," arXiv preprint arXiv:2009.06410, 2020. · arXiv
  99. Levent Yilmaz is the Alumni Distinguished Professor of Computer Science and Software Engineering at Auburn University with a courtesy appointment in Industrial and Sys- tems Engineering. He holds M.S. and Ph.D. degrees in Computer Science from Virginia Tech. His research interests are in theory and methodology of modeling and simulation, agent-directed simulation, cognitive systems, and model-driven science and engineering for complex adaptive systems. He is the former Editor-in-Chief of Simulation: Transactions of the Society for Mod- eling and Simulation International and the founding organizer and general chair of the Agent-Directed Simulation Conference series.
  100. Bo Liu is a Tenure-Track Assistant Professor with the Department of Computer Science and Software Engineering, Auburn Univer- sity, Auburn, AL, USA. He received the Ph.D. degree from the University of Mas- sachusetts Amherst, Amherst, MA, USA, in 2015. He has over 30 publications on several notable venues. He is the recipient of the UAI'2015 Facebook best student paper award and the Amazon research award in 2018. He is an Associate Editor of IEEE Transactions on Neural Networks and Learning Systems (IEEE-TNN), a senior member of IEEE, and a member of AAAI, ACM, and INFORMS.

Authors
Zhang, Z., Yilmaz, L., & Liu, B.
Affiliations
  • Auburn University (US)
  • Department of Computer Science and Software Engineering, Auburn University, Auburn, AL, USA
Venue
IEEE Transactions on Neural Networks and Learning Systems
Published
2024-08
Volume
35
Issue
8
Pages
10220-10236
ISSN
  • 2162-237X (ISSN-L)
  • 2162-2388
OpenAlex
W4362602165
Cited by
25
References
114
Citations by year
2026: 9 · 2025: 9 · 2024: 5 · 2023: 2
FWCI
2.21
Type
journal article
Language
en
Open access
Locations
  • IEEE Transactions on Neural Networks and Learning Systems (publishedVersion)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Computer Science › Artificial Intelligence › Logic, Reasoning, and Knowledge
  • Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
Keywords
inductive logic programming, explainable AI, abductive reasoning, statistical relational learning, neuro-symbolic learning, interpretable explanations, solution space, noise sensitivity

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review