CoChat
Sign Up
Cited by 6Explainable Artificial Intelligence (XAI)Advanced Neural Network ApplicationsLogic, Reasoning, and Knowledge

The DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.

EXplainable Neural-Symbolic Learning (X-NeSyL) methodology to fuse deep learning representations with expert knowledge graphs: The MonuMAI cultural heritage use case

Open access
Díaz-Rodríguez, Natalia, Lamas, Alberto, Sanchez, Jules, Franchi, Gianni, Donadello, Ivan, Tabik, Siham, Filliat, David, Cruz, Policarpo, Montes, Rosana, Herrera, Francisco·2022·Information Fusion
DOI 10.1016/j.inffus.2021.09.022
Open this paper in CoChat — free accountOpen PDF

Abstract

The latest Deep Learning (DL) models for detection and classification have achieved an unprecedented performance over classical machine learning algorithms. However, DL models are black-box methods hard to debug, interpret, and certify. DL alone cannot provide explanations that can be validated by a non technical audience such as end-users or domain experts. In contrast, symbolic AI systems that convert concepts into rules or symbols – such as knowledge graphs – are easier to explain. However, they present lower generalization and scaling capabilities. A very important challenge is to fuse DL representations with expert knowledge. One way to address this challenge, as well as the performance-explainability trade-off is by leveraging the best of both streams without obviating domain expert knowledge. In this paper, we tackle such problem by considering the symbolic knowledge is expressed in form of a domain expert knowledge graph. We present the eXplainable Neural-symbolic learning (X-NeSyL) methodology, designed to learn both symbolic and deep representations, together with an explainability metric to assess the level of alignment of machine and human expert explanations. The ultimate objective is to fuse DL representations with expert domain knowledge during the learning process so it serves as a sound basis for explainability. In particular, X-NeSyL methodology involves the concrete use of two notions of explanation, both at inference and training time respectively: (1) EXPLANet: Expert-aligned eXplainable Part-based cLAssifier NETwork Architecture, a compositional convolutional neural network that makes use of symbolic representations, and (2) SHAP-Backprop, an explainable AI-informed training procedure that corrects and guides the DL process to align with such symbolic representations in form of knowledge graphs. We showcase X-NeSyL methodology using MonuMAI dataset for monument facade image classification, and demonstrate that with our approach, it is possible to improve explainability at the same time as performance.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.inffus.2021.09.022

Cites

11 of 11 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Natural language explanations, quantitative measures of explanation quality, and models with formal symbolic basis (neural-symbolic learning and reasoning) are key for explanations to be understood by non-expert audience.

    “Natural language explanations [27,67], quantitative measures of the quality of explanations [68,69], and models that support their learning process with formal symbolic basis such as language, rules, compositional relationships or knowledge graphs (neural-symbolic learning and reasoning [66]) are key for explanations to be understood by non-expert audience.”

    From Connecting the dots in trustworthy Artificial Intelligence: From AI principles, ethics, and key requirements to responsible AI systems and regulation (2023) · cites this paper as [66]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 55

  1. Hinton, 2006, Reducing the dimensionality of data with neural networks, Science, 313, 504, 10.1126/science.1127647 · DOI · Paper page
  2. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin, Attention is all you need, in: Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017.
  3. Guidotti, 2018, A survey of methods for explaining black box models, ACM Comput. Surv., 51, 1, 10.1145/3236009 · DOI · Paper page
  4. Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, Dhruv Batra, Grad-CAM: Visual explanations from deep networks via gradient-based localization, in: Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 618–626. · DOI · Paper page
  5. Olah, 2017, Feature visualization, Distill, 2, 10.23915/distill.00007 · DOI · Paper page
  6. Sarthak Jain, Byron C. Wallace, Attention is not explanation, in: Annual Conference of the North American Chapter of the Association for Computational Linguistics, NAACL-HLT, 2019.
  7. He, 2020, Sample-efficient deep learning for COVID-19 diagnosis based on ct scans, medRxiv
  8. Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, Been Kim, Sanity checks for saliency maps, in: Proceedings of the International Conference on Neural Information Processing Systems, 2018, pp. 9505–9515.
  9. Pisoni, 2021, Human-centred artificial intelligence for designing accessible cultural heritage, Appl. Sci., 11, 870, 10.3390/app11020870 · DOI · Paper page
  10. Natalia Díaz-Rodríguez, Galena Pisoni, Accessible cultural heritage through explainable artificial intelligence, in: Adjunct Publication of the 28th ACM Conference on User Modeling, Adaptation and Personalization, 2020, pp. 317–324. · DOI
  11. d’Avila Garcez, 2019, Neural-symbolic computing: An effective methodology for principled integration of machine learning and reasoning, J. Appl. Log. IfCoLog J. Log. Appl., 6, 611
  12. Adrien Bennetot, Jean-Luc Laurent, Raja Chatila, Natalia Díaz-Rodríguez, Towards explainable neural-symbolic visual reasoning, in: Proceedings of the Neural-Symbolic Learning and Reasoning Workshop, NeSy-2019 At International Joint Conference on Artificial Intelligence, IJCAI, Macau, China, 2019.
  13. Lamas, 2020, MonuMAI: Dataset, deep learning pipeline and citizen science based app for monumental heritage taxonomy and classification, Neurocomputing, 420, 266, 10.1016/j.neucom.2020.09.041 · DOI
  14. Lesort, 2020, Continual learning for robotics: Definition, framework, learning strategies, opportunities and challenges, Inf. Fusion, 58, 52, 10.1016/j.inffus.2019.12.004 · DOI
  15. Marco Tulio Ribeiro, Sameer Singh, Carlos Guestrin, “Why should I trust you?” Explaining the predictions of any classifier, in: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016, pp. 1135–1144. · DOI · Paper page
  16. Ruth C. Fong, Andrea Vedaldi, Interpretable explanations of black boxes by meaningful perturbation, in: IEEE International Conference on Computer Vision, 2017, pp. 3429–3437. · DOI · Paper page
  17. Shrikumar, 2017, Learning important features through propagating activation differences, vol. 70, 3145
  18. Bolei Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, Antonio Torralba, Learning deep features for discriminative localization, in: Proceedings of the IEEE Conference on Computer Vision and Rattern Recognition, 2016, pp. 2921–2929. · DOI · Paper page
  19. Sundararajan, 2017, Axiomatic attribution for deep networks, vol. 70, 3319
  20. Austin Stone, Huayan Wang, Michael Stark, Yi Liu, D. Scott Phoenix, Dileep George, Teaching compositionality to CNNs, in: Proceedings of the IEEE Conference on Computer Vision and Rattern Recognition, 2017, pp. 5058–5067. · DOI
  21. De Kok, 1999, Object-based classification and applications in the alpine forest environment, Int. Arch. Photogramm. Remote Sens., 32
  22. Bernstein, 2005, Part-based statistical models for object classification and detection, vol. 2, 734
  23. Felzenszwalb, 2009, Object detection with discriminatively trained part-based models, IEEE Trans. Pattern Anal. Mach. Intell., 32, 1627, 10.1109/TPAMI.2009.167 · DOI · Paper page
  24. Weifeng Ge, Xiangru Lin, Yizhou Yu, Weakly supervised complementary parts models for fine-grained image classification from the bottom up, in: Proceedings of the IEEE Conference on Computer Vision and Rattern Recognition, 2019, pp. 3034–3043.
  25. Bianchi, 2019, Complementing logical reasoning with sub-symbolic commonsense, vol. 11784, 161
  26. Townsend, 2020, Extracting relational explanations from deep neural networks: A survey from a neural-symbolic perspective, IEEE Trans. Neural Netw. Learn. Syst., 31, 3456, 10.1109/TNNLS.2019.2944672 · DOI
  27. Serafini, 2016, Learning and reasoning with logic tensor networks, vol. 10037, 334
  28. Diligenti, 2016, Learning efficiently in semantic based regularization, vol. 9852, 33
  29. Ivan Donadello, Luciano Serafini, Artur D’Avila Garcez, Logic tensor networks for semantic image interpretation, in: Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI, pp. 1596–1602. · DOI · Paper page
  30. Saccà, 2014, Improved multi-level protein-protein interaction prediction with semantic-based regularization, BMC Bioinformatics, 15, 103, 10.1186/1471-2105-15-103 · DOI
  31. van Krieken, 2019, Semi-supervised learning using differentiable reasoning, J. Appl. Log. IfCoLog J. Log. Appl., 6, 633
  32. Xu, 2018, A semantic loss function for deep learning with symbolic knowledge, vol. 80, 5498
  33. Towell, 1994, Knowledge-based artificial neural networks, Artificial Intelligence, 70, 119, 10.1016/0004-3702(94)90105-8 · DOI
  34. d’Avila Garcez, 1999, The connectionist inductive learning and logic programming system, Appl. Intell., 11, 59, 10.1023/A:1008328630915 · DOI
  35. Marra, 2020, Relational neural machines, vol. 325, 1340
  36. Daniele, 2020, Neural networks enhancement through prior logical knowledge, CoRR, abs/2009.06087 · arXiv
  37. Manhaeve, 2018, DeepProbLog: Neural probabilistic logic programming, vol. 31, 3749
  38. Luc De Raedt, Angelika Kimmig, Hannu Toivonen, ProbLog: A Probabilistic Prolog and Its Application in Link Discovery, in: Proceedings of the 20th International Joint Conference on Artifical Intelligence, 2007, pp. 2462–2467.
  39. Honghua Dong, Jiayuan Mao, Tian Lin, Chong Wang, Lihong Li, Denny Zhou, Neural logic machines, in: ICLR, 2019.
  40. Yuan Yang, Le Song, Learn to explain efficiently via neural logic inductive learning, in: ICLR, 2020.
  41. Ebrahimi, 2021, Towards bridging the neuro-symbolic gap: Deep deductive reasoners, Appl. Intell., 1
  42. Lecue, 2020, On the role of knowledge graphs in explainable AI, Semantic Web, 11, 41
  43. Confalonieri, 2021, Using ontologies to enhance human understandability of global post-hoc explanations of black-box models, Artificial Intelligence, 296, 10.1016/j.artint.2021.103471 · DOI
  44. Yann Le Cun, Bernhard Boser, John S. Denker, Donnie Henderson, Richard E. Howard, Wayne Hubbard, Lawrence D. Jackel, Handwritten digit recognition with a back-propagation network, in: Proceedings of the 2nd International Conference on Neural Information Processing Systems, 1989, pp. 396–404.
  45. The Description Logic Handbook: Theory, Implementation and Applications, second ed., Cambridge University Press, 2007.
  46. Xianjie Chen, Roozbeh Mottaghi, Xiaobai Liu, Sanja Fidler, Raquel Urtasun, Alan Yuille, Detect what you can: Detecting and representing objects using holistic models and body parts, in: Proceedings of the IEEE Conference on Computer Vision and Rattern Recognition, 2014, pp. 1971–1978. · DOI · Paper page
  47. Holzinger, 2019, Interactive machine learning: Experimental evidence for the human in the algorithmic loop, Appl. Intell., 49, 2401, 10.1007/s10489-018-1361-5 · DOI
  48. Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, Piotr Dollár, Focal loss for dense object detection, in: Proceedings of the IEEE International Conference on Computer Vision, 2017, pp. 2980–2988.
  49. Sanfeliu, 1983, A distance measure between attributed relational graphs for pattern recognition, IEEE Trans. Syst. Man Cybern., 353, 10.1109/TSMC.1983.6313167 · DOI
  50. Everingham, 2010, The PASCAL visual object classes (VOC) challenge, Int. J. Comput. Vis., 88, 303, 10.1007/s11263-009-0275-4 · DOI · Paper page
  51. Natalia Díaz-Rodríguez, Aki Härmä, Rim Helaoui, Ignacio Huitzil, Fernando Bobillo, Umberto Straccia, Couch potato or gym addict? Semantic lifestyle profiling with wearables and fuzzy knowledge graphs, in: 6th Workshop on Automated Knowledge Base Construction, AKBC@NIPS 2017, Long Beach, California, 2017.
  52. Koji Maruhashi, Masaru Todoriki, Takuya Ohwa, Keisuke Goto, Yu Hasegawa, Hiroya Inakoshi, Hirokazu Anai, Learning multi-way relations via tensor decomposition with neural networks, in: Proceedings of the AAAI Conference on Artificial Intelligence, 2018. · DOI
  53. Getoor, 2005, Link mining: A survey, ACM SIGKDD Explor. Newsl., 7, 3, 10.1145/1117454.1117456 · DOI
  54. Zhao, 2020, An experimental study of state-of-the-art entity alignment approaches, IEEE Trans. Knowl. Data Eng., 1
  55. Donadello, 2016, Integration of numeric and symbolic information for semantic image interpretation, Intell. Artif., 10, 33

Cited by

Sorted by citations

Metadata

Authors
Díaz-Rodríguez, N., Lamas, A., Sanchez, J., Franchi, G., Donadello, I., Tabik, S., Filliat, D., Cruz, P., Montes, R., & Herrera, F.
Affiliations
  • Institut national de recherche en sciences et technologies du numérique (FR)
  • Universidad de Granada (ES)
  • Free University of Bozen-Bolzano (IT)
Corresponding author
Díaz-Rodríguez, Natalia
ORCID
Venue
Information Fusion
Publisher
Elsevier BV
Published
2022-03
Volume
79
Pages
58-83
ISSN
1566-2535 (ISSN-L)
DOI
10.1016/j.inffus.2021.09.022
OpenAlex
W3158835907
Cited by
2
References
11
Citations by year
2023: 2
FWCI
0.18
Type
journal article
Language
en
Open access
Licence
Locations
  • Information Fusion (publishedVersion, cc-by-nc-nd)
  • arXiv (Cornell University) (submittedVersion)
  • HAL (submittedVersion, other-oa)
  • Institutional Repository of the University of Granada (University of Granada) (submittedVersion, cc-by-nc-nd)
  • arXiv (Cornell University)
Funders
Association Nationale de la Recherche et de la Technologie
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Logic, Reasoning, and Knowledge
Keywords
knowledge graphs, expert knowledge, explainability metrics, cultural heritage

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review