On the Integration of Knowledge Graphs into Deep Learning Models for a More Comprehensible AI—Three Challenges for Future Research
Open accessAbstract
Deep learning models contributed to reaching unprecedented results in prediction and classification tasks of Artificial Intelligence (AI) systems. However, alongside this notable progress, they do not provide human-understandable insights on how a specific result was achieved. In contexts where the impact of AI on human life is relevant (e.g., recruitment tools, medical diagnoses, etc.), explainability is not only a desirable property, but it is -or, in some cases, it will be soon-a legal requirement. Most of the available approaches to implement eXplainable Artificial Intelligence (XAI) focus on technical solutions usable only by experts able to manipulate the recursive mathematical functions in deep learning algorithms. A complementary approach is represented by symbolic AI, where symbols are elements of a lingua franca between humans and deep learning. In this context, Knowledge Graphs (KGs) and their underlying semantic technologies are the modern implementation of symbolic AI—while being less flexible and robust to noise compared to deep learning models, KGs are natively developed to be explainable. In this paper, we review the main XAI approaches existing in the literature, underlying their strengths and limitations, and we propose neural-symbolic integration as a cornerstone to design an AI which is closer to non-insiders comprehension. Within such a general direction, we identify three specific challenges for future research—knowledge matching, cross-disciplinary explanations and interactive explanations.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Combining symbolic and subsymbolic approaches into neuro-symbolic systems shows significant promise by exploiting complementary strengths.
“More recently, a combination of symbolic and subsymbolic approaches based on artificial neural networks, variously known as neuro-symbolic or neural-symbolic (henceforth NS) systems, has shown significant promise in exploiting the complementary strengths of individual approaches [20,28].”
From Explainable AI (XAI): Core Ideas, Techniques, and Solutions (2023) · cites this paper as [20]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892 · see citing works
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555 · see citing works
WordNet: a lexical database for English
Miller, G. A. (1995) · Communications of the ACM · doi:10.1145/219717.219748 · Cited by 14,015 · see citing works
Toward principles for the design of ontologies used for knowledge sharing?
Gruber, T. R. (1995) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.1995.1081 · Cited by 7,753 · see citing works
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · see citing works
The mythos of model interpretability
Lipton, Z. C. (2018) · Communications of the ACM · doi:10.1145/3233231 · Cited by 5,427 · see citing works
Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author)
Breiman, L. (2001) · Statistical Science · doi:10.1214/ss/1009213726 · Cited by 4,340 · see citing works
DBpedia - A large-scale, multilingual knowledge base extracted from Wikipedia
Lehmann, J., Isele, R., Jakob, M., Jentzsch, A., Kontokostas, D., Mendes, P. N., Hellmann, S., Morsey, M., van Kleef, P., Auer, S., et al. (2015) · Semantic Web · doi:10.3233/sw-140134 · Cited by 3,737 · see citing works
Traffic Flow Prediction With Big Data: A Deep Learning Approach
Lv, Y., Duan, Y., Kang, W., Li, Z., & Wang, F.-Y. (2014) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2014.2345663 · Cited by 3,000 · see citing works
Linked Data: Evolving the Web into a Global Data Space
Heath, T., & Bizer, C. (2011) · Synthesis lectures on the semantic web · doi:10.2200/s00334ed1v01y201102wbe001 · Cited by 2,029 · see citing works
Understanding Black-box Predictions via Influence Functions
Koh, P. W., & Liang, P. (2017) · arXiv · doi:10.48550/arxiv.1703.04730 · Cited by 1,514 · see citing works
Sanity Checks for Saliency Maps
Adebayo, J., Gilmer, J., Muelly, M., Goodfellow, I., Hardt, M., & Kim, B. (2018) · arXiv · doi:10.48550/arxiv.1810.03292 · Cited by 1,294 · see citing works
DeepTox: toxicity prediction using deep learning
Mayr, A., Klambauer, G., Unterthiner, T., & Hochreiter, S. (2016) · Frontiers in Environmental Science · doi:10.3389/fenvs.2015.00080 · Cited by 1,081 · see citing works
Link prediction using supervised learning
Hasan, M. A., Chaoji, V., Salem, S., et al. (2006) · Cited by 829 · see citing works
Mittelstadt, B., Russell, C., & Wachter, S. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287574 · Cited by 794 · see citing works
Datta, A., Sen, S., & Zick, Y. (2016) · 2016 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2016.42 · Cited by 728 · see citing works
Lepri, B., Oliver, N., Letouzé, E., Pentland, A., & Vinck, P. (2018) · Philosophy & Technology · doi:10.1007/s13347-017-0279-x · Cited by 709 · see citing works
Lee, H., Yune, S., Mansouri, M., Kim, M., Tajmir, S. H., Guerrier, C. E., Ebert, S. A., Pomerantz, S. R., Romero, J. M., Kamalian, S., et al. (2018) · Nature Biomedical Engineering · doi:10.1038/s41551-018-0324-9 · Cited by 441 · see citing works
Heath, T., & Bizer, C. (2011) · Synthesis lectures on data, semantics and knowledge · doi:10.1007/978-3-031-79432-2 · Cited by 435 · see citing works
Towards a Definition of Knowledge Graphs.
Ehrlinger, L., & Wöß, W. (2016) · Cited by 410 · see citing works
What Does Explainable AI Really Mean? A New Conceptualization of Perspectives
Doran, D., Schulz, S., & Besold, T. R. (2017) · arXiv · doi:10.48550/arxiv.1710.00794 · Cited by 333 · see citing works
Towards explainable deep neural networks (xDNN)
Angelov, P., & Soares, E. (2020) · Neural Networks · doi:10.1016/j.neunet.2020.07.010 · Cited by 292 · see citing works
Auditing black-box models for indirect influence
Adler, P., Falk, C., Friedler, S. A., Nix, T., Rybeck, G., Scheidegger, C., Smith, B., & Venkatasubramanian, S. (2018) · Knowledge and Information Systems · doi:10.1007/s10115-017-1116-3 · Cited by 280 · see citing works
Prediction of crime occurrence from multi-modal data using deep learning
Kang, H.-W., & Kang, H.-B. (2017) · PLoS ONE · doi:10.1371/journal.pone.0176244 · Cited by 255 · see citing works
Miller, T., Howe, P., & Sonenberg, L. (2017) · arXiv · doi:10.48550/arxiv.1712.00547 · Cited by 218 · see citing works
Saradhi, V. V., & Palshikar, G. K. (2011) · Expert Systems with Applications · doi:10.1016/j.eswa.2010.07.134 · Cited by 188 · see citing works
On The Role of Knowledge Graphs in Explainable AI
Lécué, F. (2020) · PROFILES/SEMEX@ISWC · doi:10.3233/sw-190374 · Cited by 165 · see citing works
Haugeland, J. (1989) · The MIT Press eBooks · doi:10.7551/mitpress/1170.001.0001 · Cited by 144 · see citing works
The Bayesian Case Model: A Generative Approach for Case-Based Reasoning and Prototype ClassificationPreprint
Kim, B., Rudin, C. D., & Shah, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1503.01161 · Cited by 116 · see citing works
Explainable Deep Neural Networks for Multivariate Time Series Predictions
Assaf, R., & Schumann, A. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/932 · Cited by 103 · see citing works
Explicit Knowledge-based Reasoning for Visual Question AnsweringPreprint
Wang, P., Wu, Q., Shen, C., et al. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.02570 · Cited by 96 · see citing works
Semantic Web for the Legal Domain: The next step
Casanovas, P., Palmirani, M., Peroni, S., van Engers, T., & Vitali, F. (2016) · Semantic Web · doi:10.3233/sw-160224 · Cited by 73 · see citing works
Interpretable Multimodal Retrieval for Fashion Products
Liao, L., He, X., Zhao, B., Ngo, C., & Chua, T. (2018) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3240508.3240646 · Cited by 71 · see citing works
Interpreting Black-Box Classifiers Using Instance-Level Visual Explanations
Tamagnini, P., Krause, J., Dasgupta, A., & Bertini, E. (2017) · Workshop on Human-In-the-Loop Data Analytics (HILDA) · doi:10.1145/3077257.3077260 · Cited by 70 · see citing works
The knowledge graph as the default data model for learning on heterogeneous knowledge
Wilcke, X., Bloem, P., & de Boer, V. (2017) · Data Science · doi:10.3233/ds-170007 · Cited by 68 · see citing works
The Truth is in There : Rule Extraction from Opaque Models Using Genetic Programming
Johansson, U., König, R., & Niklasson, L. (2004) · Cited by 56 · see citing works
Marrying Uncertainty and Time in Knowledge Graphs
Chekol, M. W., Pirrò, G., Schoenfisch, J., & Stuckenschmidt, H. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10495 · Cited by 51 · see citing works
Bio2RDF release 3: a larger connected network of linked data for the life sciences
Dumontier, M. J., Callahan, A., Cruz-Toledo, J., et al. (2014) · Research Publications (Maastricht University) · Cited by 50 · see citing works
Assessing the Local Interpretability of Machine Learning ModelsPreprint
Slack, D., Friedler, S. A., Scheidegger, C., & Roy, C. D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.03501 · Cited by 50 · see citing works
Data Science and symbolic AI: Synergies, challenges and opportunities
Hoehndorf, R., & Queralt-Rosiñach, N. (2017) · Data Science · doi:10.3233/ds-170004 · Cited by 48 · see citing works
Semantic Web Technologies for Explainable Machine Learning Models: A Literature Review.
Seeliger, A., Pfaff, M., & Krcmar, H. (2019) · Cited by 46 · see citing works
Choose Your Neuron: Incorporating Domain Knowledge Through Neuron-Importance
Selvaraju, R. R., Chattopadhyay, P., Elhoseiny, M., Sharma, T., Batra, D., Parikh, D., & Lee, S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01261-8_32 · Cited by 43 · see citing works
Deep learning, dark knowledge, and dark matter
Sadowski, P., Collado, J., Whiteson, D., & Baldi, P. (2014) · Cited by 29 · see citing works
Explaining Trained Neural Networks with Semantic Web Technologies: First StepsPreprint
Sarker, M. K., Xie, N., Doran, D., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.04324 · Cited by 28 · see citing works
Semantic Clustering Using Multiple Ontologies
Batet, M., Valls, A., Gibert, K., et al. (2010) · Frontiers in artificial intelligence and applications · doi:10.3233/978-1-60750-643-0-207 · Cited by 14 · see citing works
Explainable Cross-Domain Recommendations Through Relational Learning
Sopchoke, S., Fukui, K.-I., & Numao, M. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12176 · Cited by 10 · see citing works
Auditing Black-box Models by Obscuring FeaturesPreprint
Adler, P., Falk, C., Friedler, S. A., Rybeck, G., Scheidegger, C., Smith, B., & Venkatasubramanian, S. (2016) · arXiv (Cornell University) · Cited by 9 · see citing works
Choose Your Neuron: Incorporating Domain Knowledge through Neuron-ImportancePreprint
Selvaraju, R. R., Chattopadhyay, P., Elhoseiny, M., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.02861 · Cited by 5 · see citing works
Human-centric Transfer Learning Explanation via Knowledge Graph [Extended Abstract]Preprint
Geng, Y., Chen, J., Jiménez-Ruiz, E., & Chen, H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.08547 · Cited by 4 · see citing works
Interactive Causal Discovery in Knowledge Graphs
Münch, M., Dibie-Barthélemy, J., Wuillemin, P.-H., et al. (2019) · HAL (Le Centre pour la Communication Scientifique Directe) · Cited by 3 · see citing works
Flagged first, then by citations
Research works printed but not linked: 5
- Ding, X.; Zhang, Y.; Liu, T.; Duan, J. Deep learning for event-driven stock prediction. In Proceedings of the Twenty-Fourth International Joint Conference on Artificial intelligence, Buenos Aires, Argentina, 25-31 July 2015.
- Banerjee, S. A Semantic Web Based Ontology in the Financial Domain. In Proceedings of World Academy of Science, Engineering and Technology; World Academy of Science, Engineering and Technology (WASET): Paris, France, 2013; Number 78, p. 1663.
- Goodfellow, I.; Bengio, Y.; Courville, A. Deep Learning; MIT Press: Cambridge, MA, USA, 2016.
- Rumelhart, D.E. Parallel distributed processing: Explorations in the microstructure of cognition. Learn. Intern. Represent. Error Propag. 1986, 1, 318-362.
- Hall, P.; Gill, N. An Introduction to Machine Learning Interpretability-Dataiku Version; O'Reilly Media, Incorporated: Sebastopol, CA, USA, 2018.
Cited by
Sorted by citations
Explainable AI (XAI): Core Ideas, Techniques, and Solutions
Dwivedi, R., Dave, D., Naik, H., Singhal, S., Omer, R., Patel, P., Qian, B., Wen, Z., Shah, T., Morgan, G., et al. (2023) · ACM Computing Surveys · doi:10.1145/3561048 · Cited by 1,386 · see citing works
A Review on Explainability in Multimodal Deep Neural Nets
Joshi, G., Walambe, R., & Kotecha, K. (2021) · IEEE Access · doi:10.1109/access.2021.3070212 · Cited by 204 · see citing works
- Authors
- Futia, G., & Vetrò, A.
- Affiliations
- Politecnico di Torino (IT)
- Corresponding author
- Futia, Giuseppe
- Venue
- Information
- Publisher
- MDPI AG
- Published
- 2020-02-22
- Volume
- 11
- Issue
- 2
- Pages
- 122
- ISSN
- 2078-2489 (ISSN-L)
- OpenAlex
- W3008128620
- Cited by
- 94
- References
- 50
- Citations by year
- 2026: 3 · 2025: 17 · 2024: 17 · 2023: 22 · 2022: 20 · 2021: 10 · 2020: 5
- FWCI
- 5.59
- Type
- journal article
- Language
- en
- Open access
- gold
- dx.doi.org
- Licence
- Locations
- Information (publishedVersion, cc-by)
- PORTO@Iris (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Semantic Web and Ontologies
- Physical Sciences › Computer Science › Artificial Intelligence › Advanced Graph Neural Networks
- Keywords
- knowledge graphs, explainable AI, neural-symbolic integration, symbolic AI, knowledge matching, interactive explanations, semantic technologies
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
