CoChat
Sign Up
Cited by 5Explainable Artificial Intelligence (XAI)Logic, Reasoning, and KnowledgeTopic Modeling

ProSLM: A Prolog Synergized Language Model for explainable Domain Specific Knowledge Based Question Answering

Open access
Vakharia, Priyesh, Kufeldt, Abigail, Meyers, Max, Lane, Ian, Gilpin, Leilani H.·2024·Lecture Notes in Computer Science
DOI 10.1007/978-3-031-71170-1_23
Open this paper in CoChat — free accountOpen PDF

Abstract

Neurosymbolic approaches can add robustness to opaque neural systems by incorporating explainable symbolic representations. However, previous approaches have not used formal logic to contextualize queries to and validate outputs of large language models (LLMs). We propose \systemname{}, a novel neurosymbolic framework, to improve the robustness and reliability of LLMs in question-answering tasks. We provide \systemname{} with a domain-specific knowledge base, a logical reasoning system, and an integration to an existing LLM. This framework has two capabilities (1) context gathering: generating explainable and relevant context for a given query, and (2) validation: confirming and validating the factual accuracy of a statement in accordance with a knowledge base (KB). Our work opens a new area of neurosymbolic generative AI text validation and user personalization.

AI snapshot

Written from the paper’s full text.

TL;DR

ProSLM is a neurosymbolic framework that couples a Prolog knowledge base and backward-chaining inference with an LLM to provide explainable context gathering and fact validation for domain-specific KBQA. It demonstrates these capabilities on UC Santa Cruz campus examples using GPT-3.5, without LLM retraining.

Key findings

  • ProSLM has two capabilities: (1) generating explainable, relevant context for a query and (2) validating the factual accuracy of a statement against a knowledge base.
  • In context-gathering examples, ProSLM returns grounded context such as `{'truth': True, 'Y': ['quiet', 'wifi']}` for “Where can I study?”, a drop deadline of `['29', 'jan']` for “Can I drop classes?”, and `'Y': 'stevenson'` for “Where should I dine today?”.
  • In fact validation, for the ChatGPT claim “UCSC has three dining halls: College Nine/Ten Dining Hall, Cowell/Stevenson Dining Hall, and Crown/Merrill Dining Hall,” ProSLM returns `number_of_dining_halls(3) (False)`, `dining_hall(college_nine_ten) (True)`, `dining_hall(cowell_stevenson) (False)`, and `dining_hall(crown_merrill) (True)`.
  • No quantitative performance metrics or benchmark results are reported; the demonstrations are qualitative.

Methods

  • Design: a neurosymbolic framework combining a GPT-3.5 neural generator, a GPT-3.5 neural translator, and a symbolic component consisting of a domain-specific Prolog knowledge base and an SWI-Prolog backward-chaining inference agent.
  • Data/sample: a UC Santa Cruz campus knowledge base and illustrative queries about studying, dropping classes, dining, and the number of dining halls.
  • Analysis: qualitative walkthroughs of explainable context gathering and fact validation, including goal trees and Prolog truth values; no statistical or benchmark evaluation is described.

Limitations

  • Stated: an incomplete knowledge base can make ProSLM return an inaccurate answer; the authors plan user-interactive KB updates to mitigate this.
  • Stated: goal trees are accurate to the inference engine’s internal workings but are not currently returned by the symbolic component; the authors plan a user-accessible visualization.
  • Evident from the text: evaluation is limited to illustrative examples with no quantitative benchmark or comparison, and a `False` validation result may mean either the sentence is factually false or the KB/rule base is incomplete (the latter ambiguity is also stated in the paper).

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1007/978-3-031-71170-1_23

Cites

31 of 31 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

8 of 10 cited sources checked · 5 supported · 0 flagged · accuracy not yet measured

  • Premisecitation

    Inaccurate model outputs, known as model hallucinations, contribute to LLMs' lack of reliability.

    “Further, inaccurate model outputs, known as model hallucinations [9], contribute to LLMs' lack of reliability.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [9] Ji, Z., et al.: Survey of hallucination in natural language generation. ACM Comput. Surv. 55(12) (2023). https://doi.org/10.1145/3571730 source
  • Premisecitation

    LLMs are not trustworthy.

    “Despite their demonstrated power and social influence, these models are not trustworthy [21].”
    • Also stated in: § 4 Method · plain
      “As previously discussed, LLMs lack trustworthy explainability [21].”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [21] Wang, B., et al.: Decodingtrust: a comprehensive assessment of trustworthiness in GPT models. arXiv preprint arXiv:2306.11698 (2023) source

      The Conclusion reports trustworthiness vulnerabilities in GPT-3.5 and GPT-4, particularly under misleading prompts, supporting concern about their reliability. But the paper primarily evaluates specific versions of those models and does not establish that LLMs generally are categorically untrustworthy. (LLM, conf=0.87)

      Experimental — accuracy not yet measured
  • Interpretationreasoning

    The symbolic component in ProSLM produces an explainable chain of reasoning to contextualize a query or to fact check a statement.

    “The symbolic component in ProSLM produces an explainable chain of reasoning to contextualize a query, as is the case in the context gathering task, or to fact check a statement, as is the case in fact validation, while leveraging the inherently uninterpretable, powerful generative abilities of an LLM.”
    • Also stated in: § 4.2 Explainable Context Gathering · plain
      “the symbolic component deterministically gathers and provides relevant context to the LLM for an explainable response generation.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The neural translator accurately converts natural language queries to Prolog and Prolog context back to natural language.
    • Rests on an assumption: Every fact in an LLM response can be converted into a Prolog query by the translator.
  • Interpretationreasoning

    ProSLM is compute-efficient because the LLM does not require any additional training or finetuning for the domain of interest.

    “Further, this framework is compute-efficient, as the LLM does not require any additional training or finetuning for the domain of interest.”
    • Also stated in: § 6 Discussion and Conclusion · plain
      “especially given the fact our framework does not need any LLM retraining”
    Quote in the paperSubstantial claim
  • Interpretationown data

    Providing context built on the user's preferences allows ProSLM to personalize the response for a user.

    “As seen in Query 3, by providing context built on the users preferences, this also allows us to personalize the response for a user.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The neural translator accurately converts natural language queries to Prolog and Prolog context back to natural language.
  • May combine two claims. 12
  • Classified as probably not a substantial claim. 13

  • May combine two claims.

    Premisecitation

    Although LLMs can be prompted to self-explain, these explanations are unreliable.

    “Although LLMs can be prompted to self-explain [2], these explanations are unreliable [8].”
    Quote in the paperSubstantial claim2 of 2 sources checked
    • cites [2] Camburu, O.M., Rocktäschel, T., Lukasiewicz, T., Blunsom, P.: e-snli: Natural language inference with natural language explanations (2018) source

      Section 4 (p. 3) describes models trained on e-SNLI to predict a label and generate an explanation, supporting the general point that a model can explain its own prediction. It does not describe prompting LLMs to self-explain. The separate claim that explanations are unreliable is attributed to [8], not to this paper. (LLM, conf=0.86)

      Experimental — accuracy not yet measured
    • cites [8] Huang, S., Mamidanna, S., Jangam, S., Zhou, Y., Gilpin, L.H.: Can large language models explain themselves? A study of LLM-generated self-explanations. arXiv preprint arXiv:2310.11207 (2023) source

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Retrieval augmented generation (RAG) architectures directly feed useful information to the QA model, thus grounding the response, reducing factuality hallucinations, and improving explainability of results.

    “As opposed to CoT, this architecture directly feeds useful information to the QA model, thus grounding the response, reducing factuality hallucinations, and improving explainability of results.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [10] Lewis, P., et al.: Retrieval-augmented generation for knowledge-intensive NLP tasks (2020) source

      The Introduction (p. 2) says RAG conditions generation on retrieved documents, and §4.2 (p. 6) reports that RAG hallucinated less and generated factually correct text more often than BART. The paper does not establish that RAG improves explainability of results; its Abstract (p. 1) describes providing provenance for decisions as an open research problem. It also does not compare RAG with CoT. (LLM, conf=0.86)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Interpretationreasoning

    Neither the Neuro-Symbolic Concept Learner [11] nor Logic-LM [13] pose a viable approach for the current work due to the complexity of questions expected for a KBQA system and the authors' interest in leveraging the generative power of an LLM.

    “Neither [11] nor [13] pose a viable approach in this current work, however, due to the complexity of questions expected for a KBQA system, and our subsequent interest in leveraging the generative power of an LLM to answer inputted queries.”
    Quote in the paperSubstantial claim2 of 2 sources checked
    • cites [11] Mao, J., Gan, C., Kohli, P., Tenenbaum, J.B., Wu, J.: The neuro-symbolic concept learner: interpreting scenes, words, and sentences from natural supervision (2019) source
    • cites [13] Pan, L., Albalak, A., Wang, X., Wang, W.: Logic-LM: empowering large language models with symbolic solvers for faithful logical reasoning. In: Bouamor, H., Pino, J., Bali, K. (eds.) Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 3806–3824. Association for Computational Linguistics, Singapore (2023). https://doi.org/10.18653/v1/2023.findings-emnlp.248, https://aclanthology.org/2023.findings-emnlp.248 source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The PredictAndExplain method requires access to a reliable explanation-augmented dataset for training.

    “The PredictAndExplain method, introduced in [2], illustrates that models can be trained to generate explanations of their outputs without sacrificing performance, but this approach requires access to a reliable explanation-augmented dataset for training.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [2] Camburu, O.M., Rocktäschel, T., Lukasiewicz, T., Blunsom, P.: e-snli: Natural language inference with natural language explanations (2018) source

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    In the fact validation example, ProSLM returned False for number_of_dining_halls(3), True for dining_hall(college_nine_ten), False for dining_hall(cowell_stevenson), and True for dining_hall(crown_merrill).

    “validation: number_of_dining_halls(3) (False) dining_hall(college_nine_ten) (True) dining_hall(cowell_stevenson) (False) dining_hall(crown_merrill) (True)”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The neural translator accurately converts natural language queries to Prolog and Prolog context back to natural language.
    • Rests on an assumption: Every fact in an LLM response can be converted into a Prolog query by the translator.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    In Query 1, ProSLM output that one can study at the McHenry Library on campus, with context that the place must be quiet and have wifi.

    “You can study at the McHenry Library on campus.It is a quiet place with wifi available for students to use.Output: prolog result: {'truth': True, 'Y': ['quiet', 'wifi']}”
    • Also stated in: § 3. Neural Translator (Prolog context to Natural Language context) · plain
      “Output: ['The place must be quiet', 'The place must have wifi']”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The neural translator accurately converts natural language queries to Prolog and Prolog context back to natural language.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingreasoning

    In the pool example, ProSLM returned p_weather as context because it confirms the pool is open at the current time and checks if it is an outdoor pool, which would make the weather affect its availability.

    “What should be noted is the reasoning behind returning 'p_weather' as context -it confirms the pool is open at the current time and also checks if it is an outdoor pool, which would make the weather affect its availability.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The neural translator accurately converts natural language queries to Prolog and Prolog context back to natural language.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingreasoning

    In Query 2, ProSLM output that one can drop classes, with the deadline to drop classes for the quarter being 29 January, after which a petition process is needed.

    “Yes, you can drop classes, but please keep in mind that the deadline to drop classes for this quarter is on the 29th of January.After this date, you will need to go through the petition process to drop a class.”
    • Also stated in: § Output from Logic LLM · plain
      “The deadline to drop classes is 29 Jan if and only if it is the winter quarter, which is checked by looking at the current month of the year.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The neural translator accurately converts natural language queries to Prolog and Prolog context back to natural language.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    In Query 3, ProSLM output that one should dine at Stevenson, taking the user's culinary preferences into account.

    “You should dine at Stevenson.Stevenson College offers a variety of dining options including a dining hall, cafe, and market.The reasoning here, seen in the goal tree above, takes in the user's culinary preferences into account to provide an accurate context -in this case which dining hall to eat at.”
    • Also stated in: § 6 Discussion and Conclusion · plain
      “Output: ['You should dine at Stevenson']”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The neural translator accurately converts natural language queries to Prolog and Prolog context back to natural language.

  • May combine two claims.

    Interpretationnone given

    ProSLM has two capabilities: context gathering, which generates explainable and relevant context for a given query, and validation, which confirms and validates the factual accuracy of a statement in accordance with a knowledge base.

    “This framework has two capabilities (1) context gathering: generating explainable and relevant context for a given query, and (2) validation: confirming and validating the factual accuracy of a statement in accordance with a knowledge base (KB).”
    • Also stated in: § 1 Introduction · plain
      “has two capabilities: (1) generating a retrievable and explainable chain of reasoning to contextualize the input query or (2) validating the accuracy of a given statement.”
    • Also stated in: § 4 Method · boosted
      “We demonstrate the utility of ProSLM for two situations -explainable context gathering and explainable fact validation.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationreasoning

    The goal trees are accurate to the internal workings of the inference engine.

    “while the goal trees are accurate to the internal workings of the inference engine”
    • Also stated in: § 4.3 Fact Validation · plain
      “A goal tree complements every fact validation, making the response and its validation explainable and trustworthy.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationreasoning

    A goal tree complements every fact validation, making the response and its validation explainable and trustworthy.

    “A goal tree complements every fact validation, making the response and its validation explainable and trustworthy.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    ProSLM is able to provide correct and reasonably sound 'opinions' and 'verified factual responses' due to either the curated context or the fact validation respectively.

    “ProSLM is able to provide correct and reasonably sound 'opinions' and 'verified factual responses' due to either the curated context or the fact validation respectively.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The neural translator accurately converts natural language queries to Prolog and Prolog context back to natural language.
    • Rests on an assumption: Every fact in an LLM response can be converted into a Prolog query by the translator.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given

    The KB is able to hold real-time information and user-specific preferences, and can be updated independently of the rest of the framework.

    “This KB is also able to hold real-time information and user-specific preferences, and can be updated independently of the rest of the framework.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Conditioning an LLM response on ProSLM's context makes the response trustworthy and explainable.

    “This makes the LLM response, that is conditioned on this context, trustworthy and explainable.”
    • Also stated in: § 4.2 Explainable Context Gathering · plain
      “This setting is most useful where the reasoning behind the context grounds the LLM response, making it trustworthy.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The neural translator accurately converts natural language queries to Prolog and Prolog context back to natural language.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    The goal trees are not currently returned by the symbolic component.

    “they are not currently returned by the symbolic component.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    An LLM response can be asserted true if each fact in that response is individually asserted true.

    “an LLM response can be asserted true if each fact in that response is individually asserted true.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    ProSLM's symbolic component mitigates hallucinations by grounding the model with logical, relevant context, and has the potential to detect them by determining the truth value of a claim.

    “in our case, ProSLM's symbolic component mitigates hallucinations by grounding the model with logical, relevant context, and has the potential to detect them by determining the truth value of a claim.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Chain of thought (CoT) prompting outperforms baselines in arithmetic, commonsense, and symbolic reasoning tasks, but requires sufficiently large LLMs.

    “This method outperforms baselines in arithmetic, commonsense, and symbolic reasoning tasks, but requires sufficiently large LLMs in order to be successful.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [24] Wei, J., et al.: Chain-of-thought prompting elicits reasoning in large language models (2022)

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

19 of 19

  • The paper relies on this without support.

    Previous approaches have not used formal logic to contextualize queries to and validate outputs of large language models.

    “However, previous approaches have not used formal logic to contextualize queries to and validate outputs of large language models (LLMs).”

  • The paper relies on this without support.

    The neural translator accurately converts natural language queries to Prolog and Prolog context back to natural language.

    “We use this translator to facilitate the conversion from natural language (English) prose to Prolog and from Prolog to natural language prose.”
    “the Translator converts the user query Q(n) to a prolog query Q(p).”

  • The paper relies on this without support.

    Every fact in an LLM response can be converted into a Prolog query by the translator.

    “This response R is fed into our Translator, which converts every fact in R into a list of Prolog queries.”
3 of 3

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. George, A.S., George, A.H.: A review of ChatGPT AI’s impact on several business sectors. Partners Universal Int. Innov. J. 1(1), 9–23 (2023)

Cited by

Sorted by citations

Related

Authors
Vakharia, P., Kufeldt, A., Meyers, M., Lane, I., & Gilpin, L. H.
Affiliations
University of California, Santa Cruz (US)
Corresponding author
Vakharia, Priyesh
ORCID
Gilpin, Leilani H.
Venue
Lecture Notes in Computer Science
Publisher
Springer Nature Switzerland
Published
2024
Pages
291-304
ISSN
  • 0302-9743 (ISSN-L)
  • 1611-3349
DOI
10.1007/978-3-031-71170-1_23
OpenAlex
W4402366652
Semantic Scholar
590f22a64902922676753777bad9c07e41f125a7
Cited by
5
References
21
Citations by year
2026: 2 · 2025: 3
FWCI
1.47
Type
conference paper
Language
en
Open access
Licence
Locations
  • Lecture notes in computer science (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by-nc-sa)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Computer Science › Artificial Intelligence › Logic, Reasoning, and Knowledge
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords
neuro-symbolic AI, Prolog, large language models, knowledge base question answering, explainable AI, domain knowledge base, logical reasoning, LLM robustness, user personalization

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-19.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-19.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review