CoChat
Sign Up
PreprintCited by 1Business Process Modeling and AnalysisLogic, Reasoning, and KnowledgeAI-based Problem Solving and Planning

VADAOrchestra: Neurosymbolic Orchestration of Adaptive Reasoning Workflows

Open access
Baldazzi, Teodoro, Bellomarini, Luigi, Coletta, Andrea, Iezzi, Michela, Maple, Carsten, Pesare, Alessandro, Sallinger, Emanuel·2026·arXiv
DOI 10.48550/arxiv.2606.22485arXiv 2606.22485
Open this paper in CoChat — free account

Abstract

Decision-making in real-world settings rarely follows a fixed script. Instead, it unfolds as a dynamic reasoning process in which the appropriate course of action evolves as new context and data become available. Traditional Business Process Management systems provide rigor, determinism, and auditability, yet they generally struggle to adapt their execution at runtime. Conversely, agentic systems based on Large Language Models (LLMs) bring flexibility to decision-making, but they are inherently opaque, often unreliable, and suffer from significant scalability constraints when operating over large datasets. To combine these complementary paradigms, we introduce VADAOrchestra, a neurosymbolic framework that models complex workflows as evolving reasoning processes. The framework adopts a hybrid approach: given a user query and a collection of data sources, an LLM-based orchestrator incrementally plans and adapts the workflow. This is encoded as a logic program in a fragment of Datalog+/- where predicates correspond to tool invocations and rules represent both predefined domain dependencies and logic constructs synthesized on demand to manipulate intermediate results. All logical inference tasks are then executed by a state-of-the-art Datalog+/- symbolic engine. This approach provides a verifiable reasoning trace, supporting the auditability and reproducibility of the entire process. Furthermore, by decoupling high-level orchestration from symbolic inference, it addresses scalability concerns, enabling complex reasoning over large datasets through targeted data querying. We evaluate VADAOrchestra on real-world financial use cases, demonstrating faithfulness, scalability, and explainability compared to standard agentic architectures.

AI snapshot

Written from the paper’s full text.

TL;DR

VADAOrchestra is a neurosymbolic framework that combines an LLM-based orchestrator with a Vadalog symbolic engine to turn adaptive reasoning workflows into auditable logic programs. It is evaluated on financial question-answering tasks, where it improves accuracy and scalability over standard agentic and RAG baselines, especially with a stronger LLM backbone.

Key findings

  • On the Bank dataset, VADAOrchestra with GPT-4o reaches 0.65 Exact Match and 0.78 LLM-as-a-Judge accuracy, surpassing LLM+MCP at 0.63 EM and 0.77 Judge; with Llama-3.3-70B, LLM+MCP slightly outperforms VADAOrchestra (0.58 EM vs. 0.50 EM) due to errors in Vadalog rule synthesis.
  • In scalability tests on the augmented BANK+ dataset, VADAOrchestra remains stable with not less than 0.63 EM and 0.8 LLM-as-a-Judge accuracy, while at complexity threshold X = 50 RAG-based approaches drop to around 0.35 EM and LLM+MCP declines from approximately 0.7 to about 0.55.
  • In the running concentration-risk example, DeepLogic Ltd (16.375%) and NeuralLinker SpA (14.375%) are individually below the 25% regulatory limit, but group-level aggregation through their common controller Synthetix Global Fund reaches 30.75%, revealing a breach.
  • The framework produces a complete logical trace of tool invocations and synthesized Vadalog rules, enabling deterministic reproducibility without LLM involvement and providing auditability that the compared LLM+RAG and LLM+MCP approaches lack.

Methods

  • Design: a neurosymbolic client-server architecture mediated by MCP, where an LLM orchestrator performs dependency-aware planning, targeted data retrieval, replanning, and on-demand Vadalog rule synthesis, while a Vadalog engine executes all symbolic inference.
  • Data: two anonymized QA datasets derived from a large European central bank knowledge graph with approximately 10k triples and nine relations: the Bank dataset of 278 template-based questions and the augmented BANK+ scalability stress test with count queries requiring up to 1,000 different KG entities.
  • Analysis: compared against LLM-only, ReFactX, LLM+RAG, and LLM+MCP baselines using Exact Match and LLM-as-a-Judge (Llama-3.3-70B judge), with Llama-3.3-70B-Instruct and GPT-4o backbones, plus cumulative accuracy plotted against query complexity thresholds from 1 to 1,000 entity-relations.

Limitations

  • Stated: when runtime entities have many relevant relationships, token generation increases due to multiple LLM invocations, and overall execution time grows relative to traditional agentic approaches because rule synthesis adds overhead.
  • Stated: the quality of generated data manipulation rules can degrade in complex financial scenarios when smaller language models are used as the backbone; future work plans additional validation mechanisms.
  • Evident from the text: evaluation is limited to anonymized financial QA datasets and a central bank knowledge graph, and the codebase is available only upon request, so broader domain generalization and reproducibility outside these settings are not demonstrated.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

DataCite · 10.48550/arxiv.2606.22485

Cites

22 of 22 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

11 of 19 cited sources checked · 5 supported · 0 flagged · accuracy not yet measured

  • Premisecitation
    § 1 Introduction · p. 1

    LLMs frequently generate post-hoc explanations that do not reflect their actual reasoning paths.

    “Research on faithfulness has shown that LLMs frequently generate posthoc explanations that do not reflect their actual reasoning paths (Lanham et al. 2023; Turpin et al. 2023; Tanneru et al. 2024; Matton et al. 2025).”
    Quote in the paperSubstantial claim3 of 3 sources checked
    • cites Lanham et al. 2023 source

      Section 2.3.1 reports substantial signs of post-hoc reasoning on some tasks, but says its extent varies widely: chain of thought changed the answer less than 10% of the time on three tasks and more than 60% on another. Section 2.3 also cautions that post-hoc reasoning does not necessarily mean the explanation is unfaithful. The claim overgeneralizes these task-dependent findings to LLMs frequently producing explanations that do not reflect their actual reasoning. (LLM, conf=0.88)

      Experimental — accuracy not yet measured
    • cites Turpin et al. 2023 source
    • cites Tanneru et al. 2024 source
  • Premisereasoning
    § 1 Introduction · p. 2

    Recent multi-agent paradigms such as chain of agents and agent loops do not fully resolve explainability and scalability issues.

    “Even recent multi-agent paradigms, such as chain of agents (Zhang et al. 2024) and agent loops (Wu et al. 2024), where multiple specialized agents collaborate or iterate to handle long-horizon tasks, do not fully resolve all these issues.”
    Quote in the paperSubstantial claim1 of 2 sources checked
    • cites Zhang et al. 2024 source

      The Introduction (p. not provided) identifies Chain-of-Agents as collaborating worker agents for long-context tasks. The Limitations section (p. not provided) acknowledges remaining cost, latency, and API-traffic concerns, which supports the scalability qualification. It does not say that explainability remains unresolved; the Introduction instead calls the framework highly interpretable. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
    • cites Wu et al. 2024

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Premisecitation
    § 1 Introduction · p. 1

    BPM systems' largely static nature is a major limitation, as they struggle to accommodate the data-driven branching that characterizes dynamic workflows.

    “However, their largely static nature proves to be a major limitation, as these frameworks struggle to accommodate the data-driven branching that characterizes dynamic workflows (Di Ciccio, Marrella, and Russo 2015).”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Di Ciccio, Marrella, and Russo 2015 source

      The Introduction and Discussion describe conventional BPM approaches as relying on prescribed execution flows and identify serious limitations in supporting evolving, knowledge-intensive processes. However, §6.1 says YAWL can evaluate branching conditions using variables defined at configuration time. The paper supports difficulty with runtime flexibility and data access, but not an unqualified inability to accommodate data-driven branching. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
  • Premisecitation
    § 1 Introduction · p. 2

    As context length increases, models exhibit degraded attention to information in the middle of the context (the 'lost in the middle' phenomenon).

    “Empirical studies show that, as context length increases, models exhibit degraded attention to information in the middle of the context (the so-called ' lost in the middle ' phenomenon (Liu et al. 2024)), leading to overlooked details and compromised decisions.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Liu et al. 2024 source
  • Findingown data
    § 5.1 Results and Discussion · p. 9

    LLM+MCP declines significantly, from approximately 0.7 to about 0.55, in the high-complexity regime.

    “Similarly, LLM+MCP declines significantly, from approximately 0.7 to about 0.55, demonstrating that delegating all reasoning and aggregation steps to the LLM may degrade performance in complex workflows.”
    Quote in the paperSubstantial claim
  • May combine two claims. 30
  • Classified as probably not a substantial claim. 21

  • May combine two claims.

    Premisecitation
    § Abstract · p. 1

    Agentic systems based on Large Language Models bring flexibility to decision-making, but they are inherently opaque, often unreliable, and suffer from significant scalability constraints when operating over large datasets.

    “Conversely, agentic systems based on Large Language Models (LLMs) bring flexibility to decisionmaking, but they are inherently opaque, often unreliable, and suffer from significant scalability constraints when operating over large datasets.”
    • Also stated in: § 1 Introduction · p. 1 · plain
      “Despite their capabilities, LLM-based systems still face a fundamental limitation in explainability.”
    Quote in the paperSubstantial claim2 of 2 sources checked
    • cites Lanham et al. 2023 source

      The Introduction says LLM-generated explanations can be unfaithful to the model’s actual reasoning, supporting a narrower concern about opacity. The paper does not establish that agentic systems are inherently opaque or often unreliable, and it does not address decision-making flexibility or scalability over large datasets. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
    • cites Liu et al. 2024 source

      The Introduction describes language models as flexible building blocks and says Transformer memory and compute grow quadratically with sequence length. Section 4 reports that models struggle to use information reliably in long input contexts. The paper does not establish that agentic decision-making systems are inherently opaque, and its scalability finding concerns long input sequences rather than large datasets generally. (LLM, conf=0.87)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Premisecitation
    § Abstract · p. 1

    Traditional Business Process Management systems provide rigor, determinism, and auditability, yet they generally struggle to adapt their execution at runtime.

    “Traditional Business Process Management systems provide rigor, determinism, and auditability, yet they generally struggle to adapt their execution at runtime.”
    • Also stated in: § 1 Introduction · p. 1 · plain
      “However, their largely static nature proves to be a major limitation, as these frameworks struggle to accommodate the data-driven branching that characterizes dynamic workflows (Di Ciccio, Marrella, and Russo 2015).”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Di Ciccio, Marrella, and Russo 2015 source

  • Classified as probably not a substantial claim.

    Premisecitation
    § 1 Introduction · p. 1

    LLM-based agentic systems exhibit strong capabilities in context-aware decision-making, planning, and tool orchestration.

    “Leveraging Large Language Models (LLMs), agentic systems exhibit strong capabilities in context-aware decision-making, planning, and tool orchestration (Shinn et al. 2023; Patil et al. 2024).”
    Quote in the paperSubstantial claim2 of 2 sources checked
    • cites Shinn et al. 2023 source
    • cites Patil et al. 2024 source

      The Introduction and §4.3 describe Gorilla selecting an API for a task while accounting for user-specified constraints, supporting context-sensitive tool selection. But the API Call Task section evaluates a single call, and §3.3 says the paper considers only one API call; these passages do not establish broader planning or coordination of multiple tools by agentic systems. (LLM, conf=0.88)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Findingown data
    § Abstract · p. 1

    VADAOrchestra was evaluated on real-world financial use cases, demonstrating faithfulness, scalability, and explainability compared to standard agentic architectures.

    “We evaluate VADAOrchestra on real-world financial use cases, demonstrating faithfulness, scalability, and explainability compared to standard agentic architectures.”
    • Also stated in: § 1 Introduction · p. 3 · boosted
      “We experimentally showcase the effectiveness of our approach on a real-world financial use case , demonstrating its scalability and the faithfulness of its reasoning traces over standard agentic architectures.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning
    § System conclusions: · p. 10

    VADAORCHESTRA produces the correct answer: total exposure of $24.6M under Synthetix Global Fund is computed deterministically and flagged as exceeding the regulatory limit.

    “Finally, only VADAORCHESTRA produces the correct answer. After identifying all exposures and discovering the ultimate controllers via dedicated tool invocations, it delegates the aggregation and threshold check to the VADALOG engine: the total exposure of $24.6M under Synthetix Global Fund is computed deterministically and flagged as exceeding the regulatory limit.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 5.1 Results and Discussion · p. 8

    With GPT-4o backbone, VADAORCHESTRA reaches 0.65 EM and 0.78 Judge accuracy, surpassing LLM+MCP's 0.63 EM and 0.77 Judge.

    “However, using the more capable GPT-4o backbone, VADAORCHESTRA reaches 0.65 EM and 0.78 Judge accuracy, surpassing the 0.63 EM and 0.77 Judge obtained by LLM+MCP .”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 5.1 Results and Discussion · p. 9

    In the low-complexity regime, VADAORCHESTRA achieves the best performance, followed by LLM+MCP and RAG-based solutions.

    “In the low-complexity regime , the results confirm those of the quantitative analysis, with VADAORCHESTRA achieving the best performance, followed by the LLM+MCP and the RAG-based solutions.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 5.1 Results and Discussion · p. 8

    The LLM-only baseline achieves around 0.18 EM and 0.23 LLM-as-a-Judge accuracy on the Bank dataset.

    “As expected, the LLM-only baseline achieves around 0.18 EM and 0.23 LLM-as-a-Judge accuracy, as the anonymized financial data lies largely outside the model's parametric knowledge, highlighting the need for external knowledge retrieval.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning
    § System conclusions: · p. 10

    When the system processes entities with many relationships relevant to the question, token generation increases due to multiple LLM invocations, and overall execution time grows compared to a traditional agentic approach.

    “First, when the system processes entities that, at runtime, turn out to have many relationships relevant to the question, we observe an increase in token generation due to the multiple LLM invocations. Consequently, the overall execution time of VADAORCHESTRA grows with respect to a traditional agentic approach, since rule synthesis introduces further overhead.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 5.1 Results and Discussion · p. 8

    ReFactX achieves 0.36 EM and 0.43 LLM-as-a-Judge accuracy on the Bank dataset.

    “For example, ReFactX raises accuracy to 0.36 (EM) and 0.43 (LLMas-a-Judge), which represents a meaningful gain over the LLM-only, however, it falls short of the RAG and agentic approaches.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 5.1 Results and Discussion · p. 8

    LLM+RAG reaches 0.39 EM and 0.60 LLM-as-a-Judge with Llama, and 0.42 EM / 0.60 LLM-as-a-Judge with GPT-4o.

    “LLM+RAG , which retrieves and injects relevant passages, reaches 0.39 EM and 0.60 LLM-as-a-Judge with Llama, and 0.42 EM / 0.60 LLM-as-a-Judge with GPT-4oindicating that the benefit of retrieval is relatively stable across models.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning
    § 5.1 Results and Discussion · p. 9

    LLM+RAG never examines the ownership structure of the borrowers and incorrectly concludes that no concentration risk exists.

    “We observe that LLM+RAG never examines the ownership structure of the borrowers and therefore incorrectly concludes that no concentration risk exists.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisereasoning
    § 1 Introduction · p. 2

    This opacity fundamentally undermines trust in high-stakes domains.

    “This opacity fundamentally undermines trust in high-stakes domains.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: High-stakes domains require auditable reasoning traces.

  • Classified as probably not a substantial claim.

    Premisereasoning
    § 1 Introduction · p. 1

    Automating such workflows is challenging because they are partially structured, following well-defined procedures from domain expertise, and partially dynamic, branching unpredictably based on runtime findings.

    “The central challenge in automating such workflows arises from their dual nature: they are partially structured, following well-defined procedures derived from domain expertise, and partially dynamic, branching unpredictably based on runtime findings.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationnone given
    § Algorithm 1 Orchestration Pipeline · p. 6

    The orchestrator never accesses full results; it operates exclusively on targeted data samples.

    “Crucially, the orchestrator never accesses full results: it operates exclusively on targeted data samples.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning
    § 1 Introduction · p. 3

    Unlike purely agentic approaches that may hallucinate, the framework enforces deterministic behavior typical of logic-based systems, ensuring that high-stakes analyses follow well-defined procedures.

    “Unlike purely agentic approaches that may 'hallucinate', our framework enforces the deterministic behavior typical of logic-based systems, ensuring that high-stakes analyses (e.g., concentration risk) follow well-defined procedures.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given
    § 1 Introduction · p. 2

    Even if an LLM-based agent correctly identifies that a specific counterparty poses a concentration risk, the reasoning behind this decision would remain opaque.

    “Even if an LLM-based agent correctly identifies that a specific counterparty poses a concentration risk, the reasoning behind this decision would remain opaque.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § Algorithm 1 Orchestration Pipeline · p. 6

    VADAORCHESTRA's on-demand data manipulation ensures computational correctness for joins, aggregations, and arithmetic while leveraging the LLM's ability to understand user intent and identify relevant data relationships.

    “VADAORCHESTRA addresses this through on-demand data manipulation: the LLM generates Vadalog rules at runtime that are executed by the symbolic engine, ensuring computational correctness for joins, aggregations, and arithmetic while leveraging the LLM's ability to understand the user's intent and identify the relevant data relationships.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Many analytical questions require combining results from multiple tools in ways that cannot be anticipated at design time.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Premisenone given
    § 1 Introduction · p. 1

    Real-world decision-making processes, particularly in complex domains such as finance, cannot be reduced to static, predefined workflows.

    “Real-world decision-making processes, particularly in complex domains such as finance, cannot be reduced to static, predefined workflows.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 3 Preliminaries · p. 4

    Vadalog is a declarative language for ontological reasoning based on Warded Datalog+/- that guarantees PTIME data complexity for query answering.

    “Vadalog is a declarative language for ontological reasoning based on Warded Datalog ± , a member of the Datalog family that extends Datalog with existential quantifiers while guaranteeing PTIME data complexity for query answering (Gottlob and Pieris 2015).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Gottlob and Pieris 2015

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Premisenone given
    § 1 Introduction · p. 2

    Without an auditable trace, the analyst cannot verify the reasoning process or assess whether a correct conclusion resulted from the right reasons.

    “Without an auditable trace, the analyst cannot verify the reasoning process or assess whether a correct conclusion resulted from the right reasons.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Analysts need to verify the reasoning process and assess whether a correct conclusion resulted from the right reasons.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisenone given
    § 1 Introduction · p. 1

    LLMs can interpret high-level goals, decompose them into sub-tasks, and interact with external resources such as web search engines, databases, and software tools.

    “LLMs can interpret high-level goals, decompose them into sub-tasks, and interact with external resources such as web search engines, databases, and software tools.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given
    § 4.1 System Architecture · p. 5

    Generated rules undergo syntactic validation before execution by the Vadalog engine.

    “Generated rules undergo syntactic validation before execution by the VADALOG engine.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given
    § 1 Introduction · p. 2

    The orchestrator can synthesize Vadalog rules on the fly to perform data manipulations such as joins and aggregations that only become relevant at runtime.

    “Furthermore, the orchestrator can synthesize Vadalog rules on the fly to perform data manipulations, such as joins and aggregations, that only become relevant at runtime.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The ownership structure in the concentration-risk example cannot be known in advance; it emerges only at runtime.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given
    § 1 Introduction · p. 1

    BPM systems provide essential rigor, transparency, and compliance guarantees required for highly structured enterprise workflows.

    “Historically, Business Process Management (BPM) systems have provided the essential rigor, transparency, and compliance guarantees required for highly structured enterprise workflows.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § 1 Introduction · p. 2

    Once a high-level concept is identified from the user's query, the system consults the dependency graph to define an initial execution plan, ensuring the execution order of MCP tools is enforced rather than left to the LLM to infer.

    “Once a high-level concept (e.g., bank's exposures) is identified from the user's query, the system consults the dependency graph to define an initial execution plan. This ensures the execution order of MCP tools is enforced, rather than leaving it up to the LLM to infer.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given
    § Algorithm 1 Orchestration Pipeline · p. 6

    Generated Vadalog rules undergo syntactic validation before being dispatched to the VADALOG engine.

    “The generated rules undergo syntactic validation before being dispatched to the VADALOG engine via the data manipulation tool (line 15).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisereasoning
    § 1 Introduction · p. 2

    Defining retrieval criteria in advance is non-trivial, as overly restrictive filters risk omitting critical information.

    “At the same time, defining retrieval criteria in advance is non-trivial, as overly restrictive filters risk omitting critical information.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 1 Introduction · p. 1

    Standardized protocols such as the Model Context Protocol (MCP) facilitate interaction by exposing tools through a uniform interface, enabling LLMs to select and compose tool invocations dynamically at runtime.

    “Standardized protocols, such as the Model Context Protocol (MCP), further facilitate this interaction by exposing tools through a uniform interface, enabling LLMs to select and compose tool invocations dynamically at runtime (Anthropic 2024).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Anthropic 2024

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationreasoning
    § 1 Introduction · p. 3

    Adaptivity is preserved: the LLM-based orchestrator manages planning complexity by invoking tools, adapting the reasoning flow, and synthesizing logic operations on demand to handle contingencies that cannot be fully predefined.

    “At the same time, adaptivity is preserved: the LLM-based orchestrator manages planning complexity by invoking tools, adapting the reasoning flow, and synthesizing logic operations on demand to handle contingencies that cannot be fully predefined.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Premisereasoning
    § 1 Introduction · p. 1

    In knowledge-intensive settings, a workflow is a dynamic process in which the appropriate course of action depends on newly available information, contextual factors, and intermediate outcomes.

    “In these knowledge-intensive settings, a workflow is not merely a sequence of tasks, but a dynamic process in which the appropriate course of action depends on newly available information, contextual factors, and intermediate outcomes.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 1 Introduction · p. 1

    Agentic AI has emerged as a foundational paradigm for automating knowledge-intensive workflows.

    “More recently, agentic AI has emerged as a foundational paradigm for automating knowledge-intensive workflows (Wei et al. 2026).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Wei et al. 2026

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationreasoning
    § 1 Introduction · p. 2

    Every step of the workflow is captured in a logical trace, enabling full auditability and reproducibility of the reasoning process.

    “In this way, every step of the workflow is captured in a logical trace, enabling full auditability and reproducibility of the reasoning process.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: High-stakes domains require auditable reasoning traces.
    • Rests on an assumption: Analysts need to verify the reasoning process and assess whether a correct conclusion resulted from the right reasons.

  • May combine two claims.

    Interpretationreasoning
    § Abstract · p. 1

    The approach provides a verifiable reasoning trace, supporting the auditability and reproducibility of the entire process.

    “This approach provides a verifiable reasoning trace, supporting the auditability and reproducibility of the entire process.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: High-stakes domains require auditable reasoning traces.
    • Rests on an assumption: Analysts need to verify the reasoning process and assess whether a correct conclusion resulted from the right reasons.

  • Classified as probably not a substantial claim.

    Interpretationnone given
    § 2 System Overview · p. 3

    The orchestrator is equipped with reasoning tools and support tools whose composition is guided by a dependency graph to retrieve information and answer user queries.

    “The orchestrator is equipped with a set of tools, including reasoning tools (e.g., computing exposures) and support tools (e.g., navigating and manipulating intermediate data) whose composition is guided by a dependency graph to retrieve information and answer user queries.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning
    § 1 Introduction · p. 3

    The resulting logical trace makes every reasoning step explicit, enabling auditability and allowing analysts to verify not only the final outcome but also the correctness of the reasoning that led to it.

    “The resulting logical trace makes every reasoning step explicit, enabling auditability and allowing analysts to verify not only the final outcome, but also the correctness of the reasoning that led to it.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: High-stakes domains require auditable reasoning traces.
    • Rests on an assumption: Analysts need to verify the reasoning process and assess whether a correct conclusion resulted from the right reasons.

  • May combine two claims.

    Interpretationreasoning
    § Abstract · p. 1

    Decoupling high-level orchestration from symbolic inference addresses scalability concerns, enabling complex reasoning over large datasets through targeted data querying.

    “Furthermore, by decoupling high-level orchestration from symbolic inference, it addresses scalability concerns, enabling complex reasoning over large datasets through targeted data querying.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Realistic concentration-risk deployments scale to thousands of entities.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationreasoning
    § 1 Introduction · p. 3

    The orchestrator inspects the cardinality and statistical distribution of intermediate results to formulate fine-grained data querying strategies.

    “The orchestrator inspects the cardinality and statistical distribution of the intermediate results to formulate fine-grained data querying strategies.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationnone given
    § 2 System Overview · p. 3

    The framework automates knowledge-intensive workflows through an LLM-based orchestrator for dynamic decision planning and a symbolic reasoning engine for logical inference, providing adaptive yet auditable executions.

    “It automates such workflows through an LLM-based orchestrator for dynamic decision planning and a symbolic reasoning engine for logical inference, providing adaptive yet auditable executions.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationnone given
    § Algorithm 1 Orchestration Pipeline · p. 6

    VADAORCHESTRA's two-phase retrieval approach works with representative, query-relevant data, avoids brittleness of fixed-k strategies, and maintains strict bounds on information in the LLM context window.

    “This two-phase approach results in working with representative, query-relevant data avoiding the brittleness of fixedk strategies and maintaining strict bounds on the amount of information within LLM's context window.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Tools may produce result sets with thousands or millions of facts, far exceeding what can be injected into an LLM's context window.

  • May combine two claims.

    Premisereasoning
    § 1 Introduction · p. 2

    Individual agents still operate within finite context windows, inter-agent communication introduces coordination overhead and information loss, and the reasoning process remains distributed across opaque generative steps.

    “Indeed, individual agents still operate within finite context windows, inter-agent communication introduces coordination overhead and information loss, and the reasoning process remains distributed across opaque generative steps.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationnone given
    § 4.3 Logical Trace · p. 7

    The logical trace provides a faithful, human-readable explanation of the decision-making process and supports deterministic reproducibility without any LLM involved.

    “This trace provides a faithful, human-readable explanation of the decision-making process, and supports deterministic reproducibility without any LLM involved.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § 5.1 Results and Discussion · p. 8

    Rule generation is a complex task that smaller models often handle imperfectly, producing rules that are syntactically correct but semantically flawed, thereby reducing final answer accuracy.

    “Rule generation is a complex task that smaller models often handle imperfectly, producing rules that are syntactically correct but semantically flawed, thereby reducing final answer accuracy.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning
    § 4.3 Logical Trace · p. 7

    The trace is a human-readable explanation of the entire decision-making process, providing full auditability to domain analysts.

    “The trace is also a human-readable explanation of the entire decision-making process, providing full auditability to domain analysts.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning
    § Algorithm 1 Orchestration Pipeline · p. 7

    To ensure termination, the system limits the execution loop to a predefined maximum number of iterations and imposes a strict constraint against invoking the same tool with identical parameters twice.

    “To ensure termination, the system limits the execution loop to a predefined maximum number of iterations and imposes a strict constraint against invoking the same tool with identical parameters twice.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning
    § 4.4 Answer Generation · p. 7

    In answer generation, instructing the LLM to rely on computed results rather than re-deriving from raw data ensures that the precision of the VADALOG engine's computations guides the final answer, preventing the LLM from introducing errors through in-context reasoning.

    “When predicates from data manipulation exist, the LLMis instructed to rely on the computed results rather than re-deriving conclusions from raw data. This ensures that the precision of the VADALOG engine's computations guides the final answer, preventing the LLM from introducing errors through in-context reasoning.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning
    § Algorithm 1 Orchestration Pipeline · p. 6

    The design achieves a clear separation of responsibilities: the LLM provides semantic understanding (what to compute), while the VADALOG engine provides computational guarantees (how to compute).

    “This design achieves a clear separation of responsibilities: the LLM provides semantic understanding (what to compute), while the VADALOG engine provides computational guarantees (how to compute).”
    Quote in the paperSubstantial claim
47 of 47

  • The paper relies on this without support.

    Realistic concentration-risk deployments scale to thousands of entities.

    “Indeed, realistic deployments scale this complexity to thousands of entities.”
    “In our concentration risk example, a realistic analysis may involve thousands of exposure records across multiple counterparties, each possibly requiring an investigation of corporate group structures.”

  • The paper relies on this without support.

    High-stakes domains require auditable reasoning traces.

    “Without an auditable trace, the analyst cannot verify the reasoning process or assess whether a correct conclusion resulted from the right reasons. This opacity fundamentally undermines trust in high-stakes domains.”

  • The paper relies on this without support.

    Analysts need to verify the reasoning process and assess whether a correct conclusion resulted from the right reasons.

    “Without an auditable trace, the analyst cannot verify the reasoning process or assess whether a correct conclusion resulted from the right reasons.”

  • The paper relies on this without support.

    The ownership structure in the concentration-risk example cannot be known in advance; it emerges only at runtime.

    “This ownership structure cannot be known in advance: it emerges only at runtime, as the investigation traverses corporate linkages among seemingly independent counterparties.”

  • The paper relies on this without support.

    Tools may produce result sets with thousands or millions of facts, far exceeding what can be injected into an LLM's context window.

    “A central challenge in LLMorchestrated decision-making is scalability: tools may produce result sets with thousands or millions of facts, far exceeding what can be injected into an LLM's context window.”

  • The paper relies on this without support.

    Many analytical questions require combining results from multiple tools in ways that cannot be anticipated at design time.

    “Pre-registered reasoning tools encapsulate domain-specific inference, but many analytical questions require combining results from multiple tools in ways that cannot be anticipated at design time.”

  • The paper relies on this without support.

    LLMs frequently approximate incorrectly when performing multi-way joins with arithmetic expressions and aggregations in-context.

    “Operations such as multi-way joins with arithmetic expressions and aggregations, which LLMs frequently approximate incorrectly when performed in-context, are instead executed symbolically with formal correctness guarantees.”

  • The paper relies on this without support.

    RAG is the most established technique to mitigate context-window limitations.

    “RAG is particularly relevant in our experiments, as it represents the most established technique to mitigate context-window limitations.”
8 of 8

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Dependency-graph edge direction is defined both ways

    Major

    The condition says pred_i is a prerequisite of pred_j, but the accompanying explanation says the tool producing pred_i requires pred_j. These opposite directions would produce different execution orders.

  • Pipeline omits small result sets from collected data

    Minor

    Algorithm 1 adds results to D only inside the branch for results exceeding the threshold. The prose instead says results below the threshold are loaded directly into memory, which the shown algorithm never does.

2 of 2

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. Gottlob, G., and Pieris, A. 2015. Beyond sparql under owl 2 ql entailment regime: Rules to the rescue. In Twenty-Fourth International Joint Conference on Artificial Intelligence .

Cited by

Sorted by citations

Authors
Baldazzi, T., Bellomarini, L., Coletta, A., Iezzi, M., Maple, C., Pesare, A., & Sallinger, E.
Affiliations
  • TU Wien (AT)
  • Bank of Italy (IT)
  • University of Warwick (GB)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2026
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2606.22485
arXiv
2606.22485
OpenAlex
W7165693486
Semantic Scholar
d36ff0c01bd8e0d11fa305432171573541a70598
Cited by
0
References
17
Type
posted content
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
arXiv (Cornell University)
Topics
  • Social Sciences › Business, Management and Accounting › Management Information Systems › Business Process Modeling and Analysis
  • Physical Sciences › Computer Science › Artificial Intelligence › Logic, Reasoning, and Knowledge
  • Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
Keywords
neuro-symbolic reasoning, Datalog, logic programming, agentic systems, business process management, auditability, explainability, financial decision-making

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-19.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-19.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review