Agentic Governance and Adversarial Verification for Policy-Constrained LLM Healthcare Appeal Generation
Open accessAbstract
Claim denial management costs U.S. healthcare approximately $260 billion annually in administrative overhead. Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) can produce fluent clinical text, but single-agent architectures fail in high-stakes healthcare: they introduce unsupported clinical details and lose the logical structure of hierarchical payer policy. We propose AGVF (Agentic Governance and Adversarial Verification Framework), a multi-agent architecture for medical-necessity appeal generation under explicit policy and evidence constraints. AGVF models appeal synthesis as a Constrained Markov Decision Process (CMDP) over five agents: policy formalization, evidence retrieval, gap analysis, adversarial critique, and gated synthesis. We prove that refinement over a fixed policy constraint graph monotonically reduces evidence-deficiency and terminates with either a complete satisfying frontier or a localized evidence gap. A deterministic citation- grounding gate prevents assertions without admissible evidence from entering shared state. We provide a reference implementation and validate it on 1,000 synthetic appeal cases parameterized from de-identified public hospital discharge data. The validation confirms zero citation-grounding violations across all AGVF cases and monotone deficiency reduction in every episode; ablating the gate raises violations to 100%, confirming it is load-bearing. The study uses no real patient records and does not measure clinical efficacy. AGVF thus contributes a theory-backed agentic architecture and verified reference implementation for policy-constrained LLM generation in healthcare.
AI snapshot
Written from the paper’s full text.
TL;DR
AGVF is a multi-agent, CMDP-based framework for policy-constrained medical-necessity appeal generation, with deterministic citation-grounding gates and adversarial refinement over a payer policy graph. On 1,000 synthetic cases, the reference implementation upheld its formal invariants—zero citation-grounding violations and monotone deficiency reduction—while the paper explicitly does not measure clinical efficacy or payer-overturn outcomes.
Key findings
- Across all 1,000 AGVF cases, mean citation-ungrounded assertions were exactly 0.000 and the citation-grounding violation rate was 0.0%; removing the deterministic gate (AGVF-Gates) produced exactly 2.000 fabricated unsupported assertions per case and a 100% grounding-violation rate.
- Mean deficiency fell monotonically from 11.1 at initialization to 1.68 at the final state, with non-increasing deficiency in 100.0% of AGVF cases; the single-pass RAG baseline violated monotonicity in 4.0% of cases (40 cases) and produced 40 exclusion-blocked (D = ∞) cases.
- Under AGVF, 39.6% of cases reached D = 0 and 60.4% terminated at a finite positive deficiency, with no AGVF case terminating in the infeasible D = ∞ state.
- Removing the critic produced no measurable change on the synthetic, mock-LLM configuration; the paper states the critic’s value remains an open empirical question requiring a real language model and semantically noisy cases.
Methods
- The paper formalizes appeal synthesis as a Constrained Markov Decision Process over five agent roles: policy formalization, evidence retrieval, gap analysis, adversarial critique, and gated synthesis.
- Validation used 1,000 synthetic appeal cases, 250 for each of four hand-formalized policy graphs: CO-50 General (6 leaf obligations), CO-197 Prior Authorization (9), CO-50 Surgical (10), and CO-50 DME (9).
- Cases used oracle-labeled evidence pools, exact-key retrieval, a deterministic mock LLM, a maximum of N = 8 refinement rounds, a one-obligation-per-round reveal limit, and a fixed seed of 42; diagnosis, procedure, DRG, length-of-stay, and charge distributions were parameterized from aggregate de-identified SPARCS 2024 discharge data profiled over 2,196,737 records.
Limitations
- The paper states that no real denial cases or patient records are evaluated, all cases and labels are fabricated, and the study speaks to controlled system behavior rather than clinical reality.
- The paper states that it uses a deterministic stand-in rather than a production language model, so end-to-end LLM retrieval quality, adversarial critique, synthesis fluency, semantic faithfulness, latency, and cost are not fully exercised.
- The paper states that guarantees are constructive and do not show improved appeal outcomes; efficacy is unmeasured, the critic is non-differentiable in this configuration, and convergence relies on idealized assumptions of a static complete policy graph and a sound complete critic/gap signal. The text also makes evident that the citation gate checks admissibility rather than full semantic entailment, and that statutory regulatory compliance is not evaluated.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
0 of 8 cited sources checked · 0 supported · 0 flagged
- Findingown data
The RAG baseline yielded 40 cases blocked by exclusion (D = ∞).
“The RAG baseline, by contrast, produces 40 exclusion-blocked (𝐷 = ∞) cases, again reflecting the brittleness of undisciplined single-pass evidence surfacing in this synthetic setup.”
Quote in the paperNot a substantial claim - Findingown data
No case in the AGVF arm ended in the infeasible D = ∞ state.
“No AGVF case terminates in the infeasible 𝐷 = ∞ state.”
Quote in the paperSubstantial claim - Findingown data
For all 1,000 AGVF cases, the average deficiency dropped from 11.1 initially to 1.68 at the end.
“The mean deficiency falls monotonically from 11.1 at initialization to 1.68 at the final state”
Quote in the paperSubstantial claim - Findingown data
As refinement rounds advanced, the interquartile and 10th-90th-percentile bands became narrower.
“The interquartile and 10th-90th-percentile bands narrow as rounds proceed”
Quote in the paperSubstantial claim - Findingown data
In the AGVF arm, the other 60.4% ended with a finite positive deficiency, an evidentiary gap shown to the human reviewer as a localized list of missing obligations.
“the remaining 60.4% terminate at a finite positive deficiency-an evidentiary gap, surfaced to the human reviewer as a localized list of missing obligations.”
Quote in the paperSubstantial claim
- Findingreasoning
In this synthetic, mock-LLM configuration, removing the critic caused no measurable change.
“Removing the critic produces no measurable change on this synthetic, mock-LLM configuration.”
Quote in the paperNot a substantial claim - Findingown data
Ablating the deterministic citation gate raises the grounding-violation rate to 100%.
“ablating the deterministic gate raises the grounding-violation rate to 100%.”
- Also stated in: § 1.5 Key Contributions of This Paper · boosted
“An ablation confirms the deterministic citation gate is load-bearing (its removal drives the grounding-violation rate to 100%).”
Quote in the paperSubstantial claim - Also stated in: § 1.5 Key Contributions of This Paper · boosted
- Findingown data
In the AGVF arm, 39.6% of cases reached D = 0, a complete and fully anchored satisfying frontier.
“Under AGVF, 39.6% of cases reach 𝐷 = 0 (a complete, fully anchored satisfying frontier)”
Quote in the paperSubstantial claim - Study design and conductnone given
The study employed a deterministic stand-in instead of a production language model.
“This study uses a deterministic stand-in rather than a production language model.”
Quote in the paperSubstantial claim - Interpretationown data
The gate governs citation admissibility and nothing else.
“the gate governs citation admissibility and nothing else, exactly as designed.”
Quote in the paperSubstantial claim - Interpretationreasoning
Under the coverage-monotonicity assumption, minimizing the deficiency potential is a formal surrogate for the CMDP reward.
“Under Assumption 4, minimizing 𝐷 is a formal surrogate for the CMDP reward R (𝑠) = 𝑃 (Overturn | 𝑠).”
Quote in the paperSubstantial claim- The authors state this as an assumption: Coverage monotonicity: the probability of overturn is non-decreasing as satisfied obligations on a viable frontier accrue; equivalently, reward is non-increasing in deficiency.
- Study design and conductreasoning
The synthetic/mock configuration cannot exercise the critic.
“the synthetic/mock configuration cannot exercise the critic”
Quote in the paperSubstantial claim - Study design and conductown data
The evaluation did not use real denial cases or patient records.
“No real denial cases or patient records are evaluated.”
Quote in the paperSubstantial claim - Interpretationown data
The band narrowing indicates consistent behavior across cases and policies rather than an averaging artifact.
“showing the behavior is consistent across cases and policies, not an averaging artifact.”
Quote in the paperSubstantial claim - Interpretationreasoning
Lemma 3.4 gives a construction that keeps citation-ungrounded assertions out of shared state.
“a construction (Lemma 3.4) that prevents citationungrounded assertions from entering shared state.”
Quote in the paperSubstantial claim - Interpretationreasoning
Among all trajectories that respect the grounding invariant and are reachable from the initial state under monotone retrieval over verified patient facts, the terminal trajectory attains the minimum achievable deficiency.
“Combining the above: among all trajectories that (a) respect the grounding invariant of Lemma 3.4 and (b) are reachable from 𝑠 0 under monotone retrieval over F 𝑃 , 𝜏 * attains the minimum achievable deficiency.”
Quote in the paperSubstantial claim- The authors state this as an assumption: Coverage monotonicity: the probability of overturn is non-decreasing as satisfied obligations on a viable frontier accrue; equivalently, reward is non-increasing in deficiency.
- Study design and conductreasoning
Gated Synthesis applies a filter that emits an assertion only if it is citation-grounded; candidate assertions without admissible citations are dropped before the draft is formed.
“The Gated Synthesis operator applies a filter G that emits an assertion 𝑐 only if 𝑐 is citation-grounded; candidate assertions without admissible citations are dropped before the draft is formed.”
Quote in the paperSubstantial claim- Rests on an assumption: Stochastic LLM outputs alone cannot guarantee that each generated assertion is tied to an admissible evidence source.
- Interpretationreasoning
Situating the citation gate on the write path rather than as an advisory score makes the implemented citation-grounding constraint hold with probability one rather than in expectation.
“Situating the citation gate on the write path-rather than as an advisory score-is what makes the implemented citation-grounding constraint hold with probability one rather than in expectation.”
Quote in the paperSubstantial claim- Rests on an assumption: Stochastic LLM outputs alone cannot guarantee that each generated assertion is tied to an admissible evidence source.
- Study design and conductreasoning
The separation of agents behind narrow typed interfaces keeps the convergence and citation-grounding invariants auditable because every state transition is mediated by the orchestrator and recorded.
“This separation keeps the convergence and citation-grounding invariants of §3.4- §3.5 auditable, because every state transition is mediated by the orchestrator and recorded, rather than emerging from opaque agent-to-agent chatter.”
Quote in the paperSubstantial claim - Interpretationreasoning
The implementation matches properties that had already been proven.
“This section confirms the implementation is faithful to properties that are already proven.”
Quote in the paperSubstantial claim - Interpretationnone given
The paper formalizes appeal synthesis as a Constrained Markov Decision Process on a policy constraint graph.
“appeal synthesis formalized as a Constrained Markov Decision Process over a policy constraint graph”
Quote in the paperSubstantial claim - Interpretationown data
The exclusion-blocked cases from the RAG baseline indicate brittleness in undisciplined single-pass evidence surfacing within this synthetic setup.
“The RAG baseline, by contrast, produces 40 exclusion-blocked (𝐷 = ∞) cases, again reflecting the brittleness of undisciplined single-pass evidence surfacing in this synthetic setup.”
Quote in the paperNot a substantial claim - Interpretationreasoning
AGVF converts an intractable stochastic optimization over free-form text into a monotone, finite, evidence-bounded search whose admissibility properties are explicit and auditable.
“AGVF thereby converts an intractable stochastic optimization over free-form text into a monotone, finite, evidence-bounded search whose admissibility properties are explicit and auditable.”
Quote in the paperSubstantial claim - Interpretationown data
The observed monotone convergence matches Theorem 3.2 under the stated assumptions.
“matching Theorem 3.2 under the stated assumptions.”
Quote in the paperSubstantial claim- The authors state this as an assumption: Within an episode, the policy graph is generated once, remains fixed, and represents the operative criteria.
- The authors state this as an assumption: Retrieval adds verified facts only, and no verified fact is withdrawn during an episode.
- The authors state this as an assumption: The Critic does not contest obligations that are already anchored and does not conceal open obligations.
- Interpretationown data
The implemented admissibility guarantee depends on the citation gate.
“This result confirms that the gate is load-bearing for the implemented admissibility guarantee.”
Quote in the paperSubstantial claim - Interpretationreasoning
For every state reachable under the gated Synthesis operator, the citation-grounding violation count is zero.
“Lemma 3.4 (Citation-grounding invariant).For every state 𝑠 reachable under the gated Synthesis operator, 𝐶 cite (𝑠) = 0.”
Quote in the paperSubstantial claim- Rests on an assumption: Stochastic LLM outputs alone cannot guarantee that each generated assertion is tied to an admissible evidence source.
- Interpretationreasoning
The terminal trajectory is a constrained optimum of the objective only with respect to the evidence made reachable by the configured retrieval process and the implemented admissibility gates.
“Hence 𝜏 * is a constrained optimum of objective (3) only with respect to the evidence made reachable by the configured retrieval process and the implemented admissibility gates.”
Quote in the paperSubstantial claim - Interpretationnone given
AGVF is a multi-agent framework that converts healthcare appeal generation from unconstrained text production into constrained refinement bounded by evidence.
“This paper presented AGVF, a multi-agent framework that turns healthcare appeal generation from unconstrained text production into constrained, evidence-bounded refinement.”
Quote in the paperSubstantial claim - Interpretationreasoning
Because the gate is a deterministic predicate over verified patient facts, the citation-grounding guarantee holds with probability one, independent of the underlying LLM's sampling.
“Because G is a deterministic predicate over F 𝑃 -not a probabilistic scorer-the guarantee holds with probability one, independent of the underlying LLM's sampling.”
Quote in the paperSubstantial claim- Rests on an assumption: Stochastic LLM outputs alone cannot guarantee that each generated assertion is tied to an admissible evidence source.
- Interpretationown data
Monotonicity is a property of the disciplined refinement loop rather than of retrieval generally.
“This illustrates that the invariant is a property of the disciplined refinement loop, not of retrieval in general.”
Quote in the paperSubstantial claim - Interpretationreasoning
Theorem 3.2 provides a convergence result: refinement on the graph reduces deficiency monotonically and ends in well-characterized states under explicit assumptions.
“a convergence result (Theorem 3.2) showing that refinement on that graph reduces deficiency monotonically and terminates in well-characterized states under explicit assumptions”
Quote in the paperSubstantial claim- The authors state this as an assumption: Within an episode, the policy graph is generated once, remains fixed, and represents the operative criteria.
- The authors state this as an assumption: Retrieval adds verified facts only, and no verified fact is withdrawn during an episode.
- The authors state this as an assumption: The Critic does not contest obligations that are already anchored and does not conceal open obligations.
- May combine two claims. 32
- Classified as probably not a substantial claim. 17
- The quote only partly states this claim. 1
May combine two claims.
Findingown dataAcross all 1,000 AGVF cases, the mean number of citation-ungrounded assertions was 0.000 and the citation-grounding violation rate was 0.0%.
“Across all 1,000 AGVF cases the mean number of citation-ungrounded assertions is exactly 0.000, and the citationgrounding violation rate is 0.0%-every emitted assertion cites at least one verified fact, as Lemma 3.4 requires (Figure 4).”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataWhen the gate was removed and two fabricated unsupported assertions were injected per case, the AGVF-Gates arm had exactly 2.000 citation-ungrounded assertions per case and a 100% grounding-violation rate.
“The AGVF-Gates arm, with the gate removed and two fabricated unsupported assertions injected per case, shows exactly 2.000 citation-ungrounded assertions per case and a 100% grounding-violation rate.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataWhen the citation-grounding gate was removed, the grounding-violation rate rose by 100 percentage points and fabricated assertions increased by 2.0 per case, with convergence and solved-rate unchanged.
“Removing the citation-grounding gate degrades the grounding-violation rate by +100 percentage points and adds +2.0 fabricated assertions per case, while leaving convergence and solved-rate untouched”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataValidation on 1,000 synthetic appeal cases confirms zero citation-grounding violations across all AGVF cases and monotone deficiency reduction in every episode.
“The validation confirms implementation fidelity for the formal invariants: zero citation-grounding violations across all AGVF cases and monotone deficiency reduction in every episode”
- Also stated in: § 1.5 Key Contributions of This Paper · plain
“validate on 1,000 synthetic cases that the implementation upholds its formal invariants: zero citation-grounding violations and monotone convergence in all AGVF episodes.”
Quote in the paperSubstantial claim- Also stated in: § 1.5 Key Contributions of This Paper · plain
May combine two claims.
FindingreasoningThe RAG baseline and the AGVF-Critic arm showed no citation-grounding violations.
“The RAG baseline and the AGVF-Critic arm likewise show no violations, because neither disables the gate.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataIn every case and at every step, D(s) did not increase; the monotone-convergence rate was 100.0%.
“𝐷 (𝑠) is non-increasing at every step of every case: the monotone-convergence rate is 100.0%,matching Theorem 3.2 under the stated assumptions.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningUnder explicit assumptions, AGVF's refinement over a fixed policy constraint graph monotonically reduces a formal evidence-deficiency potential and terminates either with a complete satisfying policy frontier or with a localized residual evidence gap.
“We prove that, under explicit assumptions, refinement over a fixed policy constraint graph monotonically reduces a formal evidence-deficiency potential and terminates either with a complete satisfying policy frontier or with a localized residual evidence gap.”
- Also stated in: § 1.5 Key Contributions of This Paper · plain
“We prove that refinement over a static policy graph with append-only evidence monotonically reduces a formal deficiency potential and terminates in either a complete satisfying frontier or a residual evidence gap.”
Quote in the paperSubstantial claim- The authors state this as an assumption: Policy ingestion is invoked once per episode.
- The authors state this as an assumption: The evidence store is monotone, meaning verified facts are append-only and cannot be retracted mid-episode.
- The authors state this as an assumption: Coverage monotonicity: the probability of overturn is non-decreasing as satisfied obligations on a viable frontier accrue; equivalently, reward is non-increasing in deficiency.
- Also stated in: § 1.5 Key Contributions of This Paper · plain
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenRegulatory-template validation is a separate deterministic checklist interface in the reference implementation and is not evaluated as a statutory compliance guarantee in Section 5.
“Regulatory-template validation is a separate deterministic checklist interface in the reference implementation; it is not evaluated as a statutory compliance guarantee in Section 5.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningBecause the citation-grounding invariant holds at every reachable state, it holds at the terminal state: the unsupported-assertion constraint is zero for the implemented predicate, discharging the corresponding CMDP constraint by construction rather than in expectation.
“Because Lemma 3.4 holds at every reachable state, it holds at the terminal 𝜏 * : C unsupported (𝜏 * ) = 0 for the implemented citation-admissibility predicate, discharging the corresponding CMDP constraint by construction rather than in expectation.”
Quote in the paperSubstantial claim- Rests on an assumption: Stochastic LLM outputs alone cannot guarantee that each generated assertion is tied to an admissible evidence source.
Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.
Interpretationnone givenIn practice, policy ingestion is imperfect, and a real critic may be unsound or incomplete.
“Real policy ingestion is imperfect and a real critic may be unsound or incomplete”
Quote in the paperNot a substantial claimMay combine two claims.
Study design and conductreasoningThe validation is a validation of implementation fidelity on a controlled synthetic benchmark, not a demonstration of real-world efficacy.
“This is a validation of implementation fidelity on a controlled synthetic benchmark, not a demonstration of real-world efficacy-we return to that distinction in Section 5.7.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S4 (based). May combine two claims.
Interpretationnone givenThe convergence guarantee depends on Assumption 1, a static and complete policy graph, and Assumption 3, a sound and complete critic/gap signal.
“The convergence guarantee relies on Assumptions 1 (a static, complete policy graph) and 3 (a sound and complete critic/gap signal).”
Quote in the paperNot a substantial claim- The authors state this as an assumption: Within an episode, the policy graph is generated once, remains fixed, and represents the operative criteria.
- The authors state this as an assumption: The Critic does not contest obligations that are already anchored and does not conceal open obligations.
May combine two claims.
PremisecitationSelf-refinement methods offer no termination or admissibility guarantee; the loop may oscillate, plateau, or improve toward a fluent but unsupported output.
“Second, self-refinement methods offer no termination or admissibility guarantee: the loop may oscillate, plateau, or”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [7]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [12]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S2 (checkable).
Interpretationnone givenTo measure efficacy credibly, one needs clinician- or payer-adjudicated outcomes on real de-identified cases.
“A credible efficacy measure requires clinician-adjudicated or payer-adjudicated outcomes on real de-identified cases”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningAGVF's refinement game is played on a fixed, finite policy-constraint board with a monotone evidence store, which forces non-increasing deficiency.
“Unlike unconstrained debate-where convergence is not guaranteed [4]-AGVF's game is played on the fixed, finite board 𝐺 𝐾 with a monotone evidence store.This structure forces non-increasing deficiency, as we now show.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [4]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- The authors state this as an assumption: Policy ingestion is invoked once per episode.
- The authors state this as an assumption: The evidence store is monotone, meaning verified facts are append-only and cannot be retracted mid-episode.
May combine two claims.
PremisecitationGenerate-critique-revise loops measurably improve LLM output, motivating AGVF's refinement structure.
“These methods establish that a generate-critique-revise loop measurably improves LLM output, which motivates AGVF's refinement structure.”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [7]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [12]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims.
Study design and conductreasoningBecause each agent contract is checkable at the orchestrator boundary, a misbehaving agent can be caught before it silently corrupts the loop, providing an operational safeguard for the assumptions the proofs rely on.
“Because each contract is checkable at the orchestrator boundary, a misbehaving agent (e.g., a Critic that violates soundness or hides open obligations) can be caught before it silently corrupts the loop-an operational safeguard for the assumptions the proofs rely on.”
Quote in the paperSubstantial claimMay combine two claims.
Study design and conductown dataThe study uses no real patient records and does not measure clinical efficacy or payer-overturn outcomes.
“The study uses no real patient records and does not measure clinical efficacy or payer-overturn outcomes.”
Quote in the paperSubstantial claimMay combine two claims.
Interpretationown dataWhen the deterministic gate is active, assertions lacking valid citations cannot enter shared state; when it is absent, every injected case becomes contaminated.
“The contrast is categorical rather than incremental: with the deterministic gate in place, citation-free or invalidly cited assertions cannot enter shared state; without it, every injected case is contaminated.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S4 (based).
InterpretationreasoningFor an agentic LLM system facing hard external constraints, representing those constraints as a formal graph and using deterministic gates for admissibility converts an unbounded generation problem into a bounded, auditable one with explicit assumptions and testable invariants.
“when an agentic LLM system must satisfy hard external constraints, encoding those constraints as a formal graph and enforcing admissibility through deterministic gates-instead of relying only on prompting or post-hoc review-turns an unbounded generation problem into a bounded, auditable one with explicit assumptions and testable invariants.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenThe one-obligation-per-round reveal limit affects solved-rate and round-count differences between arms; these should be interpreted as descriptive of loop behavior, not as efficacy comparisons.
“Solved-rate and round-count differences across arms are influenced by the one-obligation-per-round reveal limit and should be read as descriptive of loop behavior, not as efficacy comparisons.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationNeither debate nor Tree-of-Thought offers convergence or compliance guarantees; free-form debate may not converge.
“Neither method, however, offers convergence or compliance guarantees; even the debate literature acknowledges that free-form debate may not converge.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [4]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductreasoningSection 5 does not evaluate statutory compliance because no jurisdiction-specific checklist is populated.
“a regulatory-template checker, but Section 5 does not evaluate statutory compliance because no jurisdiction-specific checklist is populated.”
Quote in the paperNot a substantial claimMay combine two claims.
Study design and conductreasoningThe study neither shows nor can show that AGVF improves appeal outcomes.
“It does not, and cannot, show that AGVF improves appeal outcomes.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningAll cases, denial framings, clinical evidence items, and obligation labels were fabricated, so the study addresses controlled system behavior rather than clinical reality.
“every case, denial framing, clinical evidence item, and obligation label is fabricated; the study therefore speaks to controlled system behavior, not clinical reality.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningAGVF restricts critique to a fixed policy-constraint board and pairs refinement with deterministic gates, which produces the guarantees in Section 3.
“AGVF restricts critique to a fixed policy-constraint board and pairs refinement with deterministic gates, which is what produces the guarantees in Section 3.”
Quote in the paperSubstantial claim- Rests on an assumption: Single-agent LLM/RAG architectures are poorly matched to high-stakes administrative healthcare because they may introduce unsupported clinical details, lose hierarchical policy structure, and provide weak post-hoc evidence.
May combine two claims.
InterpretationreasoningUnder the implemented gates, the citation-grounding and convergence invariants do not depend on the LLM, so they carry over at the control-logic level.
“The citation-grounding and convergence invariants are LLM-independent under the implemented gates, so they transfer at the control-logic level”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningBecause the deficiency engine is deterministic and the model is a mock, the critic's contested-obligation set matched the gap-analysis output, making the critic indistinguishable in this setting.
“with a deterministic deficiency engine and a mock model, the critic's contested-obligation set coincides with the gap-analysis output, so the critic cannot be differentiated here.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningWith the benchmark evidence source, the system either fully satisfies the criterion or reports the remaining gap; it does not fabricate to close a gap.
“operational expression of Theorem 3.2's dichotomy under the benchmark evidence source: the system either fully substantiates the criterion or reports what remains missing; it does not close a gap by fabrication.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductown dataThe paper does not claim that its formal result proves real-world overturn probability.
“We do not claim that this proves real-world overturn probability.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
Interpretationnone givenFraming medical-necessity appeal generation as constrained multi-agent refinement over a policy graph with deterministic admissibility gates is novel to this work.
“To our knowledge, framing medical-necessity appeal generation as constrained multi-agent refinement over a policy graph with deterministic admissibility gates is novel to this work.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationMultiple LLM instances debating over several rounds improves factual accuracy, and Tree-of-Thought search lets a model explore alternative reasoning paths before committing.
“Having multiple LLM instances debate over several rounds improves factual accuracy [4], and Tree-of-Thought search lets a model explore alternative reasoning paths before committing [13].”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [4]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [13]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Classified as probably not a substantial claim.
Study design and conductreasoningThe merge step is append-only for verified facts, so no agent can retract a previously grounded fact mid-episode.
“This makes the monotone-evidence assumption (Assumption 2) enforceable in code: the merge step is append-only for verified facts, so no agent can retract a previously grounded fact mid-episode.”
Quote in the paperSubstantial claim- The authors state this as an assumption: The evidence store is monotone, meaning verified facts are append-only and cannot be retracted mid-episode.
Probably not a substantial claim: may not pass S4 (based). May combine two claims. The quote only partly states this claim.
Study design and conductreasoningAGVF checks admissibility on the write path; no assertion enters the draft unless it cites verified evidence in the record, so the human reviewer audits a document whose claims are citation-grounded by construction.
“No assertion enters the draft unless it cites verified evidence in the record, so the human reviewer audits a document whose claims are citation-grounded by construction.”
Quote in the paperNot a substantial claim- Rests on an assumption: In a regulatory setting, a fluent but unsupported statement is a liability that can implicate fraud statutes when it enters a payer-facing document.
May combine two claims.
InterpretationreasoningAGVF replaces open-ended self-feedback with refinement bounded by a formal constraint graph and a deficiency potential, yielding monotone convergence of Theorem 3.2, and replaces soft self-critique with a deterministic citation-grounding gate, yielding the admissibility invariant of Lemma 3.4.
“AGVF replaces open-ended self-feedback with refinement bounded by a formal constraint graph and a deficiency potential, yielding the monotone convergence of Theorem 3.2, and replaces soft self-critique with a deterministic citation-grounding gate, yielding the admissibility invariant of Lemma 3.4.”
Quote in the paperNot a substantial claim- Rests on an assumption: Single-agent LLM/RAG architectures are poorly matched to high-stakes administrative healthcare because they may introduce unsupported clinical details, lose hierarchical policy structure, and provide weak post-hoc evidence.
Classified as probably not a substantial claim.
InterpretationreasoningThe guarantees are constructive.
“The guarantees are constructive”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Study design and conductnone givenThe implementation evaluated in Section 5 is intentionally narrower: hand-formalized policy graphs, synthetic oracle-labeled evidence pools, exact-key retrieval, a deterministic mock LLM, and a sound deterministic critic.
“The implementation evaluated in Section 5, however, is intentionally narrower: hand-formalized policy graphs, synthetic oracle-labeled evidence pools, exact-key retrieval, a deterministic mock LLM, and a sound deterministic critic.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Study design and conductnone givenThe unsupported-assertion constraint is enforced as a hard, deterministic citation-admissibility gate inside the Synthesis operator.
“The unsupported-assertion constraint C unsupported (𝜏) = 0 of the CMDP is enforced not as a soft penalty but as a hard, deterministic citation-admissibility gate inside the Synthesis operator.”
Quote in the paperSubstantial claim- Rests on an assumption: Stochastic LLM outputs alone cannot guarantee that each generated assertion is tied to an admissible evidence source.
Classified as probably not a substantial claim. May combine two claims.
Study design and conductreasoningThe citation-grounding guarantee is a syntactic admissibility guarantee and should not be confused with full semantic entailment; a separate verifier is needed to prove that the wording of a cited assertion is completely entailed by the cited span.
“It should not be confused with full semantic entailment: a separate verifier is needed to prove that the wording of a cited assertion is completely entailed by the cited span.”
Quote in the paperSubstantial claimMay combine two claims.
Study design and conductreasoningThe study did not fully exercise end-to-end LLM retrieval quality, adversarial critique, synthesis fluency, or semantic faithfulness; reported latencies reflect loop overhead, not production model inference.
“end-to-end LLM retrieval quality, adversarial critique, synthesis fluency, and semantic faithfulness are not fully exercised, and reported latencies reflect loop overhead rather than production model inference.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Study design and conductnone givenAGVF's deterministic citation-grounding gate prevents assertions without admissible evidence citations from entering shared state.
“We also define a deterministic citation-grounding gate that prevents assertions without admissible evidence citations from entering shared state.”
Quote in the paperSubstantial claim- Rests on an assumption: Stochastic LLM outputs alone cannot guarantee that each generated assertion is tied to an admissible evidence source.
May combine two claims.
InterpretationreasoningThe citation-grounding and monotone-deficiency properties hold by construction from the formal results; the purpose of the study is to verify that the implementation honors them and to quantify the effect of removing the mechanisms that enforce them.
“The citation-grounding and monotone-deficiency properties hold by construction (Lemmas 3.1-3.4,Theorem 3.2); the purpose of the study is therefore to verify that the implementation honors them and to quantify the effect of removing the mechanisms that enforce them.”
Quote in the paperSubstantial claim- The authors state this as an assumption: Policy ingestion is invoked once per episode.
- The authors state this as an assumption: The evidence store is monotone, meaning verified facts are append-only and cannot be retracted mid-episode.
- The authors state this as an assumption: Coverage monotonicity: the probability of overturn is non-decreasing as satisfied obligations on a viable frontier accrue; equivalently, reward is non-increasing in deficiency.
The authors state this as an assumption.
Policy ingestion is invoked once per episode.
“Invoked once per episode (Assumption 1).”
The authors state this as an assumption.
The evidence store is monotone, meaning verified facts are append-only and cannot be retracted mid-episode.
“This makes the monotone-evidence assumption (Assumption 2) enforceable in code”
The paper relies on this without support.
Single-agent LLM/RAG architectures are poorly matched to high-stakes administrative healthcare because they may introduce unsupported clinical details, lose hierarchical policy structure, and provide weak post-hoc evidence.
“single-agent architectures are poorly matched to high-stakes administrative healthcare: they may introduce unsupported clinical details, lose the logical structure of hierarchical payer policy, and provide only weak post-hoc evidence that generated assertions are admissible.”
The authors state this as an assumption.
Coverage monotonicity: the probability of overturn is non-decreasing as satisfied obligations on a viable frontier accrue; equivalently, reward is non-increasing in deficiency.
“Assumption 4 (Coverage monotonicity).𝑃 (Overturn | 𝑠) is non-decreasing as satisfied obligations on a viable frontier accrue; equivalently, the reward R (𝑠) is non-increasing in 𝐷 (𝑠).”
The paper relies on this without support.
Generative models can extrapolate clinical details, exposing providers to legal liability and fraud allegations under the False Claims Act.
“Generative models can extrapolate clinical details (e.g., asserting a patient failed a conservative therapy that was never documented), exposing providers to legal liability and fraud allegations under the False Claims Act.”
The paper relies on this without support.
Standard semantic retrieval may fail to preserve nested logical dependencies, retrieving fragments that are topically relevant but procedurally insufficient.
“Standard semantic retrieval alone may fail to preserve these logical dependencies, retrieving fragments that are topically relevant but procedurally insufficient.”
The paper relies on this without support.
Standard text generators create passive summaries and fail to anticipate the rebuttal strategies of insurance medical directors trained to deny appeals based on minor evidentiary omissions.
“A standard text generator creates passive summaries.It fails to anticipate the rebuttal strategies employed by insurance company medical directors trained to deny appeals based on minor evidentiary omissions.”
The paper relies on this without support.
Stochastic LLM outputs alone cannot guarantee that each generated assertion is tied to an admissible evidence source.
“Stochastic LLM outputs alone cannot guarantee that each generated assertion is tied to an admissible evidence source.”
The paper relies on this without support.
Real denial packets combine protected health information, payer contracts, and operationally sensitive adjudication outcomes, making public benchmarking difficult.
“Real denial packets combine protected health information, payer contracts, and operationally sensitive adjudication outcomes, making public benchmarking difficult.”
The paper relies on this without support.
In a regulatory setting, a fluent but unsupported statement is a liability that can implicate fraud statutes when it enters a payer-facing document.
“In a regulatory setting, a fluent but unsupported statement is not a minor error; it is a liability that can implicate fraud statutes when it enters a payer-facing document.”
The authors state this as an assumption.
Within an episode, the policy graph is generated once, remains fixed, and represents the operative criteria.
“Assumption 1 (Theorem 3 . 2 ( Static, complete policy graph).Within an episode, I produces 𝐺 𝐾 once and it does not change; Φ 𝐺 captures the operative criteria.”
The authors state this as an assumption.
Retrieval adds verified facts only, and no verified fact is withdrawn during an episode.
“Assumption 2 (Monotone evidence).Retrieval only adds verified facts, 𝐹 𝑡 +1 ⊇ 𝐹 𝑡 ; a verified fact is never retracted mid-episode.”
The authors state this as an assumption.
The Critic does not contest obligations that are already anchored and does not conceal open obligations.
“Assumption 3 (Sound and complete critiqe).The Critic never contests an already-anchored obligation and does not hide open obligations: 𝑊 𝑡 = Δ(𝑠 𝑡 ), and therefore 𝑣 ∈ 𝑊 𝑡 ⇒ sat 𝐹𝑡 (𝑣) = 0.”
Cites
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.11401 · Cited by 3,101 · see citing works
Constrained Markov Decision Processes
Altman, E. (1999) · Routledge eBooks · doi:10.1201/9781315140223 · Cited by 2,265 · see citing works
Tree of Thoughts: Deliberate Problem Solving with Large Language Models
Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T. L., Cao, Y., & Narasimhan, K. (2023) · arXiv · doi:10.48550/arxiv.2305.10601 · Cited by 1,158 · see citing works
Reflexion: Language Agents with Verbal Reinforcement Learning
Shinn, N., Cassano, F., Berman, E., Gopinath, A., Narasimhan, K., & Yao, S. (2023) · arXiv · doi:10.48550/arxiv.2303.11366 · Cited by 1,087 · see citing works
Self-Refine: Iterative Refinement with Self-Feedback
Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., Alon, U., Dziri, N., Prabhumoye, S., Yang, Y., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.17651 · Cited by 219 · see citing works
From local to global: a graph rag approach to query-focused summarization
Edge, D., Trinh, H., Cheng, N., Bradley, J., Chao, A., Mody, A., Truitt, S., Metropolitansky, D., Ness, R. O., & Larson, J. (2024) · arXiv · doi:10.48550/arxiv.2404.16130 · Cited by 159 · see citing works
Improving Factuality and Reasoning in Language Models through Multiagent Debate
Du, Y., Li, S., Torralba, A., Tenenbaum, J. B., & Mordatch, I. (2023) · arXiv · doi:10.48550/arxiv.2305.14325 · Cited by 96 · see citing works
RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval
Sarthi, P., Abdullah, S., Tuli, A., Khanna, S., Goldie, A., & Manning, C. D. (2024) · arXiv · doi:10.48550/arxiv.2401.18059 · Cited by 36 · see citing works
Flagged first, then by citations
4 of the 12 references couldn’t be matched to a known paper.
- Authors
- Lodhiya, H., McManus, A., & Walker, R.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2026
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2609.27844
- OpenAlex
- W7214244524
- Semantic Scholar
- defc7015cef817eea205e179e307d1a174e8f70d
- Cited by
- 0
- References
- 12
- Type
- posted content
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (cc-by-nc-nd)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- multi-agent systems, large language models, retrieval-augmented generation, adversarial verification, constrained Markov decision process, evidence retrieval, citation grounding
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-07.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-07.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review