GIER: Gap-Driven Self-Refinement for Large Language Models

Probably not a substantial claim: may not pass S3 (load-bearing).

The conceptual gaps for e-SNLI are drawn from the inference ontology developed by Nie et al. (2020) in the ANLI dataset, resulting in five gap types.

“To define conceptual gaps for this task, we draw on the inference ontology developed by Nie et al. (2020) in the ANLI dataset. This results in five gap types, capturing diverse reasoning pathways: quantitative and comparative reasoning , logical inferences , lexical inferences , pragmatic inferences , and reference resolution .”

• cites Nie et al. (2020) sourcePartly supported

  Section 5 (“Linguistic analysis”; page unavailable) explicitly says the ANLI authors devised an inference ontology, so the citation supports drawing on that ontology. But the paper defines six inference types, not the citing sentence’s five gap types; the five are the citing authors’ adaptation, and their labels do not map exactly onto those in ANLI. (LLM, conf=0.86)

  Experimental — accuracy not yet measured

May combine two claims.

Rationale supervision work such as e-SNLI and ERASER provides gold explanations for post hoc evaluation or training but does not embed quality criteria into the generation process itself.

“Rationale supervision work, such as e-SNLI (Camburu et al., 2018) and ERASER (DeYoung et al., 2020), provides gold explanations for post hoc evaluation or training but does not embed quality criteria into the generation process itself.”

• cites Camburu et al., 2018 sourcePartly supported

  The Introduction (p. 2) identifies e-SNLI as a corpus of human-annotated explanations intended for training and generation, and §4.3 (p. 6) reports manual evaluation of generated explanations. The paper does not establish the broader claim that quality criteria are absent from the generation process: §3 (p. 3) describes criteria used to collect and filter explanations, while §4.3 reports an attention-based model designed to improve explanation quality. (LLM, conf=0.80)

  Experimental — accuracy not yet measured
• cites DeYoung et al., 2020 sourceSupported

Probably not a substantial claim: may not pass S3 (load-bearing).

Self-refinement methods show that models can improve through iterative critique or feedback, often using instructive signals or revision exemplars.

“Self-refinement methods (Madaan et al., 2023) show that models can improve through iterative critique or feedback, often using instructive signals or revision exemplars.”

• cites Madaan et al., 2023 sourceSupported

May combine two claims.

Across three reasoning-intensive tasks (SciFact, PrivacyQA, and eSNLI) and four LLMs (GPT-4.1, GPT-4o Mini, Gemini 1.5 Pro, and Llama 3.3 70B), GIER improves rationale quality, grounding, and reasoning alignment without degrading task accuracy.

“Across three reasoningintensive tasks (SciFact, PrivacyQA, and eSNLI) and four LLMs (GPT-4.1, GPT-4o Mini, Gemini 1.5 Pro, and Llama 3.3 70B), GIER improves rationale quality, grounding, and reasoning alignment without degrading task accuracy.”

• AssumptionRests on an assumption: Conceptual gaps are human-readable, generalizable, and semantically grounded, enabling both LLMs and human evaluators to reason about quality in principled terms.

May combine two claims.

On e-SNLI, Llama improves most in attribution quality (+50.0%), followed by GPT-4o Mini (+21.5%), both from lower baselines.

“On the reasoning-centric e-SNLI task, Llama improves most in attribution quality ( +50 . 0% ), followed by GPT-4o Mini ( +21 . 5% ), both from lower baselines.”

• AssumptionRests on an assumption: GPT-4.1 can segment human rationale, LLM rationale, and gap-level explanations into reasoning chunks.
• AssumptionRests on an assumption: The DeBERTa-v3-large-mnli-fever-anli-ling-wanli model provides valid NLI scores for attributing human reasoning chunks to LLM chunks.

May combine two claims.

In SciFact, among 45 correctly labeled samples with three or more sentences contributing to the combined gold rationale, 24 rationales simply enumerate sentences using surface-level transitions, while 21 display connective reasoning.

“There were 45 such cases. In 24 of the 45 samples, the rationale simply enumerates sentences using surface-level transitions such as 'The text states..., ' 'Additionally..., ' or 'The study notes.... ' In contrast, the remaining 21 rationales display connective reasoning”

May combine two claims.

In PrivacyQA, GIER improves recall across models but often at the cost of precision and thematic stability.

“In PrivacyQA, GIER improves recall across models but often at the cost of precision and thematic stability.”

May combine two claims.

In PrivacyQA, switching to gap-informed prompting maintains decision accuracy and selection recall, while iterative GIER refinements substantially boost the selection of correct sentences (+46.7% gain).

“Switching to gap-informed prompting maintains decision accuracy and selection recall, while iterative GIER refinements substantially boost the selection of correct sentences (+46 . 7% gain).”

May combine two claims.

GPT-4.1 benefits most in SciFact, showing a +14.9% gain in rationale recall and +8.6% in grounding ratio.

“GPT-4.1, with strong reasoning and instruction-following skills, benefits most in SciFact, showing a +14 . 9% gain in rationale recall and +8 . 6% in grounding ratio.”

May combine two claims.

In the first iteration, rationale recall increases by up to +8.3% in SciFact and sentence recall by +16.6% in PrivacyQA, with moderate gains continuing through the third iteration.

“In the first iteration, rationale recall increases by up to +8 . 3% in SciFact and sentence recall by +16 . 6% in PrivacyQA, with moderate gains continuing through the third iteration.”

May combine two claims.

In e-SNLI, decision accuracy remains stable, but human-aligned reasoning discovery improves markedly with GIER.

“In e-SNLI, decision accuracy remains stable, but human-aligned reasoning discovery improves markedly with GIER.”

May combine two claims.

Iterative GIER refinements lead to further improvement in SciFact rationales recall (+14.9% gain) and grounding ratio (+8.6%), while decision accuracy remains stable.

“Iterative GIER refinements then lead to further improvement, particularly in rationales recall ( +14 . 9% gain) and grounding ratio ( +8 . 6% ), indicating stronger textual anchoring and conciseness, while decision accuracy remains stable.”

May combine two claims.

Despite GPT-4.1 achieving high attribution scores in e-SNLI, it produced incorrect labels for 45 samples.

“Despite GPT-4.1 achieving high attribution scores in e-SNLI, it produced incorrect labels for 45 samples.”

• AssumptionRests on an assumption: GPT-4.1 can segment human rationale, LLM rationale, and gap-level explanations into reasoning chunks.
• AssumptionRests on an assumption: The DeBERTa-v3-large-mnli-fever-anli-ling-wanli model provides valid NLI scores for attributing human reasoning chunks to LLM chunks.

May combine two claims.

In PrivacyQA final GIER iteration, FPCU questions show 82.6% thematic alignment, and TPSC questions show a high concentration of on-theme selections with spillovers into FPCU and UCC.

“We observe strong thematic alignment for questions about FPCU ( 82 . 6% alignment rate), and TPSC questions show a high concentration of on-theme selections, with frequent but conceptually adjacent spillovers into FPCU and UCC.”

May combine two claims.

In e-SNLI, using GPT-4.1, 827 of 1,000 randomly sampled instances are classified correctly, suggesting many examples may be easy cases for the LLM.

“Using GPT-4.1, we run a standard NLI task to determine which instances it can or cannot label correctly. 827 are classified correctly, suggesting that many examples may be 'easy' cases for the LLM.”

May combine two claims.

The minor decline in reasoning attribution on e-SNLI is not significant, while the decline in selection precision on PrivacyQA is; the confidence interval for mean change in thematic drift ([+0.33, +1.09]) suggests that spurious sentences with multiple inferential steps beyond the question are rarely introduced.

“The minor decline in reasoning attribution on e-SNLI is not significant, while the decline in selection precision on PrivacyQA is-though the corresponding confidence interval for mean change in thematic drift ( [+0 . 33 , +1 . 09] ) suggests that spurious sentences with multiple inferential steps beyond the question are rarely introduced.”

May combine two claims.

The conciseness gap in SciFact controls verbosity and grounding, while thematic overreach acts as a soft constraint to suppress conceptual drift in PrivacyQA.

“The conciseness gap in SciFact controls verbosity and grounding, while thematic overreach acts as a soft constraint to suppress conceptual drift in PrivacyQA.”

May combine two claims.

In PrivacyQA, the accuracy drop (70% to 63%) is not due to failures in identifying truly answerable questions, which rises from 41 to 46 of the 50 answerable questions; rather, it is due to a rise in cases where unanswerable questions are marked as answerable (21 to 33 cases).

“Also, the accuracy drop ( 70% → 63% ) is not due to failures in identifying truly answerable questions, which rises from 41 → 46 of the 50 answerable questions. Rather, it is due to a rise in cases were unanswerable questions are marked as answerable ( 21 → 33 cases).”

May combine two claims.

Introducing conceptual gap definitions to the prompt yields a +20.9% relative gain (62% to 75%) in capturing human reasoning chunks in e-SNLI, with the final iteration sustaining this improvement.

“Introducing conceptual gap definitions to the prompt yields a +20 . 9% relative gain ( 62% → 75% ) in capturing human reasoning chunks, with the final iteration sustaining this improvement.”

• AssumptionRests on an assumption: GPT-4.1 can segment human rationale, LLM rationale, and gap-level explanations into reasoning chunks.
• AssumptionRests on an assumption: The DeBERTa-v3-large-mnli-fever-anli-ling-wanli model provides valid NLI scores for attributing human reasoning chunks to LLM chunks.

May combine two claims. The quote only partly states this claim.

Among the 45 incorrectly labeled e-SNLI samples, 20 were aligned, 15 were counter, and the rest misaligned.

“20 cases were aligned , e.g., the gold rationale 'Not all beaches are sandy' supports a neutral label, while the model chose entailment, stating 'The context says the dog is on a beach, but does not specify if it is sandy; most beaches are sandy, so the entailment is very strong but not certain. '”

• Also stated in: § 7 Discussion · p. 8 · plain

  “15 were counter , e.g., the model refutes the gold rationale 'Partially-drunk beverages suggests one is sitting at a table drunk' (entailment) with 'The context only shows young people with partially-consumed drinks, not that they are drunk or that the drinks are alcoholic; the statement claims more than is supported' (neutral).”

• AssumptionRests on an assumption: GPT-4.1 can segment human rationale, LLM rationale, and gap-level explanations into reasoning chunks.
• AssumptionRests on an assumption: The DeBERTa-v3-large-mnli-fever-anli-ling-wanli model provides valid NLI scores for attributing human reasoning chunks to LLM chunks.

May combine two claims.

Dropping the coverage gap (in SciFact and PrivacyQA) notably reduces retrieval breadth but sometimes improves decision accuracy by encouraging more conservative outputs (PrivacyQA).

“Dropping the coverage gap (in SciFact and PrivacyQA) notably reduces retrieval breadth but sometimes improves decision accuracy by encouraging more conservative outputs (PrivacyQA).”

May combine two claims.

Across all tasks, most metrics stabilize by the third iteration, with only minor and statistically insignificant variations thereafter.

“Across all tasks, we observe that most metrics stabilize by the third iteration, with only minor and statistically insignificant variations thereafter.”

May combine two claims.

By iteration 3, most metrics plateau, with gains becoming smaller and p-values increasing.

“By iteration 3, most metrics plateau, with gains becoming smaller and p -values increasing.”

May combine two claims.

Sentence precision degrades steadily over the same period, suggesting a tradeoff between recall and focus.

“Notably, sentence precision degrades steadily over the same period, suggesting a tradeoff between recall and focus.”

May combine two claims.

Grounding ratio increases through iteration 4 but stalls afterward.

“Grounding ratio increases through iteration 4 but stalls afterward.”

May combine two claims.

Thematic drift has the highest jump of +62.7% in the initial prompt, while later stages show statistically insignificant changes.

“Thematic drift has the highest jump of +62 . 7% in the initial prompt, while later stages show statistically insignificant changes.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The highest input token size of 20,296 appears in iteration 5 for PrivacyQA, but is within the context window length of all models.

“The highest input token size of 20,296 appears in iteration 5 for PrivacyQA, but is within the context window length of all models.”

May combine two claims.

In SciFact, rationale recall improves significantly from GIER initial to iteration 3 (+8.3%, +5.4%, +1.7%), but then flattens (+0.1%) and declines (-0.9%).

“In SciFact, rationale recall improves significantly from GIER initial to iteration 3 ( +8 . 3% , +5 . 4% , +1 . 7% ), but then flattens ( +0 . 1% ) and declines ( -0 . 9% ).”

May combine two claims.

No metric exhibits late-stage inflection or recovery, indicating absence of latent improvements.

“No metric exhibits late-stage inflection or recovery, indicating absence of latent improvements.”

May combine two claims.

In PrivacyQA, sentence recall improves at every step, but with diminishing returns.

“In PrivacyQA, sentence recall improves at every step, but with diminishing returns.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In Example 8, a relevant answer was provided for a gold unanswerable question.

“Example 8: Relevant answer provided for a gold unanswerable question”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

In Example 3, an incorrect baseline conclusion was corrected during GIER revision.

“Example 3: Incorrect baseline conclusion corrected during GIER revision”

Probably not a substantial claim: may not pass S3 (load-bearing).

In Example 4, GIER's decision contradicted the gold label.

“Example 4: Decision contradicting the gold label”

Probably not a substantial claim: may not pass S3 (load-bearing).

In Example 2, GIER surfaced a gold evidence set consisting of two sentences.

“Example 2: Gold evidence set with two sentences surfaced by GIER”

May combine two claims.

The authors observed strong alignment between the gap analysis and the revisions, with no misaligned cases.

“We observed strong alignment between the gap analysis and the revisions, with no misaligned cases.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Gap explanations make up 34.5% of attribution in Figure 3.

“Gap explanations make up for 34.5% of attribution.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In Example 6, non-gold but thematically aligned sentences were retrieved for a UAED question.

“Example 6: Non-gold but thematically aligned sentences retrieved for a UAED question”

May combine two claims.

In e-SNLI, 19 decisions changed-7 correct, 12 incorrect.

“In e-SNLI, 19 decisions changed-7 correct, 12 incorrect-demonstrating a capacity for revision.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In Example 13, complex multi-line reasoning was explicitly surfaced by GIER.

“Example 13: Complex multi-line reasoning explicitly surfaced by GIER”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

In Example 11, the model label conflicted with the gold label but was accompanied by detailed gap-based reasoning.

“Example 11: Model label conflicting with gold but with detailed gap-based reasoning”

Probably not a substantial claim: may not pass S3 (load-bearing).

In Example 7, thematic drift into UCC sentences occurred for a TPSC question.

“Example 7: Thematic drift into UCC sentences for a TPSC question”

Probably not a substantial claim: may not pass S3 (load-bearing).

In Example 9, a poor answer was retrieved for a gold unanswerable question.

“Example 9: Poor answer retrieved for a gold unanswerable question”

Probably not a substantial claim: may not pass S3 (load-bearing).

In Example 12, an incorrect label flip was accompanied by a plausible multi-gap explanation.

“Example 12: Incorrect label flip accompanied by plausible multi-gap explanation”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

In Example 5, connective inferences emerged from the GIER initial to the final iteration.

“Example 5: Emergence of connective inferences from GIER initial to final iteration”

May combine two claims.

In e-SNLI, there were 27 none labels and 70 fully aligned cases.

“e-SNLI had the most none labels (27), indicating no revisions were needed, while 70 were fully aligned .”

May combine two claims.

In SciFact, all instances involved suggested improvements, all fully aligned.

“In SciFact, all instances involved suggested improvements, all fully aligned .”

May combine two claims.

A manual analysis of the 33 samples reveals that 11 were clearly poor or irrelevant, while 9 included supporting sentences that were at least topically related to the question.

“A manual analysis of the 33 samples reveals that 11 were clearly poor or irrelevant, while 9 included supporting sentences that were at least topically related to the question.”

May combine two claims.

The pragmatic inferences gap accounted for about 11% of the gold reasoning chunks, followed by logical inferences at 9%.

“The pragmatic inferences gap accounted for about 11% of the chunks, followed by logical inferences at 9% , consistent with the inferential demands of e-SNLI.”

May combine two claims.

In PrivacyQA, 19 decisions switched, 13 of them incorrect, with 12 of those reflecting shifts from 'unanswerable' to 'answerable,' often due to broad reinterpretations.

“PrivacyQA also had 19 switches, 13 of them incorrect, with 12 of those reflecting shifts from 'unanswerable' to 'answerable,' often due to broad reinterpretations.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In Example 10, the label was corrected after gap-aware rationale refinement.

“Example 10: Corrected label after gap-aware rationale refinement”

May combine two claims.

Out of 50 unanswerable questions in PrivacyQA, GPT-4.1 produced answers for 33 of them from the first GIER iteration onward, compared to 21 with the initial prompt.

“Out of 50 unanswerable questions in PrivacyQA, GPT-4.1 produced answers for 33 of them from the first GIER iteration onward, compared to 21 with the initial prompt.”

May combine two claims.

GIER utilizes natural language descriptions of reasoning gaps and prompts a model to iteratively critique and refine its own outputs to better satisfy these criteria, unlike prompting strategies that rely on demonstrations, examples, or chain-of-thought templates.

“Unlike prompting strategies that rely on demonstrations, examples, or chain-of-thought templates, GIER utilizes natural language descriptions of reasoning gaps, and prompts a model to iteratively critique and refine its own outputs to better satisfy these criteria.”

• AssumptionRests on an assumption: Conceptual gaps are human-readable, generalizable, and semantically grounded, enabling both LLMs and human evaluators to reason about quality in principled terms.

May combine two claims.

GIER's gap-driven scaffolding not only improves explanation quality but also actively shapes the model's reasoning and decision-making behavior.

“These results underscore that GIER's gap-driven scaffolding not only improves explanation quality but also actively shapes the model's reasoning and decisionmaking behavior.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

The shift toward more inclusive answering in PrivacyQA reflects the inherent subjectivity and ambiguity in interpreting privacy policies.

“This shift toward more inclusive answering reflects the inherent subjectivity and ambiguity in interpreting privacy policies.”

May combine two claims.

GIER eliminates both instructive signals and revision exemplars, requiring models to infer flaws and corrective actions directly from abstract quality criteria.

“GIER differs by eliminating both: models must infer flaws and corrective actions directly from abstract quality criteria.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The gap definitions were iteratively refined based on the pilot study to ensure conceptual clarity and practical utility.

“Based on these findings, we iteratively refined the gap definitions to ensure conceptual clarity and practical utility.”

May combine two claims.

GIER achieves gains without exemplars or pattern-based corrections, positioning it as a criterion-driven alternative to methods that rely on demonstration-based or pattern-learning refinements.

“We note that, although these gains mirror those of prior refinement methods, GIER achieves them without exemplars or pattern-based corrections. This positions GIER as a criterion-driven alternative to methods that rely on demonstration-based or pattern-learning refinements.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

In SciFact, the 100 sampled instances ensure a balanced label distribution (50 support, 50 refute) and diversity of gold rationale complexity (25 single, 29 two, 46 three or more).

“We sample 100 instances, ensuring a balanced label distribution (50 support, 50 refute) and a diversity of gold rationale complexity: 25 instances with a single evidence rationale set, 29 with two, and 46 with three or more.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

GIER's iterative process was initially run for five iterations after the initial prompt; more iterations were not attempted because results indicated diminishing returns.

“We initially run GIER's iterative process for five iterations after the initial prompt. We did not attempt more iterations since results at this stage indicated diminishing returns.”

May combine two claims.

Incorrect decisions can still reflect grounded reasoning, underscoring the need for evaluation frameworks that go beyond label agreement to assess reasoning quality.

“These cases highlight that incorrect decisions can still reflect grounded reasoning, underscoring the need for evaluation frameworks that go beyond label agreement to assess reasoning quality.”

• AssumptionRests on an assumption: GPT-4.1 can segment human rationale, LLM rationale, and gap-level explanations into reasoning chunks.
• AssumptionRests on an assumption: The DeBERTa-v3-large-mnli-fever-anli-ling-wanli model provides valid NLI scores for attributing human reasoning chunks to LLM chunks.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

In PrivacyQA, the final selection includes 26 FPCU, 22 TPSC, 21 DS, 24 DR, 21 UCC, and 18 UAED questions, covering all 27 privacy policies.

“Since some questions span multiple themes, our final selection includes 26 FPCU questions, 22 TPSC, 21 DS, 24 DR, 21 UCC, and 18 UAED questions, covering all 27 privacy policies.”

Probably not a substantial claim: may not pass S4 (based).

GIER relies on well-specified gap definitions.

“While GIER incurs additional computational cost and relies on well-specified gap definitions, it offers a scalable, training-free pathway for enhancing explanation fidelity.”

May combine two claims.

GIER offers a lightweight alternative that aligns model reasoning through prompts rather than fine-tuning or reward signals.

“Our approach also complements alignment methods like RLHF (Ouyang et al., 2022) and preference modeling (Rafailov et al., 2023), offering a lightweight alternative that aligns model reasoning through prompts rather than fine-tuning or reward signals.”

• cites Ouyang et al., 2022 sourceNot checked — no full text of the source
• cites Rafailov et al., 2023 sourceNot checked — no full text of the source

Classified as probably not a substantial claim.

The study selects a balanced sample of 100 instances per task.

“Given the exploratory nature of this work, the high computational cost of iterative LLM experimentation, and the human effort required to validate reasoning alignment, we select a balanced sample of 100 instances per task.”

Probably not a substantial claim: may not pass S3 (load-bearing).

A few cases where gold answers omitted semantically relevant sentences were manually corrected by adding, but not deleting, theme-aligned sentences in the gold answer set prior to evaluation.

“We identified a few cases where the gold answers omitted semantically relevant sentences (e.g., a question about DS contained only cookie reference sentences). These were manually corrected by adding, but not deleting, theme-aligned sentences in the gold answer set prior to evaluation.”

May combine two claims.

Conceptual scaffolding alone improves reasoning, and structured revisiting of gaps enables further alignment with both evidence and interpretive structure.

“These results suggest that conceptual scaffolding alone improves reasoning, and that structured revisiting of gaps enables further alignment with both evidence and interpretive structure.”

Probably not a substantial claim: may not pass S4 (based).

Textual grounding (quoting text from the abstract) is essential in SciFact for automatic overlap evaluation.

“Textual grounding (quoting text from the abstract) is essential in this task for automatic overlap evaluation.”

May combine two claims.

GIER uses only natural language descriptions of reasoning gaps, requiring models to self-diagnose and revise without examples.

“In contrast, GIER uses only natural language descriptions of reasoning gaps, requiring models to self-diagnose and revise without examples.”

May combine two claims.

GPT-4.1 achieves the highest recall gain in PrivacyQA, though with broader sentence inclusion, indicating a strong response to abstract prompts that sometimes leads to over-selection in ambiguous policy contexts.

“GPT-4.1 achieves the highest recall gain, though with broader sentence inclusion, indicating a strong response to abstract prompts that sometimes leads to over-selection in ambiguous policy contexts.”

Classified as probably not a substantial claim. May combine two claims.

The e-SNLI evaluation focuses on harder instances, selecting 75 failure cases and 25 success cases, uniformly distributed across the three label types.

“We focus our evaluation on harder instances, selecting 75 failure cases and 25 success cases, uniformly distributed across the three label types.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

DS questions frequently draw from TPSC and FPCU content, reflecting the reality that security guarantees are often embedded in broader discussions about data handling.

“Similarly, DS questions frequently draw from TPSC and FPCU content, reflecting the reality that security guarantees are often embedded in broader discussions about data handling.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Source faithfulness is critical in scientific domains.

“Source faithfulness is critical in scientific domains, while the other gaps ensure the rationale is focused, balanced, and justified.”

May combine two claims.

The observed drift in PrivacyQA may reflect not just limitations in model selection, but also structural constraints in the source material, as privacy policies are relatively scarce in UCC/UAED/DR sentences.

“Privacy policies are relatively scarce in UCC/UAED/DR sentences; the observed drift may therefore reflect not just limitations in model selection, but also structural constraints in the source material.”

May combine two claims.

Selection of thematically unrelated sentences such as ISA, PC, PCI, and others are rare, indicating that outputs remain anchored in relevant zones of privacy discourse.

“Selection of thematically unrelated sentences such as ISA, PC, PCI, and others are rare, indicating that the outputs remain anchored in relevant zones of privacy discourse.”

May combine two claims.

Llama's smaller recall gain (+7.5%) and milder drift suggest a more conservative retrieval style.

“Llama's smaller recall gain ( +7 . 5% ) and milder drift suggest a more conservative retrieval style.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

The systematic preference for FPCU sentences is likely driven by their abundance in policies, general applicability across many privacy concerns, and perhaps a bias toward safe, foundational disclosures.

“This reveals a systematic preference for FPCU sentences, likely driven by their abundance in policies (collection/use disclosures are the most verbose), general applicability across many privacy concerns, and perhaps a bias toward safe, foundational disclosures.”

May combine two claims.

In e-SNLI, while 47% of samples achieve their most human-aligned reasoning in the initial gap-informed prompt, 30% peak in the final GIER iteration, indicating recovery of latent reasoning quality that single-shot prompting could not elicit.

“Best alignment further shows that, while 47% of samples achieve their most human-aligned reasoning in the initial gapinformed prompt, 30% peak in the final GIER iteration, indicating the recovery of latent reasoning quality that single-shot prompting could not elicit.”

• AssumptionRests on an assumption: GPT-4.1 can segment human rationale, LLM rationale, and gap-level explanations into reasoning chunks.
• AssumptionRests on an assumption: The DeBERTa-v3-large-mnli-fever-anli-ling-wanli model provides valid NLI scores for attributing human reasoning chunks to LLM chunks.

May combine two claims.

Gemini and Llama show smaller recall gains (+7.9%, +12.2%) and reductions in grounding, suggesting a tendency to paraphrase rather than quote directly, possibly due to weaker instruction tuning or a preference for coherence over fidelity.

“In contrast, Gemini and Llama show smaller recall gains ( +7 . 9% , +12 . 2% ) and reductions in grounding, suggesting a tendency to paraphrase rather than quote directly, possibly due to weaker instruction tuning or a preference for coherence over fidelity.”

Classified as probably not a substantial claim.

A pilot study using 25 randomly sampled instances per task was conducted to determine which conceptual gaps are most relevant for each task.

“To determine which conceptual gaps are most relevant for each task, we conducted a pilot study using 25 randomly sampled instances from the datasets per task.”

May combine two claims.

There is strong co-selection between TPSC, FPCU, and UCC sentences, reflecting the document-level practice of grouping data sharing with collection procedures and user opt-out options.

“Other broader patterns include strong co-selection between TPSC, FPCU, and UCC sentences. This trend reflects the documentlevel practice of grouping data sharing with collection procedures and user opt-out options.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The three tasks (SciFact, PrivacyQA, e-SNLI) span abstractive and extractive settings, structured and unstructured evidence, and both objective and subjective reasoning challenges.

“Together, these tasks span abstractive and extractive settings, structured and unstructured evidence, and both objective and subjective reasoning challenges.”

Probably not a substantial claim: may not pass S3 (load-bearing).

GIER incurs additional computational cost.

“While GIER incurs additional computational cost and relies on well-specified gap definitions, it offers a scalable, training-free pathway for enhancing explanation fidelity.”

Classified as probably not a substantial claim.

GIER introduces a novel gap-driven formulation.

“In summary, GIER integrates elements from prompting, critique-based revision, and explanation-centered evaluation but introduces a novel gap-driven formulation.”

Probably not a substantial claim: may not pass S4 (based).

The quality of GIER revisions is tightly coupled to how well the conceptual gaps are operationalized in natural language.

“The quality of revisions is tightly coupled to how well the conceptual gaps are operationalized in natural language.”

Probably not a substantial claim: may not pass S3 (load-bearing).

This study primarily evaluates GIER using models with relatively strong reasoning capabilities.

“This study primarily evaluates GIER using models with relatively strong reasoning capabilities.”

May combine two claims.

GIER operationalizes abstract reasoning flaws as interpretable prompts and uses these prompts to elicit iterative self-improvement without examples or rewards.

“It operationalizes abstract reasoning flaws as interpretable prompts and uses these prompts to elicit iterative selfimprovement without examples or rewards.”

May combine two claims.

In tasks where the model lacks domain knowledge or exhibits poor reasoning skills, GIER may yield limited gains or fail to close critical performance gaps.

“In tasks where the model lacks domain knowledge or exhibits poor reasoning skills, GIER may yield limited gains or fail to close critical performance gaps.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The thematic drift metric used in the PrivacyQA experiments is a task-specific heuristic informed by domain expertise.

“Our thematic drift metric, used in the PrivacyQA experiments, is a task-specific heuristic informed by domain expertise.”

May combine two claims.

GIER eliminates both instructive feedback and revision exemplars.

“GIER differs from these methods by eliminating both instructive feedback and revision exemplars; models must independently infer both the flaws and the corrective actions with respect to given gap definitions.”

May combine two claims.

In SciFact, conciseness functions as a key constraint on verbosity and plays an important role in maintaining textual grounding.

“In contrast, dropping the conciseness gap leads to a drop in grounding ratio ( -11 . 1% ) and a small gain in rationale recall ( +4 . 7% ), indicating that conciseness functions as a key constraint on verbosity and plays an important role in maintaining textual grounding.”

Classified as probably not a substantial claim.

We evaluated the statistical significance of improvements from the initial GIER iteration to the final GIER iteration using the Wilcoxon signed-rank test on paired per-example metric values.

“We evaluated the statistical significance of improvements observed from the initial GIER iteration (with gap definitions) to the final GIER iteration using the Wilcoxon signed-rank test on paired perexample metric values.”

May combine two claims.

The conceptual guidance embedded in the GIER prompt is sufficient for improvement in e-SNLI, with iterations offering no consistent value.

“This suggests that the conceptual guidance embedded in the GIER prompt is sufficient for improvement in this task, with iterations offering no consistent value.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Terminating too early may leave residual gaps unaddressed.

“Conversely, terminating too early may leave residual gaps unaddressed.”

Classified as probably not a substantial claim.

GIER's effectiveness depends critically on the clarity, precision, and completeness of the human-authored gap definitions.

“GIER's effectiveness depends critically on the clarity, precision, and completeness of the human-authored gap definitions.”

Classified as probably not a substantial claim.

We use 100 samples per task.

“We use 100 samples per task, reflecting the high computational cost of multi-step iterative revision.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

GIER does not impose an explicit hierarchy or prioritization among gaps; the model must navigate trade-offs autonomously during revision.

“GIER does not impose an explicit hierarchy or prioritization among gaps; the model must navigate trade-offs autonomously during revision.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

The boundary between explicit gap-driven self-reflection and implicit self-correction via CoT is not always sharply defined.

“While GIER differs from few-shot or zero-shot chain-of-thought prompting that provides explicit, conceptually grounded feedback, the boundary between explicit gap-driven selfreflection and implicit self-correction via CoT is not always sharply defined.”

Classified as probably not a substantial claim.

Gaps like textual grounding and source faithfulness are designed to mitigate the risk of hallucinated compliance by anchoring reasoning to evidence.

“Gaps like textual grounding and source faithfulness are designed to mitigate this risk by anchoring reasoning to evidence.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In e-SNLI, the reasoning chunks covered under the pragmatic inference gap are potentially also identified in other gaps.

“Dropping the pragmatic inference gap produces a modest relative gain in decision accuracy ( +1 . 8% ) and a slight reduction in reasoning attribution ( -1 . 4% ); the reasoning chunks covered under this gap are potentially also identified in other gaps.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

Crafting high-quality conceptual scaffolds requires both domain understanding and careful prompt engineering.

“Crafting high-quality conceptual scaffolds requires both domain understanding and careful prompt engineering to ensure that the gaps meaningfully guide revision.”

May combine two claims.

In PrivacyQA, enforcing coverage pushes the model to broader interpretation of questions, as well as retrieving broader content.

“This suggests that enforcing coverage pushes the model to broader interpretation of questions, as well as retrieving broader content.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

In some cases, unnecessary iterations may produce diminishing returns, redundant changes, or even slight regressions.

“In some cases, unnecessary iterations may produce diminishing returns, redundant changes, or even slight regressions.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The thematic drift metric captures broad patterns of semantic alignment but simplifies the complex ways that meaning evolves in privacy policies.

“While it captures broad patterns of semantic alignment, it simplifies the complex ways that meaning evolves in privacy policies.”

Classified as probably not a substantial claim.

The risk of hallucinated compliance can lead to confident but inaccurate explanations.

“This risk of hallucinated compliance-where the model optimizes for perceived satisfaction rather than factual soundness-can lead to confident but inaccurate explanations.”

May combine two claims.

GIER shares the goal of aligning model behavior with abstract human-centric standards but does so without reward tuning or label-based supervision.

“GIER shares the goal of aligning model behavior with abstract human-centric standards but does so without reward tuning or label-based supervision.”

May combine two claims. The quote only partly states this claim.

Iterative enhancement with GIER reliably improves reasoning-related metrics, with statistically significant and confidently bounded effect sizes.

“These results demonstrate that iterative enhancement with GIER reliably improves reasoningrelated metrics, with statistically significant and confidently bounded effect sizes.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Because gap definitions are abstract and do not inherently enforce factual correctness, models may sometimes generate superficially plausible but incorrect rationales that appear to satisfy the gaps.

“Because gap definitions are abstract and do not inherently enforce factual correctness, models may sometimes generate superficially plausible but incorrect rationales that appear to satisfy the gaps.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The remaining cases reflect broad or reinterpretive reasoning.

“The remaining cases reflect broad or reinterpretive reasoning.”

May combine two claims.

Longer gap definitions or overlapping gaps made the prompt harder to effectively act on.

“Longer gap definitions or overlapping gaps made the prompt harder to effectively act on.”

Probably not a substantial claim: may not pass S2 (checkable).

The most effective abstractions are those that LLMs can act on, not merely recognize.

“This refinement reflects a broader theme in prompt engineering and LLM alignment-the most effective abstractions are those that LLMs can act on, not merely recognize.”

May combine two claims.

The model's revisions consistently follow its own gap analyses, demonstrating high intent-action consistency.

“These results show that the model's revisions consistently follow its own gap analyses, demonstrating high intent-action consistency.”

May combine two claims.

The influence of individual gaps is more task-specific, functioning either as strong behavioral drivers-shaping retrieval breadth, grounding, or reasoning discovery-or as soft regularizers that constrain peripheral content.

“The influence of individual gaps is more task-specific, functioning either as strong behavioral drivers-shaping retrieval breadth, grounding, or reasoning discovery-or as soft regularizers that constrain peripheral content.”

May combine two claims.

GIER's revision process can promote flexible reinterpretation when questions are topically extensible, but does not universally override unanswerability.

“These patterns suggest that GIER's revision process can promote flexible reinterpretation when questions are topically extensible, but does not universally override unanswerability.”

May combine two claims.

Across all three tasks, the model did revise decisions, indicating that GIER is not purely a post hoc justification mechanism.

“Across all three tasks, the model did revise decisions, indicating that GIER is not purely a post hoc justification mechanism.”