MESH-Harness: Self-Improving Agent Harnesses via Bandit-Guided Compositional Evolution

Classified as probably not a substantial claim.

AutoHarness synthesizes action validators and policy code through iterative refinement using environment feedback.

“AutoHarness synthesizes action validators and policy code through iterative refinement using environment feedback (Lou et al., 2026).”

• cites Lou et al., 2026 sourceSupported

May combine two claims.

Both costs are lower on every task; the largest test-time savings occur on LiveCodeBench (27.0%) and ScienceWorld (59.5%).

“Both costs are lower on every task; the largest test-time savings occur on LiveCodeBench (27.0%) and ScienceWorld (59.5%).”

May combine two claims.

MESH-Harness exceeds the strongest fixed references by 6.33 points on Text and 10.83 on Math.

“MESH-Harness exceeds the strongest fixed references by 6.33 points on Text and 10.83 on Math (Tables 2-3).”

May combine two claims.

The text improvement over Meta-Harness is driven by LawBench (+8.00) and USPTO (+11.00), offsetting a decrease on Symptom-to-Diagnosis (84.90 to 83.00).

“The text improvement over Meta-Harness is driven by LawBench ( +8 . 00 ) and USPTO ( +11 . 00 ), offsetting a decrease on Symptom-to-Diagnosis (84.90 to 83.00).”

May combine two claims.

MESH-Harness is particularly strong on Text and ScienceWorld, while reaching the performance level of the strongest Math competitor, DGM-H (53.21 versus 53.00).

“MESH-Harness is particularly strong on Text and ScienceWorld, while reaching the performance level of the strongest Math competitor, DGM-H (53.21 versus 53.00).”

May combine two claims.

Keeping the initial pools for all slots fixed yields 51.20, a 5.13-point drop.

“Keeping the initial pools for all slots fixed yields 51.20, a 5.13-point drop (Table 5).”

May combine two claims.

On ScienceWorld, MESH-Harness rises from 51.90 to 58.10, despite beginning below the handcrafted reference at 55.50.

“On ScienceWorld, it rises from 51.90 to 58.10, despite beginning below the handcrafted reference at 55.50.”

May combine two claims.

Performance increases from α=0 to 0.8 and then falls at 1.6, favoring a moderate exploration bonus among the tested values.

“Performance increases from α = 0 to 0 . 8 and then falls at 1 . 6 (Figure 4), favoring a moderate exploration bonus among the tested values.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The dense statistical state occupies O(p^2) space; with a cached inverse, a rank-one update and a quadratic-form score each require O(p^2) arithmetic.

“The dense statistical state occupies O ( p 2 ) space. With a cached inverse, a rank-one update and a quadratic-form score each require O ( p 2 ) arithmetic; the agent implementation uses this update.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The selected Math combination centers on the requested mathematical quantity and its output form.

“The selected combination centers on the requested mathematical quantity and its output form.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Its analyzer distinguishes counting, finding all solutions, extremal questions, and exact-value questions, then extracts constraints and corresponding obligations.

“Its analyzer distinguishes counting, finding all solutions, extremal questions, and exact-value questions, then extracts constraints and corresponding obligations.”

May combine two claims.

Among the measured configurations, MESH-Harness's DeepSeek and Sonnet variants are the two nondominated points, offering the lowest cost and highest score, respectively.

“Among the measured configurations, its DeepSeek and Sonnet variants are the two nondominated points (Figure 5), offering the lowest cost and highest score, respectively.”

May combine two claims.

Retiring an implementation changes the feasible set of compositions but does not invalidate the stored observations.

“Retiring an implementation changes the feasible set of compositions but does not invalidate the stored observations.”

May combine two claims.

Full covariance does not introduce explicit pairwise reward features; rather, its off-diagonal blocks retain which slot representations have been observed together.

“Full covariance does not introduce explicit pairwise reward features. Rather, its off-diagonal blocks retain which slot representations have been observed together:”

May combine two claims.

Text ablations support both guided selection and module evolution.

“Text ablations support both guided selection and module evolution, while further experiments on these tasks show transfer across execution models and gains from refining a Meta-Harness solution.”

Probably not a substantial claim: may not pass S3 (load-bearing).

A JSON field carries the final answer, with separate rules for a complete label, a combination of labels, or a structured string.

“A JSON field carries the final answer, with separate rules for a complete label, a combination of labels, or a structured string.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The composer displays at most four supplied examples and adds a short 'Decisive difference' annotation to each: features shared with the query but absent from the other displayed cases are identified explicitly.

“The composer displays at most four supplied examples and adds a short 'Decisive difference' annotation to each: features shared with the query but absent from the other displayed cases are identified explicitly.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The Text harness separates seven decisions: maintaining training examples, defining the answer space, retrieving evidence, composing context, allocating computation, checking the response, and extracting the final answer.

“The Text harness separates seven decisions: maintaining training examples, defining the answer space, retrieving evidence, composing context, allocating computation, checking the response, and extracting the final answer.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

These counts concern inexpensive surrogate comparisons, not benchmark executions; only the selected complete harness consumes a candidate-evaluation slot.

“These counts concern inexpensive surrogate comparisons, not benchmark executions. Only the selected complete harness consumes a candidate-evaluation slot.”

Classified as probably not a substantial claim.

For slot pool sizes N1,...,Nk and S sweeps, the coordinate comparisons require S∑j Nj acquisition evaluations, plus final scoring and any start-selection or duplicate-handling overhead.

“For slot pool sizes N 1 , . . . , N k and S sweeps, the coordinate comparisons require S ∑ j N j acquisition evaluations, plus final scoring and any start-selection or duplicate-handling overhead.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The retriever ranks examples using weighted feature overlap, input-length proximity, and script-dependent features, while suppressing near-duplicates and repeated answer labels.

“The retriever ranks examples using weighted feature overlap, input-length proximity, and script-dependent features, while suppressing near-duplicates and repeated answer labels.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

It is a conditional statement under Eq. (18); the empirical α is a search coefficient rather than an asserted calibrated confidence radius.

“It is a conditional statement under Eq. (18); the empirical α is a search coefficient rather than an asserted calibrated confidence radius.”

• AssumptionThe authors state this as an assumption: The surrogate satisfies the uniform confidence envelope (Eq. 18) for some q_t ≥ 0.

Probably not a substantial claim: may not pass S3 (load-bearing).

The budget policy identifies the requested answer type from the input and stored examples, allowing downstream components to preserve the semantics of each task.

“The budget policy identifies the requested answer type from the input and stored examples, allowing downstream components to preserve the semantics of each task.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

The selected memory module removes exact input-target duplicates and, when its pool exceeds capacity, preferentially removes redundant examples from large signature groups.

“The selected memory module removes exact input-target duplicates and, when its pool exceeds capacity, preferentially removes redundant examples from large signature groups.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The Math harness contains seven slots for query analysis, budget allocation, retrieval, context composition, solving, verification, and answer parsing.

“The Math harness contains seven slots for query analysis, budget allocation, retrieval, context composition, solving, verification, and answer parsing.”

May combine two claims.

With Nj=10 and S=2, this is 140 coordinate comparisons instead of 10^7 for the seven-slot tasks, and 100 instead of 10^5 for the five-slot ScienceWorld controller.

“With N j = 10 and S = 2 , this is 140 coordinate comparisons instead of 10 7 for the seven-slot tasks, and 100 instead of 10 5 for the five-slot ScienceWorld controller.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The verifier checks specific completion defects, including a missing answer after truncation and an unresolved symbol where a definite result is requested.

“The verifier checks specific completion defects, including a missing answer after truncation and an unresolved symbol where a definite result is requested.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The parser first uses explicit answer fields, then applies answer-region and exact-form fallback rules.

“The parser first uses explicit answer fields, then applies answer-region and exact-form fallback rules.”

May combine two claims.

Starting from a Meta-Harness solution, 20 additional MESH-Harness evaluations improve the text macro-average from 50.63 to 54.37, with gains on all three datasets.

“Starting from a Meta-Harness solution, 20 additional MESH-Harness evaluations improve the text macro-average from 50.63 to 54.37, with gains on all three datasets (Table 8).”

Classified as probably not a substantial claim.

Exhaustive comparison would require ∏j Nj evaluations.

“Exhaustive comparison would require ∏ j N j evaluations.”

Probably not a substantial claim: may not pass S3 (load-bearing).

A shared answer contract connects these decisions across single-label prediction, multi-label prediction, and structured-string generation.

“A shared answer contract connects these decisions across single-label prediction, multi-label prediction, and structured-string generation.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Novelty is a tie breaker, not an additional reward, and bounded random sampling is the duplicate fallback rather than an exhaustive search for an unseen composition.

“Novelty is a tie breaker, not an additional reward, and bounded random sampling is the duplicate fallback rather than an exhaustive search for an unseen composition.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The selected composer does not insert the retrieved passages, even though retrieval is available through the task interface.

“It does not insert the retrieved passages, even though retrieval is available through the task interface.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The resulting prompt keeps the problem, its constraints, and the required form of the answer together.

“Here, the resulting prompt keeps the problem, its constraints, and the required form of the answer together.”

May combine two claims.

Its creation does not add an observation, and it receives neither its parent's reward nor a synthetic reward.

“Its creation does not add an observation, and it receives neither its parent's reward nor a synthetic reward.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Structured strings remain open-ended outputs rather than entries in a finite label inventory.

“Structured strings, for example, remain open-ended outputs rather than entries in a finite label inventory.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The selected composer preserves the complete problem and adds compact planning notes and an exact-answer contract.

“The selected composer preserves the complete problem and adds compact planning notes and an exact-answer contract.”

May combine two claims.

Including the default executor, MESH-Harness retains a 4.36-5.70-point advantage and a 2.8-7.2% cost reduction across all four execution models.

“Including the default executor, MESH-Harness retains a 4.36-5.70-point advantage and a 2.8-7.2% cost reduction across all four models (Table 7).”

Classified as probably not a substantial claim.

The evaluated composition is in the active product space in either case.

“The evaluated composition is in the active product space in either case.”

Classified as probably not a substantial claim. May combine two claims.

Pool refresh changes the feasible set without resetting the statistical state; protecting the incumbent preserves an already evaluated configuration, while retaining historical embeddings allows subsequent selection to reuse all completed observations.

“Pool refresh changes the feasible set without resetting the statistical state. Protecting the incumbent preserves an already evaluated configuration, while retaining historical embeddings allows subsequent selection to reuse all completed observations.”

May combine two claims.

Uncertainty cannot increase at a fixed vector after an observation, and its reduction is largest along directions coupled to the evaluated composition.

“Uncertainty cannot increase at a fixed vector after an observation, and its reduction is largest along directions coupled to the evaluated composition.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

These obligations include completeness for a solution set, or an attaining construction and a matching bound for an extremum.

“These include completeness for a solution set, or an attaining construction and a matching bound for an extremum.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Equation (19) separates three sources of selection error: uncertainty in the chosen composition, the range of the acquisition adjustment, and the gap left by approximate search.

“Equation (19) separates three sources of selection error: uncertainty in the chosen composition, the range of the acquisition adjustment, and the gap left by approximate search.”

• AssumptionThe authors state this as an assumption: The surrogate satisfies the uniform confidence envelope (Eq. 18) for some q_t ≥ 0.

Probably not a substantial claim: may not pass S3 (load-bearing).

For structured strings, the prompt requests preservation of stereochemistry, charges, and ring closures.

“For the latter, the prompt requests preservation of stereochemistry, charges, and ring closures.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

For long derivations, the solver requests forward progress without repeated restarts and a complete answer once a defensible value is available.

“For long derivations it requests forward progress without repeated restarts and a complete answer once a defensible value is available.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

The solver adds a problem-dependent instruction: maintain a ledger when many named quantities occur, substitute explicitly given numerical constants when applicable, or proceed through the governing identity.

“The solver adds a problem-dependent instruction: maintain a ledger when many named quantities occur, substitute explicitly given numerical constants when applicable, or proceed through the governing identity.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The planner constructs an ordered chain of preconditions: perform a permitted focus action, navigate toward an absent target, inspect a visible object's exact identity, establish a missing circuit element, measure an unresolved property, or execute a report branch supported by an observation.

“The planner constructs an ordered chain of preconditions: perform a permitted focus action, navigate toward an absent target, inspect a visible object's exact identity, establish a missing circuit element, measure an unresolved property, or execute a report branch supported by an observation.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The analyzer extracts the entry-point contract, requested output, input-size bounds, and boundary conditions.

“The analyzer extracts the entry-point contract, requested output, input-size bounds, and boundary conditions.”

Probably not a substantial claim: may not pass S3 (load-bearing).

When progress stalls, the planner can move to a less-attempted later precondition from its chain.

“When progress stalls, the planner can move to a less-attempted later precondition from its chain.”

Classified as probably not a substantial claim.

Each draft is evaluated on the public tests.

“Each draft is evaluated on the public tests.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The repair prompt includes a small set of failing rows, requests preservation of passing behavior, and warns against special-casing public examples.

“Its prompt includes a small set of failing rows, requests preservation of passing behavior, and warns against special-casing public examples.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The focus guard and recovery module check proposed actions through their own interfaces.

“The focus guard and recovery module then check proposed actions through their own interfaces.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The feedback interface retains the evidence needed for this comparison, including outputs and failures for individual public-test rows.

“The feedback interface retains the evidence needed for this comparison, including outputs and failures for individual public-test rows.”

Classified as probably not a substantial claim. May combine two claims.

Retention and evolution operate at the level of individual slots, while evaluation measures the behavior of the assembled program.

“Retention and evolution operate at the level of individual slots, while evaluation measures the behavior of the assembled program.”

• AssumptionRests on an assumption: The fixed-capacity pool with lexicographic eviction preserves the most useful modules.

Probably not a substantial claim: may not pass S3 (load-bearing).

The selected budget requests two drafts with repair enabled.

“The selected budget requests two drafts with repair enabled.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The LiveCodeBench harness separates problem analysis, budget selection, prompt construction, draft generation, public-test feedback, repair selection, and code parsing.

“The LiveCodeBench harness separates problem analysis, budget selection, prompt construction, draft generation, public-test feedback, repair selection, and code parsing.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The refreshed planner was selected at iterations 27, 29, 32, 35, and 36, and the final iteration-36 combination joins it with four components from the initial pool.

“The refreshed planner was selected at iterations 27, 29, 32, 35, and 36, and the final iteration36 combination joins it with four components from the initial pool.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Syntax, timeout, and runtime defects take precedence when choosing the repair instruction.

“Syntax, timeout, and runtime defects take precedence when choosing the repair instruction.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Recovery uses the same state and permission interfaces to replace an invalid or repeatedly ineffective action.

“Recovery uses the same state and permission interfaces to replace an invalid or repeatedly ineffective action.”

May combine two claims.

The final Text, Math, and LiveCodeBench harnesses each combine seven implementations from the initial pool; ScienceWorld combines four initial implementations with the refreshed planner.

“The final Text, Math, and LiveCodeBench harnesses each combine seven implementations from the initial pool; ScienceWorld combines four initial implementations with the refreshed planner.”

• AssumptionRests on an assumption: The initial pool of LLM-generated implementations provides a sufficiently diverse and useful starting point for search.

Probably not a substantial claim: may not pass S3 (load-bearing).

Evidence memory records attempted actions, observed changes, score increments, locations, and focus history.

“Evidence memory records attempted actions, observed changes, score increments, locations, and focus history.”

Classified as probably not a substantial claim. May combine two claims.

The repair selector compares failures across drafts: a failing row is local if another draft passes it, contested if the drafts produce different wrong outputs, and shared otherwise.

“The repair selector compares failures across drafts: a failing row is local if another draft passes it, contested if the drafts produce different wrong outputs, and shared otherwise.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The ScienceWorld harness uses five slots: evidence memory, goal-phase planning, action-candidate ranking, a focus guard, and recovery.

“The ScienceWorld harness uses five slots: evidence memory, goal-phase planning, action-candidate ranking, a focus guard, and recovery.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Both drafts use temperature zero, with different instructions: one models the process directly; the other is asked to re-derive the requested output quantity independently, using requirement sentences extracted from the statement when available.

“Both drafts use temperature zero, with different instructions: one models the process directly; the other is asked to re-derive the requested output quantity independently, using requirement sentences extracted from the statement when available.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The selected ScienceWorld planner was introduced at search iteration 24 by a trace-aware, failure-driven refresh targeting incomplete goals.

“It was introduced at search iteration 24 by a trace-aware, failure-driven refresh targeting incomplete goals.”

• AssumptionRests on an assumption: Recurring validation failure groups are indicative of general failure modes that can be fixed by module proposals.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The planner converts this record into a concrete next subgoal; ranking and guard modules constrain which actions are proposed and accepted.

“The planner converts this record into a concrete next subgoal; ranking and guard modules constrain which actions are proposed and accepted.”

May combine two claims.

Across tasks, the selected programs organize different forms of evidence: comparisons among examples in Text, mathematical constraints and answer forms in Math, public-test outcomes in LiveCodeBench, and observed preconditions in ScienceWorld.

“Across tasks, the selected programs organize different forms of evidence: comparisons among examples in Text, mathematical constraints and answer forms in Math, public-test outcomes in LiveCodeBench, and observed preconditions in ScienceWorld.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The prompt distinguishes a function or class method from a standard-input program and reproduces public examples.

“The prompt distinguishes a function or class method from a standard-input program and reproduces public examples.”

Classified as probably not a substantial claim. May combine two claims.

Contextual bandits use shared features to generalize from observed rewards and balance predicted performance against uncertainty.

“Contextual bandits use shared features to generalize from observed rewards and balance predicted performance against uncertainty (Li et al., 2010; Chu et al., 2011).”

• cites Li et al., 2010Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Chu et al., 2011Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim.

Meta-Harness identifies candidate evaluation as its main computational bottleneck.

“Meta-Harness likewise identifies candidate evaluation as its main computational bottleneck (Lee et al., 2026).”

• cites Lee et al., 2026Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim.

Large language models can act as agents by retrieving information, calling tools, and interacting with environments.

“Large language models (LLMs) can act as agents by retrieving information, calling tools, and interacting with environments (Yao et al., 2023).”

• cites Yao et al., 2023Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim.

Feature-based bandits provide a way to allocate evaluations by sharing reward information across candidates and balancing exploitation with uncertainty-driven exploration.

“Feature-based bandits (Li et al., 2010; Chu et al., 2011) provide a way to allocate evaluations by sharing reward information across candidates and balancing exploitation with uncertainty-driven exploration.”

• cites Li et al., 2010Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Chu et al., 2011Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

MASPOB selects prompt combinations using bandit-guided coordinate ascent but does not update its candidate pools.

“MASPOB selects prompt combinations using bandit-guided coordinate ascent but does not update its candidate pools (Hong et al., 2026).”

• cites Hong et al., 2026 sourceNot checked — no full text of the source

May combine two claims.

Guided selection without the uncertainty term improves on random selection, but still trails the full acquisition rule by 4.63 points.

“Thus, guided selection without the uncertainty term improves on random selection, but still trails the full acquisition rule by 4.63 points.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Both AutoHarness and Meta-Harness make the surrounding program an object of optimization rather than a fixed implementation.

“Both approaches make the surrounding program an object of optimization rather than a fixed implementation.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Meta-Harness searches over task-specific Python programs, allowing a proposer agent to inspect previous candidates' code, scores, and execution traces.

“Meta-Harness searches over task-specific Python programs, allowing a proposer agent to inspect previous candidates' code, scores, and execution traces (Lee et al., 2026).”

• cites Lee et al., 2026Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim.

The harness governs how model calls work together.

“Because the harness governs how model calls work together, optimizing it offers a way to improve an agent without updating model weights.”

May combine two claims.

The advantage of MESH-Harness spans context construction, retrieval, execution feedback, and sequential interaction without changing the execution model.

“The advantage therefore spans context construction, retrieval, execution feedback, and sequential interaction without changing the execution model.”

Classified as probably not a substantial claim. May combine two claims.

The final harness is chosen by the prespecified validation criterion and its implementations are frozen for testing.

“The final harness is chosen by the prespecified validation criterion and its implementations are frozen for testing.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Meta-Harness evaluates 20 × 2 candidates, and MESH-Harness evaluates 40 × 1 compositions.

“Meta-Harness evaluates 20 × 2 candidates, and MESH-Harness evaluates 40 × 1 compositions.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Generation and embedding costs count toward search expenditure; deployment executes only the selected harness, without further search.

“Generation and embedding costs count toward search expenditure; deployment executes only the selected harness, without further search.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Retrieved solutions belong to the external corpus; development and test solutions are not added to it.

“Retrieved solutions belong to the external corpus; development and test solutions are not added to it.”

Classified as probably not a substantial claim.

All reported test scores are means over three test runs of the same frozen harness.

“All reported test scores are means over three test runs of the same frozen harness.”

Classified as probably not a substantial claim.

All comparisons use the shared task splits and scoring protocols described in the paper.

“All comparisons use the shared task splits and scoring protocols described in the paper.”

Classified as probably not a substantial claim.

A candidate's verifier can inspect and refine an answer using the available context, but cannot consult the reference answer.

“A candidate's verifier can inspect and refine an answer using the available context, but cannot consult the reference answer.”

Classified as probably not a substantial claim. May combine two claims.

Search and final harness selection use validation data; the selected implementations are frozen before held-out testing.

“Search and final harness selection use validation data; the selected implementations are frozen before held-out testing.”

Classified as probably not a substantial claim.

The selected code, executor settings, and retrieval resources are fixed before testing.

“The selected code, executor settings, and retrieval resources are fixed before testing.”

• AssumptionRests on an assumption: Validation performance predicts held-out test performance for the selected harness.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The development and test questions are disjoint, and answers are evaluated by the corresponding source-specific programmatic graders.

“The development and test questions are disjoint, and answers are evaluated by the corresponding source-specific programmatic graders.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The corpus is filtered against the evaluation questions and deduplicated before search; the same corpus is used for validation and testing.

“The corpus is filtered against the evaluation questions and deduplicated before search; the same corpus is used for validation and testing.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

A harness can use public examples to test and repair its program; private tests are reserved for final scoring.

“A harness can use public examples to test and repair its program; private tests are reserved for final scoring.”

• AssumptionRests on an assumption: Public test cases are informative for repairing code such that private test performance improves.

Classified as probably not a substantial claim.

Each record retains its originating evaluation and component identities, so the proposer can distinguish an observed result from the stated intent of an untested edit.

“Each record retains its originating evaluation and component identities, so the proposer can distinguish an observed result from the stated intent of an untested edit.”

• AssumptionRests on an assumption: The LLM proposer can generate valid and useful modules from the bounded history context.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Reuse does not require assigning an inherited or synthetic reward to a new implementation.

“This is the operational counterpart of Proposition A.1: reuse does not require assigning an inherited or synthetic reward to a new implementation.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Initialization and refresh share the same admissibility principle: a candidate must implement its assigned responsibility and be executable through the declared interface.

“Initialization and refresh share the same admissibility principle: a candidate must implement its assigned responsibility and be executable through the declared interface.”

Classified as probably not a substantial claim.

All methods use the same splits.

“All methods use the same splits (Appendix C.1).”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Executor identity and scoring remain fixed while the optimizer changes module implementations.

“Executor identity and scoring remain fixed while the optimizer changes module implementations.”

May combine two claims.

This is surrogate-level information transfer; it is not an assignment of the observed reward to every selected module as an independent contribution.

“This is surrogate-level information transfer; it is not an assignment of the observed reward to every selected module as an independent contribution.”

Classified as probably not a substantial claim.

Held-out answers remain with the evaluator throughout execution.

“Held-out answers remain with the evaluator throughout execution.”

Classified as probably not a substantial claim.

Test data are not used for generation, selection, or stopping.

“Test data are not used for generation, selection, or stopping.”

Classified as probably not a substantial claim.

An example is useful to the solver only through the context and decision rules with which it is presented.

“These connected choices make complete-program evaluation relevant: an example is useful to the solver only through the context and decision rules with which it is presented.”

Probably not a substantial claim: may not pass S3 (load-bearing).

CHECK in Algorithm 5 combines syntax and signature validation, the permitted-capability checks, task-specific return-shape and state checks, and small execution fixtures.

“CHECK in Algorithm 5 combines syntax and signature validation, the permitted-capability checks, task-specific return-shape and state checks, and small execution fixtures.”

Classified as probably not a substantial claim.

Each selected harness is frozen and evaluated three times on the same held-out data.

“Each selected harness is frozen and evaluated three times on the same held-out data.”

Classified as probably not a substantial claim.

Validation and test predictions use this training-derived memory without adding held-out labels.

“Validation and test predictions use this training-derived memory without adding held-out labels.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

SELECTVAL applies the prespecified validation criterion in Section C.5; neither the acquisition score nor a test score is substituted for that criterion.

“SELECTVAL applies the prespecified validation criterion in Section C.5; neither the acquisition score nor a test score is substituted for that criterion.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Retrieval and composition serve complementary roles: one selects examples, while the other exposes the distinctions the model should consider.

“Retrieval and composition thus serve complementary roles: one selects examples, while the other exposes the distinctions the model should consider.”

May combine two claims.

Reusing and refining implementations within fixed functional slots can improve task performance while reducing the execution cost of the resulting harnesses.

“These results show that reusing and refining implementations within fixed functional slots can improve task performance while reducing the execution cost of the resulting harnesses.”

Probably not a substantial claim: may not pass S3 (load-bearing).

We use the same task-specific output and scoring rules for all harnesses.

“We use the same task-specific output and scoring rules for all harnesses.”

Classified as probably not a substantial claim.

The optimized program joins evidence selection, comparison, and answer formatting through explicit interfaces.

“The optimized program therefore joins evidence selection, comparison, and answer formatting through explicit interfaces.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

The monotonicity result in Proposition A.2 applies whenever the current coordinate is among the feasible replacements.

“The monotonicity result in Proposition A.2 applies whenever the current coordinate is among the feasible replacements.”

Classified as probably not a substantial claim. May combine two claims.

Coverage and lineage regularize exploration of an evolving candidate space, while coordinate search bounds the cost of comparing those candidates.

“Coverage and lineage regularize exploration of an evolving candidate space, while coordinate search bounds the cost of comparing those candidates.”

• AssumptionThe authors state this as an assumption: The surrogate satisfies the uniform confidence envelope (Eq. 18) for some q_t ≥ 0.

Probably not a substantial claim: may not pass S3 (load-bearing).

CHECK rejects duplicate identifiers and duplicate implementations within the slot's historical catalog.

“It also rejects duplicate identifiers and duplicate implementations within the slot's historical catalog.”

Classified as probably not a substantial claim.

Each test repetition starts with fresh task memory and environment episodes.

“Each test repetition starts with fresh task memory and environment episodes.”

Probably not a substantial claim: may not pass S3 (load-bearing).

A component may determine which available information enters the solver context.

“This is a concrete choice within the optimization space: a component may determine which available information enters the solver context.”

May combine two claims.

Combining module-level design reuse with feedback-driven compositional search can systematically improve agent harnesses while keeping overall optimization cost under control.

“These results show that combining module-level design reuse with feedback-driven compositional search can systematically improve agent harnesses while keeping overall optimization cost under control.”

• AssumptionRests on an assumption: The four evaluated task domains are representative of agent harness applications generally.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Only public-test results can be used for repair.

“Only public-test results can be used for repair.”

• AssumptionRests on an assumption: Public test cases are informative for repairing code such that private test performance improves.

Classified as probably not a substantial claim.

The splits remain fixed across methods and repeated test runs.

“The splits remain fixed across methods and repeated test runs.”

Classified as probably not a substantial claim. May combine two claims.

All optimization methods use the same execution model and a budget of 40 complete candidate evaluations per task.

“All optimization methods use the same execution model and a budget of 40 complete candidate evaluations per task.”

Classified as probably not a substantial claim. May combine two claims.

Final harnesses are selected on validation data and frozen before testing; reported test scores average three executions of each frozen harness.

“Final harnesses are selected on validation data and frozen before testing; reported test scores average three executions of each frozen harness.”

Classified as probably not a substantial claim.

The resulting division of labor carries the answer specification from problem analysis through derivation and final extraction, while preserving a compact solver context.

“The resulting division of labor carries the answer specification from problem analysis through derivation and final extraction, while preserving a compact solver context.”

May combine two claims.

Reusable components can exchange structured execution evidence: draft generation supplies alternatives, feedback characterizes them, and repair turns those observations into a targeted instruction.

“This case illustrates how reusable components can exchange structured execution evidence: draft generation supplies alternatives, feedback characterizes them, and repair turns those observations into a targeted instruction.”

May combine two claims.

The repair module can use a passing alternative to guide a local correction, or direct attention to the required output when multiple drafts disagree or fail together.

“The repair module can therefore use a passing alternative to guide a local correction, or direct attention to the required output when multiple drafts disagree or fail together.”

May combine two claims.

The pool has two roles: preserving alternatives for combination selection and admitting new implementations during search.

“Their provenance illustrates the two roles of the pool: preserving alternatives for combination selection and admitting new implementations during search.”

• AssumptionRests on an assumption: The initial pool of LLM-generated implementations provides a sufficiently diverse and useful starting point for search.