CoChat
Sign Up
PreprintCited by 7Artificial Intelligence ApplicationsNatural Language Processing TechniquesTopic Modeling

Better Harnesses, Smaller Models: Building 90% Cheaper Agents via Automated Harness Adaptation

Open access
Yang, Chenyang, Zhao, Xinran, Wu, Tongshuang, Kästner, Christian·2026·arXiv
DOI 10.48550/arxiv.2607.08938arXiv 2607.08938
Open this paper in CoChat — free accountOpen PDF

Abstract

Frontier LLM agents are automating many business tasks, but their high inference cost makes large-scale deployment unsustainable. Small language models (SLMs) offer a cheaper alternative, yet they typically fall short when swapped into a harness designed for a frontier LLM. We show that for many routine business tasks, SLM agents can match LLM performance at 90% lower cost, when paired with an adapted harness that can be automatically discovered by a meta agent. The key insight is that much of the task difficulty is shared across instances and can be lifted from the model into the harness via tailored instructions, tools, and orchestration loops. To study this systematically, we create a framework that maps agent failure modes to harness adaptation strategies, and build a harness optimizer that automatically discovers effective adaptations from failure trajectories. Across seven business-oriented agentic tasks and three SLM families, we found optimized harnesses significantly improve performance on 16 of 21 task-SLM pairs, with seven pairs closing the SLM-LLM performance gap and the best SLM agent recovering 89.7% of LLM performance at 4% of the cost. Our analysis further shows that adaptation works best for tasks with more repetitive workflows and for SLMs with sufficient base capabilities. Together, these results suggest that harness adaptation can expand the practical deployment range of SLM agents in routine business tasks.

AI snapshot

Written from the paper’s full text.

TL;DR

Automatically adapting agent harnesses lets small language models match frontier LLM performance on routine business tasks at roughly 90% lower cost. Across seven tasks and three SLMs, optimized harnesses improved 16 of 21 task-SLM pairs, with the best SLM recovering 89.7% of LLM performance at 4% of the cost.

Key findings

  • Optimized harnesses significantly improved performance on 16 of 21 task-SLM pairs, and seven pairs closed the SLM-LLM performance gap. The best SLM, gemma-4-26b-a4b, recovered 89.7% of LLM performance at 4% cost, with 25% latency reduction.
  • Harness adaptation was more effective on less diverse tasks: across seven tasks, task diversity negatively correlated with optimized harness performance (Spearman ρ = -0.96); in diversity-controlled variants, performance dropped from 89.1% to 68.0% as diversity increased. The paper also reports +21.1% accuracy when moving from the most diverse to least diverse task setting.
  • More capable SLMs benefited more from adaptation: +48.8% vs. +15.5% for less capable models.
  • The most commonly addressed failure modes were instruction-following (81%) and knowledge (81%), followed by tool-use (62%) and long-context (33%). Dominant adaptation strategies were adding contexts (86%), creating tools (43%), and managing tools (29%); no successful harness adaptation created sub-agents.

Methods

  • Empirical study across 7 curated business-oriented agentic tasks and 3 SLMs (qwen3-coder-30b-a3b, ministral3-8b, gemma-4-26b-a4b), compared against frontier LLM gemini-3.1-pro-preview with a generic harness. Each task used a 20/20/60 train/validation/test split; each task-model pair was optimized three times, with the best-validation harness evaluated on a held-out test set.
  • Tasks were curated from public benchmarks (TheAgentCompany, LOCA-Bench, WebGenBench, WebArena CMS, RefactorBench), with 100 instances per task except website management (50). Evaluation used objective metrics such as exact-match decisions, executable checks, and runnable code; cost was computed from tracked token usage and official API pricing.
  • A meta-agent harness optimizer searched over contexts, tools, hooks, context management, and sub-agents using software-agent-sdk, with a GEPA-style genetic search procedure and a $20 budget per optimization run. Analyses compared accuracy, cost, and latency; measured task diversity via normalized Levenshtein distance between tool-call sequences; used Artificial Analysis benchmark scores as a model-capability proxy; and manually categorized optimized harness edits by failure mode and adaptation strategy.

Limitations

  • Stated: agent runs and optimization are highly stochastic despite repeated runs; external validity is limited to a particular set of tasks, models, and one optimizer implementation, and may not transfer to other tasks, future models, or different optimizers; tasks have clear success metrics and clean environments, not fully capturing complex deployment scenarios; reproducibility is limited because closed-source frontier LLM APIs are black-box and may evolve or be deprecated.
  • Stated: the optimizer used gemini-3.1-pro-preview as meta-agent; replacing it with cheaper gemini-3-flash-preview yielded worse final harnesses despite more iterations. Optimization budget was $20 per task-model pair, total $1260 for main experiments.
  • Evident from the text: no successful harness adaptation created sub-agents, which the authors hypothesize may reflect current SLMs’ limited agent-management capabilities or a limitation of the meta-agent. The study covers only seven tasks and three SLM families, so broader generalization is not established.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

DataCite · 10.48550/arxiv.2607.08938

Cites

0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 16 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    Optimized harnesses significantly improve performance on 16 of 21 task-SLM pairs.

    “we found optimized harnesses significantly improve performance on 16 of 21 task-SLM pairs”
    • Also stated in: § Step-by-Step Workflow **1.Discover Contacts & Initial Instructions:** […] get_convers ation_histroy send_ message ≠ no same message sent gemma-4-26b-a4b + adapted harness gemini-3.1-proHigh cost Low performance Adapted harness moves SLM agents to low-cost, highperformance regions Discover documents Communicate with people Create and manipulate artifacts · plain
      “Optimized harnesses significantly improve performance on 16 out of 21 SLM-task pairs”
    Quote in the paperSubstantial claim
  • Findingown data

    Harness adaptation works best for tasks with more repetitive (less diverse) workflows.

    “Our analysis further shows that adaptation works best for tasks with more repetitive workflows”
    • Also stated in: § Step-by-Step Workflow **1.Discover Contacts & Initial Instructions:** […] get_convers ation_histroy send_ message ≠ no same message sent gemma-4-26b-a4b + adapted harness gemini-3.1-proHigh cost Low performance Adapted harness moves SLM agents to low-cost, highperformance regions Discover documents Communicate with people Create and manipulate artifacts · plain
      “Harness adaptation is more effective for less diverse tasks.”
    Quote in the paperSubstantial claim
  • Findingown data

    Seven of the 21 task-SLM pairs close the SLM-LLM performance gap.

    “with seven pairs closing the SLM-LLM performance gap”
    • Also stated in: § Step-by-Step Workflow **1.Discover Contacts & Initial Instructions:** […] get_convers ation_histroy send_ message ≠ no same message sent gemma-4-26b-a4b + adapted harness gemini-3.1-proHigh cost Low performance Adapted harness moves SLM agents to low-cost, highperformance regions Discover documents Communicate with people Create and manipulate artifacts · plain
      “with seven pairs closing the SLM-LLM performance gap”
    Quote in the paperSubstantial claim
  • Findingown data

    Optimized harnesses work best when task instances share a stable workflow, with +21.1% accuracy going from the most diverse to the least diverse task setting.

    “Optimized harnesses work best when task instances share a stable workflow (+21.1% accuracy when we go from the most diverse task setting to the least diverse setting).”
    Quote in the paperSubstantial claim
  • Findingown data

    The best adapted SLM agent recovers 89.7% of LLM performance at 4% of the cost (96% cost reduction).

    “the best SLM agent recovering 89.7% of LLM performance at 4% of the cost”
    • Also stated in: § Step-by-Step Workflow **1.Discover Contacts & Initial Instructions:** […] get_convers ation_histroy send_ message ≠ no same message sent gemma-4-26b-a4b + adapted harness gemini-3.1-proHigh cost Low performance Adapted harness moves SLM agents to low-cost, highperformance regions Discover documents Communicate with people Create and manipulate artifacts · plain
      “The best adapted SLM can recover 89% of LLM-agent performance with 96% cost reduction.”
    Quote in the paperSubstantial claim
  • May combine two claims. 25
  • Classified as probably not a substantial claim. 17

  • May combine two claims.

    Findingown data

    The dominant harness adaptation strategies include adding contexts (86%), creating new tools (43%), and managing tools (29%).

    “The dominant strategies include adding contexts (86%), creating new tools (43%), and managing tools (29%).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    For playwright-testing, the optimized harness for gemma-4-26b-a4b adds contexts to externalize knowledge and enforce instructions, and creates hooks to enforce successful pytest runs.

    “For example, for the playwright-testing task, the optimized harness for gemma-4-26b-a4b adds contexts to externalize knowledge and enforce instructions, and create hooks to enforce successful pytest runs.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Findingnone given

    Without logging previous edits and their outcomes, the meta-agent tends to rediscover similar fixes, wasting budget on redundant local exploration.

    “Without logging previous edits and their outcomes, the meta-agent tends to rediscover similar fixes, wasting budget on redundant local exploration.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The best gemma-4-26b harness for anomaly-detection combines creating a custom query_mock_bigquery tool, filtering active tools from 40+ to seven, and externalizing the table-naming convention in the system prompt.

    “For example, the best gemma-4-26b harness for the anomaly-detection task combines three strategies together: it creates a custom query_mock_bigquery tool that addresses long-context issues from the default bigquery_run_query MCP tool; it filters the agent's active tool set from 40+ MCP tool down to seven, and externalizes the environment's table-naming convention in the system prompt.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    In the budget-approval example, the SLM gemma-4-26b-a4b achieves 75.0% accuracy and cannot make a reliable replacement.

    “The agent, however, performs worse (75.0% with gemma-4-26b-a4b) and cannot make a reliable replacement (Figure 1).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Ministral-3-8b sometimes struggles with using file editing tools, while qwen3-coder-30b-a3b occasionally fails to emit correct JSON tool calls.

    “For example, ministral-3-8b sometimes struggles with using file editing tools, while qwen3-coder-30b-a3b occasionally fails to emit correct JSON tool calls (but instead emit raw XMLs).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning

    Frontier LLMs are often strong enough to perform well with general-purpose harnesses, whereas SLMs more often exhibit capability gaps that make specialized harness adaptation especially useful.

    “frontier LLMs are often strong enough that they can perform well with general-purpose harnesses.SLMs, by contrast, more often exhibit capability gaps that make creating specialized harness adaptation especially useful.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The adapted budget-approval harness narrowed the action space, rewrote the system prompt into an explicit step-by-step procedure, and introduced a custom anti_loop_hook.py safeguard that prevents the model from getting stuck in a loop.

    “It narrowed the action space, rewrites the system prompt into an explicit step-by-step procedure, and introduces a custom anti_loop_hook.pysafeguard that prevents the model from getting stuck in a loop”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The best performing SLM, gemma-4-26b-a4b, recovers 89% of LLM performance at 4% cost, with 25% latency reduction.

    “The best performing SLM, gemma-4-26b-a4b, can recover 89% of LLM performance at 4% cost, with 25% latency reduction.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Across SLMs, models with stronger capabilities perform better and improve more with optimized harnesses.

    “Across SLMs, we observe a clear trend that models with stronger capabilities perform better and improve more with optimized harnesses (Figure 6).”
    • Also stated in: § VII. CONCLUSION · plain
      “Less capable models are harder to optimize.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The average benchmark scores reported by Artificial Analysis are a valid proxy for models' general capability.

  • May combine two claims.

    Findingown data

    These failure modes are most commonly addressed by adding contexts (86%), creating tools (43%), and managing tools (29%).

    “These failure modes are most commonly addressed by adding contexts (86%), creating tools (43%), and managing tools (29%).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Replacing gemini-3.1-pro-preview with gemini-3-flash-preview affords more iterations but yields worse final harnesses.

    “Replacing gemini-3.1-pro-previewwith a cheaper model gemini-3-flash-preview affords more iterations, but the lower-quality diagnoses and edits more than offset the extra exploration, yielding worse final harnesses.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    In the budget-approval example, a frontier LLM (gemini-3.1-pro) achieves 97.3% accuracy on curated evaluation data at $0.22 per query.

    “We can deploy an agent with a frontier LLM that will perform very well (97.3% accuracy with gemini-3.1-proon curated evaluation data), but is also expensive ($0.22 per query) to deploy in production.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationreasoning

    Multiple independent searches cover more distinct regions of the design space and increase the chance that at least one run discovers a strong adaptation.

    “Multiple runs cover more distinct regions of the design space and increase the chance that at least one run discovers a strong adaptation.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S1 (asserted), S4 (based). May combine two claims.

    Interpretationnone given

    If task instances share recurring workflows and failure patterns, a single adapted harness can encode those patterns and improve many instances; otherwise it is harder to create a comprehensive harness to cover diverse failures.

    “If task instances share recurring workflows and failure patterns, a single adapted harness can encode those patterns and improve many instances; otherwise, it is harder to create a comprehensive harness to cover all diverse failures.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    Given a well-described design space, a harness optimizer can automatically identify effective harness adaptations that significantly boost SLM performance, making it possible to build effective agents at a fraction of the cost of LLMs.

    “We show that, given a welldescribed design space, a harness optimizer can automatically identify effective harness adaptations that significantly boost SLM performance, making it possible to build effective agents at a fraction of the cost of LLMs.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    Adaptation matters most when a task's capability demands exceed what a model can reliably provide, and this mismatch is more common for SLMs.

    “This is because adaptation matters most when a task's capability demands exceed what a model can reliably provide, and this mismatch is more common for SLMs.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    Harness optimization is bottlenecked by diagnosis quality; spending budget on evaluation or additional iterations is useful only when the meta-agent can propose good harness candidates.

    “Harness optimization is bottlenecked by diagnosis quality -spending budget on evaluation or additional iterations is useful only when the meta-agent can propose good harness candidates.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    For each task-model pair, the optimization procedure was run three times and the harness with the best validation score was kept.

    “For each task-model pair, we run the optimization procedure three times and keep the harness with the best validation score.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    A common observation is that self-improving agents often make new tools to improve task performance.

    “A common observation is that these agent often make new tools to improve task performance.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [10]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites [47]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    The test set was held out for the final evaluation of the selected harness.

    “The test set is held out for the final evaluation of the selected harness.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisereasoning

    Adding contexts helps with knowledge gaps, planning gaps, tool-use gaps, and instruction-following gaps.

    “It helps with knowledge gaps by externalizing implicit knowledge into instructions, with planning gaps by providing explicit plans, with tool-use gaps by providing more detailed tool descriptions and examples, and with instruction-following gaps by reinforcing key constraints.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    Harnesses can absorb substantial task difficulty but cannot fully replace missing core model capabilities.

    “suggesting that harnesses can absorb substantial task difficulty but cannot fully replace missing core model capabilities.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given

    Failures in agentic task outcomes come from one or multiple local failures in the trajectory.

    “Failures in agentic task outcomes come from one or multiple local failures in the trajectory.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisereasoning

    Adding contexts can lead to new long-context and instruction-following failures due to growing context complexity.

    “while adding contexts is straightforward and often effective, it can lead to new failures on long-context and instruction-following due to growing context complexity.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Filtering out irrelevant tools makes tool selection easier and more efficient.

    “By filtering out irrelevant tools, we can make tool selection easier and more efficient [38].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [38]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationown data

    This works best for more repetitive tasks and SLMs with strong base capabilities.

    “Our analysis found that this works best for more repetitive tasks and SLM with strong base capabilities.”
    Quote in the paperSubstantial claim
    • The authors state this as an assumption: The average benchmark scores reported by Artificial Analysis are a valid proxy for models' general capability.

  • May combine two claims.

    Interpretationown data

    SLMs can be particularly effective when the task is more repetitive and more task demands can be offloaded to the harness.

    “SLMs can be particularly effective, when the task is more repetitive and more task demands can be offloaded to the harness.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Each task comes with an objective evaluation metric, such as exact-match decisions, executable checks, or runnable code.

    “Each task comes with an objective evaluation metric, such as exact-match decisions, executable checks, or runnable code.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    Stock-alert tasks require strong long-context capabilities that are difficult to fully capture within a harness.

    “Similarly, stock-alert tasks require strong long-context capabilities, which are difficult to fully capture within a harness.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Each task-model configuration was run three times and the average result reported to account for variation across agent runs.

    “Each configuration is run three times, and we report the average result to account for variations across agent runs.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Concurrent work on Life-Harness adapts agent harnesses for frozen models but focuses on evolving a harness that transfers across benchmark environments and model backbones, whereas this work studies harness adaptation as a cost-performance strategy.

    “Concurrent work on Life-Harness [62] also adapts agent harnesses for frozen models, but focuses on evolving a harness that transfers across benchmark environments and model backbones, whereas we study harness adaptation as a cost-performance strategy for making specialized SLM agents competitive with frontier-LLM agents on routine business tasks.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [62]

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning

    Our work intentionally does not make assumptions on what SLMs are capable of, but employs a meta-agent to empirically diagnose their trajectories and create tailored harnesses.

    “In contrast, our work intentionally does not make assumptions on what SLMs are capable of, but employs a meta-agent to empirically diagnose their trajectories and create tailored harnesses.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    There is no one-size-fits-all harness, and harness optimization needs to be re-run each time we migrate to a new SLM.

    “This indicates that, there is no one-size-fits-all harness, and that harness optimization needs to be re-run each time we migrate to a new SLM.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    Because we use closed-source frontier LLM APIs that are black-box and may evolve over time, exact replication of the optimization may be difficult once these APIs change or get deprecated.

    “Because we use closed-source frontier LLM APIs that are black-box and may evolve over time, exact replication of the optimization may be difficult once these APIs change or get deprecated.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Study design and conductnone given

    The agent runs and the optimization process are highly stochastic, even with multiple repeated runs.

    “The agent runs and the optimization process are highly stochastic, even with our mitigation of multiple repeated runs.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    We focus on tasks with clear success metrics and clean environments, which do not fully capture the complex deployment scenarios of specialized agents.

    “We focus on tasks with clear success metrics and clean environments, which do not fully capture the complex deployment scenarios of specialized agents.”
    Quote in the paperNot a substantial claim
37 of 37

  • The paper relies on this without support.

    The seven curated agentic tasks are representative of routine business tasks in general, so results generalize to many routine business tasks.

    “We purposefully select seven agentic tasks designed to reflect real-world business deployment scenarios.”

  • The paper relies on this without support.

    The initial generic harness is representative of a harness designed for a frontier LLM, so SLM underperformance reflects a model-harness mismatch rather than a poor harness.

    “Small language models often perform poorly when naively swapped into an agent harness designed for frontier LLMs.”

  • The authors state this as an assumption.

    The average benchmark scores reported by Artificial Analysis are a valid proxy for models' general capability.

    “To analyze the role of model capability, we use the average benchmark scores reported by Artificial Analysis [55] as a proxy for models' general capability, and use these scores to contextualize performance trends.”

  • The paper relies on this without support.

    The optimized harness can be reused across agent runs.

    “While there is a one-time offline optimization cost per task ($20 in our experiments), this cost is amortized after deployment because the optimized harness can be reused across agent runs.”

  • The paper relies on this without support.

    The per-run cost savings observed in the experiments will persist after deployment.

    “Based on the per-run cost savings we observe, the optimization cost is already recovered after 13 runs on average across tasks and models.”
5 of 5

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The BusinessBench dataset contains 477 eligible instances across seven task families.

    “The dataset contains 477 eligible instances across seven task families.”

    From StateM: Reaching 95.3% Raw Accuracy, or a \$15 Frontier Run, on Terminal-Bench 2.1 via Harness Scaling (2026) · cites this paper as Yang et al., 2026

    The Experiment Setups and Table I in Methodology describe seven business-relevant tasks. Table I lists 100 instances for each of six tasks and 50 for the seventh—a total of 650, not 477. The cited passages do not identify this suite as BusinessBench or describe 477 eligible instances. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • BusinessBench contains 477 eligible instances across seven families.

    “BusinessBench (Yang et al., 2026) tests generalization at the task-family level.”

    From StateM: Reaching 95.3% Raw Accuracy, or a \$15 Frontier Run, on Terminal-Bench 2.1 via Harness Scaling (2026) · cites this paper as Yang et al., 2026

    Table I in B. Methodology lists seven business-relevant tasks, supporting the seven-family part of the claim. But its instance counts total 650 (six tasks with 100 each and one with 50), not 477, and the cited passages do not call the collection BusinessBench. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • Better Harnesses, Smaller Models studies automated harness adaptation as a way for lower-cost models to recover much of the performance of larger models.

    “Better Harnesses, Smaller Models studies automated harness adaptation as a way for lower-cost models to recover much of the performance of larger models (Yang et al., 2026).”

    From StateM: Reaching 95.3% Raw Accuracy, or a \$15 Frontier Run, on Terminal-Bench 2.1 via Harness Scaling (2026) · cites this paper as Yang et al., 2026

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 2

  1. S. G. Patil, H. Mao, C. Cheng-Jie Ji, F. Yan, V. Suresh, I. Stoica, and J. E. Gonzalez, "The Berkeley function calling leaderboard (bfcl): From tool use to agentic evaluation of large language models," in Forty-second International Conference on Machine Learning, 2025.
  2. X. Wang, S. Rosenberg, J. Michelini, C. Smith, H. Tran, E. Nyst, R. Malhotra, X. Zhou, V. Chen, R. Brennan, and G. Neubig, "The openhands software agent sdk: A composable and extensible foundation for production agents," 2025. [Online]. Available: https: //arxiv.org/abs/2511.03690

Cited by

Sorted by citations

Authors
Yang, C., Zhao, X., Wu, T., & Kästner, C.
Affiliations
Carnegie Mellon University (US)
ORCID
Zhao, Xinran
Venue
arXiv
Publisher
arXiv
Published
2026
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2607.08938
arXiv
2607.08938
OpenAlex
W7168150711
Semantic Scholar
eafe5396b3cff91737e49a9264dbe913e5ded998
Cited by
0
References
0
Type
posted content
Open access
Licence
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords
small language models, LLM agents, meta-agents, agent failure modes, business automation, inference cost, agent orchestration

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-08.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-08.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review