Single-Agent LLMs Outperform Multi-Agent Systems on Multi-Hop Reasoning Under Equal Thinking Token Budgets

Classified as probably not a substantial claim.

Recent budget-aware studies suggest that, when computation is normalized, many such strategies underperform strong single-agent baselines.

“Recent budget-aware studies suggest that, when computation is normalized, many such strategies underperform strong single-agent baselines (Wang et al., 2024; Han et al., 2025).”

• cites Wang et al., 2024 sourceSupported
• cites Han et al., 2025 sourceCould not verify

  Section 6.2 compares token-budget-aware reasoning with vanilla chain-of-thought on accuracy and token use, but does not compare strategies with strong single-agent baselines at normalized computation. Searches for single-agent comparisons and compute-matched evaluations found no passage establishing the attributed conclusion. (LLM, conf=0.75)

  Experimental — accuracy not yet measured

Classified as probably not a substantial claim.

Many elaborate prompting or search strategies fail to outperform simpler baselines once token or compute budgets are matched.

“Reasoning in Token Economies and follow-up work show that many elaborate prompting or search strategies fail to outperform simpler baselines once token or compute budgets are matched (Wang et al., 2024; Han et al., 2025; Li et al., 2025b; Wen et al., 2025).”

• cites Wang et al., 2024 sourceSupported
• cites Han et al., 2025 sourcePartly supported

  Section 6.2 reports that the more elaborate TALE-EP method has slightly lower accuracy than vanilla chain-of-thought prompting, while using substantially fewer tokens. This supports a narrower comparison, but the paper does not show that many prompting or search strategies fail to outperform simple baselines when token or compute budgets are matched. (LLM, conf=0.77)

  Experimental — accuracy not yet measured
• cites Li et al., 2025bNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim.

Once computation is normalized, agentic benefits tend to concentrate in weaker model or harder-regime settings and diminish or even reverse as base model capability increases.

“On the other hand, Kim et al. (2025) shows that once computation is normalized, agentic benefits tend to concentrate in weaker model or harder-regime settings and diminish (or even reverse) as base model capability increases, highlighting coordination overhead as a first-order constraint.”

• cites Kim et al., 2025 sourceSupported

Classified as probably not a substantial claim. May combine two claims.

Orchestration can induce drift, information loss, or misleading improvements from evaluation artifacts.

“Additionally, Cemri et al. (2025) introduce multi-agent tracing and a failure taxonomy that explains how orchestration can induce drift, information loss, or misleading improvements from evaluation artifacts.”

• cites Cemri et al., 2025 sourcePartly supported

  The Introduction describes MAST-Data traces and a failure taxonomy, while Section 4 identifies task derailment and breakdowns in information flow between agents. Section 5.2 says aggregate success rates can obscure the effects of optimizations, but the paper does not attribute misleading improvements to evaluation artifacts. (LLM, conf=0.84)

  Experimental — accuracy not yet measured

Classified as probably not a substantial claim.

Coordination overhead is a first-order constraint.

“On the other hand, Kim et al. (2025) shows that once computation is normalized, agentic benefits tend to concentrate in weaker model or harder-regime settings and diminish (or even reverse) as base model capability increases, highlighting coordination overhead as a first-order constraint.”

• cites Kim et al., 2025 sourceSupported

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

MAS advantages are task-specific and not universal.

“Benchmarks for collaborative agents (e.g., MedAgentBoard) further reveal that MAS advantages are task-specific and not universal (Zhu et al., 2025).”

• cites Zhu et al., 2025 sourceSupported

May combine two claims.

On Gemini-2.5-Flash, SAS-L is the strongest single-agent configuration in every MuSiQue panel, and its block averages remain above standard SAS throughout.

“On Gemini-2.5-Flash, SAS-L is the strongest single-agent configuration in every MuSiQue panel, and its block averages remain above standard SAS throughout.”

May combine two claims.

Deletion produces a weaker version of the same trend: SAS remains stronger at mild degradation, the systems overlap at moderate degradation, and SAS regains a small edge at the heaviest deletion level.

“Deletion produces a weaker version of the same trend. SAS remains stronger at mild degradation, the systems overlap at moderate degradation, and SAS regains a small edge at the heaviest deletion level.”

May combine two claims.

Under masking, SAS leads at mild degradation (α = 0.3) and the systems are roughly tied at moderate degradation, but Sequential becomes better at heavy degradation (α = 0.7).

“Under masking, SAS leads at mild degradation ( α = 0.3) and the systems are roughly tied at moderate degradation, but Sequential becomes better at heavy degradation ( α = 0.7).”

May combine two claims.

Sequential MAS writes longer thoughts and canvasses 2 times more entities than SAS does.

“Sequential MAS writes longer thoughts and canvasses 2 times more entities than SAS does”

Probably not a substantial claim: may not pass S3 (load-bearing).

Gemini-2.5-Pro is the strongest overall model.

“Other observations include that Gemini-2.5-Pro is the strongest overall”

Probably not a substantial claim: may not pass S3 (load-bearing).

For MuSiQue, confidence intervals tend to be wider or more overlapping than for FRAMES.

“For MuSiQue, the confidence intervals also tend to be wider or more overlapping than FRAMES”

May combine two claims.

Parallel-roles is especially competitive on FRAMES and on Gemini-Pro settings.

“is especially competitive on FRAMES and on Gemini-Pro settings.”

May combine two claims.

SAS consumes much less thinking token than any MAS variants while achieving the same or better results.

“SAS also consumes much less thinking token than any MAS variants while achieving the same or better results (refer to Appendix F for full results with thinking token spent).”

May combine two claims.

SAS remains competitive with, and usually slightly stronger than, Sequential throughout the Gemini model version sweep.

“Second, SAS remains competitive with, and usually slightly stronger than, Sequential throughout the sweep.”

May combine two claims.

Under substitution, SAS is ahead at α = 0.3, the two systems move closer at α = 0.5, and Sequential is clearly better at α = 0.7.

“Under substitution, which both removes signal and injects misleading content, the crossover is even clearer: SAS is ahead at α = 0.3, the two systems move closer at α = 0.5, and Sequential is clearly better at α = 0.7.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

DeepSeek-R1-Distill-Llama-70B is the stronger of the two open-source families.

“DeepSeek-R1-Distill-Llama-70B is the stronger of the two open-source families.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Our observed failure modes include over-exploration, aggregation/extraction errors, and opaque API accounting.

“These failure modes align with our observed over-exploration, aggregation/extraction errors, and opaque API accounting.”

May combine two claims.

Adding distractor sentences is milder still: both systems degrade, but SAS remains ahead throughout, albeit by a smaller margin than in the clean setting.

“Adding distractor sentences is milder still: both systems degrade, but SAS remains ahead throughout, albeit by a smaller margin than in the clean setting.”

May combine two claims.

Across model families and datasets, SAS is the best-performing system or statistically indistinguishable from the best for all budgets except the lowest one (100 tokens), in which case the model does not produce a useful reasoning trace at all for either approach.

“Across model families and datasets, SAS is the best-performing system or statistically indistinguishable from the best for all budgets except the lowest one (100 tokens), in which case the model does not produce a useful reasoning trace at all (for either approach).”

May combine two claims.

Starting from Gemini-2.5-Flash, performance increases monotonically with model capability for both SAS and Sequential MAS.

“First, starting from Gemini-2.5-Flash, performance increases monotonically with model capability for both SAS and Sequential MAS”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Qwen3 saw slight performance gains in both SAS and Sequential MAS under Deep Paraphrase, though not significant.

“Qwen3 also saw slight performance gains in both SAS and Sequential MAS, even though it is not significant.”

May combine two claims.

For Qwen3-30B-A3B, SAS-L is sometimes competitive in the low-budget panel, but standard SAS is usually stronger in the middle and high-budget ranges.

“For Qwen3-30B-A3B, SAS-L is sometimes competitive in the low-budget panel, but standard SAS is usually stronger in the middle and high-budget ranges.”

Probably not a substantial claim: may not pass S3 (load-bearing).

MuSiQue is clearly the harder benchmark.

“MuSiQue is clearly the harder benchmark.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Qwen3-30B-A3B shows a similar picture, with Sequential MAS exploring more entities than SAS across buckets.

“Qwen3-30B-A3B shows a similar picture, with Sequential MAS exploring more entities than SAS across buckets”

May combine two claims.

Both systems pursue disjoint candidates and fail to reconcile.

“Both systems pursue disjoint candidates and fail to reconcile.”

May combine two claims.

Both systems frequently contain the gold in thoughts (SAS 61.9%, Sequential MAS 64.5%, both 50.9% for Gemini; SAS 73.7%, Sequential MAS 78.5%, both 69.9% for Qwen3).

“Both systems frequently contain the gold in thoughts (SAS 61.9%, Sequential MAS 64.5%, both 50.9% for Gemini; SAS 73.7%, Sequential MAS 78.5%, both 69.9% for Qwen3).”

May combine two claims.

For Gemini-2.5-Flash with a requested 10k budget, the SAS pipeline reported an average of 1,687 thinking tokens, while the visible thought content contained an average of 251 words, a proxy for 359 tokens.

“For example, using Gemini-2.5-Flash with a requested 10k budget, the SAS pipeline reported an average of 1,687 thinking tokens. However, the visible thought content only contained an average of 251 words, a proxy for 359 tokens (see Table 7).”

May combine two claims.

The gold string appears in SAS's thoughts in 42.7% vs. 18.6% for MAS for Gemini, and 63.5% vs. 28.1% for Qwen3.

“the gold string appears in SAS's thoughts in 42.7% vs. 18.6% for MAS for Gemini, and 63.5% vs. 28.1% for Qwen 3.”

Probably not a substantial claim: may not pass S3 (load-bearing).

For Qwen3-30B-A3B, the relative token lengths between Sequential MAS and SAS are closer in some buckets.

“the relative token lengths are closer in some buckets”

The quote only partly states this claim.

Failures for both SAS and Sequential MAS are dominated by never writing the correct surface form.

“Failures for both are dominated by: (i) never writing the correct surface form”

• AssumptionThe authors state this as an assumption: The ground truth answers used for evaluation are correct.

The quote only partly states this claim.

Failures for both SAS and Sequential MAS are dominated by losing a previously correct span at finalization.

“losing a previously correct span at finalization”

• AssumptionThe authors state this as an assumption: The ground truth answers used for evaluation are correct.

Probably not a substantial claim: may not pass S3 (load-bearing).

Backtracking markers occur on both sides.

“Backtracking markers occur on both sides”

May combine two claims.

For Gemini-2.5-Flash SAS, the visible thought proxy plateaus at approximately 350 tokens, being 354 tokens at a 1k budget and 359 tokens at a 10k budget.

“For Gemini2.5-Flash SAS, the visible thought proxy plateaus at ≈ 350 tokens. It is 354 tokens at a 1k budget and 359 tokens at a 10k budget (see Table 7).”

Probably not a substantial claim: may not pass S4 (based).

MAS typically consume more tokens through longer reasoning traces or multiple agent interactions.

“MAS typically consume more tokens through longer reasoning traces or multiple agent interactions, making it unclear whether their gains arise from architectural advantages or simply from increased compute.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Multi-agent LLM architectures have demonstrated strong empirical performance across a range of tasks.

“Multi-agent LLM architectures (MAS), including planners, role-playing systems, debate frameworks, and tool-specialized swarms, have demonstrated strong empirical performance across a range of tasks.”

May combine two claims.

MAS can be advantageous when a single agent's effective context utilization is degraded, or when MAS benefit from additional unaccounted computation through extended interactions.

“This perspective also clarifies when MAS can be advantageous: specifically, when a single agent's effective context utilization is degraded (e.g., due to long or noisy contexts), or when MAS benefit from additional unaccounted computation through extended interactions.”

May combine two claims.

Multi-agent systems become competitive when a single agent's effective context utilization is degraded, or when more compute is expended.

“This perspective further predicts that multi-agent systems become competitive when a single agent's effective context utilization is degraded, or when more compute is expended.”

• Also stated in: § 1 Introduction · p. 2 · plain

  “This perspective also clarifies when MAS can be advantageous: specifically, when a single agent's effective context utilization is degraded (e.g., due to long or noisy contexts), or when MAS benefit from additional unaccounted computation through extended interactions.”

Classified as probably not a substantial claim.

Recent work has emphasized that comparisons between reasoning strategies are often confounded by unequal test-time computation.

“Recent work has emphasized that comparisons between reasoning strategies are often confounded by unequal test-time computation.”

May combine two claims.

In sufficiently degraded regimes, MAS and SAS may become comparable, and carefully structured MAS pipelines may occasionally surpass SAS by imposing useful factorization, filtering, or verification structure on the reasoning process.

“In sufficiently degraded regimes, MAS and SAS may become comparable, and carefully structured MAS pipelines may occasionally surpass SAS by imposing useful factorization, filtering, or verification structure on the reasoning process.”

• AssumptionThe authors state this as an assumption: T0 is the identity map and degradation is monotone in the sense that for 0 ≤ α1 ≤ α2, there exists a stochastic map S α1→α2 such that the degradation transformation composes appropriately.

May combine two claims.

When effective single-agent context utilization deteriorates enough, structured multi-agent reasoning becomes competitive and can occasionally be better.

“The degradation experiments identify the boundary of that claim: when effective single-agent context utilization deteriorates enough, structured multi-agent reasoning becomes competitive and can occasionally be better.”

Probably not a substantial claim: may not pass S3 (load-bearing).

SAS with few-shot prompting outperformed MAS for reflection assessment in education analytics.

“In education analytics, SAS with few-shot prompting outperformed MAS for reflection assessment (Li et al., 2025a).”

• cites Li et al., 2025aNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim. May combine two claims.

The judge's rubric and prompt are used for all datasets and for both SAS and MAS configurations, so any differences in accuracy arise from the systems being evaluated, not from the judge.

“The judge's rubric and prompt are used for all datasets and for both SAS and MAS configurations, so any differences in accuracy arise from the systems being evaluated, not from the judge.”

May combine two claims.

The SAS ≥ Sequential pattern appears to be a stable property of the comparison itself, with both architectures benefiting from stronger base models but SAS remaining at least as competitive as the best multi-agent analogue.

“Instead, it appears to be a stable property of the comparison itself, with both architectures benefiting from stronger base models but SAS remaining at least as competitive as the best multi-agent analogue.”

May combine two claims.

Once effective single-agent context utilization deteriorates sufficiently, a well-designed MAS may recover task-relevant information more reliably than a degraded single pass, even though it still cannot exceed the ideal information available in C.

“In other words, once effective single-agent context utilization deteriorates sufficiently, a well-designed MAS may recover task-relevant information more reliably than a degraded single pass, even though it still cannot exceed the ideal information available in C .”

• AssumptionThe authors state this as an assumption: T0 is the identity map and degradation is monotone in the sense that for 0 ≤ α1 ≤ α2, there exists a stochastic map S α1→α2 such that the degradation transformation composes appropriately.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

In practice, modern LLMs may not utilize all of C equally well.

“In practice, however, modern LLMs may not utilize all of C equally well.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The wider confidence intervals on MuSiQue reflect greater instability and a harder reasoning problem.

“reflecting greater instability and a harder reasoning problem.”

Classified as probably not a substantial claim.

Improvements in MAS depend on task structure and verification protocols rather than MAS being universally superior.

“Finally, Ke et al. (2026) study learned orchestration via holistic configuration selection and propose controlled benchmarking axes for MAS, emphasizing that improvements depend on task structure and verification protocols rather than MAS being universally superior.”

• cites Ke et al., 2026Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

Across two datasets (FRAMES, MuSiQue), three model families (Qwen3, DeepSeek, Gemini), and five different MAS architectures (Sequential, Debate, Ensemble, Parallel-roles, Subtaskparallel), SAS matches or outperforms MAS when computation is normalized, unless context utilization is degraded to a certain point.

“Our results across two datasets (FRAMES, MuSiQue), three model families (Qwen3, DeepSeek, Gemini), and five different MAS architectures (Sequential, Debate, Ensemble, Parallel-roles, Subtaskparallel) consistently show that SAS matches or outperforms MAS when computation is normalized, unless context utilization is degraded to a certain point.”

May combine two claims.

Conditioning on M leaves more residual uncertainty about Y than conditioning on C; the multi-agent architecture cannot increase mutual information with the true answer.

“Thus, conditioning on M leaves more residual uncertainty about Y than conditioning on C ; the multi-agent architecture cannot increase mutual information with the true answer.”

• AssumptionThe authors state this as an assumption: The multi-agent messages M are a (possibly stochastic) function of the full context C.

May combine two claims.

Even when the explicit token cap is removed and across multiple Gemini generations, stronger models improve both architectures, but they do not create a regime in which Sequential becomes systematically superior.

“Thus, even when we remove the explicit token cap and move across multiple Gemini generations, the same qualitative pattern persists: stronger models improve both architectures, but they do not create a regime in which Sequential becomes systematically superior.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The simple regex-based changes may have obfuscated the questions or removed helpful phrasing cues that the models were relying on.

“This suggests that the simple regex-based changes may have obfuscated the questions or removed helpful phrasing cues that the models were relying on.”

May combine two claims.

SAS-L is mainly a Gemini effect: it helps when the Gemini models' thinking channel appears underutilized, but it does not produce the same consistent benefit for the open-source runs.

“In conclusion, SAS-L is mainly a Gemini effect: it helps when the Gemini models' thinking channel appears underutilized, but it does not produce the same consistent benefit for the open-source runs.”

May combine two claims.

Modern LLMs do not use long context perfectly: attention dilution, noise sensitivity, context confusion, and positional bias can all degrade reasoning even when relevant information is present.

“Independent of architecture, modern LLMs do not use long context perfectly: attention dilution, noise sensitivity, context confusion, and positional bias can all degrade reasoning even when relevant information is present.”

• cites Du et al., 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Li et al., 2025cNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Lu et al., 2024Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S1 (asserted).

Much of the apparent advantage of MAS comes from increased compute.

“Anthropic (2025) argues that much of the apparent advantage of MAS comes from increased compute.”

• cites Anthropic, 2025Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim. May combine two claims.

As frontier models improve, the benefits of orchestration diminish and SAS can match or surpass MAS.

“Empirical evidence increasingly suggests that as frontier models improve, the benefits of orchestration diminish and SAS can match or surpass MAS (Gao et al., 2025).”

• cites Gao et al., 2025 sourceNot checked — no full text of the source

May combine two claims.

SAS under-explores and sometimes fails to copy the correct span from its own chain; Sequential MAS's breadth plus occasional backtracking rescues the final.

“Interpretation: SAS under-explores and sometimes fails to copy the correct span from its own chain; Sequential MAS's breadth plus occasional backtracking rescues the final.”

May combine two claims.

Success typically follows a two-stage path: (i) surface a plausible span, (ii) perform a late constraint re-check that avoids switching away from the correct span.

“Interpretation: Success typically follows a two-stage path: (i) surface a plausible span, (ii) perform a late constraint re-check that avoids switching away from the correct span.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The study only studies the effect on performance across model families and datasets while increasing the thinking token cap.

“(iii) We only study the effect on performance across model families and datasets while increasing the thinking token cap.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The study restricts only the upper thinking budget cap, not the lower thinking budget cap.

“As a result, we restrict only upper thinking budget cap, not lower thinking budget cap.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Gemini thinking accounting is approximate.

“(ii) Gemini thinking accounting is approximate”

Classified as probably not a substantial claim.

The study does not enforce the models to actually use up all of those budgets.

“We do not enforce the models to actually use up all of those budgets.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The study focuses on text-only multi-hop reasoning; MAS advantages with tools/vision or safety constraints are out of scope.

“(i) We focus on text-only multi-hop reasoning; MAS advantages with tools/vision or safety constraints are out of scope.”

May combine two claims.

Sequential MAS's breadth, absent pruning, degrades precision and sometimes loses a correct span at finalization.

“Sequential MAS's breadth, absent pruning, degrades precision and sometimes loses a correct span at finalization.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The study reports both API and content-based proxies for Gemini thinking and emphasizes accuracy under matched requested budgets.

“we report both API and content-based proxies and emphasize accuracy under matched requested budgets”

May combine two claims.

The dominant failure mode is mutual drift with insufficient late binding to the question's constraints; neither path surfaces an extractable gold span.

“Interpretation: The dominant failure mode is mutual drift with insufficient late binding to the question's constraints; neither path surfaces an extractable gold span.”

Classified as probably not a substantial claim.

The primary analysis compares configurations based on their matched requested budget B.

“Therefore, our primary analysis intentionally and necessarily compares configurations based on their matched requested budget B .”

Classified as probably not a substantial claim. May combine two claims.

The Sequential MAS pipeline executes k separate agent calls, and the final thought text is a concatenation of k distinct (and potentially truncated) thought blocks, leading to more total text output.

“This is likely because the Sequential MAS pipeline executes k separate agent calls, and the final thought text is a concatenation of k distinct (and potentially truncated) thought blocks, leading to more total text output.”

Classified as probably not a substantial claim.

The matched requested budget B is the only variable that can be directly controlled by the researcher.

“This is the only variable that can be directly controlled by the researcher.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

All four context degradation methods use the same generation workflow and differ only in how the generated thought text is corrupted before answer generation.

“All four methods use the same generation workflow and differ only in how the generated thought text is corrupted before answer generation.”