ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The Map domain was excluded from the evaluation because of website issues, following Miyai et al. (2025), to keep the comparison fair.

“We exclude the domain of Map due to website issues followingMiyai et al. (2025) for a fair comparison.”

• cites Miyai et al. (2025) sourceSupported

May combine two claims.

At k = 2, MaTTS without aggregation scores 50.8, compared with 47.6 for MaTTS without memory.

“For example, at 𝑘 = 2, MaT TS w/o aggregation achieves 50.8 compared to 47.6 from MaT TS w/o memory, extracting more value from each rollout as noted in (Yue et al., 2025).”

• cites Yue et al., 2025 sourceCould not verify

  Section 2.2 (p. 5) discusses pass@k, and Appendix C.5 (p. 23) reports results for RL algorithms, not MaTTS ablations. Searches for MaTTS, aggregation, memory, 50.8, and 47.6 found no passage supporting the claimed comparison. (LLM, conf=0.75)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

ReasoningBank exhibits robustness to LLM-as-a-Judge calibration.

“Re as on i ng Ban k exhibits robustness to LLM-as-a-Judge calibration.”

May combine two claims.

Strong baselines such as AWM fail to provide gains and even degrade on the WebArena Multi subset.

“In contrast, strong baselines such as AWM fail to provide gains and even degrade in this setting.”

May combine two claims.

ReasoningBank provides far more benefits: Pass@1 rises from 49.7 to 53.0, showing high-quality memory can harness diversity of scaling to extract constructive contrastive signals.

“In contrast, Re as on i ng Ban k provides far more benefits: Pass@1 rises from 49.7 to 53.0, showing that high-quality memory can harness the diversity of scaling to extract constructive contrastive signals.”

May combine two claims.

At k=5, MaTTS achieves 55.1 in parallel scaling compared with 52.4 for vanilla TTS, and 54.5 versus 51.9 in sequential scaling.

“Specifically, at 𝑘 = 5, MaT TS achieves 55.1 in parallel scaling compared with 52.4 for vanilla TTS, and 54.5 versus 51.9 in sequential scaling.”

May combine two claims.

Strategies in ReasoningBank are not flat or monolithic but evolve over time, exhibiting emergent behaviors that resemble RL learning dynamics.

“We find that the strategies in Re as on i ng Ban k are not flat or monolithic, but instead evolve over time, exhibiting emergent behaviors that resemble the learning dynamics of RL (Wang et al., 2025a).”

• cites Wang et al., 2025aNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

MaTTS on WebArena demonstrates additional strong performance improvement and efficiency gains with ReasoningBank.

“We first present the overall results of MaT TS on WebArena in Table 1, which demonstrate additional strong performance improvement and efficiency gains with ReasoningBank.”

May combine two claims.

Scaling actually reduces performance for weaker memories: Synapse slightly increases from 40.6 to 41.2, and AWM from 44.4 to 45.5.

“The trend is depicted in pick bars and is striking: scaling actually reduces performance for weaker memories, where Synapse slightly increases from 40.6 to 41.2, and AWM from 44.4 to 45.5.”

May combine two claims.

On SWE-Bench-Verified, ReasoningBank saves 2.8 and 1.3 steps on average compared to baselines.

“The average step on SWE-Bench-Verified is also smaller by saving 2.8 and 1.3 steps respectively.”

May combine two claims.

MaTTS with ReasoningBank delivers the strongest benefit, with BoN climbing from 49.7 to 55.1.

“In contrast, MaT TS with Reasoning-Ban k delivers the strongest benefit, with BoN climbing from 49.7 to 55.1.”

May combine two claims.

ReasoningBank improves overall success rate on WebArena by +8.3, +7.2, and +4.6 with three different backbone LLMs compared to memory-free agents.

“Specifically, Re as on i ng Ban k improves the overall success rate on WebArena (Table 1) by +8.3, +7.2, and +4.6 with three different backbone LLMs compared to memory-free agents.”

May combine two claims.

ReasoningBank consistently outperforms existing memory mechanisms that store raw trajectories or only successful task routines on web browsing and software engineering benchmarks, improving both effectiveness and efficiency.

“Across web browsing and software engineering benchmarks, Re as on i ng Ban k consistently outperforms existing memory mechanisms that store raw trajectories or only successful task routines, improving both effectiveness and efficiency; M aT T S further amplifies these gains.”

• Also stated in: § 1. Introduction · boosted

  “We demonstrate that Re as on i ng -Ban k outperforms baselines in both effectiveness (up to 20% relative improvement, Table 1) and efficiency (up to 16% fewer interaction steps, Table 1).”

• Also stated in: § 4.2. Results of Re as on i ng Ban k · plain

  “Re as on i ng Ban k consistently outperforms baselines across LLM backbones on all datasets.”

May combine two claims.

For MaTTS w/o memory, gains are smaller and less consistent (parallel fluctuates 39.0-42.2, sequential 37.4-40.6).

“For the baseline of MaT TS w/o memory, the gains are smaller and less consistent (e.g., parallel scaling fluctuates between 39.0 and 42.2 and sequential scaling fluctuates between 37.4 and 40.6).”

May combine two claims.

ReasoningBank enables distillation of reasoning patterns from both successes and failures, achieving 46.5 on success-only traces and further improving to 49.7 when failures are included.

“In contrast, the design of ReasoningBank enables distillation of reasoning patterns from both successes and failures, achieving 46.5 on success-only traces and further improving to 49.7 when failures are included.”

May combine two claims.

With MaTTS, parallel scaling grows from 49.7 (k=1) to 55.1 (k=5), while sequential scaling rises from 49.7 to 54.5.

“With M aT T S, parallel scaling grows from 49.7 (𝑘 = 1) to 55.1 (𝑘 = 5), while sequential scaling rises from 49.7 to 54.5.”

May combine two claims.

When failures are added, Synapse increases only from 40.6 to 41.7, while AWM drops from 44.4 to 42.2.

“As a result, when failures are added, their performance is limited or even degraded: Synapse increases only from 40.6 (success only) to 41.7 (with failures), while AWM drops from 44.4 to 42.2.”

May combine two claims.

ReasoningBank consistently improves success rates on Mind2Web, with gains especially pronounced in the cross-domain setting.

“On Mind2Web (Table 3), which includes cross-task, cross-website, and cross-domain evaluations that impose progressively higher demands, Re as on i ng Ban k consistently improves success rates.The gains are especially pronounced in the cross-domain setting, which requires the highest level of generalization.”

May combine two claims.

ReasoningBank consistently reduces the number of steps compared to the vanilla baseline, with notably larger reductions on successful instances.

“Re as on i ng Ban k consistently reduces the number of steps compared to the vanilla baseline, with notably larger reductions on successful instances.”

May combine two claims.

Both parallel scaling and sequential scaling boost performance.

“Both parallel scaling and sequential scaling boost performance.”

May combine two claims.

For vanilla TTS without memory, sequential scaling yields little or no benefit as scaling goes on, and parallel scaling consistently dominates.

“In contrast, for vanilla TTS which is not equipped with memory module, sequential scaling yields little or even no benefit as scaling goes on, and parallel scaling consistently dominates.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

ReasoningBank makes good use of failure trajectories.

“Reasoning Ban k makes good use of failure trajectories.”

May combine two claims.

With stronger memory mechanisms such as ReasoningBank, sequential refinement brings higher gains at small k, but its benefit quickly saturates; parallel scaling continues to provide diverse rollouts and surpasses sequential at larger k.

“With stronger memory mechanisms such as Re as on i ng -Ban k, sequential refinement brings higher gains at small 𝑘, but its benefit quickly saturates-once the model either succeeds or fails decisively, further refinements add little new insight.In contrast, parallel scaling continues to provide diverse rollouts that allow the model to critique and improve upon its own generations, leading it to su”

May combine two claims.

Memory items induced by ReasoningBank from reflecting on past trajectories help prevent similar errors, producing emergent improvement; the original trajectory failed because of an imprecise search query that returned many irrelevant items.

“Figure 17 shows how memory items induced by Re as oni ng Ban k through reflecting on past trajectories helps to prevent similar errors from happening again, which enables emergent improvement.”

• Also stated in: § Additional information · plain

  “In this case, the original trajectory actually fails because of the imprecise search query that leads to numerous returned items, and irrelevant objects.”

May combine two claims.

Relative to no memory, ReasoningBank raises total token consumption by only about 4.3% while improving overall performance by 20.5%.

“compared with “No memory”, while the total token consumption is increased only by around 4.3%, the overall performance is boosted by 20.5%.”

May combine two claims.

ReasoningBank consistently improves task performance while reducing redundant exploration.

“Extensive experiments show that Re as on i ng Ban k consistently improves performance while reducing redundant exploration.”

May combine two claims. The quote only partly states this claim.

On SWE-Bench-Verified with Gemini-2.5-pro, ReasoningBank resolves 57.4% of issues versus 54.0% for no memory.

“| Re as on i ng Ban k | 57.4 | 19.8 |”

• Also stated in: § E. Limitations · plain

  “| No Memory | 54.0 | 21.1 |”

May combine two claims. The quote only partly states this claim.

On Mind2Web with Gemini-2.5-flash, ReasoningBank improves element accuracy, action F1, step success rate and task-level success rate over no memory across cross-task, cross-website and cross-domain settings (e.g., cross-task element accuracy 52.1 vs. 46.0).

“| Re as on i ng Ban k 52.1 60.4 44.9 4.8 44.3 52.6 33.9 2.3 40.6 41.3 36.6 1.6 |”

• Also stated in: § E. Limitations · plain

  “| No Memory | 46.0 59.1 40.3 3.3 39.8 45.1 31.7 1.7 35.8 37.9 31.9 1.0 |”

Probably not a substantial claim: may not pass S3 (load-bearing).

The 100% (ground-truth) verifier accuracy setting produces the best performance among the simulated accuracy levels.

“Intuitively, the 100% (ground-truth) accuracy setting yields the best performance.”

• AssumptionRests on an assumption: Simulating verifier accuracy by probabilistically flipping the judge's labels faithfully represents the noise of a real LLM-as-a-Judge verifier.

May combine two claims. The quote only partly states this claim.

On SWE-Bench-Verified with Gemini-2.5-flash, ReasoningBank resolves 38.8% of issues versus 34.2% for no memory.

“| Re as on i ng Ban k | 38.8 | 27.5 |”

• Also stated in: § E. Limitations · plain

  “| No Memory | 34.2 | 30.3 |”

May combine two claims.

MaTTS with memory-aware scaling scores 51.3 at k = 2 and 62.1 at k = 5, versus 52.4 at k = 5 for MaTTS without memory.

“MaT TS not only preserves efficiency at small 𝑘 (51.3 at 𝑘 = 2) but also sustains strong growth with scaling, reaching 62.1 at 𝑘 = 5, compared to only 52.4 for MaT TS w/o memory.”

May combine two claims.

Memory baselines such as Synapse and AWM incur substantially greater computation overhead than ReasoningBank while yielding smaller performance gains.

“baselines such as Synapse and AWM will greatly increase the computation overhead while achieving less performance gains compared with ReasoningBan k, demonstrating the cost-effectiveness of Re as on i ng Ban k.”

May combine two claims.

In a navigation-heavy shopping task, the baseline needs 29 steps because of repeated inefficient browsing, while ReasoningBank completes the task in 10 steps.

“In a navigation-heavy shopping task, the baseline requires 29 steps due to repeated inefficient browsing.”

• Also stated in: § Additional information · plain

  “In contrast, Re as on i ng Ban k leverages stored reasoning about category filtering, enabling the agent to directly reach the relevant items and complete the task in only 10 steps.”

May combine two claims.

For vanilla test-time scaling without a memory module, sequential scaling yields little or no benefit as scaling increases, and parallel scaling consistently dominates.

“In contrast, for vanilla TTS which is not equipped with memory module, sequential scaling yields little or even no benefit as scaling goes on, and parallel scaling consistently dominates.”

May combine two claims.

In a case study, the no-memory baseline inspects only the Recent Orders table and outputs the most recent purchase date, whereas ReasoningBank uses recalled reasoning hints to explore the full purchase history and identify the earliest order.

“While the baseline agent (without memory) only checks the “Recent Orders” table and mistakenly outputs the most recent purchase date, Re as on i ng Ban k recalls from past reasoning hints to explore the full purchase history and correctly identifies the earliest order.”

May combine two claims.

For ReasoningBank, sequential test-time scaling has a short-term advantage but parallel scaling dominates at larger scaling factors (e.g., 55.1 vs. 54.5 at k = 5).

“Sequential scaling shows shortterm advantage, but parallel dominates at larger scales for Reasoni ng Ban k.”

• Also stated in: § E. Limitations · plain

  “leading it to surpass sequential at larger 𝑘 (e.g., 55.1 vs. 54.5 at 𝑘 = 5).”

May combine two claims.

Memory-aware scaling improves sample efficiency and sustains stronger performance gains.

“Memory-aware scaling improves sample efficiency and sustains stronger performance gains.”

May combine two claims.

On the WebArena-Shopping subset with Gemma-3-12B-Instruct, ReasoningBank raises success rate from 17.1% without memory to 24.1%, exceeding Synapse (16.0%) and AWM (21.4%).

“While the baseline success rate without memory is 17.1%, ReasoningBank improves this to 24.1%, outperforming other memory-based methods such as Synapse (16.0%) and AWM (21.4%).”

May combine two claims.

ReasoningBank enables agents to solve tasks more efficiently by reusing and refining reasoning knowledge, avoiding unnecessary or redundant exploration.

“This indicates that Re as on i ng Ban k enables agents to solve tasks more efficiently by reusing and refining reasoning knowledge, thus avoiding unnecessary or redundant exploration.”

May combine two claims.

There is a synergy between memory and test-time scaling: high-quality memory steers scaled exploration toward more promising paths, and the rich experiences generated forge stronger memories.

“It creates a synergy between memory and test-time scaling: high-quality memory steers the scaled exploration toward more promising paths, while the rich experiences generated forge even stronger memories.”

• Also stated in: § Abstract · boosted

  “The better memory in turn guides more effective scaling, establishing a powerful synergy between memory and test-time scaling.”

May combine two claims.

Unlike baselines, ReasoningBank can transform failures into constructive signals rather than noise, enabling more robust generalization.

“This highlights that, unlike baselines, Re as on i ng Ban k can transform failures into constructive signals rather than noise, enabling more robust generalization.”

May combine two claims.

Baseline methods such as Synapse and AWM build memory solely from successful trajectories and thus are not equipped to benefit from failures.

“Baseline methods such as Synapse and AWM build memory solely from successful trajectories, and thus are not equipped to benefit from failures.”

May combine two claims.

MaTTS generates diverse exploration that provides contrastive signals, enabling ReasoningBank to synthesize better memories.

“We introduce memory-aware test-time scaling (MaTTS) in both parallel and sequential settings, which generates diverse exploration to provide contrastive signals, enabling ReasoningBank to synthesize better memories.”

• Also stated in: § 3.3. MaTTS: Memory-aware Test-Time Scaling · plain

  “Unlike the vanilla approach, MaT TS deliberately learns from the abundant successful and failure trajectories generated during scaling for more effective memory curation.”

May combine two claims.

ReasoningBank operates in a closed loop: it retrieves relevant memories to guide actions, then analyzes, distills, and consolidates new experiences back into memory, enabling continuous evolution.

“This process operates in a closed loop: when facing a new task, the agent retrieves relevant memories from ReasoningBank to guide its actions.Afterward, the new experience is analyzed, distilled, and consolidated back into the ReasoningBank, allowing the agent to continuously evolve and improve its strategic capabilities.”

• Also stated in: § Integrating Re as on i ng Ban k with Agents. · plain

  “Together, these steps form a closed-loop process: the agent leverages past experiences, constructs new memory from current tasks, and continually updates its memory, enabling sustained evolvement in test-time learning scenarios.”

May combine two claims.

ReasoningBank enables agents to generalize from both successful and failed experiences, providing richer guidance for test-time learning.

“By abstracting experiences into reusable reasoning units, ReasoningBank enables agents to generalize not only from successful cases (Alazraki et al., 2025;Fu et al., 2024;Zhao et al., 2024) but also by learning from failures, providing richer guidance for test-time learning.”

May combine two claims.

Scaling alone is insufficient; only when paired with a good memory mechanism does it contribute to curation of more effective memory.

“This asymmetry highlights that scaling alone is insufficient; only when paired with good memory mechanism does it contribute to curation of more effective memory, closing the virtuous cycle.”

The quote only partly states this claim.

ReasoningBank is a memory framework that extracts generalizable reasoning strategies from an agent's self-judged successful and failed experiences, without ground-truth labels.

“We propose ReasoningBank, a novel memory framework that distills generalizable reasoning strategies from an agent's self-judged successful and failed experiences.”

May combine two claims.

The baselines were run with the same memory retrieval and consolidation mechanisms as ReasoningBank, differing only in memory extraction, so the comparison is fair.

“To ensure a fair comparison, the baselines are implemented with the same “Memory Retrieval” and “Memory Consolidation” mechanisms.”

• Also stated in: § B.1. Web Browsing · plain

  “The only difference is about “Memory Extraction”, which is exactly how Re as on i ng Ban k different from baselines in terms of memory formulations.”

May combine two claims.

MaTTS unlocks more of the potential of agent systems and encourages diversified generation, improving pass@k performance.

“Overall, we see that M aT TS unlocks more potential of agent systems and encourages diversified generation for better pass@𝑘 performance.”

May combine two claims.

Using simple embedding-based retrieval and straightforward consolidation isolates the effect of content quality, so the observed gains can be attributed directly to the design of reasoning-oriented memory content.

“We intentionally adopt simple embeddingbased retrieval and straightforward consolidation to better isolate the effect of content quality.”

• Also stated in: § Additional information · plain

  “This choice ensures that the observed gains can be attributed directly to the design of reasoning-oriented memory content.”

May combine two claims.

Newly extracted memory items were added to the pool without pruning, a minimal consolidation choice made to avoid confounding from complex consolidation algorithms.

“We adopt a minimal consolidation strategy: newly generated items are directly added without additional pruning.”

• Also stated in: § Memory Consolidation. · plain

  “This choice highlights the contribution of Re as on i ng Ban k itself without introducing confounding factors from complex consolidation algorithms.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

All research ideas, methodology design, experiments, analyses, and final writing decisions were conceived, implemented, and validated solely by the authors.

“All research ideas, methodology design, experiments, analyses, and final writing decisions were conceived, implemented, and validated solely by the authors.”

Classified as probably not a substantial claim. May combine two claims.

Other memory architectures such as episodic or hierarchical memory address the orthogonal concern of memory form or structure, whereas this work's contribution concerns what should be stored and reused.

“These designs address orthogonal concerns (memory form/structure), while our contribution targets what should be stored and reused.”

Classified as probably not a substantial claim.

The model's thinking process is used as an approximation of the observation sequence, following Wang et al. (2025d), because observation representations are lengthy.

“We use the thinking process of 𝜋 L as the approximation of 𝑜 0:𝑡 due to lengthy observation representations followingWang et al. (2025d).”

• cites Wang et al. (2025d)Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

Memory and test-time scaling act synergistically: ReasoningBank steers scaling toward more promising rollouts, and diverse rollouts supply richer contrastive signals for memory.

“Further results reveal a strong synergy between memory and scaling: ReasoningBank guides scaling toward more promising rollouts, while diverse rollouts enrich memory with valuable contrastive signals.”