CoChat
Sign Up
Cited by 192AI-based Problem Solving and PlanningDomain Adaptation and Few-Shot LearningTopic Modeling

Expel: LLM agents are experiential learners

Open access
Zhao, Andrew, Huang, Daniel, Xu, Quentin, Lin, Matthieu, Liu, Yong-Jin, Huang, Gao·2024·Proceedings of the AAAI Conference on Artificial Intelligence
DOI 10.1609/aaai.v38i17.29936
Open this paper in CoChat — free accountOpen PDF

Abstract

The recent surge in research interest in applying large language models (LLMs) to decision-making tasks has flourished by leveraging the extensive world knowledge embedded in LLMs. While there is a growing demand to tailor LLMs for custom decision-making tasks, finetuning them for specific tasks is resource-intensive and may diminish the model's generalization capabilities. Moreover, state-of-the-art language models like GPT-4 and Claude are primarily accessible through API calls, with their parametric weights remaining proprietary and unavailable to the public. This scenario emphasizes the growing need for new methodologies that allow learning from agent experiences without requiring parametric updates. To address these problems, we introduce the Experiential Learning (ExpeL) agent. Our agent autonomously gathers experiences and extracts knowledge using natural language from a collection of training tasks. At inference, the agent recalls its extracted insights and past experiences to make informed decisions. Our empirical results highlight the robust learning efficacy of the ExpeL agent, indicating a consistent enhancement in its performance as it accumulates experiences. We further explore the emerging capabilities and transfer learning potential of the ExpeL agent through qualitative observations and additional experiments.

AI snapshot

Written from the paper’s full text.

TL;DR

ExpeL is an LLM agent that learns from its own task experiences without parameter updates by storing trajectories, extracting natural-language insights, and retrieving similar successful trajectories for inference. It improves over ReAct/Act baselines across HotpotQA, ALFWorld, and WebShop and shows forward transfer from HotpotQA to FEVER.

Key findings

  • ExpeL consistently outperformed ReAct and Act across HotpotQA, ALFWorld, and WebShop; in ablations, ExpeL reached 39.0 ± 1.7 success on HotpotQA vs. ReAct 28.0 ± 1.4, and 59.0 ± 0.3 on ALFWorld vs. ReAct 40.0 ± 0.3.
  • Both insight extraction and retrieval mattered: insights-only/retrieval-only variants scored 36%/31% on HotpotQA, 50%/55% on ALFWorld, and 37%/38% on WebShop, with WebShop scores of 0.675/0.67, respectively.
  • Cross-task learning matched Reflexion on HotpotQA (40% at R3 vs. 39%) and outperformed it on ALFWorld (54% at R3 vs. 59%) without repeated attempts.
  • Transfer from HotpotQA to FEVER improved performance for both transfer agents, with the agent using few in-context examples improving more; ablations showed learned insights beat hand-crafted ones (32.0 ± 1.1), reflections hurt (29.0 ± 0.4), gpt-3.5 insights were lower (32.0 ± 0.4), and task-similarity retrieval was best (59.0 ± 0.3 vs. reasoning similarity 48.5 ± 2.1 and random 42.5 ± 0.8).

Methods

  • ExpeL uses ReAct as its base planner; during training it gathers experiences by retrying tasks up to Z times with Reflexion, stores trajectories in an experience pool, and extracts natural-language insights via LLM operators ADD/EDIT/UPVOTE/DOWNVOTE from fail/success pairs and lists of successes. At evaluation, it augments task specifications with insights and retrieves top-k successful trajectories by task similarity using Faiss/kNN/all-mpnet-base-v2.
  • Evaluated on HotpotQA, ALFWorld, WebShop, and FEVER transfer; four-fold validation, success rate (exact match for HotpotQA/FEVER, task completion for ALFWorld, matching item for WebShop), plus WebShop reward and ALFWorld task-type breakdown. Baselines were ReAct, Act, and imitation learning; actions used gpt-3.5-turbo-0613, insights used gpt-4-0613, with temperature 0 and greedy decoding.
  • Transfer used HotpotQA as source and FEVER as target, adapting source insights with target fewshot examples and comparing against ReAct, Act, and transfer without task demonstrations.

Limitations

  • Stated: the work investigates textual observations only, limiting real-world applicability; closed-source API LLMs may be off-limits in some applications; insights currently fit the context window but lifelong learning may need extra retrieval; prompting techniques lack the theoretical underpinnings of reinforcement learning.
  • Evident: only deterministic environments are considered; evaluation is limited to four text-based benchmarks with four-fold validation; transfer assumes source and target tasks share common knowledge.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1609/aaai.v38i17.29936

Cites

56 of 56 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

7 of 20 cited sources checked · 4 supported · 0 flagged · accuracy not yet measured

  • Findingown data

    ExpeL consistently outperformed strong baselines on three vastly different domains.

    “We evaluated ExpeL on three vastly different domains and consistently outperformed strong baselines.”
    • Also stated in: § 5.2 Main Results · plain
      “Experiential learning Augmenting agents with abstracted insights and the ability to recall successful trajectories improve performance across all environments compared to baseline agents.”
    Quote in the paperSubstantial claim
  • Findingown data

    The ExpeL agent's performance consistently enhances as it accumulates experiences.

    “Our empirical results highlight the robust learning efficacy of the ExpeL agent, indicating a consistent enhancement in its performance as it accumulates experiences.”
    Quote in the paperSubstantial claim
  • Findingown data

    ExpeL showed performance gain compared to vanilla ReAct and Act agents.

    “We demonstrated its learning abilities by showing its performance gain compared to vanilla ReAct and Act agents.”
    Quote in the paperSubstantial claim
  • Findingown data

    The ExpeL agent updated its beliefs through the insights and over its gained experience.

    “We noticed our ExpeL agent updated its beliefs through the insights and over its gained experience.”
    Quote in the paperSubstantial claim
  • Findingown data

    Learned insights by the agent are more advantageous than hand-crafted ones.

    “Results in Tab. 3 show several significant findings: (1) learned insights by the agent are more advantageous than hand-crafted ones;”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim. 18
  • May combine two claims. 24
  • The quote only partly states this claim. 1

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Retrieval allows LLMs to access databases, mitigating hallucinations.

    “Retrieval allows LLMs to access databases, mitigating hallucinations (Li et al. 2022;Wang, Yang, and Wei 2023;Rubin, Herzig, and Berant 2022;Liu et al. 2022).”
    Quote in the paperNot a substantial claim1 of 3 sources checked
    • cites Li et al. 2022

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Wang, Yang, and Wei 2023

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Rubin, Herzig, and Berant 2022 source

      The Background: Prompt Retrieval Problem setup section describes retrieving examples from a training set and supplying them to a language model as a prompt. This supports retrieval of stored examples, but the paper does not establish that retrieval gives LLMs access to databases or mitigates hallucinations. (LLM, conf=0.81)

      Experimental — accuracy not yet measured

  • Classified as probably not a substantial claim.

    Premisereasoning

    A relatively less explored area is facilitating agents to learn autonomously from their own experiences.

    “A relatively less explored area is facilitating agents to learn autonomously from their own experiences, similar to a student gaining insights from practicing for an exam.”
    Quote in the paperSubstantial claim1 of 3 sources checked
    • cites Lin et al. 2023a

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Shaw et al. 2023 source

      Section 3.1 (page not available) describes improving an agent iteratively using successful episodes from its own tree-search policy, supporting the idea of learning from experience. It does not say that autonomous learning from experience is a relatively less explored area; it notes that prior work has explored reinforcement learning for agent improvement. (LLM, conf=0.79)

      Experimental — accuracy not yet measured
    • cites Shinn et al. 2023

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Interpretationown data

    ExpeL leverages cross-task learning by accumulating task experience, whereas Reflexion improves results by iteratively refining insights through repeated task execution.

    “While Reflexion improves results by iteratively refining insights through repeated task execution (R1, R2, R3...), our ExpeL agent leverages cross-task learning by accumulating task experience.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Shinn et al. 2023 source

  • May combine two claims.

    Interpretationcitation

    ExpeL enables inter-task learning, whereas self-improvement methods like Reflexion facilitate intra-task improvements.

    “Furthermore, self-improvement methods like Reflexion (Shinn et al. 2023) facilitate intra-task improvements, but ExpeL enables inter-task learning.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Shinn et al. 2023 source

  • May combine two claims.

    Premisecitation

    Agents like Reflexion showcase feedback-based improvement, yet often lack cross-task memory.

    “Agents like Reflexion showcase feedback-based improvement, yet often lack cross-task memory (Shinn et al. 2023).”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Shinn et al. 2023 source

      The Introduction and Self-reflection sections describe Reflexion improving through feedback and storing reflections for later trials. They describe memory across trials, but do not establish that Reflexion lacks cross-task memory; that limitation is unsupported by our copy. (LLM, conf=0.83)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Premisecitation

    Previous works on learning in LLM agents have primarily been trained on extensive human-labeled datasets or improved via iterative retries on a single task.

    “Additionally, previous works on learning in LLM agents have primarily been trained on extensive human-labeled datasets (Lin et al. 2023a;Shaw et al. 2023) or improved via iterative retries (Shinn et al. 2023) on a single task.”
    Quote in the paperSubstantial claim1 of 3 sources checked
    • cites Lin et al. 2023a

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Shaw et al. 2023 source
    • cites Shinn et al. 2023

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • May combine two claims.

    Premisecitation

    Current LLMs are bounded by context window size, so these agents have no recollections of what they have seen, and therefore no learning can be done outside of a few demonstrations.

    “However, since current LLMs are bounded by context window size (Tworkowski et al. 2023), these agents have no recollections of what they have seen, and therefore no learning can be done outside of a few demonstrations.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Tworkowski et al. 2023 source

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Findingnone given

    The extracted insights do not exceed the current LLM's token limit, so they can fit into the agent's context window.

    “Furthermore, since our extracted insights do not exceed the current LLM's token limit, we can fit them into the agent's context window.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    ExpeL developed several unexpected emerged abilities at the end of its training.

    “Lastly, we presented several unexpected emerged abilities our agent developed at the end of its training.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    In HotpotQA, a likely influential insight was that the agent should consider the answer might be in the observations already made.

    “This ability was particularly observed in HotpotQA where a likely influential insight was stating that the agent should “consider the answer might be in the observations already made”.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingreasoning

    The ExpeL agent promptly and accurately found the correct item at the first step while the ReAct agent could not find it in time.

    “This behavior emerged from the extracted insight claiming that “when searching for an item” it needs to “consider its nature and its typical usage”, leading the agent to promptly and accurately find the correct item at the first step while the ReAct agent could not find it in time.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    ExpeL paired with Reflexion shows a notable improvement in success rate, with the success rate increasing as the number of task reattempts grows.

    “The results demonstrate a notable improvement in the success rate when ExpeL is paired with Reflexion, with the success rate increasing as the number of task reattempts grows.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Reason similarity is advantageous but slightly drops in performance.

    “Reason similarity is still advantageous but slightly drops in performance, possibly due to dynamically changing fewshots during a single trajectory, causing instabilities.”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data

    Using gpt-4-0613 to extract insights outperforms gpt-3.5-turbo-0613.

    “We empirically found that gpt-4-0613 is better than gpt-3.5-turbo-0613at following instructions on how to use the insight extraction operators and hallucinated less.”
    • Also stated in: § 4.5 ExpeL's Strengths · boosted
      “our experiments show that using gpt-4 to extract insights outperforms gpt-3.5-turbo(refer to Sec. 5.6).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Randomly selected baseline has a significant drop in performance while ranking using reason similarity also has a noticeable dip.

    “Randomly selected baseline has a significant drop in performance while ranking using reason similarity also has a noticeable dip.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Hand-crafted insights enjoyed a performance boost over ReAct but were less effective than LLM-generated ones.

    “Hand-crafted insights enjoyed a performance boost over ReAct but were less effective than LLMgenerated ones.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    ExpeL has the capacity to recover from errors and stay on course without hallucinating when completing tasks.

    “This highlights ExpeL's capacity to recover from errors and stay on course without hallucinating when completing tasks.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    Adjusting prompts rather than altering LLM parameters ensures the LLM's inherent common sense knowledge remains intact, allowing for superior generalization.

    “Rather than altering the LLM parameters, adjusting the prompts may be more beneficial: this strategy ensures that the LLM's inherent common sense knowledge remains intact, allowing for superior generalization (Liu et al. 2023a).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Liu et al. 2023a

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given

    ExpeL is not restricted to specific language models and complements existing strategies aimed at enhancing LLM agent planning capabilities.

    “It is not restricted to specific language models and complements existing strategies aimed at enhancing LLM agent planning capabilities.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    ExpeL does not rely on retries during deployment.

    “ExpeL does not rely on retries during deployment, which certain domains require.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationnone given

    The trial and error way of gathering experiences improves the chances of getting more positive examples for experience recall during evaluation and allows for collecting valuable success/failure pairs used for comparisons during insight extraction.

    “To highlight, this trial and error way of gathering experiences not only improves the chances of getting more positive examples for experience recall during evaluation but also allows for collecting valuable success/failure pairs used for comparisons during insight extraction (Sec.4.2).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationreasoning

    The self-correction behavior is possibly encouraged by the generated insight "reassess the situation and consider alternative actions" if "an attempt does not progress the task".

    “This behavior is possibly encouraged by the generated insight “reassess the situation and consider alternative actions” if “an attempt does not progress the task”.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    Comparing a failed trajectory with a successful trajectory for the same task offers a concrete understanding of the agent's shortcomings, highlighting the correct and incorrect actions.

    “First, we let the agent compare a failed trajectory with a successful trajectory for the same task.This comparison offers a concrete understanding of the agent's shortcomings, highlighting the correct and incorrect actions.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    Reflections sometimes output hallucinations, misleading the insight extraction stage.

    “(2) using reflections in addition to success/failure pairs and lists of successes is disadvantageous, possibly due to reflections sometimes outputting hallucinations, therefore misleading the insight extraction stage;”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning

    Extra retrieval steps for insights might be needed for truly lifelong learning agents to ensure a manageable context window size.

    “However, extra retrieval steps for insights might be needed for truly lifelong learning agents to ensure a manageable context window size.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given

    ExpeL demands less data, reduces computational resources, and is straightforward to implement.

    “Additionally, our learning approach is highly accessible; it demands less data, reduces computational resources, and is straightforward to implement.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Finetuning an LLM restricts its functionalities and can hurt its generalization abilities.

    “Furthermore, finetuning an LLM restricts its functionalities and can hurt its generalization abilities (Du et al. 2022).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Du et al. 2022

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    The work investigated tasks with textual observation, which is limiting in real-world scenarios.

    “In this work, we investigated tasks with textual observation, which is limiting in real-world scenarios.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationreasoning

    The slight drop in performance for reason similarity is possibly due to dynamically changing fewshots during a single trajectory, causing instabilities.

    “Reason similarity is still advantageous but slightly drops in performance, possibly due to dynamically changing fewshots during a single trajectory, causing instabilities.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    ExpeL offers inherent interpretability because both the extracted experiences and successful trajectories are presented in natural language, allowing users to easily inspect, modify, or remove potentially harmful trajectories/insights.

    “First and foremost, ExpeL offers inherent interpretability, as both the extracted experiences and successful trajectories are presented in natural language.This design allows users to easily inspect, modify, or remove potentially harmful trajectories/insights -a challenge in finetuned models.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    Identifying patterns within a set of successful trajectories from different tasks sheds light on common 'good practices' that the agent can adopt to ensure success in evaluation tasks.

    “Second, we let the agent identify patterns within a set of successful trajectories from different tasks.This approach sheds light on common”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    Prompting techniques lack theoretical underpinnings that could potentially impact the efficiency of the resulting policies.

    “Lastly, unlike reinforcement learning methods, prompting techniques lack theoretical underpinnings that could potentially impact the efficiency of the resulting policies.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    The belief update enables the agent to avoid unnecessary actions and increase efficiency in solving a given task.

    “This belief thereby update enables the agent to avoid unnecessary actions and increase efficiency in solving a given task.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    There is a synergistic interplay between abstraction and recollection in experiential learning, with ExpeL showing a quantitative advantage over baseline/restricted learning mode agents.

    “These observations highlight the synergistic interplay between abstraction and recollection in experiential learning, with ExpeL showing a quantitative advantage over baseline/restricted learning mode agents.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationnone given

    The method was investigated using closed-source API LLMs, which can be off-limits in some applications.

    “Additionally, we investigated the efficacy of our method by using closed-source API LLMs, which can be off-limits in some applications.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    Both insight extraction and task similarity retrieval are essential and synergistic.

    “Additionally, we compare ExpeL with ExpeL (retrieve-only) and ExpeL (insights-only) to highlight that both insight extraction and task similarity retrieval are essential and synergistic.”
    Quote in the paperSubstantial claim
36 of 36

  • The authors state this as an assumption.

    The source and target tasks exhibit common knowledge.

    “Like most transfer learning settings, we assume that the source and target tasks exhibit common knowledge.”

  • The paper relies on this without support.

    Some of the most potent language models are proprietary.

    “Furthermore, some of the most potent language models are proprietary.”

  • The paper relies on this without support.

    WebShop presents a unique challenge, requiring both website-based reasoning and precise execution of actions.

    “Furthermore, WebShop presents a unique challenge, requiring both website-based reasoning (price comparisons, query reformulation, etc.) and precise execution of actions (searching, clicking, option selection, etc.).”

  • The paper relies on this without support.

    Human learning occurs mainly either by storing successful trajectories in memory or by extracting high-level insights from experiences.

    “Human learning occurs mainly either by storing successful trajectories in memory, which can be later recalled as specific examples, or by extracting high-level insights from experiences, enabling generalization to novel situations.”

  • The paper relies on this without support.

    The source and target tasks are inherently different, so no experience pool can be retrieved from for transfer.

    “Notice that since source and target tasks are inherently different, we do not have an experience pool to retrieve from; thus, the ExpeL Transfer agents use the existing fixed fewshot examples as in-context examples.”
5 of 5

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Agent memory systems are instantiated within an agent's ongoing interaction with an environment, continuously incorporating new information generated by the agent's own actions and environmental feedback into a persistent memory base.

    “In contrast, agent memory systems are instantiated within an agent's ongoing interaction with an environment , continuously incorporating new information generated by the agent's own actions and environmental feedback into a persistent memory base (Wang et al., 2024m; Zhao et al., 2024; Sun et al., 2025e).”

    From Memory in the Age of AI Agents (2025) · cites this paper as Zhao et al., 2024

    Section 4.1, “Gathering Experiences” (page not indicated), says ExpeL adds trajectories from its environment interactions to an experience pool after each training trial and uses failed attempts for self-reflection. Section 4, “ExpeL: An Experiential Learning Agent” (page not indicated), distinguishes this training stage from evaluation, where the agent uses experiences already gathered. The paper therefore supports an agent-generated memory base, but not the claim that it continuously incorporates new information throughout ongoing use. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • The resulting summaries capture the global, high-level information of the original data rather than specific factual or experiential details.

    “The resulting summaries capture the global, high-level information of the original data, rather than specific factual or experiential details (Zhao et al., 2024; Anokhin et al., 2024).”

    From Memory in the Age of AI Agents (2025) · cites this paper as Zhao et al., 2024

    Section 4.2 describes extracting high-level insights from experiences, supporting the claim that the extracted information is abstract rather than a set of specific examples. But Sections 4.2–4.3 also say ExpeL retains and retrieves specific successful trajectories; the paper does not support treating the resulting information as exclusively high-level or devoid of experiential detail. Page numbers are unavailable in our copy. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • Agent memory systems are instantiated within an agent's ongoing interaction with an environment, continuously incorporating new information from the agent's own actions and environmental feedback into a persistent memory base.

    “In contrast, agent memory systems are instantiated within an agent's ongoing interaction with an environment , continuously incorporating new information generated by the agent's own actions and environmental feedback into a persistent memory base (Wang et al., 2024m; Zhao et al., 2024; Sun et al., 2025e).”

    From Memory in the Age of AI Agents (2025) · cites this paper as Zhao et al., 2024

    Section 4.1, “Gathering Experiences” (page not provided), says ExpeL stores trajectories from its actions and environmental observations in an experience pool during training. Sections 4.3–4.4 describe retrieving from that pool during evaluation, not continuously adding evaluation experiences to it. The claim therefore overstates how continuously the persistent memory is updated. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • The resulting summaries capture the global, high-level information of the original data, rather than specific factual or experiential details.

    “The resulting summaries capture the global, high-level information of the original data, rather than specific factual or experiential details (Zhao et al., 2024; Anokhin et al., 2024).”

    From Memory in the Age of AI Agents (2025) · cites this paper as Zhao et al., 2024

    Section 4.2 distinguishes high-level insights extracted from experiences from successful trajectories recalled as specific examples. That supports the contrast with specific experiential details, but the paper describes extracted insights rather than summaries of original data and does not establish that they omit specific factual details. Section 4.3 also says the agent uses both insights and retrieved trajectories. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • Most existing approaches construct an experience pool or memory bank and focus on improving the retrieval of useful past information.

    “Most existing approaches construct an experience pool or memory bank and focus on improving the retrieval of useful past information (Zhao et al., 2024; Gao et al., 2024; Xu et al., 2025b).”

    From 3DLLM-Mem: Long-Term Spatial-Temporal Memory for Embodied 3D Large Language Model (2025) · cites this paper as Zhao et al., 2024

    Section 4.2 describes ExpeL’s experience pool and its retrieval of relevant successful trajectories. But the paper does not establish that most existing approaches use an experience pool or memory bank and focus on improving retrieval; ExpeL itself also emphasizes extracting insights from experiences. The indexed passages provide section labels but no page numbers. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

Cites

Flagged first, then by citations

Research works printed but not linked: 9

  1. NeurIPS.
  2. Du, M.; He, F.; Zou, N.; Tao, D.; and Hu, X. 2022. Shortcut Learning of Large Language Models in Natural Language Understanding: A Survey. arXiv preprint.
  3. NeurIPS.
  4. Maas; Carey; Wheeler; Saatchi; Billington; and Shamash. 2023. To Infinity and Beyond: SHOW-1 and Showrunner Agents in Multi-Agent Simulations. arXiv preprint.
  5. Sumers, T. R.; Yao, S.; Narasimhan, K.; and Griffiths, T. L. 2023. Cognitive Architectures for Language Agents. arXiv preprint.
  6. Thoppilan, R.; De Freitas, D.; Hall, J.; Shazeer, N.; Kul- shreshtha, A.; Cheng, H.-T.; Jin, A.; Bos, T.; Baker, L.; Du, Y.; et al. 2022. LaMDA: Language Models for Dialog Ap- plications. arXiv preprint.
  7. Touvron, H.; Lavril, T.; Izacard, G.; Martinet, X.; Lachaux, M.-A.; Lacroix, T.; Rozière, B.; Goyal, N.; Hambro, E.; Azhar, F.; et al. 2023a. LLaMA: Open and Efficient Foun- dation Language Models. arXiv preprint.
  8. Touvron, H.; Martin, L.; Stone, K.; Albert, P.; Almahairi, A.; Babaei, Y.; Bashlykov, N.; Batra, S.; Bhargava, P.; Bhosale, S.; et al. 2023b. Llama 2: Open Foundation and Fine-Tuned Chat Models. arXiv preprint.
  9. Zhang, Z.; Zhang, A.; Li, M.; and Smola, A. 2023. Auto- matic Chain of Thought Prompting in Large Language Mod- els. In ICLR.

Cited by

Sorted by citations

Related

Authors
Zhao, A., Huang, D., Xu, Q., Lin, M., Liu, Y.-J., & Huang, G.
Affiliations
Tsinghua University (CN)
ORCID
Venue
Proceedings of the AAAI Conference on Artificial Intelligence
Publisher
Association for the Advancement of Artificial Intelligence (AAAI)
Published
2024-03-24
Volume
38
Issue
17
Pages
19632-19642
ISSN
  • 2159-5399 (ISSN-L)
  • 2374-3468
DOI
10.1609/aaai.v38i17.29936
OpenAlex
W4393160747
Cited by
153
References
31
Citations by year
2026: 61 · 2025: 63 · 2024: 28 · 2023: 1
FWCI
44.76
Type
conference paper
Language
en
Open access
Locations
  • Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Funders
  • National Natural Science Foundation of China (U2336214, 62332019, 62022048)
  • Tsinghua University
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords
large language models, LLM agents, experiential learning, decision-making, transfer learning, nonparametric learning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-16.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-16.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review