LLMs Can't Plan, But Can Help Planning in LLM-Modulo Frameworks

Probably not a substantial claim: may not pass S3 (load-bearing).

Simulators take on the roles of plan evaluation and critiques performed by the respective critics in the LLM-Modulo framework.

“Simulators takes on the roles of plan evaluation and critiques performed by the respective critics in the LLM-Modulo framework”

• cites Rajvanshi et al., 2023 sourceDisputed

  Section 4.3 uses AI2-Thor as the simulator for SayNav’s experiments, but does not assign it a plan-critique role. More decisively, Section 5 (Limitations and Discussion; page unavailable) says that mechanisms to verify and provide feedback on LLM-generated plans are future work, rather than roles already performed by the simulator. The paper does not discuss the LLM-Modulo framework. (LLM, conf=0.84)

  Checked by a reasoning model — may be wrong.

  Experimental — accuracy not yet measured

May combine two claims.

Works claiming planning capabilities for LLMs either work in domains where subgoal interactions can be safely ignored or delegate interaction resolution to humans in the loop.

“our close examination of several works claiming planning capabilities for LLMs (Kambhampati et al., 2023) suggests that they either work in domains/tasks where subgoal interactions can be safely ignored”

• cites Kambhampati et al., 2023 sourcePartly supported

  The Abstract (p. 1) says most claims about LLM planning capabilities are based on commonsense tasks, and Related Work (p. 2) describes some reasoning tasks as shallow and notes systems that use user feedback. The paper does not say that subgoal interactions can safely be ignored in those tasks or that humans resolve those interactions for the LLMs. (LLM, conf=0.72)

  Experimental — accuracy not yet measured

May combine two claims.

LLM-Modulo based agentification with automated critics in the loop significantly improves performance (6x of baselines) on the travel planning benchmark, even with a limit of 10 back prompting cycles and weaker models such as GPT-3.5turbo.

“LLM-Modulo based agentification with automated critics in the loop significantly improves the performance (6x of baselines) even with a limit of 10 back prompting cycles, and weaker models such as GPT-3.5turbo.”

• cites Gundawar et al., 2024 sourcePartly supported

  The Introduction and Experiments and Results report improved performance with LLM-Modulo and critics, including a rise from 0% to 5% final pass rate for GPT-3.5-Turbo. The Plan Backboard and Reformatter section sets a maximum of 10 iterations. However, the paper reports a 4.6× improvement for GPT-4-Turbo, not the claimed 6×; page numbers are unavailable in our copy. (LLM, conf=0.91)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing).

Approximate symbolic models can help drastically improve performance.

“the well known fact that even approximate symbolic models can help drastically improve the performance (c.f. ( Guan et al., 2022))”

• cites Guan et al., 2022 sourceSupported

Probably not a substantial claim: may not pass S3 (load-bearing).

LLMs can be used for designing a reward model or shaping the reward.

“LLMs can also be used for designing a reward model or shaping the reward”

• cites Bhambri et al., 2024Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Kwon et al., 2022Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Hao et al., 2023 sourceSupported

Probably not a substantial claim: may not pass S3 (load-bearing).

There has been a performance revolution in RL due to LLMs.

“there has been a performance revolution of sorts there (Yao et al., 2023b;Liang et al., 2023;Wang et al., 2023b).”

• cites Yao et al., 2023bNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Liang et al., 2023 sourceCould not verify

  The Introduction discusses LLMs planning robot actions using skills that may have been pretrained with reinforcement learning, and the Method describes LLM-generated robot policies. Neither passage reports a performance revolution in reinforcement learning due to LLMs; searches for RL performance improvements and a revolution found no such statement in our copy. (LLM, conf=0.72)

  Experimental — accuracy not yet measured
• cites Wang et al., 2023bNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

SayCan explicitly filters the action choices suggested by the LLM with the help of simulator.

“SayCan (Ahn et al., 2022) the earliest use of LLMs in generating policies in an RL-with-Simulator scenario, explicitly filters the action choices suggested by the LLM with the help of simulator.”

• cites Ahn et al., 2022 sourcePartly supported

  Section 3 (p. 4) and Algorithm 1 (p. 5) show that SayCan selects actions by combining LLM scores with learned affordance scores. They do not say a simulator filters the LLM's choices: Appendix C.2 (p. 20) uses simulation for RL training, while Appendix D.3 (p. 24) uses a language-based simulator to test planning. The paper also does not establish the claim that SayCan was the earliest such use of LLMs. (LLM, conf=0.87)

  Experimental — accuracy not yet measured

May combine two claims.

LLMs can successfully implement functions corresponding to hard critics and several common-sense critics.

“we also find that LLMs can successfully implement functions corresponding to hard critics and several common-sense critics.”

• cites Gundawar et al., 2024 sourceSupported

Probably not a substantial claim: may not pass S3 (load-bearing).

The fact that simulators play the role of verifiers is often not explicitly recognized in cases where LLMs are used as an actor to generate an admissible plan by interacting with a simulator, e.g., AlfWorld and Minecraft.

“The fact that simulators play the role of verifiers is often not explicitly recognized in cases where LLMs are used as an actor to generate an admissible plan by interacting with a simulator, for example in the case of AlfWorld (Yao et al., 2023b;Shinn et al., 2023) and Minecraft (Wang et al., 2023b).”

• cites Yao et al., 2023bNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Shinn et al., 2023 sourceSupported
• cites Wang et al., 2023bNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

Contrary to their claim of 'self-improvement', works like Wang et al., 2022 actually heavily depend on external knowledge (crafted seed examples) and critics (filtering step).

“Contrary to their claim of”

• cites Wang et al., 2022 sourcePartly supported

  Section 2.2 (p. 3) says Self-Instruct starts with 175 human-written seed tasks and includes a filtering step; Appendix A.1 (p. 14) says the method relies on those seeds. But Section 2.2 describes the filtering as similarity checks and heuristics, not external critics, so that characterization goes beyond what the paper supports. (LLM, conf=0.83)

  Experimental — accuracy not yet measured

May combine two claims.

Domains like AlfWorld do have sub-goal interactions for successful task completion, but Yao et al., 2023b and Shinn et al., 2023 largely ignore these interactions by either focusing on single subgoals or relying on the ergodic nature of the domain when prompting LLMs for generating plans.

“Although domains like AlfWorld(Shridhar et al., 2021) do have sub-goal interactions for successful task completion,(Yao et al., 2023b) and(Shinn et al., 2023) largely ignore these interactions by either focusing on single subgoals or relying on the ergodic nature of the domain when prompting LLMs for generating plans(Verma et al., 2024a).”

• cites Yao et al., 2023bNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Shinn et al., 2023 sourcePartly supported

  Split into 2 atoms — 1 confirmed, 0 partially supported, 0 disputed, 0 misattributed, 1 unverifiable. Expand the claim to see per-atom verdicts.

  Experimental — accuracy not yet measured
• cites Verma et al., 2024aNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

With back prompting from VAL acting as the external verifier and critic, LLM performance in Blocks World improves to 82% within 15 back prompting rounds, while in Logistics, it improves to 70%.

“with back prompting from VAL (Howey et al., 2004) acting as the external verifier and critic, LLM performance in Blocks World improves to 82% within 15 back prompting rounds, while in Logistics, it improves to 70%.”

• cites Valmeekam et al., 2023cNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Howey et al., 2004Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S4 (based).

LLM-Modulo Frameworks combine LLMs with external model-based verifiers in a tighter bi-directional interaction regime.

“We present a vision of LLM-Modulo Frameworks that combines the strengths of LLMs with external model-based verifiers in a tighter bi-directional interaction regime.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

LLMs are giant external non-veridical memories that can serve as powerful cognitive orthotics for human or machine agents, if rightly used.

“we take the stance that LLMs are amazing giant external non-veridical memories that can serve as powerful cognitive orthotics for human or machine agents, if rightly used.”

• AssumptionRests on an assumption: LLMs are pre-trained on web-scale language corpora, effectively our collective consciousness.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Real-world planning problems are often specified incompletely, leaving it to human commonsense to refine the specification.

“Often the planning problems in real world situations are specified incompletely, leaving it to the human commonsense to refine the specification.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

The underlying n-gram nature makes LLMs effortlessly intermix disparate fields of study.

“The underlying ngram nature makes them effortlessly intermix what would be considered disparate fields of study”

• AssumptionRests on an assumption: LLMs are pre-trained on web-scale language corpora, effectively our collective consciousness.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

LLMs make it easy to get problem-specific knowledge as long as we are willing to relax the correctness requirements of that knowledge.

“Indeed, LLMs make it easy to get problem-specific knowledge as long as we are willing to relax the correctness requirements of that knowledge.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

In general, the verifier may be more complex and can involve substantial work.

“In general though, the verifier may be more complex and can involve substantial work”

May combine two claims.

In Tree of Thoughts, the tree is essentially a way to generate diverse priming prompts set up in a problem-specific way, with no deeper connection to search-based agents.

“there is really no deeper connection to search-based agents.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

LLMs are approximate knowledge sources, without guarantees, trained on collective consciousness.

“They are a kind of approximate knowledge source (albeit sans guarantees) trained on our collective consciousness.”

• AssumptionRests on an assumption: LLMs are pre-trained on web-scale language corpora, effectively our collective consciousness.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

LLMs should be viewed as universal approximate knowledge sources that have meaningful roles in planning/reasoning beyond simple format translators.

“We also argue that LLMs should be viewed as universal approximate knowledge sources that have much more meaningful roles to play in planning/reasoning tasks beyond simple front-end/back-end format translators.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

A prompt diversification strategy helps increase the completeness of LLM candidate generation.

“This last strategy helps increase the completeness of the LLM candidate generation”

May combine two claims.

Claims about LLM self-verification in the literature are either made in tacit knowledge tasks with little possibility of a verifier or use external verification by simulators or simple operating-system calls.

“those claims are either (i) made in the context of tacit knowledge tasks for which there is little possibility of a verifier (e.g.essay writing)-making it hard to evaluate whether LLM's critiquing actually helped or (ii) the external verification is carried out either by simulators (Wang et al., 2023b;Yao et al., 2023b) or simple calls to the underlying operating system.”

• cites Wang et al., 2023bNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites Yao et al., 2023bNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

LLMs cannot self-critique their plans, and therefore cannot self-improve by generating synthetic data via self-critique and fine-tuning.

“One important corollary of the fact that LLMs cannot selfcritique their plans is that they also can't self-improve by generating synthetic data, e.g. by generating plans themselves, critiquing the plans by themselves to improve them, and then using those to fine-tune themselves”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

A system that takes constant time to produce the next token cannot possibly be doing principled reasoning on its own.

“Even from a pure engineering perspective, a system that takes constant time to produce the next token cannot possibly be doing principled reasoning on its own.”

Probably not a substantial claim: may not pass S4 (based).

The completeness of the LLM-Modulo system depends on the LLM's ability to generate all potentially relevant candidates.

“The completeness of the system depends on the LLM's ability to generate all potentially relevant candidates.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Because 24 puzzle solutions can be verified by simple arithmetic operations, it is trivial to implement an external verifier for these problems.

“Because the 24 puzzle's solutions can be verified by simple arithmetic operations, it is trivial to implement an external verifier for these problems.”

May combine two claims.

Using LLMs this way means humans need only be present when the dynamics/preference model is being teased out and refined, and actual planning can be left to sounder planning frameworks with correctness guarantees.

“This way of using LLMs has the advantage that the humans need only be present when the dynamics/preference model is being teased out and refined, and the actual planning after that can be left to sounder planning frameworks with correctness guarantees, such as the LLM-Modulo framework we propose.”

Probably not a substantial claim: may not pass S4 (based).

LLMs can generate approximate ideas not just about plan candidates, but also about domain models, problem reduction strategies, and refinements to the problem specification.

“the framework explicitly recognizes that the LLMs can generate approximate ideas not just about plan candidates, but domain models, problem reduction strategies, and refinements to the problem specification.”

May combine two claims.

LLMs are unlikely to have System 2 competencies by themselves, but can be valuable resources in solving System 2 tasks.

“While it is unlikely that they will have System 2 competencies by themselves, they can nevertheless be valuable resources in solving System 2 tasks.”

May combine two claims.

Auto-regressive LLMs cannot, by themselves, do planning or self-verification.

“We argue that auto-regressive LLMs cannot, by themselves, do planning or self-verification (which is after all a form of reasoning)”

• Also stated in: § 1. Introduction · plain

  “The central position of the paper is that LLMs cannot plan themselves”

May combine two claims.

LLMs cannot take on the role of hard critics with soundness guarantees, but can help simulate some aspects of the role of soft style critics.

“while LLMs cannot take on the role of hard critics with soundness guarantees,foot_10 they can help simulate some aspects of the role of soft (style) critics.”

Probably not a substantial claim: may not pass S4 (based).

The only way to guarantee that a course of actions achieves a desired state of affairs is to support robust planning capabilities.

“The only way to guarantee the latter is to to support robust planning capabilities”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

We can do the check easily because of the formal specification of correctness.

“Note that we can do this check easily because of the formal specification of correctness.”

Probably not a substantial claim: may not pass S4 (based).

Viewing LLMs as an approximate knowledge source trained over civilizational knowledge models improves the ability of LLMs to guess executable plans.

“Viewing LLMs as an approximate knowledge source trained over civilizational knowledge models, then the ability of LLMs to guess executable plans improves.”

• AssumptionRests on an assumption: LLMs can be viewed as an approximate knowledge source trained over civilizational knowledge models.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

The essence of LLM-Modulo framework is equally applicable to other scenarios involving planning and reasoning, such as Reinforcement Learning with Simulators.

“we believe that the essence of LLM-Modulo framework is equally applicable to other scenarios involving planning and reasoning-such as Reinforcement Learning with Simulators.”

May combine two claims.

LLMs cannot plan themselves but can play a variety of constructive roles in solving planning tasks, especially as approximate knowledge sources and candidate plan generators in LLM-Modulo Frameworks in conjunction with external sound model-based verifiers.

“Our position is that LLMs cannot plan themselves but can play a variety of constructive roles in solving planning tasks-especially as approximate knowledge sources and candidate plan generators in the socalled LLM-Modulo Frameworks in conjunction with external sound model-based verifiers.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Planning in LLM-Modulo framework avoids inheriting the expressiveness and search-complexity limitations of traditional symbolic planners, while retaining their soundness guarantees.

“We discussed how planning in LLM-Modulo framework avoids inheriting the expressiveness and search-complexity limitations of traditional symbolic planners, while retaining their soundness guarantees.”

May combine two claims.

If a plan is returned in Mystery BW, it must have passed VAL and is thus guaranteed correct by its model.

“if a plan is returned, it must have passed muster with VAL, and is thus guaranteed correct by its model”

• cites Howey et al., 2004Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

The fact that LLMs can help come up with approximate quasi-symbolic transition models, reward models and models of high level actions has made a bigger splash in RL.

“the fact that LLM's can help come up with approximate quasi-symbolic transition models, reward models and models of high level actions has made a bigger splash in RL.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

DRL methods have horrendous sample complexity even in reaching a single subgoal.

“Given the horrendous sample complexity of the DRL methods even in reaching a single subgoal”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

Multi-modality increases the coverage of LLMs' System 1 imagination but it is not clear that this gives them System 2 competence.

“While multi-modality is a great addition that increases the coverage of their System 1 imagination (Figure 1), it is not clear that this gives them System 2 competence.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

It is easy to accommodate replanning scenarios, where the loop starts with an externally supplied candidate plan.

“Although we focus on planning from scratch, it is easy to accommodate replanning scenarios, where the loop starts with an externally supplied candidate plan.”