CoChat
Sign Up
PreprintCited by 28Topic ModelingNatural Language Processing TechniquesAI-based Problem Solving and Planning

GPT3-to-plan: Extracting plans from text using GPT-3

Open access
Olmo, Alberto, Sreedharan, Sarath, Kambhampati, Subbarao·2021·arXiv
DOI 10.48550/arxiv.2106.07131arXiv 2106.07131
Open this paper in CoChat — free accountOpen PDF

Abstract

Operations in many essential industries including finance and banking are often characterized by the need to perform repetitive sequential tasks. Despite their criticality to the business, workflows are rarely fully automated or even formally specified, though there may exist a number of natural language documents describing these procedures for the employees of the company. Plan extraction methods provide us with the possibility of extracting structure plans from such natural language descriptions of the plans/workflows, which could then be leveraged by an automated system. In this paper, we investigate the utility of generalized language models in performing such extractions directly from such texts. Such models have already been shown to be quite effective in multiple translation tasks, and our initial results seem to point to their effectiveness also in the context of plan extractions. Particularly, we show that GPT-3 is able to generate plan extraction results that are comparable to many of the current state of the art plan extraction methods.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2106.07131

Cites

9 of 9 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Reformulator modules can be supported to a large extent by LLMs because LLMs are very good at format change across syntactic representations.

    “These reformulator modules can be supported to a large extent by LLMs, given that one thing LLMs are very good at is format change across different syntactic representations (Olmo et al., 2021).”

    From LLMs Can't Plan, But Can Help Planning in LLM-Modulo Frameworks (2024) · cites this paper as Olmo et al., 2021

    The Experiments section (p. 2) describes GPT-3 generating a structured plan representation from natural-language text, and the Introduction (p. 1) reports performance comparable to task-specific extractors. This supports an example of LLM-assisted format conversion, but the paper does not establish that LLMs are broadly very good at changing formats across different syntactic representations or that they can support reformulator modules to a large extent. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • LLMs are good at format and syntax changes.

    “The framework also recognizes that LLMs are good at format/syntax changes.”

    From LLMs Can't Plan, But Can Help Planning in LLM-Modulo Frameworks (2024) · cites this paper as Olmo et al., 2021

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. Radford, A.; Wu, J.; Child, R.; Luan, D.; Amodei, D.; and Sutskever, I. 2019. Language models are unsupervised multitask learners. OpenAI blog 1(8): 9.

Cited by

Sorted by citations

Related

Authors
Olmo, A., Sreedharan, S., & Kambhampati, S.
ORCID
Sreedharan, Sarath
Venue
arXiv
Publisher
arXiv
Published
2021
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2106.07131
arXiv
2106.07131
OpenAlex
W3172480024
Cited by
27
References
9
Citations by year
2026: 1 · 2025: 2 · 2024: 11 · 2023: 11 · 2022: 2
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
Keywords
GPT-3, large language models, natural language processing, workflow automation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review