Cited by 25AI-based Problem Solving and PlanningConstraint Satisfaction and OptimizationArtificial Intelligence in Games

The 2023 International Planning Competition

Open access
Taitler, Ayal, Alford, Ron, Espasa, Joan, Behnke, Gregor, Fišer, Daniel, Gimelfarb, Michael, Pommerening, Florian, Sanner, Scott +4 more·2024·AI Magazine

Abstract

Abstract In this article, we present an overview of the 2023 International Planning Competition. It featured five distinct tracks designed to assess cutting‐edge methods and explore the frontiers of planning within these settings: the classical (deterministic) track, the numeric track, the Hierarchical Task Networks (HTN) track, the learning track, and the probabilistic and reinforcement learning track. Each of these tracks evaluated planning methodologies through one or more subtracks, with the goal of pushing the boundaries of current planner performance. To achieve this objective, the competition introduced a combination of well‐established challenges and entirely novel ones. Within this article, each track offers an exploration of its historical context, justifies its relevance within the planning landscape, discusses emerging domains and trends, elucidates the evaluation methodology, and ultimately presents the results.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1002/aaai.12169

Cites

53 of 53 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Planetarium contains a dataset of 145,918 text-to-PDDL pairs derived from the International Planning Competition, across 73 unique state combinations.

    “along with a dataset of 145,918 text-to-PDDL pairs across 73 unique state combinations with varying levels of difficulty.”

    From Planetarium: A Rigorous Benchmark for Translating Text to Structured Planning Languages (2025) · cites this paper as Taitler et al., 2024

    The Classical Track history and motivation section discusses IPC planning domains and PDDL, but not Planetarium or a text-to-PDDL dataset. Searches for Planetarium, text-to-PDDL pairs, 145,918, and 73 state combinations found no passage establishing the claimed dataset size, composition, or derivation from the competition. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

1 of the 54 references couldn’t be matched to a known paper.

Research works printed but not linked: 11

  1. Behnke, G., D. Höller, and P. Bercher, eds. 2021. Proceedings of the 10th International Planning Competition: Planner and Domain Abstracts -Hierarchical Task Network (HTN) Planning Track (IPC 2020). https://ipc2020.hierarchical- task.net/publications/IPC2020Booklet.pdf.
  2. Brockman, G., V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba. 2016. "Openai Gym." arXiv preprint arXiv:1606.01540. · arXiv
  3. Fernandez-Olivares, J., I. Vellido, and L. Castillo. 2021. "Address- ing HTN Planning with Blind Depth First Search." In Proceedings of 10th International Planning Competition: Planner and Domain Abstracts -Hierarchical Task Network (HTN) Planning Track (IPC 2020), 1-4.
  4. Gebser, M., M. Maratea, and F. Ricca. 2016. "What's Hot in the Answer Set Programming Competition." In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, February 12- 17, 2016, Phoenix, Arizona, USA, eds. D. Schuurmans and M. P. Wellman, 4327-29. AAAI Press.
  5. C. A. C. Coello, 507-23. Springer.
  6. PMLR.
  7. Olz, C., E. Wierzba, P. Bercher, and F. Lindner. 2021. "Towards Improving the Comprehension of HTN Planning Domains by Means of Preconditions and Effects of Compound Tasks." In Proceedings of the 10th Workshop on Knowledge Engineering for Planning and Scheduling, KEPS.
  8. Seipp, J. 2019. "Planner Metrics Should Satisfy Independence of Irrel- evant Alternatives." In ICAPS 2019 Workshop on the International Planning Competition (WIPC), 40-41.
  9. Sharma, V., D. Arora, Mausam, and P. Singla. 2023. "SymNet 3.0: Exploiting Long-Range Influences in Learning Generalized Neural Policies for Relational MDPs." In Proceedings of the Thirty- Ninth Conference on Uncertainty in Artificial Intelligence, eds. R. J. Evans and I. Shpitser, volume 216 of Proceedings of Machine Learning Research, 1921-31. PMLR.
  10. Taitler, A., M. Gimelfarb, S. Gopalakrishnan, M. Mladenov, X. Liu, and S. Sanner. 2023. "pyRDDLGym: From RDDL to Gym Envi- ronments." In ICAPS 2023 Workshop on Bridging the Gap Between AI Planning and Reinforcement Learning (PRL).
  11. Vellido, I., J. Fdez-Olivares, and R. Pérez. 2020. "A Knowledge Based Process for the Generation of HTN Domains from VGDL Video Game Descriptions." In ICAPS2020 Workshop on Knowledge Engineering for Planning and Scheduling, 11-20.

Cited by

Sorted by citations

Authors
Taitler, A., Alford, R., Espasa, J., Behnke, G., Fišer, D., Gimelfarb, M., Pommerening, F., Sanner, S., Scala, E., Schreiber, D., Segovia‐Aguas, J., & Seipp, J.
Affiliations
  • University of Toronto (CA)
  • Mitre (United States) (US)
  • University of St Andrews (GB)
  • University of Amsterdam (NL)
  • Saarland University (DE)
  • University of Basel (CH)
  • University of Brescia (IT)
  • Karlsruhe Institute of Technology (DE)
  • Universitat Pompeu Fabra (ES)
  • Linköping University (SE)
Corresponding author
Taitler, Ayal
Venue
AI Magazine
Publisher
Wiley
Published
2024-04-05
Volume
45
Issue
2
Pages
280-296
ISSN
  • 0738-4602 (ISSN-L)
  • 2371-9621
OpenAlex
W4393988223
Cited by
24
References
54
Citations by year
2026: 3 · 2025: 4 · 2024: 3 · 2023: 2 · 2021: 3 · 2020: 1 · 2017: 1 · 2015: 3
FWCI
1.48
Type
journal article
Language
en
Open access
Locations
  • AI Magazine (publishedVersion, cc-by)
  • Repositori digital de la UPF (Universitat Pompeu Fabra) (publishedVersion, cc-by)
  • Repository KITopen (Karlsruhe Institute of Technology) (submittedVersion, cc-by)
  • KTH Publication Database DiVA (KTH Royal Institute of Technology) (submittedVersion)
  • UvA-DARE (University of Amsterdam) (submittedVersion, cc-by)
  • e-Archivo (Carlos III University of Madrid) (submittedVersion, cc-by-nc-nd)
  • Institutional Research Information System (Università degli Studi di Brescia) (submittedVersion)
  • St Andrews Research Repository (St Andrews Research Repository) (submittedVersion, other-oa)
  • KITopen (cc-by)
Funders
  • Vetenskapsrådet
  • University Of Edinburgh
  • Engineering and Physical Sciences Research Council (EP/P020267/1)
  • Natural Sciences and Engineering Research Council of Canada
  • European Research Council (882500)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
  • Physical Sciences › Computer Science › Computer Networks and Communications › Constraint Satisfaction and Optimization
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
Keywords
International Planning Competition, automated planning, planning competitions, hierarchical task networks, probabilistic planning, learning-based planning, deterministic planning, numeric planning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review