CoChat
Sign Up
Cited by 112AI-based Problem Solving and PlanningConstraint Satisfaction and OptimizationLogic, Reasoning, and Knowledge

The 2014 International Planning Competition: Progress and Trends

Open access
Vallati, Mauro, Chrpa, Lukáš, Grześ, Marek, McCluskey, Thomas L., Roberts, Mark, Sanner, Scott·2015·AI Magazine
DOI 10.1609/aimag.v36i3.2571
Open this paper in CoChat — free accountOpen PDF

Abstract

This article reviews the 2014 International Planning Competition (IPC‐2014), the eighth in a series of competitions starting in 1998. IPC‐2014 was held in three separate parts to assess the state of the art in three prominent areas of planning research: the deterministic (classical) part (IPCD), the learning part (IPCL), and the probabilistic part (IPPC). Each part evaluated planning systems in ways that pushed the edge of existing planner performance by introducing new challenges, novel tasks, or both. The competition surpassed again the number of competitors that participated in its predecessor, highlighting the competition's central role in shaping the landscape of ongoing developments in evaluating planning systems.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1609/aimag.v36i3.2571

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Planetarium contains a dataset of 145,918 text-to-PDDL pairs derived from the International Planning Competition, across 73 unique state combinations.

    “along with a dataset of 145,918 text-to-PDDL pairs across 73 unique state combinations with varying levels of difficulty.”

    From Planetarium: A Rigorous Benchmark for Translating Text to Structured Planning Languages (2025) · cites this paper as Vallati et al., 2015

    The 'Domain Models Introduced' section discusses IPC planning domains and PDDL features, but not a Planetarium dataset. Searches for Planetarium, 145,918 text-to-PDDL pairs, and 73 state combinations found no passage addressing the dataset or those figures. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 41 · Cited by 112

Research works printed but not linked: 31

  1. Alcázar, V.; Veloso, M. M.; and Borrajo, D. 2011. Adapting a Rapidly-Exploring Random Tree for Automated Planning. In Proceedings, the Fourth International Symposium on Combina- torial Search, ed. D. Borrajo, M. Likhachev, and C. Linares Lopez. Palo Alto, CA: AAAI Press.
  2. Bonet, B., and Geffner, H. 2003. Labeled RTDP: Improving the Convergence of Real-Time Dynamic Programming. In Proceedings of the 13th International Conference on Automated Planning and Scheduling (ICAPS-03), 12-21. Menlo Park, CA: AAAI Press.
  3. Coles, A. J.; Coles, A.; Olaya, A. G.; Celorrio, S. J.; López, C. L.; Sanner, S.; and Yoon, S. 2012. A Survey of the Seventh International Planning Competition. AI Magazine 33(1): 83-88.
  4. Dietterich, T. G.; Taleghan, M. A.; and Crowley, M. 2013. Pac Optimal Planning for Invasive Species Management: Improved Exploration for Reinforcement Learning from Simulator-Defined MDPS. In Proceedings of the 27th Nation- al Conference on Artificial intelligence (AAAI-13). Palo Alto, CA: AAAI Press.
  5. Erol, K.; Hendler, J. A.; and Nau, D. S. 1994. HTN planning: Complexity and expressivity. In Proceedings of the 12th National Conference on Artificial Intelligence (AAAI), 1123- 1128. Palo Alto, CA: AAAI Press.
  6. Gerevini, A.; Saetti, A.; and Serina, I. 2003. Planning through Stochastic Local Search and Temporal Action Graphs. Journal of Artificial Intelligence Research (JAIR) 20: 239-290.
  7. Gerevini, A.; Saetti, A.; and Serina, I. 2010. An Empirical Analysis of Some Heuristic Features for Planning Through Local Search and Action Graphs. Fundamenta Informaticae 105(2010): 1-31 doi: 10.3233/FI-1010-350. · DOI
  8. Helmert, M. 2006. The Fast Downward Planning System. Journal of Artificial Intelligence Research (JAIR) 26: 191-246.
  9. Hoffmann, J., and Nebel, B. 2001. The FF Planning System: Fast Plan Generation Through Heuristic Search. Journal of Artificial Intelligence Research (JAIR) 14: 253-302.
  10. Howe, A., and Dahlman, E. 1993. A Critical Assessment of Benchmark Comparison In Planning. Journal of Artificial Intelligence Research (JAIR) 1:1-15.
  11. Howey, R.; Long, D.; and Fox, M. 2004. Automatic Plan Val- idation, Continuous Effects and Mixed Initiative Planning Using PDDL. In Proceedings of the Sixteenth IEEE Internation- al Conference on Tools with Artificial Intelligence, 294-301.
  12. Karafyllidis, I., and Thanailakis, A. 1997. A Model for Pre- dicting Forest Fire Spreading Using Gridular Automata. Eco- logical Modelling 99(1): 87-97.
  13. Katz, M.; Hoffmann, J.; and Domshlak, C. 2013. Red-Black Relaxed Plan Heuristics. In Proceedings of the 27th AAAI Con- ference of the American Association for Artificial Intelligence (AAAI13), 489-495. Palo Alto, CA: AAAI Press.
  14. Keller, T., and Helmert, M. 2013. Trial-Based Heuristic Tree Search for Finite Horizon MDPs. In Proceedings of the 23rd International Conference on Automated Planning and Schedul- ing (ICAPS-13), 135-143. Palo Alto, CA: AAAI Press.
  15. Kissmann, P., and Edelkamp, S. 2011. Improving Cost-Opti- mal Domain-Independent Symbolic Planning. In Proceed- ings of the Twenty-Fifth AAAI Conference on Artificial Intelli- gence (AAAI 2011). Palo Alto, CA: AAAI Press.
  16. Koller, A., and Hoffmann, J. 2010. Waking Up a Sleeping Rabbit: On Natural-Language Sentence Generation with FF. In Proceedings of the 20th International Conference on Auto- Competition Report FALL 2015 97
  17. mated Planning and Scheduling (ICAPS10), 238-241. Palo Alto, CA: AAAI Press.
  18. Kolobov, A.; Mausam; and Weld, D. 2012. LRTDp Versus UCT for Online Probabilistic Planning. In Proceedings of the 26th National Conference on Artificial intelligence (AAAI-12).
  19. Muneepeerakul, R.; Weitz, J. S.; Levin, S.; Rinaldo, A.; and Rodriguez-Iturbe, I. 2007. A Neutral Metapopulation Model of Biodiversity in River Networks. Journal of Theoretical Biol- ogy 245(2): 35-63.
  20. Parkinson, S.; Longstaff, A. P.; Crampton, A.; and Gregory, P. 2014. Automated Planning for Multiobjective Machine Tool Calibration: Optimising Makespan and Measurement Uncertainty. In Proceedings of the 24th International Confer- ence on Automated Planning and Scheduling (ICAPS14). Palo Alto, CA: AAAI Press
  21. Pollack, M. E., and Horty, J. F. 1999. There's More to Life Than Making Plans: Plan Management in Dynamic, Multi- agent Environments. AI Magazine 20(4): 71.
  22. Richter, S., and Westphal, M. 2010. The LAMA Planner: Guiding Cost-Based Anytime Planning with Landmarks. Journal of Artificial Intelligence Research (JAIR) 39: 127-177.
  23. Shah, M. M. S.; Chrpa, L.; Kitchin, D. E.; McCluskey, T. L.; and Vallati, M. 2013. Exploring Knowledge Engineering Strategies in Designing and Modeling a Road Traffic Acci- dent Management Domain. In Proceedings of the 23rd Inter- national Joint Conference on Artificial Intelligence. Palo Alto, CA: AAAI Press.
  24. Shivashankar, V.; Alford, R.; Kuter, U.; and Nau, D. 2013. The GoDeL Planning System: A More Perfect Union of Domain-Independent and Hierarchical Planning. In Pro- ceedings of the Twenty-Third international Joint Conference on Artificial Intelligence, 2380-2386. Palo Alto, CA: AAAI Press.
  25. Silver, D., and Veness, J. 2010. Monte-Carlo Planning in Large POMDPs. In Proceedings of 24th Conference on Neural Information Processing Systems (NIPS-10), 2164-2172.
  26. Sim, H. S.; Kim, K.-E.; Kim, J. H.; Chang, D.-S.; and Koo, M.- W. 2008. Symbolic Heuristic Search Value Iteration for Fac- tored POMDPs. In Proceedings of the 23rd National Conference on Artificial Intelligence (AAAI-08), 1088-1093.
  27. Smith, D. E.; Frank, J.; and Cushing, W. 2008. The ANML language. Paper presented at the ICAPS 2008 Workshop on Knowledge Engineering for Planning and Scheduling (KEPS). 14 September, Sydney, Australia.
  28. Somani, A.; Ye, N.; Hsu, D.; and Lee, W. S. 2013. Despot: Online POMDP Planning with Regularization. Paper pre- sented at 27th Conference on Neural Information Process- ing Systems (NIPS-13), 5-8 December, Lake Tahoe, Nevada.
  29. Teichteil-Königsbuch, F.; Kuter, U.; and Infantes, G. 2010. Incremental Plan Aggregation for Generating Policies in MDPs. In Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems (AAMAS-10): Vol- ume 1, 1231-1238.
  30. Yoon, S.; Fern, A.; and Givan, R. 2007. FF-Replan: A Baseline for Probabilistic Planning. In Proceedings of the 17th Interna- tional Conference on Automated Planning and Scheduling (ICAPS-07), 352-359. Palo Alto, CA: AAAI Press.
  31. Younes, H. L. S.; Littman, M. L.; Weissman, D.; and Asmuth, J. 2005. The First Probabilistic Track of the International Planning Competition. Journal of Artificial Intelligence Research (JAIR) 24: 851-887.

Authors
Vallati, M., Chrpa, L., Grześ, M., McCluskey, T. L., Roberts, M., & Sanner, S.
Affiliations
  • University of Huddersfield (GB)
  • University of Kent (GB)
  • United States Naval Research Laboratory (US)
  • Data61 (AU)
ORCID
Venue
AI Magazine
Publisher
Wiley
Published
2015-09
Volume
36
Issue
3
Pages
90-98
ISSN
  • 0738-4602 (ISSN-L)
  • 2371-9621
DOI
10.1609/aimag.v36i3.2571
OpenAlex
W2147277731
Cited by
112
References
41
Citations by year
2026: 2 · 2025: 7 · 2024: 9 · 2023: 6 · 2022: 5 · 2021: 12 · 2020: 12 · 2019: 12 · 2018: 20 · 2017: 19 · 2016: 6 · 2015: 2
FWCI
15.36
Type
journal article
Language
en
Open access
Licence
onlinelibrary.wiley.com/termsAndConditions#vor (vor)
Locations
  • AI Magazine (publishedVersion)
  • University of Huddersfield Repository (University of Huddersfield) (acceptedVersion)
  • Kent Academic Repository (University of Kent) (submittedVersion)
  • Huddersfield Research Portal (University of Huddersfield) (submittedVersion)
Funders
Amazon Web Services
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
  • Physical Sciences › Computer Science › Computer Networks and Communications › Constraint Satisfaction and Optimization
  • Physical Sciences › Computer Science › Artificial Intelligence › Logic, Reasoning, and Knowledge
Keywords
International Planning Competition, planning systems, deterministic planning, learning-based planning, probabilistic planning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review