Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 218 of the 223 references it cites were found in a registry or an index.

A practical guide to multi-objective reinforcement learning and planning

Open access

Abstract

Real-world sequential decision-making tasks are generally complex, requiring trade-offs between multiple, often conflicting, objectives. Despite this, the majority of research in reinforcement learning and decision-theoretic planning either assumes only a single objective, or that multiple objectives can be adequately handled via a simple linear combination. Such approaches may oversimplify the underlying problem and hence produce suboptimal results. This paper serves as a guide to the application of multi-objective methods to difficult problems, and is aimed at researchers who are already familiar with single-objective reinforcement learning and planning methods who wish to adopt a multi-objective perspective on their research, as well as practitioners who encounter multi-objective decision problems in practice. It identifies the factors that may influence the nature of the desired solution, and illustrates by example how these influence the design of multi-objective decision-making systems for complex problems.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/s10458-022-09552-y

Cites

218 of 218 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 Visual representation of the solution sets in multiobjective planning and learning. Note that a minimum requirement is that the utility function is monotonically increasing, hence the Pareto Front is the largest set of undominated values
Fig. 1 Visual representation of the solution sets in multiobjective planning and learning. Note that a minimum requirement is that the utility function is monotonically increasing, hence the Pareto Front is the largest set of undominated valuesCC-BY · From the PDF
Fig. 2 Work flow diagram for multi-objective reinforcement learning and planning
Fig. 2 Work flow diagram for multi-objective reinforcement learning and planningCC-BY · From the PDF
Fig. 3 The six motivating scenarios for MOMDPs: a the unknown utility function scenario, b the decision support scenario, c the known utility function scenario, d the interactive decision support scenario, e the dynamic utility function scenario, and f the review and adjust scenario
Fig. 3 The six motivating scenarios for MOMDPs: a the unknown utility function scenario, b the decision support scenario, c the known utility function scenario, d the interactive decision support scenario, e the dynamic utility function scenario, and f the review and adjust scenarioCC-BY · From the PDF
Fig. 4 Multi-objective multi-agent decision making taxonomy and mapping of solution concepts [126]
Fig. 4 Multi-objective multi-agent decision making taxonomy and mapping of solution concepts [126]CC-BY · From the PDF
Image 5
CC-BY · From the PDF
Fig. 6 Comparison of returns (left) with non-dominated returns (right). In order to enhance presentation, the right plot's horizontal axis was clipped to a smaller interval
Fig. 6 Comparison of returns (left) with non-dominated returns (right). In order to enhance presentation, the right plot's horizontal axis was clipped to a smaller intervalCC-BY · From the PDF
Fig. 8 Evaluation of MONES using 2 multi-objective metrics: the Hypervolume (left) and Expected Utility Metric (right)
Fig. 8 Evaluation of MONES using 2 multi-objective metrics: the Hypervolume (left) and Expected Utility Metric (right)CC-BY · From the PDF

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 223 references

Research works printed but not linked: 10

  1. Aissani, N., Beldjilali, B., & Trentesaux, D. (2008). Efficient and effective reactive scheduling of manufacturing system using sarsa-multi-objective agents. In: MOSIM’08: 7th Conference Internationale de Modelisation et Simulation, pp. 698–707.
  2. Dazeley, R., Vamplew, P., & Cruz, F. (2021). Explainable reinforcement learning for broad-xai: A conceptual framework and survey. arXiv preprint arXiv:2108.09003. · arXiv
  3. Juozapaitis, Z., Koul, A., Fern, A., Erwig, M., & Doshi-Velez, F. (2019). Explainable reinforcement learning via reward decomposition. In: IJCAI/ECAI Workshop on Explainable Artificial Intelligence.
  4. Mannion, P., Heintz, F., Karimpanal, T.G., & Vamplew, P. (2021). Multi-objective decision making for trustworthy ai. In: Proceedings of the Multi-Objective Decision Making (MODeM) Workshop.
  5. Rollon, E., & Larrosa, J. (2008). Constraint optimization techniques for multiobjective branch and bound search. In: International conference on logic programming, ICLP.
  6. Rowe, J., Smith, A., Pokorny, B., Mott, B., & Lester, J. (2018). Toward automated scenario generation with deep reinforcement learning in gift. In: Proceedings of the Sixth Annual GIFT User Symposium, pp. 65–74.
  7. Smith, B. J., Klassert, R., & Pihlakas, R. (2021). Soft maximin approaches to multi-objective decision-making for encoding human intuitive values. In: Multi-Objective Decision Making Workshop.
  8. Van Dijk, M.T., van Wingerden, J.W., Ashuri, T., Li, Y., & Rotea, M.A. (2016). Yaw-misalignment and its impact on wind turbine loads and wind farm power output. Journal of Physics: Conference Series, 753(6). · DOI
  9. Van der Maaten, L., & Hinton, G. (2008). Visualizing data using t-SNE. Journal of machine learning research, 9(11).
  10. White, C. C., & Kim, K. W. (1980). Solution procedures for vector criterion Markov decision processes. Large Scale Systems, 1, 129–140.

Cited by

See all 401

Metadata

Authors
Hayes, C. F., Rădulescu, R., Bargiacchi, E., Källström, J., Macfarlane, M., Reymond, M., Verstraeten, T., Zintgraf, L. M., Dazeley, R., Heintz, F., Howley, E., Irissappane, A. A., Mannion, P., Nowé, A., Ramos, G., Restelli, M., Vamplew, P., & Roijers, D. M.
Affiliations
  • Ollscoil na Gaillimhe – University of Galway (IE)
  • Vrije Universiteit Brussel (BE)
  • Linköping University (SE)
  • University of Amsterdam (NL)
  • University of Oxford (GB)
  • Deakin University (AU)
  • University of Washington Tacoma (US)
  • Universidade do Vale do Rio dos Sinos (BR)
  • Politecnico di Milano (IT)
  • Federation University (AU)
Corresponding author
  • Hayes, Conor F.
  • Rădulescu, Roxana
  • Bargiacchi, Eugenio
  • Källström, Johan
  • Macfarlane, Matthew
  • Reymond, Mathieu
  • Verstraeten, Timothy
  • Zintgraf, Luisa M.
  • Dazeley, Richard
  • Heintz, Fredrik
  • Howley, Enda
  • Irissappane, Athirai A.
  • Mannion, Patrick
  • Nowé, Ann
  • Ramos, Gabriel
  • Restelli, Marcello
  • Vamplew, Peter
  • Roijers, Diederik M.
Venue
Autonomous Agents and Multi-Agent Systems
Published
2022-04
Volume
36
Issue
1
ISSN
  • 1387-2532 (ISSN-L)
  • 1573-7454
OpenAlex
W3139398491
Cited by
400
References
223
Citations by year
2026: 119 · 2025: 125 · 2024: 80 · 2023: 52 · 2022: 22
FWCI
22.98
Type
journal article
Language
en
Open access
Licence
Locations
  • Autonomous Agents and Multi-Agent Systems (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • Virtual Community of Pathological Anatomy (University of Castilla La Mancha) (submittedVersion, cc-by)
  • FedUni ResearchOnline (Federation University Australia) (submittedVersion, cc-by)
  • DiVA (Linkoping University) (submittedVersion, other-oa)
  • Figshare (submittedVersion, cc-by)
  • VUBIR (Vrije Universiteit Brussel) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • Vlaamse regering (Onderzoeksprogramma Artificiële Intelligentie (AI) Vlaanderen)
  • National University Ireland, Galway
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Physical Sciences › Computer Science › Computational Theory and Mathematics › Advanced Multi-Objective Optimization Algorithms
  • Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
Keywords
multi-objective reinforcement learning, sequential decision making, conflicting objectives, trade-offs between objectives, solution design, complex decision problems

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: A practical guide to multi-objective reinforcement learning and planning. https://app.cochat.ai/papers/10.1007/s10458-022-09552-y
The paper
Hayes, C. F., Rădulescu, R., Bargiacchi, E., Källström, J., Macfarlane, M., Reymond, M., Verstraeten, T., Zintgraf, L. M., Dazeley, R., Heintz, F., Howley, E., Irissappane, A. A., Mannion, P., Nowé, A., Ramos, G., Restelli, M., Vamplew, P., & Roijers, D. M. (2022). A practical guide to multi-objective reinforcement learning and planning. Autonomous Agents and Multi-Agent Systems. https://doi.org/10.1007/s10458-022-09552-y

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.