The DOI matches its Crossref record; 218 of the 223 references it cites were found in a registry or an index.
A practical guide to multi-objective reinforcement learning and planning
Open accessAbstract
Real-world sequential decision-making tasks are generally complex, requiring trade-offs between multiple, often conflicting, objectives. Despite this, the majority of research in reinforcement learning and decision-theoretic planning either assumes only a single objective, or that multiple objectives can be adequately handled via a simple linear combination. Such approaches may oversimplify the underlying problem and hence produce suboptimal results. This paper serves as a guide to the application of multi-objective methods to difficult problems, and is aimed at researchers who are already familiar with single-objective reinforcement learning and planning methods who wish to adopt a multi-objective perspective on their research, as well as practitioners who encounter multi-objective decision problems in practice. It identifies the factors that may influence the nature of the desired solution, and illustrates by example how these influence the design of multi-objective decision-making systems for complex problems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures



![Fig. 4 Multi-objective multi-agent decision making taxonomy and mapping of solution concepts [126]](https://figures.cochat.ai/paper-figures/7bb607ea1aff4a2e976d4ccdd0c76294/a-practical-guide-to-multi-objective-reinforcement-learning-and-planning-figure-4.png)



Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“nl-MOMDPs are widely recognized in the reinforcement literature as particularly difficult to solve [26,27], and may admit of no computationally tractable exact solutions.”
From Phenomenal interface theory: a model for basal consciousness (2025) · cites this paper as [27]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“The task faced by the bee is thus best modelled as a nonlinear multi-objective Markov Decision Process (nl-MOMDP) [26,27].”
From Phenomenal interface theory: a model for basal consciousness (2025) · cites this paper as [27]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A fast and elitist multiobjective genetic algorithm: NSGA-II
Deb, K., Pratap, A., Agarwal, S., & Meyarivan, T. (2002) · IEEE Transactions on Evolutionary Computation · doi:10.1109/4235.996017 · Cited by 50,053 · cited 1×
“…best vector from a solution set S according to: π∗ = arg max π∈S u(Vπ), (17) so it helps if the user has a larger variety of value vectors to select from.2 In multi-objective optimisation, this idea has been used to create algorithms that explicitly look for diverse solutions [Deb et al., 2002].”
MethodHuman-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · cited 2×
“Deep reinforcement learning methods [Mnih et al., 2015, Lillicrap et al., 2015] have shown to scale beyond finite and discrete spaces to problem domains with high-dimensional, continuous state and action spaces.”
MethodReinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782 · cited 2×
“In the second stage, the learned skills are used in a hierarchical version of DDPG [85] to produce a policy coverage set for the MOMDP.”
BackgroundMethodProximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach
Zitzler, E., & Thiele, L. (1999) · IEEE Transactions on Evolutionary Computation · doi:10.1109/4235.797969 · Cited by 8,798
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
Finn, C., Abbeel, P., & Levine, S. (2017) · arXiv · doi:10.48550/arxiv.1703.03400 · Cited by 8,512
Santos, A. G., da Rocha, G. O., & de Andrade, J. B. (2019) · Scientific Reports · doi:10.1038/s41598-018-37186-2 · Cited by 8,495
Nash, J. (1951) · The Annals of Mathematics · doi:10.2307/1969529 · Cited by 7,002
Nash, J. (1989) · Cournot Oligopoly · doi:10.1017/cbo9780511528231.007 · Cited by 5,595
Research works printed but not linked: 10
- Aissani, N., Beldjilali, B., & Trentesaux, D. (2008). Efficient and effective reactive scheduling of manufacturing system using sarsa-multi-objective agents. In: MOSIM’08: 7th Conference Internationale de Modelisation et Simulation, pp. 698–707.
- Dazeley, R., Vamplew, P., & Cruz, F. (2021). Explainable reinforcement learning for broad-xai: A conceptual framework and survey. arXiv preprint arXiv:2108.09003. · arXiv
- Juozapaitis, Z., Koul, A., Fern, A., Erwig, M., & Doshi-Velez, F. (2019). Explainable reinforcement learning via reward decomposition. In: IJCAI/ECAI Workshop on Explainable Artificial Intelligence.
- Mannion, P., Heintz, F., Karimpanal, T.G., & Vamplew, P. (2021). Multi-objective decision making for trustworthy ai. In: Proceedings of the Multi-Objective Decision Making (MODeM) Workshop.
- Rollon, E., & Larrosa, J. (2008). Constraint optimization techniques for multiobjective branch and bound search. In: International conference on logic programming, ICLP.
- Rowe, J., Smith, A., Pokorny, B., Mott, B., & Lester, J. (2018). Toward automated scenario generation with deep reinforcement learning in gift. In: Proceedings of the Sixth Annual GIFT User Symposium, pp. 65–74.
- Smith, B. J., Klassert, R., & Pihlakas, R. (2021). Soft maximin approaches to multi-objective decision-making for encoding human intuitive values. In: Multi-Objective Decision Making Workshop.
- Van Dijk, M.T., van Wingerden, J.W., Ashuri, T., Li, Y., & Rotea, M.A. (2016). Yaw-misalignment and its impact on wind turbine loads and wind farm power output. Journal of Physics: Conference Series, 753(6). · DOI
- Van der Maaten, L., & Hinton, G. (2008). Visualizing data using t-SNE. Journal of machine learning research, 9(11).
- White, C. C., & Kim, K. W. (1980). Solution procedures for vector criterion Markov decision processes. Large Scale Systems, 1, 129–140.
Cited by
Explainable AI Over the Internet of Things (IoT): Overview, State-of-the-Art and Future Directions
Jagatheesaperumal, S. K., Pham, Q.-V., Ruby, R., Yang, Z., Xu, C., & Zhang, Z. (2022) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2022.3215676 · Cited by 171
“Moreover, in [163], the real-world decision from a multi-objective perspective was studied using RL and planning strategies.”
Giuliani, M., Lamontagne, J. R., Reed, P. M., & Castelletti, A. (2021) · Water Resources Research · doi:10.1029/2021wr029927 · Cited by 167
Secret Collusion among AI Agents: Multi-Agent Deception via Steganography
Motwani, S., Baranchuk, M., Strohmeier, M., Bolina, V., Torr, P., Hammond, L., & Witt, C. D. (2024) · Advances in Neural Information Processing Systems 37 · doi:10.52202/079017-2336 · Cited by 106
del Real, A. J., Andreiana, D. S., Roldán, Á. O., Bustos, A. H., & Acevedo, L. (2022) · Applied Sciences · doi:10.3390/app122312377 · Cited by 128
“An extensive analysis concerning the exploratory techniques and”
BackgroundMethodA Systematic Review of Machine-Learning Solutions in Anaerobic Digestion
Rutland, H., You, J., Liu, H., Bull, L., & Reynolds, D. M. (2023) · Bioengineering · doi:10.3390/bioengineering10121410 · Cited by 79
“Multi-objective reinforcement learning is a method for balancing control of multiple parameters and optimizing these concurrent objectives within an AD system [39].”
Method
Metadata
- Authors
- Hayes, C. F., Rădulescu, R., Bargiacchi, E., Källström, J., Macfarlane, M., Reymond, M., Verstraeten, T., Zintgraf, L. M., Dazeley, R., Heintz, F., Howley, E., Irissappane, A. A., Mannion, P., Nowé, A., Ramos, G., Restelli, M., Vamplew, P., & Roijers, D. M.
- Affiliations
- Ollscoil na Gaillimhe – University of Galway (IE)
- Vrije Universiteit Brussel (BE)
- Linköping University (SE)
- University of Amsterdam (NL)
- University of Oxford (GB)
- Deakin University (AU)
- University of Washington Tacoma (US)
- Universidade do Vale do Rio dos Sinos (BR)
- Politecnico di Milano (IT)
- Federation University (AU)
- Corresponding author
- Hayes, Conor F.
- Rădulescu, Roxana
- Bargiacchi, Eugenio
- Källström, Johan
- Macfarlane, Matthew
- Reymond, Mathieu
- Verstraeten, Timothy
- Zintgraf, Luisa M.
- Dazeley, Richard
- Heintz, Fredrik
- Howley, Enda
- Irissappane, Athirai A.
- Mannion, Patrick
- Nowé, Ann
- Ramos, Gabriel
- Restelli, Marcello
- Vamplew, Peter
- Roijers, Diederik M.
- ORCID
- Venue
- Autonomous Agents and Multi-Agent Systems
- Published
- 2022-04
- Volume
- 36
- Issue
- 1
- ISSN
- 1387-2532 (ISSN-L)
- 1573-7454
- OpenAlex
- W3139398491
- Cited by
- 400
- References
- 223
- Citations by year
- 2026: 119 · 2025: 125 · 2024: 80 · 2023: 52 · 2022: 22
- FWCI
- 22.98
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor, 12 d)
- Locations
- Autonomous Agents and Multi-Agent Systems (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- Virtual Community of Pathological Anatomy (University of Castilla La Mancha) (submittedVersion, cc-by)
- FedUni ResearchOnline (Federation University Australia) (submittedVersion, cc-by)
- DiVA (Linkoping University) (submittedVersion, other-oa)
- Figshare (submittedVersion, cc-by)
- VUBIR (Vrije Universiteit Brussel) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Vlaamse regering (Onderzoeksprogramma Artificiële Intelligentie (AI) Vlaanderen)
- National University Ireland, Galway
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Computer Science › Computational Theory and Mathematics › Advanced Multi-Objective Optimization Algorithms
- Physical Sciences › Computer Science › Artificial Intelligence › AI-based Problem Solving and Planning
- Keywords
- multi-objective reinforcement learning, sequential decision making, conflicting objectives, trade-offs between objectives, solution design, complex decision problems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A practical guide to multi-objective reinforcement learning and planning. https://app.cochat.ai/papers/10.1007/s10458-022-09552-y
- The paper
- Hayes, C. F., Rădulescu, R., Bargiacchi, E., Källström, J., Macfarlane, M., Reymond, M., Verstraeten, T., Zintgraf, L. M., Dazeley, R., Heintz, F., Howley, E., Irissappane, A. A., Mannion, P., Nowé, A., Ramos, G., Restelli, M., Vamplew, P., & Roijers, D. M. (2022). A practical guide to multi-objective reinforcement learning and planning. Autonomous Agents and Multi-Agent Systems. https://doi.org/10.1007/s10458-022-09552-y
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.