Deep Reinforcement Learning for Multiobjective Optimization
Open accessThe DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Abstract
This article proposes an end-to-end framework for solving multiobjective optimization problems (MOPs) using deep reinforcement learning (DRL), that we call DRL-based multiobjective optimization algorithm (DRL-MOA). The idea of decomposition is adopted to decompose the MOP into a set of scalar optimization subproblems. Then, each subproblem is modeled as a neural network. Model parameters of all the subproblems are optimized collaboratively according to a neighborhood-based parameter-transfer strategy and the DRL training algorithm. Pareto-optimal solutions can be directly obtained through the trained neural-network models. Specifically, the multiobjective traveling salesman problem (MOTSP) is solved in this article using the DRL-MOA method by modeling the subproblem as a Pointer Network. Extensive experiments have been conducted to study the DRL-MOA and various benchmark methods are compared with it. It is found that once the trained model is available, it can scale to newly encountered problems with no need for retraining the model. The solutions can be directly obtained by a simple forward calculation of the neural network; thereby, no iteration is required and the MOP can be always solved in a reasonable time. The proposed method provides a new way of solving the MOP by means of DRL. It has shown a set of new characteristics, for example, strong generalization ability and fast solving speed in comparison with the existing methods for multiobjective optimizations. The experimental results show the effectiveness and competitiveness of the proposed method in terms of model performance and running time.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
A fast and elitist multiobjective genetic algorithm: NSGA-II
Deb, K., Pratap, A., Agarwal, S., & Meyarivan, T. (2002) · IEEE Transactions on Evolutionary Computation · doi:10.1109/4235.996017 · Cited by 50,053
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · cited 1×
“In this article, we adopt the RNN model of the gated recurrent unit (GRU) [34] that has similar performance but fewer parameters than the long short-term memory (LSTM) which is employed in the original Pointer Network in [17].”
MethodSequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213
No free lunch theorems for optimization
Wolpert, D. H., & Macready, W. G. (1997) · IEEE Transactions on Evolutionary Computation · doi:10.1109/4235.585893 · Cited by 14,237 · cited 1×
“See https://www.ieee.org/publications/rights/index.html for more information. problems, or even new instances of a similar problem, the algorithm needs to be revised to obtain a good result, which is known as the No Free Lunch theorem [16].”
Background(2010) · Cited by 14,134
MOEA/D: A Multiobjective Evolutionary Algorithm Based on Decomposition
Zhang, Q., & Li, H. (2007) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2007.892759 · Cited by 10,069 · cited 3×
“First, the decomposition strategy [2] is adopted to decompose the MOP into a number of subproblems.”
BackgroundMethodDeb, K., & Jain, H. (2014) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2013.2281535 · Cited by 7,133
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132 · cited 1×
“Then, model parameters of all the subproblems are optimized collaboratively according to the neighborhood-based parameter-transfer strategy and the Actor–Critic [25] training algorithm.”
MethodNonlinear Multiobjective Optimization
Miettinen, K. (1998) · International Series in Operations Research & Management Science · doi:10.1007/978-1-4615-5563-6 · Cited by 4,710
An Effective Heuristic Algorithm for the Traveling-Salesman Problem
Lin, S., & Kernighan, B. W. (1973) · Operations Research · doi:10.1287/opre.21.2.498 · Cited by 3,821
Nonlinear Multiobjective Optimization
Mardle, S., & Miettinen, K. M. (2000) · The Journal of the Operational Research Society · doi:10.2307/254267 · Cited by 3,049
Platemo: a matlab platform for evolutionary multi-objective optimization [educational forum]
Tian, Y., Cheng, R., Zhang, X., & Jin, Y. (2017) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2017.2742868 · Cited by 2,777
TSPLIB—A Traveling Salesman Problem Library
Reinelt, G. (1991) · ORSA Journal on Computing · doi:10.1287/ijoc.3.4.376 · Cited by 2,616
Konda, V. R., & Tsitsiklis, J. N. (2002) · Cited by 2,202
Vinyals, O., Fortunato, M., & Jaitly, N. (2015) · neural information processing systems · Cited by 2,043
A Fast and Elitist Multiobjective Genetic Algorithm: NSGA-II
Deb, K. (2002) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.6487416 · Cited by 1,554
An Evolutionary Many-Objective Optimization Algorithm Based on Dominance and Decomposition
Li, K., Deb, K., Zhang, Q., & Kwong, S. (2015) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2014.2373386 · Cited by 1,208
Self-citationA multi-objective genetic local search algorithm and its application to flowshop scheduling
Ishibuchi, H., & Murata, T. (1998) · IEEE Transactions on Systems, Man and Cybernetics, Part C (Applications and Reviews) · doi:10.1109/5326.704576 · Cited by 1,019
Towards a Reliable and Context-Based System Architecture for Autonomous Vehicles
Kain, T., Mundhenk, P., Müller, J.-S., Tompits, H., Wesche, M., & Decke, H. (2020) · OASIcs : OpenAccess Series in Informatics · doi:10.4230/oasics.asd.2020.1 · Cited by 797
Zhang, X., Tian, Y., Cheng, R., & Jin, Y. (2018) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2016.2600642 · Cited by 660
Genetic local search for multi-objective combinatorial optimization
Jaszkiewicz, A. (2002) · European Journal of Operational Research · doi:10.1016/s0377-2217(01)00104-7 · Cited by 548
Jaszkiewicz, A. (2002) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2002.802873 · Cited by 474
Localized Weighted Sum Method for Many-Objective Optimization
Wang, R., Zhou, Z., Ishibuchi, H., Liao, T., & Zhang, T. (2018) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2016.2611642 · Cited by 362
Self-citationMOEA/D-ACO: A Multiobjective Evolutionary Algorithm Using Decomposition and AntColony
Ke, L., Zhang, Q., & Battiti, R. (2013) · IEEE Transactions on Cybernetics · doi:10.1109/tsmcb.2012.2231860 · Cited by 306
Learning Heuristics for the TSP by Policy Gradient
Deudon, M., Cournut, P., Lacoste, A., Adulyasak, Y., & Rousseau, L.-M. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-93031-2_12 · Cited by 303
Decomposition-Based Algorithms Using Pareto Adaptive Scalarizing Methods
Wang, R., Zhang, Q., & Zhang, T. (2016) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2016.2521175 · Cited by 283
Self-citationNeural Combinatorial Optimization with Reinforcement Learning
Bello, I., Pham, H., Le, Q. V., Norouzi, M., & Bengio, S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.09940 · Cited by 257
Multi-Objective Deep Reinforcement LearningPreprint
Mossalam, H., Assael, Y., Roijers, D. M., & Whiteson, S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.02707 · Cited by 252
PlatEMO: A MATLAB Platform for Evolutionary Multi-Objective Optimization
Tian, Y., Cheng, R., Zhang, X., & Jin, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.00879 · Cited by 235
Cai, X., Li, Y., Fan, Z., & Zhang, Q. (2015) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2014.2350995 · Cited by 200
Specification of Genetic Search Directions in Cellular Multi-objective Genetic Algorithms
Murata, T., Ishibuchi, H., & Gen, M. (2001) · Lecture Notes in Computer Science · doi:10.1007/3-540-44719-9_6 · Cited by 193
MONAS: Multi-Objective Neural Architecture Search using Reinforcement Learning
Hsu, C.-H., Chang, S.-H., Liang, J.-H., Chou, H.-P., Liu, C.-H., Chang, S.-C., Pan, T., Chen, Y., Wei, W., & Juan, D.-C. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.10332 · Cited by 137
Hybridization of Decomposition and Local Search for Multiobjective Optimization
Ke, L., Zhang, Q., & Battiti, R. (2014) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2013.2295886 · Cited by 113
A Grid Weighted Sum Pareto Local Search for Combinatorial Multi and Many-Objective Optimization
Cai, X., Sun, H., Zhang, Q., & Huang, Y. (2018) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2018.2849403 · Cited by 95
Comparison between MOEA/D and NSGA-II on the Multi-Objective Travelling Salesman Problem
Peng, W., Zhang, Q., & Li, H. (2008) · Studies in computational intelligence · doi:10.1007/978-3-540-88051-6_14 · Cited by 87
Chen, J., Li, J., & Xin, B. (2017) · IEEE Transactions on Evolutionary Computation · doi:10.1109/tevc.2017.2671462 · Cited by 82
Cited by
Challenges of real-world reinforcement learning: definitions, benchmarks and analysis
Dulac-Arnold, G., Levine, N., Mankowitz, D. J., Li, J., Paduraru, C., Gowal, S., & Hester, T. (2021) · Machine Learning · doi:10.1007/s10994-021-05961-4 · Cited by 645
A Review of Deep Reinforcement Learning for Smart Building Energy Management
Yu, L., Qin, S., Zhang, M., Shen, C., Jiang, T., & Guan, X. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3078462 · Cited by 476
Evolutionary Large-Scale Multi-Objective Optimization: A Survey
Tian, Y., Si, L., Zhang, X., Cheng, R., He, C., Tan, K. C., & Jin, Y. (2022) · ACM Computing Surveys · doi:10.1145/3470971 · Cited by 401
A practical guide to multi-objective reinforcement learning and planning
Hayes, C. F., Rădulescu, R., Bargiacchi, E., Källström, J., Macfarlane, M., Reymond, M., Verstraeten, T., Zintgraf, L. M., Dazeley, R., Heintz, F., et al. (2022) · Autonomous Agents and Multi-Agent Systems · doi:10.1007/s10458-022-09552-y · Cited by 400
“Using deep networks as non-linear function approximators for handling multiobjective optimisation problems has been on the rise the past few years [4, 82, 103, 109, 164].”
BackgroundArtificial Neural Network (ANN) Based Fast and Accurate Inductor Modeling and Design
Guillod, T., Papamanolis, P., & W. Kolar, J. (2020) · IEEE Open Journal of Power Electronics · doi:10.1109/ojpel.2020.3012777 · Cited by 264
Metadata
- Authors
- Li, K., Zhang, T., & Wang, R.
- Venue
- IEEE Transactions on Cybernetics
- Published
- 2021-06
- Volume
- 51
- Issue
- 6
- Pages
- 3103-3114
- ISSN
- 2168-2267 (ISSN-L)
- 2168-2275
- OpenAlex
- W2948175577
- Cited by
- 424
- References
- 38
- FWCI
- 17.27
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.