The DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Actor-Critic Deep Reinforcement Learning for Solving Job Shop Scheduling Problems
Open accessAbstract
In the past decades, many optimization methods have been devised and applied to job shop scheduling problem (JSSP) to find the optimal solution. Many methods assumed that the scheduling results were applied to static environments, but the whole environments in the real world are always dynamic. Moreover, many unexpected events such as machine breakdowns and material problems may be present to adversely affect the initial job scheduling. This work views JSSP as a sequential decision making problem and proposes to use deep reinforcement learning to cope with this problem. The combination of deep learning and reinforcement learning avoids handcraft features as used in traditional reinforcement learning, and it is expected that the combination will make the whole learning phase more efficient. Our proposed model comprises actor network and critic network, both including convolution layers and fully connected layer. Actor network agent learns how to behave in different situations, while critic network helps agent evaluate the value of statement then return to actor network. This work proposes a parallel training method, combining asynchronous update as well as deep deterministic policy gradient (DDPG), to train the model. The whole network is trained with parallel training on a multi-agent environment and different simple dispatching rules are considered as actions. We evaluate our proposed model on more than ten instances that are present in a famous benchmark problem library - OR library. The evaluation results indicate that our method is comparative in static JSSP benchmark problems, and achieves a good balance between makespan and execution time in dynamic environments. Scheduling score of our method is 91.12% in static JSSP benchmark problems, and 80.78% in dynamic environments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reinforcement Learning: An Introduction
(2005) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2004.842673 · Cited by 24,496
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Mastering the game of Go without human knowledge
Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., et al. (2017) · Nature · doi:10.1038/nature24270 · Cited by 9,084
Watkins, C. J. C. H., & Dayan, P. (1992) · Machine Learning · doi:10.1007/bf00992698 · Cited by 9,083
Markov decision processes: Discrete stochastic dynamic programming
Hazeghi, K., & Puterman, M. L. (1995) · Journal of the American Statistical Association · doi:10.2307/2291177 · Cited by 8,037
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132
Scheduling: Theory, Algorithms, and Systems
(1996) · IIE Transactions · doi:10.1080/15458830.1996.11770714 · Cited by 6,210
Grandmaster level in StarCraft II using multi-agent reinforcement learning
Vinyals, O., Babuschkin, I., Czarnecki, W. M., Mathieu, M., Dudzik, A., Chung, J., Choi, D. H., Powell, R., Ewalds, T., Georgiev, P., et al. (2019) · Nature · doi:10.1038/s41586-019-1724-z · Cited by 3,455
Markov Decision Processes: Discrete Stochastic Dynamic Programming
Baxter, L. A. (1995) · Technometrics · doi:10.1080/00401706.1995.10484354 · Cited by 2,415
Konda, V. R., & Tsitsiklis, J. N. (2002) · Cited by 2,202
OR-Library: Distributing Test Problems by Electronic Mail
Beasley, J. E. (1990) · Journal of the Operational Research Society · doi:10.1057/jors.1990.166 · Cited by 1,899
Genetic Algorithms and Random Keys for Sequencing and Optimization
Bean, J. C. (1994) · INFORMS journal on computing · doi:10.1287/ijoc.6.2.154 · Cited by 1,543
Evolution Strategies as a Scalable Alternative to Reinforcement Learning
Salimans, T., Ho, J., Chen, X., Sidor, S., & Sutskever, I. (2017) · arXiv · doi:10.48550/arxiv.1703.03864 · Cited by 1,027
A Computational Study of the Job-Shop Scheduling Problem
Applegate, D., & Cook, W. (1991) · ORSA Journal on Computing · doi:10.1287/ijoc.3.2.149 · Cited by 908
Probabilistic Learning Combinations of Local Job-Shop Scheduling Rules
Fisher, H. F. (1963) · Medical Entomology and Zoology · Cited by 729
On the Job-Shop Scheduling Problem
Manne, A. S. (1960) · Operations Research · doi:10.1287/opre.8.2.219 · Cited by 666
Review of job shop scheduling research and its new perspectives under Industry 4.0
Zhang, J., Ding, G., Zou, Y., Qin, S., & Fu, J. (2019) · Journal of Intelligent Manufacturing · doi:10.1007/s10845-017-1350-2 · Cited by 558
Cui, R., Yang, C., Li, Y., & Sharma, S. (2017) · IEEE Transactions on Systems, Man, and Cybernetics: Systems · doi:10.1109/tsmc.2016.2645699 · Cited by 538
Sequential Optimality and Coordination in Multiagent Systems
Boutilier, C. E. (1999) · Cited by 294
Ant colony optimization combined with taboo search for the job shop scheduling problem
Huang, K.-L., & Liao, C.-J. (2008) · Computers & Operations Research · doi:10.1016/j.cor.2006.07.003 · Cited by 278
Dynamic job-shop scheduling using reinforcement learning agents
Aydin, M. E., & Öztemel, E. (2000) · Robotics and Autonomous Systems · doi:10.1016/s0921-8890(00)00087-7 · Cited by 274
OR-Library: Distributing Test Problems by Electronic Mail
Beasley, J. E. (1990) · Journal of the Operational Research Society · doi:10.1038/sj/jors/0411109 · Cited by 256
Fu, Y., Ding, J., Wang, H., & Wang, J. (2017) · Applied Soft Computing · doi:10.1016/j.asoc.2017.12.009 · Cited by 168
UAV Autonomous Target Search Based on Deep Reinforcement Learning in Complex Disaster Scene
Wu, C., Ju, B., Wu, Y., Lin, X., Xiong, N. N., Xu, G., Li, H., & Liang, X. (2019) · IEEE Access · doi:10.1109/access.2019.2933002 · Cited by 168
Deep reinforcement learning for semiconductor production scheduling
Waschneck, B., Reichstaller, A., Belzner, L., Altenmüller, T., Bauernhansl, T., Knapp, A., & Kyek, A. (2018) · ASMC proceedings · doi:10.1109/asmc.2018.8373191 · Cited by 150
Deep Reinforcement Learning as a Job Shop Scheduling Solver: A Literature Review
Cunha, B., Madureira, A. M., Fonseca, B., & Coelho, D. (2019) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-14347-3_34 · Cited by 94
Frazzon, E. M., Albrecht, A., Pires, M. C., Israel, E., Kück, M., & Freitag, M. (2017) · International Journal of Production Research · doi:10.1080/00207543.2017.1355118 · Cited by 86
Ye, Y., Qiu, D., Li, J., & Štrbac, G. (2019) · IEEE Access · doi:10.1109/access.2019.2940005 · Cited by 83
Pneumatic artificial muscle-driven robot control using local update reinforcement learning
Cui, Y., Matsubara, T., & Sugimoto, K. (2017) · Advanced Robotics · doi:10.1080/01691864.2016.1274680 · Cited by 37
Asuhaimi, F. A., Bu, S., Klaine, P. V., & Imran, M. A. (2019) · IEEE Access · doi:10.1109/access.2019.2939827 · Cited by 28
Zeng, D. D., & Sycara, K. P. (2002) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/tai.1995.479378 · Cited by 10
Cited by
Flexible Job-Shop Scheduling via Graph Neural Network and Deep Reinforcement Learning
Song, W., Chen, X., Li, Q., & Cao, Z. (2023) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3189725 · Cited by 519
Deep reinforcement learning for dynamic scheduling of a flexible job shop
Liu, R., Piplani, R., & Toro, C. (2022) · International Journal of Production Research · doi:10.1080/00207543.2022.2058432 · Cited by 310
Deep reinforcement learning in production systems: a systematic literature review
Panzer, M., & Bender, B. (2022) · International Journal of Production Research · doi:10.1080/00207543.2021.1973138 · Cited by 282
Li, Y., Gu, W., Yuan, M., & Tang, Y. (2022) · Robotics and Computer-Integrated Manufacturing · doi:10.1016/j.rcim.2021.102283 · Cited by 261
Dynamic multi-objective scheduling for flexible job shop by deep reinforcement learning
Luo, S., Zhang, L., & Fan, Y. (2021) · Computers & Industrial Engineering · doi:10.1016/j.cie.2021.107489 · Cited by 255
Metadata
- Authors
- Liu, C.-L., Chang, C.-C., & Tseng, C.-J.
- Venue
- IEEE Access
- Published
- 2020
- Volume
- 8
- Pages
- 71752-71762
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W3016938688
- Cited by
- 281
- References
- 35
- FWCI
- 20.78
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.