The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Proximal Policy Optimization in StarCraft
Open accessAbstract
Deep reinforcement learning is an area of research that has blossomed tremendously in recent years and has shown remarkable potential in computer games. Real-time strategy game has become an important field of artificial intelligence in game for several years. This paper is about to introduce a kind of algorithm that used to train agents to fight against computer bots. Not only because games are excellent tools to test deep reinforcement learning algorithms for their valuable insight into how well an algorithm can perform in isolated environments without the real-life consequences, but also real-time strategy games are a very complex genre that challenges artificial intelligence agents in both short-term or long-term planning. In this paper, we introduce some history of deep learning and reinforcement learning. Then we combine them with StarCraft. PPO is the algorithm which have some of the benefits of trust region policy optimization (TRPO), but it is much simpler to implement, more general for environment, and have better sample complexity. The StarCraft environment: Blood War Application Programming Interface (BWAPI) is open source to test. The results show that PPO can work well in BWAPI and train units to defeat the opponents. The algorithm presented in the thesis is corroborated by experiments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Reinforcement Learning: An Introduction
Sutton, R. S., & Barto, A. G. (1998) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.1998.712192 · Cited by 27,225
Reducing the dimensionality of data with neural networks
Hinton, G. E., & Salakhutdinov, R. R. (2006) · Science · doi:10.1126/science.1127647 · Cited by 20,928
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336
Mastering the game of Go with deep neural networks and tree search
Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., et al. (2016) · Nature · doi:10.1038/nature16961 · Cited by 15,612
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
Reinforcement Learning: A Survey
Kaelbling, L. P., Littman, M. L., & Moore, A. W. (1996) · Journal of Artificial Intelligence Research · doi:10.1613/jair.301 · Cited by 8,972
Deep Learning: Methods and Applications
Deng, L., & Yu, D. (2014) · Foundations and Trends® in Signal Processing · doi:10.1561/2000000039 · Cited by 3,480
Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning
Williams, R. J. (1992) · Machine Learning · doi:10.1007/bf00992696 · Cited by 3,416
Deep Reinforcement Learning with Double Q-Learning
Van Hasselt, H., Guez, A., & Silver, D. (2016) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v30i1.10295 · Cited by 3,318
Convolutional Neural Networks for Speech Recognition
Abdel-Hamid, O., Mohamed, A.-R., Jiang, H., Deng, L., Penn, G., & Yu, D. (2014) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2014.2339736 · Cited by 2,262
Temporal difference learning and TD-Gammon
Tesauro, G. (1995) · Communications of the ACM · doi:10.1145/203330.203343 · Cited by 1,618
The "Wake-Sleep" Algorithm for Unsupervised Neural Networks
Hinton, G. E., Dayan, P., Frey, B. J., & Neal, R. M. (1995) · Science · doi:10.1126/science.7761831 · Cited by 1,070
Riedmiller, M. (2005) · Lecture Notes in Computer Science · doi:10.1007/11564096_32 · Cited by 757
Emergence of Locomotion Behaviours in Rich Environments
Heess, N., TB, D., Sriram, S., Lemmon, J., Merel, J., Wayne, G., Tassa, Y., Erez, T., Wang, Z., Eslami, S. M. A., et al. (2017) · arXiv · doi:10.48550/arxiv.1707.02286 · Cited by 657
Scaling Learning Algorithms toward AI
Bengio, Y., & LeCun, Y. (2007) · Large-Scale Kernel Machines · doi:10.7551/mitpress/7496.003.0016 · Cited by 455
A survey of real-time strategy game AI research and competition in StarCraft
Ontanon, S., Synnaeve, G., Uriarte, A., Richoux, F., Churchill, D., & Preuss, M. (2013) · IEEE Transactions on Computational Intelligence and AI in Games · doi:10.1109/tciaig.2013.2286295 · Cited by 423
Deep auto-encoder neural networks in reinforcement learning
Lange, S., & Riedmiller, M. (2010) · The 2010 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2010.5596468 · Cited by 329
Autonomous reinforcement learning on raw visual input data in a real world application
Lange, S., Riedmiller, M., & Voigtlander, A. (2012) · The 2012 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2012.6252823 · Cited by 202
StarCraft Micromanagement With Reinforcement Learning and Curriculum Transfer Learning
Shao, K., Zhu, Y., & Zhao, D. (2019) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2018.2823329 · Cited by 178
Wierstra, D., Förster, A., Peters, J., & Schmidhuber, J. (2009) · Logic Journal of IGPL · doi:10.1093/jigpal/jzp049 · Cited by 150
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1988) · The MIT Press eBooks · doi:10.7551/mitpress/4943.003.0042 · Cited by 146
Multiobjective exploration of the StarCraft map space
Togelius, J., Preuß, M., Beume, N., Wessing, S., Hagelbäck, J., & Yannakakis, G. N. (2010) · IEEE Conference on Computational Intelligence and Games (CIG) · doi:10.1109/itw.2010.5593346 · Cited by 111
Wender, S., & Watson, I. D. (2012) · IEEE Conference on Computational Intelligence and Games (CIG) · doi:10.1109/cig.2012.6374183 · Cited by 99
A Review of Real‐Time Strategy Game AI
Robertson, G., & Watson, I. D. (2014) · AI Magazine · doi:10.1609/aimag.v35i4.2478 · Cited by 93
Deep RTS: A Game Environment for Deep Reinforcement Learning in Real-Time Strategy Games
Andersen, P., Goodwin, M., & Granmo, O. (2018) · IEEE Conference on Computational Intelligence and Games (CIG) · doi:10.1109/cig.2018.8490409 · Cited by 62
A robot that reinforcement-leams to identify and memorize important previous observations
Bakker, B., Zhumatiy, V., Gruener, G., & Schmidhuber, J. (2004) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2003.1250667 · Cited by 61
Baird, I., & Leemon, C. (1993) · Defense Technical Information Center (DTIC) · doi:10.21236/ada280862 · Cited by 61
Learning Mutual Visibility Relationship for Pedestrian Detection with a Deep Model
Ouyang, W., Zeng, X., & Wang, X. (2016) · International Journal of Computer Vision · doi:10.1007/s11263-016-0890-9 · Cited by 57
Using plan-based reward shaping to learn strategies in StarCraft: Broodwar
Efthymiadis, K., & Kudenko⋆, D. (2013) · IEEE Conference on Computational Intelligence and Games (CIG) · doi:10.1109/cig.2013.6633622 · Cited by 27
Cited by
FinMem: A Performance-Enhanced LLM Trading Agent With Layered Memory and Character Design
Yu, Y., Li, H., Chen, Z., Jiang, Y., Li, Y., Suchow, J. W., Zhang, D., & Khashanah, K. (2025) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2025.3593370 · Cited by 23
Metadata
- Authors
- Liu, Y.
- Publisher
- University of North Texas Libraries
- Published
- 2019-05-01
- OpenAlex
- W2951360122
- Cited by
- 615
- References
- 32
- Type
- dissertation
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.