Autonomous navigation of UAV in large-scale unknown complex environment with deep reinforcement learning
The DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
Abstract
Unmanned Aerial Vehicles (UAVs) based delivery is thriving. In this paper, we model autonomous navigation of UAV in large-scale unknown complex environment as a discrete-time continuous control problem and solve it using deep reinforcement learning. Without path planning or map construction, our method enables UAVs to navigate from arbitrary departure places to destinations using only sensory information of local environment and GPS signal. We argue the navigation task is a partially observable Markov decision process (POMDP) and extant recurrent deterministic policy gradient algorithm is less efficient. Consequently, we derive a faster policy learning algorithm for POMDP based on actor-critic architecture. To validate our ideas, we simulate five virtual environments and a virtual UAV flying at a fixed altitude with constant speed. Cognition of local environment is achieved by measuring distances from UAV to obstacles in multiple directions. Simulation results demonstrate the effectiveness of our method.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
A solution to the simultaneous localization and map building (SLAM) problem
Dissanayake, M. W. M. G., Newman, P., Clark, S., Durrant-Whyte, H. F., & Csorba, M. (2001) · IEEE Transactions on Robotics and Automation · doi:10.1109/70.938381 · Cited by 2,661
Memory-based control with recurrent neural networks
Heess, N., Hunt, J. J., Lillicrap, T. P., & Silver, D. (2015) · arXiv · doi:10.48550/arxiv.1512.04455 · Cited by 211
Cited by
Unmanned Aerial Vehicles (UAVs): A Survey on Civil Applications and Key Research Challenges
Shakhatreh, H., Sawalmeh, A. H., Al-Fuqaha, A., Dou, Z., Almaita, E., Khalil, I., Othman, N. S., Khreishah, A., & Guizani, M. (2019) · IEEE Access · doi:10.1109/access.2019.2909530 · Cited by 2,529
Applications of Deep Reinforcement Learning in Communications and Networking: A Survey
Luong, N. C., Hoang, D. T., Gong, S., Niyato, D., Wang, P., Liang, Y.-C., & Kim, D. I. (2019) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2019.2916583 · Cited by 2,060
Accessing From the Sky: A Tutorial on UAV Communications for 5G and Beyond
Zeng, Y., Wu, Q., & Zhang, R. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2952892 · Cited by 1,589
Deep reinforcement learning based mobile robot navigation: A review
Zhu, K., & Zhang, T. (2021) · Tsinghua Science and Technology · doi:10.26599/tst.2021.9010012 · Cited by 499
Wang, C., Wang, J., Shen, Y., & Zhang, X. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2018.2890773 · Cited by 443
Self-citationA Survey on Millimeter-Wave Beamforming Enabled UAV Communications and Networking
Xiao, Z., Zhu, L., Liu, Y., Yi, P., Zhang, R., Xia, X.-G., & Schober, R. (2022) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3124512 · Cited by 420
Survey on computational-intelligence-based UAV path planning
Zhao, Y., Zheng, Z., & Liu, Y. (2018) · Knowledge-Based Systems · doi:10.1016/j.knosys.2018.05.033 · Cited by 418
Drone Deep Reinforcement Learning: A Review
Azar, A. T., Koubaa, A., Ali Mohamed, N., Ibrahim, H. A., Ibrahim, Z. F., Kazim, M., Ammar, A., Benjdira, B., Khamis, A. M., Hameed, I. A., et al. (2021) · Electronics · doi:10.3390/electronics10090999 · Cited by 314
“Other machine learning algorithms need labeled data to start the training process, which is not available in UAV cases as, in reality, the UAV deals with new environments during each mission [73].”
BackgroundPath-Planning for Unmanned Aerial Vehicles with Environment Complexity Considerations: A Survey
Jones, M., Djahel, S., & Welsh, K. (2023) · ACM Computing Surveys · doi:10.1145/3570723 · Cited by 258
A Gentle Introduction to Reinforcement Learning and its Application in Different Fields
Naeem, M., Rizvi, S. T. H., & Coronato, A. (2020) · IEEE Access · doi:10.1109/access.2020.3038605 · Cited by 257
Applications of Deep Reinforcement Learning in Communications and Networking: A Survey
Luong, N. C., Hoang, D. T., Gong, S., Niyato, D., Wang, P., Liang, Y.-C., & Kim, D. I. (2018) · arXiv · doi:10.48550/arxiv.1810.07862 · Cited by 17
Metadata
- Authors
- Wang, C., Wang, J., Zhang, X., & Zhang, X.
- Venue
- 2017 IEEE Global Conference on Signal and Information Processing (GlobalSIP)
- Publisher
- IEEE
- Published
- 2017-11
- Pages
- 858-862
- ISSN
- 2376-4066 (ISSN-L)
- OpenAlex
- W2774386693
- Cited by
- 106
- References
- 4
- FWCI
- 4.07
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.