Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
PreprintCited by 86Reinforcement Learning in RoboticsAir Traffic Management and OptimizationArtificial Intelligence Applications

The DOI matches its DataCite record.

A Survey on Reinforcement Learning in Aviation Applications

Open access
Razzaghi, Pouria, Tabrizian, Amin, Guo, Wei, Chen, Shulu, Taye, Abenezer, Thompson, Ellis, Bregeon, Alexis, Baheri, Ali, Wei, Peng·2022·arXiv
DOI 10.48550/arxiv.2211.02147arXiv 2211.02147
Open this paper in CoChat — free accountOpen PDF

Abstract

Compared with model-based control and optimization methods, reinforcement learning (RL) provides a data-driven, learning-based framework to formulate and solve sequential decision-making problems. The RL framework has become promising due to largely improved data availability and computing power in the aviation industry. Many aviation-based applications can be formulated or treated as sequential decision-making problems. Some of them are offline planning problems, while others need to be solved online and are safety-critical. In this survey paper, we first describe standard RL formulations and solutions. Then we survey the landscape of existing RL-based applications in aviation. Finally, we summarize the paper, identify the technical gaps, and suggest future directions of RL research in aviation.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2211.02147

Cites

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

No references could be resolved yet.

Research works printed but not linked: 118

  1. [1][1] D. Bertsekas, Dynamic programming and optimal control: Volume I, Vol. 1, Athena scientific, 2012. -
  2. [2][2] D. Ha, J. Schmidhuber, World models, arXiv preprint arXiv:1803.10122 (2018). - · arXiv
  3. [3][3] V. Feinberg, A. Wan, I. Stoica, M. I. Jordan, J. E. Gonzalez, S. Levine, Model-based value estimation for efficient model-free reinforcement learning, arXiv preprint arXiv:1803.00101 (2018). - · arXiv
  4. [4][4] S. Racani` ere, T. Weber, D. Reichert, L. Buesing, A. Guez, D. Jimenez Rezende, A. Puigdom` enech Badia, O. Vinyals, N. Heess, Y. Li, et al., Imagination-augmented agents for deep reinforcement learning, Advances in neural information processing systems 30 (2017). -
  5. [5][5] A. Nagabandi, G. Kahn, R. S. Fearing, S. Levine, Neural network dynamics for model-based deep reinforcement learning with model-free fine-tuning, 2018 IEEE International Conference on Robotics and Automation (ICRA) (2018) 7559-7566. -
  6. [6][6] C. J. C. H. Watkins, Learning from delayed rewards, Ph.D. thesis, King's College (1989). -
  7. [7][7] R. S. Sutton, A. G. Barto, Reinforcement learning: An introduction, MIT press, 2018. -
  8. [9][9] G. Qu, A. Wierman, Finite-time analysis of asynchronous stochastic approximation and q -learning, Conference on Learning Theory (2020) 3185-3205. -
  9. [11][11] G. A. Rummery, M. Niranjan, On-line Q-learning using connectionist systems, Vol. 37, Citeseer, 1994. -
  10. [14][14] S. Gu, T. Lillicrap, I. Sutskever, S. Levine, Continuous deep q-learning with model-based acceleration, International conference on machine learning (2016) 2829-2838. -
  11. [15][15] H. Van Hasselt, A. Guez, D. Silver, Deep reinforcement learning with double q-learning, Proceedings of the AAAI conference on artificial intelligence 30 (1) (2016). -
  12. [16][16] Z. Wang, T. Schaul, M. Hessel, H. Hasselt, M. Lanctot, N. Freitas, Dueling network architectures for deep reinforcement learning, International conference on machine learning (2016) 1995-2003. -
  13. [17][17] J. Duan, Y. Guan, S. E. Li, Y. Ren, Q. Sun, B. Cheng, Distributional soft actor-critic: Off-policy reinforcement learning for addressing value estimation errors, IEEE Transactions on Neural Networks and Learning Systems (2021). -
  14. [18][18] R. S. Sutton, D. McAllester, S. Singh, Y. Mansour, Policy gradient methods for reinforcement learning with function approximation, Advances in neural information processing systems 12 (1999). -
  15. [20][20] D. Silver, G. Lever, N. Heess, T. Degris, D. Wierstra, M. Riedmiller, Deterministic policy gradient algorithms, International conference on machine learning (2014) 387-395. -
  16. [21][21] T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, D. Wierstra, Continuous control with deep reinforcement learning, ICRA (Poster) (2016). -
  17. [22][22] J. Schulman, S. Levine, P. Abbeel, M. Jordan, P. Moritz, Trust region policy optimization, International conference on machine learning (2015) 1889-1897. -
  18. [23][23] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, O. Klimov, Proximal policy optimization algorithms, arXiv preprint arXiv:1707.06347 (2017). - · arXiv
  19. [24][24] G. Ke, Q. Meng, T. Finley, T. Wang, W. Chen, W. Ma, Q. Ye, T.Y. Liu, Lightgbm: A highly efficient gradient boosting decision tree, Advances in neural information processing systems 30 (2017) 31463154. -
  20. [25][25] V. Mnih, A. P. Badia, M. Mirza, A. Graves, T. Lillicrap, T. Harley, D. Silver, K. Kavukcuoglu, Asynchronous methods for deep reinforcement learning, International conference on machine learning (2016) 1928-1937. -
  21. [26][26] J. X. Wang, Z. Kurth-Nelson, D. Tirumala, H. Soyer, J. Z. Leibo, R. Munos, C. Blundell, D. Kumaran, M. Botvinick, Learning to reinforcement learn, arXiv preprint arXiv:1611.05763 (2016). - · arXiv
  22. [27][27] T. Haarnoja, A. Zhou, P. Abbeel, S. Levine, Soft actor-critic: Offpolicy maximum entropy deep reinforcement learning with a stochastic actor, International conference on machine learning (2018) 1861-1870. -
  23. [28][28] S. Gronauer, K. Diepold, Multi-agent deep reinforcement learning: a survey, Artificial Intelligence Review 55 (2) (2022) 895-943. -
  24. [30][30] S. Omidshafiei, J. Pazis, C. Amato, J. P. How, J. Vian, Deep decentralized multi-task multi-agent reinforcement learning under partial observability, International Conference on Machine Learning (2017) 26812690. -
  25. [31][31] W. Wang, Y. Liu, R. Srikant, L. Ying, 3m-rl: Multi-resolution, multiagent, mean-field reinforcement learning for autonomous uav routing, IEEE Transactions on Intelligent Transportation Systems (2021). -
  26. [32][32] M. Tan, Multi-agent reinforcement learning: Independent vs. cooperative agents, Proceedings of the tenth international conference on machine learning (1993) 330-337. -
  27. [33][33] C. Li, J. Liu, Y. Zhang, Y. Wei, Y. Niu, Y. Yang, Y. Liu, W. Ouyang, Ace: Cooperative multi-agent q-learning with bidirectional actiondependency, Proceedings of the AAAI conference on artificial intelligence 37 (7) (2023) 8536-8544. -
  28. [34][34] L. Matignon, G. J. Laurent, N. Le Fort-Piat, Hysteretic q-learning: an algorithm for decentralized reinforcement learning in cooperative multiagent teams, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems (2007) 64-69. -
  29. [35][35] J. Foerster, G. Farquhar, T. Afouras, N. Nardelli, S. Whiteson, Counterfactual multi-agent policy gradients, Proceedings of the AAAI conference on artificial intelligence 32 (1) (2018). -
  30. [36][36] S. Li, Y. Wu, X. Cui, H. Dong, F. Fang, S. Russell, Robust multiagent reinforcement learning via minimax deep deterministic policy gradient, Proceedings of the AAAI conference on artificial intelligence 33 (1) (2019) 4213-4220. -
  31. [37][37] Y. Yang, R. Luo, M. Li, M. Zhou, W. Zhang, J. Wang, Mean field multi-agent reinforcement learning, International conference on machine learning (2018) 5571-5580. -
  32. [38][38] P. Sunehag, G. Lever, A. Gruslys, W. M. Czarnecki, V. Zambaldi, M. Jaderberg, M. Lanctot, N. Sonnerat, J. Z. Leibo, K. Tuyls, et al., Value-decomposition networks for cooperative multi-agent learning, arXiv preprint arXiv:1706.05296 (2017). - · arXiv
  33. [39][39] T. Rashid, G. Farquhar, B. Peng, S. Whiteson, Weighted qmix: Expanding monotonic value function factorisation for deep multi-agent reinforcement learning, Advances in neural information processing systems 33 (2020) 10199-10210. -
  34. [40][40] M. Schmidt, A review of aircraft turnaround operations and simulations, Progress in Aerospace Sciences 92 (2017) 25-38. -
  35. [42][42] M. Conde Rocha Murca, R. DeLaura, R. J. Hansman, R. Jordan, T. Reynolds, H. Balakrishnan, Trajectory clustering and classification for characterization of air traffic flows, 16th AIAA Aviation Technology, Integration, and Operations Conference (2016) 3760. -
  36. [43][43] H. Lee, W. Malik, Y. C. Jung, Taxi-out time prediction for departures at charlotte airport using machine learning techniques, 16th AIAA Aviation Technology, Integration, and Operations Conference (2016) 3910. -
  37. [44][44] N. Takeichi, R. Kaida, A. Shimomura, T. Yamauchi, Prediction of delay due to air traffic control by machine learning, AIAA modeling and simulation technologies conference (2017) 1323. -
  38. [45][45] S. Choi, Y. J. Kim, S. Briceno, D. Mavris, Prediction of weatherinduced airline delays based on machine learning algorithms, 2016 IEEE/AIAA 35th Digital Avionics Systems Conference (DASC) (2016) 1-6. -
  39. [46][46] S. Ayhan, H. Samet, Aircraft trajectory prediction made easy with predictive analytics, Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2016) 21-30. -
  40. [47][47] R. Alligier, D. Gianazza, N. Durand, Machine learning and mass estimation methods for ground-based aircraft climb prediction, IEEE Transactions on Intelligent Transportation Systems 16 (6) (2015) 31383149. -
  41. [51][51] J. M. Hoekstra, J. Ellerbroek, Bluesky atc simulator project: an open data and open source approach, Proceedings of the 7th international conference on research in air transportation 131 (2016) 132. -
  42. [52][52] J. Berndt, Jsbsim: An open source flight dynamics model in c++, AIAA Modeling and Simulation Technologies Conference and Exhibit (2004) 4923. -
  43. [54][54] Z. Wang, W. Pan, H. Li, X. Wang, Q. Zuo, Review of deep reinforcement learning approaches for conflict resolution in air traffic control, Aerospace 9 (6) (2022) 294. -
  44. [55][55] W. H. Harman, Tcas- a system for preventing midair collisions, The Lincoln Laboratory Journal 2 (3) (1989) 437-457. -
  45. [56][56] J.-B. Jeannin, K. Ghorbal, Y. Kouskoulas, R. Gardner, A. Schmidt, E. Zawadzki, A. Platzer, Formal verification of acas x, an industrial airborne collision avoidance system, 2015 International Conference on Embedded Software (EMSOFT) (2015) 127-136. -
  46. [59][59] M. Herman, Towards explainable automation for air traffic control using deep q-learning from demonstrations and reward decomposition, Master's thesis, Aerospace Engineering (2021). -
  47. [60][60] M. Brittain, P. Wei, Autonomous separation assurance in an highdensity en route sector: A deep multi-agent reinforcement learning approach, 2019 IEEE Intelligent Transportation Systems Conference (ITSC) (2019) 3256-3262. -
  48. [61][61] M. Brittain, X. Yang, P. Wei, A deep multi-agent reinforcement learning approach to autonomous separation assurance, AIAA Journal of Aerospace Information Systems 18 (12) (2021). -
  49. [63][63] W. Guo, M. Brittain, P. Wei, Safety enhancement for deep reinforcement learning in autonomous separation assurance, 2021 IEEE International Intelligent Transportation Systems Conference (ITSC) (2021) 348-354. -
  50. [64][64] J. Hu, X. Yang, W. Wang, P. Wei, L. Ying, Y. Liu, Obstacle avoidance for uas in continuous action space using deep reinforcement learning, IEEE Access 10 (2022) 90623-90634. -
  51. [65][65] D.-T. Pham, N. P. Tran, S. K. Goh, S. Alam, V. Duong, Reinforcement learning for two-aircraft conflict resolution in the presence of uncertainty, 2019 IEEE-RIVF International Conference on Computing and Communication Technologies (RIVF) (2019) 1-6. -
  52. [69][69] M. W. Brittain, P. Wei, Towards autonomous air trac control for sequencing and separation-a deep reinforcement learning approach, 2018 Aviation Technology, Integration, and Operations Conference (2018) 3664. -
  53. [72][72] C. P. Garcia, L. Weigang, N. S. Hirata, C. Neumann, Isuam: Intelligent and safe uam with deep reinforcement learning, 2023 IEEE 29th International Conference on Parallel and Distributed Systems (ICPADS) (2023) 378-383. -
  54. [73][73] Y. Gal, Z. Ghahramani, Dropout as a bayesian approximation: Representing model uncertainty in deep learning, international conference on machine learning (2016) 1050-1059. -
  55. [75][75] R. Dalmau, E. Allard, Air traffic control using message passing neural networks and multi-agent reinforcement learning, Proceedings of the 10th SESAR Innovation Days, Virtual Event (2020) 7-10. -
  56. [76][76] P. Zhao, Y. Liu, Physics informed deep reinforcement learning for aircraft conflict resolution, IEEE Transactions on Intelligent Transportation Systems (2021). -
  57. [78][78] N. P. Tran, D.-T. Pham, S. K. Goh, S. Alam, V. Duong, An intelligent interactive conflict solver incorporating air traffic controllers' preferences using reinforcement learning, 2019 Integrated Communications, Navigation and Surveillance Conference (ICNS) (2019) 1-8. -
  58. [79][79] P. N. Tran, D.-T. Pham, S. K. Goh, S. Alam, V. Duong, An interactive conflict solver for learning air traffic conflict resolutions, Journal of Aerospace Information Systems 17 (6) (2020) 271-277. -
  59. [80][80] H. Wen, H. Li, Z. Wang, X. Hou, K. He, Application of ddpg-based collision avoidance algorithm in air traffic control, 2019 12th International Symposium on Computational Intelligence and Design (ISCID) 1 (2019) 130-133. -
  60. [82][82] M. Ribeiro, J. Ellerbroek, J. Hoekstra, Determining optimal conflict avoidance manoeuvres at high densities with reinforcement learning, Proceedings of the Tenth SESAR Innovation Days, Virtual Conference (2020) 7-10. -
  61. [83][83] R. Isufaj, D. Aranega Sebastia, M. A. Piera, Towards conflict resolution with deep multi-agent reinforcement learning, Proceedings of the 14th USA/Europe Air Traffic Management Research and Development Seminar (ATM2021), New Orleans, LA, USA (2021) 20-24. -
  62. [84][84] J. Lai, K. Cai, Z. Liu, Y. Yang, A multi-agent reinforcement learning approach for conflict resolution in dense traffic scenarios, 2021 IEEE/AIAA 40th Digital Avionics Systems Conference (DASC) (2021) 1-9. -
  63. [85][85] J. Mollinga, H. van Hoof, An autonomous free airspace en-route controller using deep reinforcement learning techniques, International Conference for Research in Air Transportation (ICRAT) (2020). -
  64. [88][88] L. E. Alvarez, M. Brittain, K. Breeden, Towards a standardized reinforcement learning framework for aam contingency management, 2023 IEEE/AIAA 42nd Digital Avionics Systems Conference (DASC) (2023) 1-6. -
  65. [89][89] M. W. Brittain, L. E. Alvarez, K. Breeden, Improving autonomous separation assurance through distributed reinforcement learning with attention networks, Proceedings of the AAAI Conference on Artificial Intelligence 38 (21) (2024) 22857-22863. -
  66. [90][90] X. Yang, P. Wei, Scalable multi-agent computational guidance with separation assurance for autonomous urban air mobility, Journal of Guidance, Control, and Dynamics 43 (8) (2020) 1473-1486. -
  67. [91][91] G. Chaslot, S. Bakkes, I. Szita, P. Spronck, Monte-carlo tree search: A new framework for game ai, Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment 4 (1) (2021) 216-217. -
  68. [94][94] A. J. Singh, A. Kumar, H. C. Lau, Approximate difference rewards for scalable multigent reinforcement learning, Proceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems (2021) 1655-1657. -
  69. [97][97] K. D. Julian, J. Lopez, J. S. Brush, M. P. Owen, M. J. Kochenderfer, Policy compression for aircraft collision avoidance systems, 2016 IEEE/AIAA 35th Digital Avionics Systems Conference (DASC) (2016) 1-10. -
  70. [100][100] C. Huang, Y. Xu, Integrated frameworks of unsupervised, supervised and reinforcement learning for solving air traffic flow management problem, 2021 IEEE/AIAA 40th Digital Avionics Systems Conference (DASC) (2021) 1-10 doi:10.1109/DASC52595.2021.9594397 . - · DOI
  71. [101][101] Y. Xie, A. Gardi, R. Sabatini, Reinforcement learning-based flow management techniques for urban air mobility and dense low-altitude air traffic operations, 2021 IEEE/AIAA 40th Digital Avionics Systems Conference (DASC) (2021) 1-10. -
  72. [102][102] Y. Tang, Y. Xu, Multi-agent deep reinforcement learning for solving large-scale air traffic flow management problem: A time-step sequential decision approach, 2021 IEEE/AIAA 40th Digital Avionics Systems Conference (DASC) (2021) 1-10. -
  73. [103][103] T. Kravaris, C. Spatharis, A. Bastas, G. A. Vouros, K. Blekas, G. Andrienko, N. Andrienko, J. M. C. Garcia, Resolving congestions in the air traffic management domain via multiagent reinforcement learning methods, arXiv preprint arXiv:1912.06860 (2019). - · arXiv
  74. [105][105] T. Duong, K. K. Todi, U. Chaudhary, H.-L. Truong, Decentralizing air traffic flow management with blockchain-based reinforcement learning, 2019 IEEE 17th International Conference on Industrial Informatics (INDIN) 1 (2019) 1795-1800. -
  75. [106][106] Y. Chen, Y. Xu, M. Hu, L. Yang, Demand and capacity balancing technology based on multi-agent reinforcement learning, 2021 IEEE/AIAA 40th Digital Avionics Systems Conference (DASC) (2021) 1-9. -
  76. [107][107] S. Chen, A. D. Evans, M. Brittain, P. Wei, Integrated conflict management for uam with strategic demand capacity balancing and learningbased tactical deconfliction, IEEE Transactions on Intelligent Transportation Systems (2024). -
  77. [109][109] C. Spatharis, T. Kravaris, G. A. Vouros, K. Blekas, G. Chalkiadakis, J. M. C. Garcia, E. C. Fernandez, Multiagent reinforcement learning methods to resolve demand capacity balance problems, Proceedings of the 10th Hellenic Conference on Artificial Intelligence (2018) 1-9. -
  78. [112][112] M. Bloem, N. Bambos, Ground delay program analytics with behavioral cloning and inverse reinforcement learning, Journal of Aerospace Information Systems 12 (3) (2015) 299-313. -
  79. [113][113] E. George, S. S. Khan, Reinforcement learning for taxi-out time prediction: An improved q-learning approach, 2015 International Conference on Computing and Network Communications (CoCoNet) (2015) 757764. -
  80. [116][116] K. T. Talluri, G. Van Ryzin, G. Van Ryzin, The theory and practice of revenue management, Vol. 1, Springer, 2004. -
  81. [117][117] A. Gosavii, N. Bandla, T. K. Das, A reinforcement learning approach to a single leg airline revenue management problem with multiple fare classes and overbooking, IIE transactions 34 (9) (2002) 729-742. -
  82. [119][119] N. Bondoux, A. Q. Nguyen, T. Fiig, R. Acuna-Agost, Reinforcement learning applied to airline revenue management, Journal of Revenue and Pricing Management 19 (5) (2020) 332-348. -
  83. [120][120] S. A. Shihab, P. Wei, A deep reinforcement learning approach to seat inventory control for airline revenue management, Journal of Revenue and Pricing Management (2021) 1-17. -
  84. [122][122] N. E. Alamdari, G. Savard, Deep reinforcement learning in seat inventory control problem: an action generation approach, Journal of Revenue and Pricing Management 20 (5) (2021) 566-579. -
  85. [123][123] P. Belobaba, Air travel demand and airline seat inventory management, Ph.D. thesis, Massachusetts Institute of Technology (1987). -
  86. [124][124] S. A. Emami, P. Castaldi, A. Banazadeh, Neural network-based flight control systems: Present and future, Annual Reviews in Control 53 (2022) 97-137. doi:10.1016/j.arcontrol.2022.04.006 . URL http://dx.doi.org/10.1016/j.arcontrol.2022.04.006 - · DOI
  87. [125][125] Z. Li, X. Chen, M. Xie, Z. Zhao, Adaptive fault-tolerant tracking control of flying-wing unmanned aerial vehicle with system input saturation and state constraints, Transactions of the Institute of Measurement and Control 44 (4) (2022) 880-891. -
  88. [128][128] S. Lee, T. Shim, S. Kim, J. Park, K. Hong, H. Bang, Vision-based autonomous landing of a multi-copter unmanned aerial vehicle using reinforcement learning, 2018 International Conference on Unmanned Aircraft Systems (ICUAS) (2018) 108-114. -
  89. [129][129] B. Xian, X. Zhang, H. Zhang, X. Gu, Robust adaptive control for a small unmanned helicopter using reinforcement learning, IEEE Transactions on Neural Networks and Learning Systems (2021). -
  90. [130][130] Y. Zhen, M. Hao, W. Sun, Deep reinforcement learning attitude control of fixed-wing uavs, 2020 3rd International Conference on Unmanned Systems (ICUS) (2020) 239-244. -
  91. [133][133] H. Shi, X. Li, K.-S. Hwang, W. Pan, G. Xu, Decoupled visual servoing with fuzzy q-learning, IEEE Transactions on Industrial Informatics 14 (1) (2016) 241-252. -
  92. [134][134] D. Kim, G. Oh, Y. Seo, Y. Kim, Reinforcement learning-based optimal flat spin recovery for unmanned aerial vehicle, Journal of Guidance, Control, and Dynamics 40 (4) (2017) 1076-1084. -
  93. [135][135] Y. Zuo, K. Deng, Y. Yang, T. Huang, Flight attitude simulator control system design based on model-free reinforcement learning method, 2019 IEEE 3rd Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC) (2019) 355361. -
  94. [138][138] M. Al-Gabalawy, Machine learning for aircraft control, Journal of Advanced Research in Dynamical and Control Systems 11 (2019) 31653191. -
  95. [139][139] E. Bøhn, E. M. Coates, S. Moe, T. A. Johansen, Deep reinforcement learning attitude control of fixed-wing uavs using proximal policy optimization, 2019 International Conference on Unmanned Aircraft Systems (ICUAS) (2019) 523-533. -
  96. [141][141] M. Blanke, M. Kinnaert, J. Lunze, M. Staroswiecki, J. Schr¨ oder, Diagnosis and fault-tolerant control, Vol. 2, Springer, 2006. -
  97. [142][142] W. Koch, R. Mancuso, R. West, A. Bestavros, Reinforcement learning for uav attitude control, ACM Transactions on Cyber-Physical Systems 3 (2) (2019) 1-21. -
  98. [143][143] Y. Wang, J. Sun, H. He, C. Sun, Deterministic policy gradient with integral compensator for robust quadrotor control, IEEE Transactions on Systems, Man, and Cybernetics: Systems 50 (10) (2019) 3713-3725. -
  99. [144][144] F. Fei, Z. Tu, D. Xu, X. Deng, Learn-to-recover: Retrofitting uavs with reinforcement learning-assisted flight control under cyber-physical attacks, 2020 IEEE International Conference on Robotics and Automation (ICRA) (2020) 7358-7364. -
  100. [145][145] L. Bhan, M. Quinones-Grueiro, G. Biswas, Fault tolerant control combining reinforcement learning and model-based control, 2021 5th International Conference on Control and Fault-Tolerant Systems (SysTol) (2021) 31-36. -
  101. [146][146] A. Wickman, Exploring feasibility of reinforcement learning flight route planning, Bachelor's thesis, Department of Computer and Information Science (2021). -
  102. [147][147] A. T. Azar, A. Koubaa, N. Ali Mohamed, H. A. Ibrahim, Z. F. Ibrahim, M. Kazim, A. Ammar, B. Benjdira, A. M. Khamis, I. A. Hameed, et al., Drone deep reinforcement learning: A review, Electronics 10 (9) (2021) 999. -
  103. [148][148] S. Ragi, E. K. Chong, Uav path planning in a dynamic environment via partially observable markov decision process, IEEE Transactions on Aerospace and Electronic Systems 49 (4) (2013) 2397-2412. -
  104. [149][149] B. Zhang, Z. Mao, W. Liu, J. Liu, Geometric reinforcement learning for path planning of uavs, Journal of Intelligent & Robotic Systems 77 (2) (2015) 391-409. -
  105. [150][150] C. Yan, X. Xiang, C. Wang, Towards real-time path planning through deep reinforcement learning for a uav in dynamic environments, Journal of Intelligent & Robotic Systems 98 (2) (2020) 297-309. -
  106. [151][151] K. Tumer, A. Agogino, Distributed agent-based air traffic flow management, Proceedings of the 6th international joint conference on Autonomous agents and multiagent systems (2007) 1-8. -
  107. [152][152] C. Huang, Y. Xu, Integrated frameworks of unsupervised, supervised and reinforcement learning for solving air traffic flow management problem, 2021 IEEE/AIAA 40th Digital Avionics Systems Conference (DASC) (2021) 1-10. -
  108. [154][154] P. Juntama, D. Delahaye, S. Chaimatanan, S. Alam, Hyperheuristic approach based on reinforcement learning for air traffic complexity mitigation, AIAA Journal of Aerospace Information Systems (2022) 1-16. -
  109. [155][155] K. Zhang, Y. Yang, C. Xu, D. Liu, H. Song, Learning-to-dispatch: Reinforcement learning based flight planning under emergency, 2021 IEEE International Intelligent Transportation Systems Conference (ITSC) (2021) 1821-1826. -
  110. [157][157] Q. Deng, B. F. Santos, Lookahead approximate dynamic programming for stochastic aircraft maintenance check scheduling optimization, European Journal of Operational Research 299 (3) (2022) 814-833. -
  111. [162][162] A. Baheri, H. Ren, B. Johnson, P. Razzaghi, P. Wei, A verification framework for certifying learning-based safety-critical aviation systems, AIAA AVIATION 2022 Forum (2022). doi:10.2514/6.2022-3965 . - · DOI
  112. [163][163] J. Garcıa, F. Fern´ andez, A comprehensive survey on safe reinforcement learning, Journal of Machine Learning Research 16 (1) (2015) 14371480. -
  113. [164][164] B. R. Kiran, I. Sobh, V. Talpaert, P. Mannion, A. A. Al Sallab, S. Yogamani, P. P´ erez, Deep reinforcement learning for autonomous driving: A survey, IEEE Transactions on Intelligent Transportation Systems (2021). -
  114. [165][165] A. Baheri, S. Nageshrao, H. E. Tseng, I. Kolmanovsky, A. Girard, D. Filev, Deep reinforcement learning with enhanced safety for autonomous highway driving, 2020 IEEE Intelligent Vehicles Symposium (IV) (2020) 1550-1555. -
  115. [167][167] J. Groot, Improving safety of vertical manoeuvres in a layered airspace with deep reinforcement learning, Master's thesis, Aerospace Engineering (2021). -
  116. [168][168] C. Lazarus, J. G. Lopez, M. J. Kochenderfer, Runtime safety assurance using reinforcement learning, 2020 AIAA/IEEE 39th Digital Avionics Systems Conference (DASC) (2020) 1-9. -
  117. [170][170] P. Wu, X. Yang, P. Wei, J. Chen, Safety assured online guidance with airborne separation for urban air mobility operations in uncertain environments, IEEE Transactions on Intelligent Transportation Systems (2022). -
  118. [171][171] M. Hawley, R. Bharadwaj, V. Venkataraman, Real-time mitigation of loss of separation events using reinforcement learning, 2019 IEEE/AIAA 38th Digital Avionics Systems Conference (DASC) (2019) 1-6.

Cited by

Metadata

Authors
Razzaghi, P., Tabrizian, A., Guo, W., Chen, S., Taye, A., Thompson, E., Bregeon, A., Baheri, A., & Wei, P.
Affiliations
  • George Washington University (US)
  • École Nationale de l’Aviation Civile (FR)
  • Rochester Institute of Technology (US)
  • Pennsylvania State University (US)
Corresponding author
Razzaghi, Pouria
ORCID
Venue
arXiv
Publisher
arXiv
Published
2022
Volume
136
Pages
108911-108911
ISSN
0952-1976 (ISSN-L)
DOI
10.48550/arxiv.2211.02147
arXiv
2211.02147
OpenAlex
W4400650558
Cited by
86
References
0
Citations by year
2026: 28 · 2025: 30 · 2024: 19 · 2023: 8 · 2022: 1
FWCI
11.34
Type
journal article
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • Engineering Applications of Artificial Intelligence (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • HAL (submittedVersion)
  • arXiv (Cornell University)
Funders
  • U.S. Department of Transportation
  • Federal Aviation Administration (692M15-21-T-00022)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Physical Sciences › Engineering › Aerospace Engineering › Air Traffic Management and Optimization
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence Applications
Keywords
reinforcement learning, aeronautical applications, flight control, air traffic management, airline revenue management, aircraft maintenance scheduling, aviation safety, sustainable aviation, decision-making

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.