Bio-Inspired Collision Avoidance in Swarm Systems via Deep Reinforcement Learning
Open accessThe DOI matches its Crossref record; 58 of the 59 references it cites were found in a registry or an index.
Abstract
Autonomous vehicles have been highlighted as a major growth area for future transportation systems and the deployment of large numbers of these vehicles is expected when safety and legal challenges are overcome. To meet the necessary safety standards, effective collision avoidance technologies are required to ensure that the number of accidents are kept to a minimum. As large numbers of autonomous vehicles, operating together on roads, can be regarded as a swarm system, we propose a bio-inspired collision avoidance strategy using virtual pheromones; an approach that has evolved effectively in nature over many millions of years. Previous research using virtual pheromones showed the potential of pheromone-based systems to maneuver a swarm of robots. However, designing an individual controller to maximise the performance of the entire swarm is a major challenge. In this paper, we propose a novel deep reinforcement learning (DRL) based approach that is able to train a controller that introduces collision avoidance behaviour. To accelerate training, we propose a novel sampling strategy calledHighlight Experience Replayand integrate it with a Deep Deterministic Policy Gradient algorithm with noise added to the weights and biases of the artificial neural network to improve exploration. To evaluate the performance of the proposed DRL-based controller, we applied it to navigation and collision avoidance tasks in three different traffic scenarios. The experimental results showed that the proposed DRL-based controller outperformed the manually-tuned controller in terms of stability, effectiveness, robustness and ease of tuning process. Furthermore, the proposed Highlight Experience Replay method outperformed than the popular Prioritized Experience Replay sampling strategy by taking 27% of training time average over three stages.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Citations
Cites
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357
Continuous control with deep reinforcement learning
Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., & Wierstra, D. (2015) · arXiv · doi:10.48550/arxiv.1509.02971 · Cited by 10,782
Proximal Policy Optimization Algorithms
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. (2017) · arXiv · doi:10.48550/arxiv.1707.06347 · Cited by 10,625
Reinforcement Learning: An Introduction
Johnson, J. D., Li, J., & Chen, Z. (2000) · Neurocomputing · doi:10.1016/s0925-2312(00)00324-6 · Cited by 8,414
Asynchronous Methods for Deep Reinforcement Learning
Mnih, V., Badia, A. P., Mirza, M., Graves, A., Lillicrap, T. P., Harley, T., Silver, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1602.01783 · Cited by 7,132 · cited 1×
“The third branch is actor-critic method [43].”
Trust region policy optimization
Schulman, J., Levine, S., Moritz, P., Jordan, M. I., & Abbeel, P. (2015) · arXiv · doi:10.48550/arxiv.1502.05477 · Cited by 4,578
Schaul, T., Quan, J., Antonoglou, I., & Silver, D. (2015) · arXiv · doi:10.48550/arxiv.1511.05952 · Cited by 3,120
Dorigo, M., Birattari, M., & Stutzle, T. (2006) · IEEE Computational Intelligence Magazine · doi:10.1109/ci-m.2006.248054 · Cited by 1,880
Reciprocal n-Body Collision Avoidance
van den Berg, J., Guy, S. J., Lin, M., & Manocha, D. (2011) · Springer Tracts in Advanced Robotics · doi:10.1007/978-3-642-19457-3_1 · Cited by 1,858
Swarm robotics: a review from the swarm engineering perspective
Brambilla, M., Ferrante, E., Birattari, M., & Dorigo, M. (2013) · Swarm Intelligence · doi:10.1007/s11721-012-0075-2 · Cited by 1,762
Research works printed but not linked: 1
- 10.1109/mci.2006.329691 · doi:10.1109/mci.2006.329691 · Paper page
Cited by
Reinforcement learning algorithms: A brief survey
Shakya, A. K., Pillai, G., & Chakrabarty, S. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.120495 · Cited by 656
Multi-Agent Deep Reinforcement Learning for Multi-Robot Applications: A Survey
Orr, J., & Dutta, A. (2023) · Sensors · doi:10.3390/s23073625 · Cited by 206
Distributed Motion Planning for Safe Autonomous Vehicle Overtaking via Artificial Potential Field
Xie, S., Hu, J., Bhowmick, P., Ding, Z., & Arvin, F. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2022.3189741 · Cited by 163
Self-citationWu, K., Hu, J., Ding, Z., & Arvin, F. (2023) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2023.3239748 · Cited by 90
Self-citationHuman-in-the-Loop Robot Learning for Smart Manufacturing: A Human-Centric Perspective
Chen, H., Li, S., Fan, J., Duan, A., Yang, C., Navarro-Alarcon, D., & Zheng, P. (2025) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2025.3528051 · Cited by 67
“Na et al. [166] designed a collision avoidance behavior for swarm systems based on a virtual pheromone, making robot navigation and collision avoidance tasks in different scenarios.”
Metadata
- Authors
- Na, S., Niu, H., Lennox, B., & Arvin, F.
- Affiliations
- University of Manchester (GB)
- Venue
- IEEE Transactions on Vehicular Technology
- Published
- 2022-03
- Volume
- 71
- Issue
- 3
- Pages
- 2511-2526
- ISSN
- 0018-9545 (ISSN-L)
- 1939-9359
- OpenAlex
- W4210530701
- Cited by
- 64
- References
- 59
- Citations by year
- 2026: 4 · 2025: 19 · 2024: 12 · 2023: 25 · 2022: 4
- FWCI
- 4.94
- Type
- journal article
- Language
- en
- Open access
- Licence
- Locations
- IEEE Transactions on Vehicular Technology (publishedVersion)
- Research Explorer (The University of Manchester) (acceptedVersion, other-oa)
- Durham Research Online (Durham University) (submittedVersion)
- Durham Research Online (Durham University) (submittedVersion, cc-by)
- Funders
- EU H2020-FET-OPEN RoboRoyale Project (964492)
- Engineering and Physical Sciences Research Council (EP/R026084/1, EP/P01366X/1)
- University of Manchester
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Physical Sciences › Engineering › Control and Systems Engineering › Traffic control and management
- Keywords
- collision avoidance, autonomous vehicles, swarm systems, virtual pheromones, bio-inspired control, deep reinforcement learning, deep deterministic policy gradient, exploration noise, traffic scenarios, vehicle navigation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Bio-Inspired Collision Avoidance in Swarm Systems via Deep Reinforcement Learning. https://app.cochat.ai/papers/10.1109/tvt.2022.3145346
- The paper
- Na, S., Niu, H., Lennox, B., & Arvin, F. (2022). Bio-Inspired Collision Avoidance in Swarm Systems via Deep Reinforcement Learning. IEEE Transactions on Vehicular Technology. https://doi.org/10.1109/tvt.2022.3145346
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.