Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
Cited by 58Reinforcement Learning in RoboticsDistributed Control Multi-Agent SystemsUAV Applications and Optimization

The DOI matches its Crossref record; 73 of the 74 references it cites were found in a registry or an index; none of the 2 sources its claims cite is checked for support yet.

Application of Deep Reinforcement Learning to UAV Swarming for Ground Surveillance

Open access
Arranz, Raúl, Carramiñana, David, Miguel, Gonzalo de, Besada, Juan A., Bernardos, Ana M.·2023·Sensors
DOI 10.3390/s23218766PMC PMC10648592
Open this paper in CoChat — free accountOpen PDF

Abstract

This paper summarizes in depth the state of the art of aerial swarms, covering both classical and new reinforcement-learning-based approaches for their management. Then, it proposes a hybrid AI system, integrating deep reinforcement learning in a multi-agent centralized swarm architecture. The proposed system is tailored to perform surveillance of a specific area, searching and tracking ground targets, for security and law enforcement applications. The swarm is governed by a central swarm controller responsible for distributing different search and tracking tasks among the cooperating UAVs. Each UAV agent is then controlled by a collection of cooperative sub-agents, whose behaviors have been trained using different deep reinforcement learning models, tailored for the different task types proposed by the swarm controller. More specifically, proximal policy optimization (PPO) algorithms were used to train the agents’ behavior. In addition, several metrics to assess the performance of the swarm in this application were defined. The results obtained through simulation show that our system searches the operation area effectively, acquires the targets in a reasonable time, and is capable of tracking them continuously and consistently.

Overview

CoChat Summary

TL;DR

The paper proposes a hybrid AI system for UAV swarm ground surveillance, combining a centralized rule-based swarm controller with deep reinforcement learning sub-agents for search, tracking, and obstacle avoidance. Simulation results show effective area search, reasonable target acquisition, and consistent tracking, with performance adapting to changing numbers of targets and UAVs.

Key findings

  • In a baseline simulation with 8 fixed-wing UAVs tracking 5 targets over a 6 km × 6 km area, run 10 times, most targets were kept in the swarm’s field of view for over 95% of the time since first identification, though some losses occurred mainly due to occlusions behind obstacles.
  • Across 10 simulations totaling 50 targets, the swarm searched the operation area effectively, but border cells had higher revisit periods, indicating more difficulty exploring edges.
  • With UAVs fixed at 8 and targets varied as 1, 3, 5, and 6, total search time generally increased with more targets, while the percentage of detected targets showed fewer differences because tracking is assigned to a single agent once a target is first detected.
  • With targets fixed at 5 and UAVs varied as 8, 9, and 10, increasing the number of UAVs significantly improved revisit time and reduced ordered target acquisition time.

Methods

  • The system uses a two-layer centralized architecture: a deterministic rule-based swarm controller assigns search and tracking tasks, while each UAV agent executes tasks through three cooperative DRL sub-agents for search, tracking, and obstacle avoidance.
  • The sub-agents were trained independently using proximal policy optimization (PPO) in Unity ML-Agents, with an artificial neural network having an input layer, two fully connected hidden layers of 128 neurons each, and an output layer.
  • Evaluation used simulation scenarios with a 6 km square operation area, fixed-wing UAVs at constant altitude, ground targets moving at 20 m/s, static obstacles, and metrics including revisit period, search time, percentage of detected targets, tracking continuity, and target acquisition time.

Limitations

  • The paper states that RL-based approaches are expensive in computation, development time, and resources, and that training a swarm control system for an imminent mission with a specific or new scenario could be difficult.
  • The paper states that performance strongly depends on the fidelity of training scenarios, that reward function design requires expert knowledge and many design iterations, and that there is no clear method for determining reward functions.
  • Evident from the text: the study assumes perfect sensors, negligible navigation error, benign weather, fixed UAV altitude, and simulation-only validation; targets in the tracking training initially move at constant speed in straight lines, and the evaluation uses constant-speed targets.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.3390/s23218766

Cites

73 of 74 references verified0 citation errors · 1 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 Swarming process layers and relationship between them.
Figure 1 Swarming process layers and relationship between them.CC BY · PubMed Central
Figure 2 Control structures. (a) Centralized structure. (b) Hierarchical structure. (c) Distributed structure.
Figure 2 Control structures. (a) Centralized structure. (b) Hierarchical structure. (c) Distributed structure.CC BY · PubMed Central
Figure 3 A 2D environment made up of a 5 × 5 grid (from [29]).
Figure 3 A 2D environment made up of a 5 × 5 grid (from [29]).CC BY · PubMed Central
Figure 4 Classification of path-planning algorithms (inspired on [30]).
Figure 4 Classification of path-planning algorithms (inspired on [30]).CC BY · PubMed Central
Figure 5 Structure of deep RL with a policy neural network.
Figure 5 Structure of deep RL with a policy neural network.CC BY · PubMed Central
Figure 6 UAV swarm system. Coordination architecture.
Figure 6 UAV swarm system. Coordination architecture.CC BY · PubMed Central
Figure 7 Deployment of the sub-agents in the different layers.
Figure 7 Deployment of the sub-agents in the different layers.CC BY · PubMed Central
Figure 8 Mission beginning, after deployment (8 UAVs; 5 targets).
Figure 8 Mission beginning, after deployment (8 UAVs; 5 targets).CC BY · PubMed Central
Figure 9 Mid-stage of mission (8 UAVs; 5 targets).
Figure 9 Mid-stage of mission (8 UAVs; 5 targets).CC BY · PubMed Central
Figure 10 Test scenario.
Figure 10 Test scenario.CC BY · PubMed Central
Figure 11 Mean revisit time by zone of the operation area throughout 10 simulations.
Figure 11 Mean revisit time by zone of the operation area throughout 10 simulations.CC BY · PubMed Central
Figure 12 Mean percentage of targets detected by the swarm throughout 10 simulations.
Figure 12 Mean percentage of targets detected by the swarm throughout 10 simulations.CC BY · PubMed Central
Figure 13 Histogram showing the percentage of tracking time that each target (50, 5 across 10 simulations) is in the swarm’s field of view.
Figure 13 Histogram showing the percentage of tracking time that each target (50, 5 across 10 simulations) is in the swarm’s field of view.CC BY · PubMed Central
Figure 14 Revisit period and search time when modifying the number of targets.
Figure 14 Revisit period and search time when modifying the number of targets.CC BY · PubMed Central
Figure 15 Percentage of targets detected by the swarm when modifying the number of targets.
Figure 15 Percentage of targets detected by the swarm when modifying the number of targets.CC BY · PubMed Central
Figure 16 Mean revisit time by zone of the operation area when modifying the number of UAVs.
Figure 16 Mean revisit time by zone of the operation area when modifying the number of UAVs.CC BY · PubMed Central
Figure 17 Target acquisition time when modifying the number of UAVs.
Figure 17 Target acquisition time when modifying the number of UAVs.CC BY · PubMed Central

Verification

Claims the paper makes

61 Own interpretation · 13 Own result · 2 No basis given · 1 No full text of the sources · 1 Not checked yet · 1 Own method

24 more claims

Create a free account to see all 79 claims and to verify the 1 reference still open.

See all claims — free account
  • The quote only partly states this claim. 1
  • It may state two separate claims as one. 20
  • Probably not a substantial claim. 41

Create a free account to see these claims.

Create a free account to see these assumptions.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Research works printed but not linked: 2

  1. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal policy optimization algorithms. arXiv.
  2. Haarnoja, T., Zhou, A., Hartikainen, K., Tucker, G., Ha, S., Tan, J., and Levine, S. (2018). Soft actor-critic algorithms and applications. arXiv.

Cited by

See all 58

Related

See all

Metadata

Authors
Arranz, R., Carramiñana, D., Miguel, G. D., Besada, J. A., & Bernardos, A. M.
Affiliations
  • Information Processing and Telecommunications Center (ES)
  • Information Processing and Telecommunications Center, Universidad Politécnica de Madrid, ETSI Telecomunicación, Av. Complutense 30, 28040 Madrid, Spain
Corresponding author
Bernardos, Ana M.
ORCID
Venue
Sensors
Publisher
MDPI AG
Published
2023-10-27
Volume
23
Issue
21
Pages
8766
ISSN
1424-8220 (ISSN-L)
DOI
10.3390/s23218766
PMC
PMC10648592
OpenAlex
W4387977146
Semantic Scholar
cfdeb972d25cf0eeba9bf73b4bea951d8b7dca6c
Cited by
58
References
72
Citations by year
2026: 24 · 2025: 25 · 2024: 9
FWCI
4.13
Type
journal article
Language
en
Open access
Licence
Locations
  • Sensors (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Hispana (acceptedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion, cc-by-nc-nd)
  • Archivo Digital UPM (Universidad Politécnica de Madrid) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • arXiv (Cornell University)
Funders
  • MCIN/AEI/10.13039/501100011033 (PID2020-118249RB-C21, 101021797)
  • European Union’s Horizon 2020 Research and Innovation Programme (PID2020-118249RB-C21, 101021797)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
  • Physical Sciences › Engineering › Control and Systems Engineering › Distributed Control Multi-Agent Systems
  • Physical Sciences › Engineering › Aerospace Engineering › UAV Applications and Optimization
Keywords
UAV swarms, aerial swarms, deep reinforcement learning, ground surveillance, proximal policy optimization, law enforcement, security applications

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.