Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 61 of the 62 references it cites were found in a registry or an index.

Empirical assessment and comparison of neuro-evolutionary methods for the automatic off-line design of robot swarms

Open access

Abstract

Neuro-evolution is an appealing approach to generating collective behaviors for robot swarms. In its typical application, known as off-line automatic design, the neural networks controlling the robots are optimized in simulation. It is understood that the so-called reality gap, the unavoidable differences between simulation and reality, typically causes neural network to be less effective on real robots than what is predicted by simulation. In this paper, we present an empirical study on the extent to which the reality gap impacts the most popular and advanced neuro-evolutionary methods for the off-line design of robot swarms. The results show that the neural networks produced by the methods under analysis performed well in simulation, but not in real-robot experiments. Further, the ranking that could be observed in simulation between the methods eventually disappeared. We find compelling evidence that real-robot experiments are needed to reliably assess the performance of neuro-evolutionary methods and that the robustness to the reality gap is the main issue to be addressed to advance the application of neuro-evolution to robot swarms.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41467-021-24642-3

Cites

61 of 61 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 The robot and its reference model. a The e-puck robot in the configuration used for the experiments presented in the paper. Details are provided in “Methods”. b The reference model RM 1.1, which formally describes the programming interface through which, in the experiments presented in the paper, the control software interacts with the underlying hardware. The range-and-bearing vector points to the aggregate position of the neighboring peers perceived; its magnitude increases with the number of neighboring peers perceived and decreases with their distance. Formally, V=(∑)m=1ⁿ(1/(1+rm),∠bm), where rm and ∠bm are range and bearing of neighbor m, respectively. If no neighboring peer is perceived, the vector points in front of the robot and has unitary magnitude; formally, V = (1, ∠0).
Fig. 1 The robot and its reference model. a The e-puck robot in the configuration used for the experiments presented in the paper. Details are provided in “Methods”. b The reference model RM 1.1, which formally describes the programming interface through which, in the experiments presented in the paper, the control software interacts with the underlying hardware. The range-and-bearing vector points to the aggregate position of the neighboring peers perceived; its magnitude increases with the number of neighboring peers perceived and decreases with their distance. Formally, V=(∑)m=1ⁿ(1/(1+rm),∠bm), where rm and ∠bm are range and bearing of neighbor m, respectively. If no neighboring peer is perceived, the vector points in front of the robot and has unitary magnitude; formally, V = (1, ∠0).CC BY · PubMed Central
Fig. 2 Arenas for the five missions. a XOR-AGGREGATION, simulation; b, real robots. c, HOMING, simulation; d, real robots. e, FORAGING, simulation; f, real robots. g, SHELTER, simulation; h, real robots. i, DIRECTIONAL-GATE, simulation; j, real robots. The 20 robots operate in a dodecagonal arena of 4.91 m2, the red glow in (e, f, g, h, i), and (j) indicates the presence of a light source at the bottom side of the arena. Dimensions (in meters) of the elements present in the arenas are given in (a, c, e, g,) and (i). Details on the experimental setup are provided in “Methods” and videos of the robot experiments are available as Supplementary Movies 1–5.
Fig. 2 Arenas for the five missions. a XOR-AGGREGATION, simulation; b, real robots. c, HOMING, simulation; d, real robots. e, FORAGING, simulation; f, real robots. g, SHELTER, simulation; h, real robots. i, DIRECTIONAL-GATE, simulation; j, real robots. The 20 robots operate in a dodecagonal arena of 4.91 m2, the red glow in (e, f, g, h, i), and (j) indicates the presence of a light source at the bottom side of the arena. Dimensions (in meters) of the elements present in the arenas are given in (a, c, e, g,) and (i). Details on the experimental setup are provided in “Methods” and videos of the robot experiments are available as Supplementary Movies 1–5.CC BY · PubMed Central
Fig. 3 Aggregated results. a Aggregate performance in simulation (white narrow boxes) and in reality (gray wide boxes) across the five missions considered, represented by notched box-and-whisker plots, where the notches represent the 95% confidence interval on the median. If notches on different boxes do not overlap, the medians of the corresponding methods differ significantly, with a confidence of at least 95%. Graphical conventions adopted in box-and-whisker plots are described in “Methods” under the heading Statistics. Prior to the aggregation and for each missions, the results are normalized between the lowest and highest performance observed in reality by any of the design methods. As a result, the normalized performance in reality ranges between 0 and 1, but the one in simulation might exceed 1 (shadowed area). Indeed, in many cases, the performance observed in simulation exceeded the best one observed in the real-robot experiments. The performance of Chocolate and RandomWalk, the two methods included in the study as yardsticks, is grayed out so as to focus the attention of the reader to the neuro-evolutionary methods under analysis. b Friedman rank-sum test on the performance in reality: expected rank and 95% confidence interval. If two segments do not overlap, the rank of the corresponding methods differ significantly, with a confidence of at least 95%. Also here, the performance of Chocolate and RandomWalk is grayed out to focus the attention to the neuro-evolutionary methods. The videos of all robot experiments are available as Supplementary Movie 1–5.
Fig. 3 Aggregated results. a Aggregate performance in simulation (white narrow boxes) and in reality (gray wide boxes) across the five missions considered, represented by notched box-and-whisker plots, where the notches represent the 95% confidence interval on the median. If notches on different boxes do not overlap, the medians of the corresponding methods differ significantly, with a confidence of at least 95%. Graphical conventions adopted in box-and-whisker plots are described in “Methods” under the heading Statistics. Prior to the aggregation and for each missions, the results are normalized between the lowest and highest performance observed in reality by any of the design methods. As a result, the normalized performance in reality ranges between 0 and 1, but the one in simulation might exceed 1 (shadowed area). Indeed, in many cases, the performance observed in simulation exceeded the best one observed in the real-robot experiments. The performance of Chocolate and RandomWalk, the two methods included in the study as yardsticks, is grayed out so as to focus the attention of the reader to the neuro-evolutionary methods under analysis. b Friedman rank-sum test on the performance in reality: expected rank and 95% confidence interval. If two segments do not overlap, the rank of the corresponding methods differ significantly, with a confidence of at least 95%. Also here, the performance of Chocolate and RandomWalk is grayed out to focus the attention to the neuro-evolutionary methods. The videos of all robot experiments are available as Supplementary Movie 1–5.CC BY · PubMed Central
Fig. 4 Results per missions. Performance obtained in simulation (white narrow boxes) and in reality (gray wide boxes) in all five missions: a XOR-AGGREGATION, b HOMING, c FORAGING, d SHELTER, e DIRECTIONAL-GATE. The results are presented using notched box-and-whiskers plots, where the notches represent the 95% confidence interval on the median. If notches on different boxes do not overlap, the medians of the corresponding methods differ significantly, with a confidence of at least 95%. Graphical conventions adopted in box-and-whisker plots are described in “Methods” under the heading Statistics. The performance of Chocolate and RandomWalk is grayed out so as to focus the attention of the reader to the neuro-evolutionary methods under analysis. The videos of all robot experiments are available as Supplementary Movie 1–5.
Fig. 4 Results per missions. Performance obtained in simulation (white narrow boxes) and in reality (gray wide boxes) in all five missions: a XOR-AGGREGATION, b HOMING, c FORAGING, d SHELTER, e DIRECTIONAL-GATE. The results are presented using notched box-and-whiskers plots, where the notches represent the 95% confidence interval on the median. If notches on different boxes do not overlap, the medians of the corresponding methods differ significantly, with a confidence of at least 95%. Graphical conventions adopted in box-and-whisker plots are described in “Methods” under the heading Statistics. The performance of Chocolate and RandomWalk is grayed out so as to focus the attention of the reader to the neuro-evolutionary methods under analysis. The videos of all robot experiments are available as Supplementary Movie 1–5.CC BY · PubMed Central
Fig. 5 Distributions. Empirical distribution of the performance of the control software generated by each method under analysis on each of the five mission considered. The last column displays the normalized performance of each method, aggregated across the five missions. Aggregation is performed using the min-max normalization technique described in Methods under the heading Statistics. The black line represents the empirical distribution of the performance observed in real-robot experiments; the gray one, the one obtained in simulation.
Fig. 5 Distributions. Empirical distribution of the performance of the control software generated by each method under analysis on each of the five mission considered. The last column displays the normalized performance of each method, aggregated across the five missions. Aggregation is performed using the min-max normalization technique described in Methods under the heading Statistics. The black line represents the empirical distribution of the performance observed in real-robot experiments; the gray one, the one obtained in simulation.CC BY · PubMed Central
Fig. 6 Trajectories of the robots throughout the entire median runs. For each method on each mission, we report the execution in simulation (top row for each mission) and reality (bottom row for each mission) of the instance of control software that obtained a median performance in reality, out of the instances produced by that method for that mission. The color of a spot represents the amount of time a robot spent on that spot during the execution. If a robot were to stay on a spot for more than a quarter of the entire execution, the color of that spot would be dark blue (value 0.25 in the color scale). The figure indicates that the control software produced by the evolutionary approaches cover the space differently in simulation and reality: in reality, the robots tend to form clusters, mostly against the walls. Differences between simulation and reality are less pronounced for Chocolate and barely noticeable for RandomWalk. For each mission and each method, a direct comparison of the behavior in simulation and reality is available in Supplementary Movie 6.
Fig. 6 Trajectories of the robots throughout the entire median runs. For each method on each mission, we report the execution in simulation (top row for each mission) and reality (bottom row for each mission) of the instance of control software that obtained a median performance in reality, out of the instances produced by that method for that mission. The color of a spot represents the amount of time a robot spent on that spot during the execution. If a robot were to stay on a spot for more than a quarter of the entire execution, the color of that spot would be dark blue (value 0.25 in the color scale). The figure indicates that the control software produced by the evolutionary approaches cover the space differently in simulation and reality: in reality, the robots tend to form clusters, mostly against the walls. Differences between simulation and reality are less pronounced for Chocolate and barely noticeable for RandomWalk. For each mission and each method, a direct comparison of the behavior in simulation and reality is available in Supplementary Movie 6.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 62 references

Research works printed but not linked: 8

  1. Berman, S., Kumar, V. & Nagpal, R. Design of control policies for spatially inhomogeneous robot swarms with application to commercial pollination in IEEE International Conference on Robotics and Automation, ICRA (pp. 378–385. IEEE, Piscataway, NJ, USA, 2011).
  2. Brambilla, M., Brutschy, A., Dorigo, M. & Birattari, M. Property-driven design for swarm robotics: a design method based on prescriptive modeling and model checking. ACM Tran. Auton. Adap. 9, 17:1–17:28 (2014).
  3. Gauci, M, Chen, J, Li, W, Dodd, T. J & Groß, R. Clustering Objects with Robots That Do Not Compute in Proceedings of the 2014 International Conference on Autonomous Agents and Multi-Agent Systems - AAMAS2014. (International Foundation for Autonomous Agents and Multiagent Systems: Richland, SC, USA, 2014) 421–428.
  4. Brooks, R. A. Artificial life and real robots in Towards a Practice of Autonomous Systems. Proceedings of the First European Conference on Artificial Life (eds. Varela, F. J. & Bourgine, P.) 3–10 (MIT Press, Cambridge, MA, USA, 1992).
  5. Caruana, R., Lawrence, S. & Giles, C. L. Overfitting in neural nets: backpropagation, conjugate gradient, and early stopping in Advances in Neural Information Processing Systems 13 (eds. Leen, T. K., Dietterich, T. G. & Tresp, V.) 402–408 (MIT Press, Cambridge, MA, USA, 2001).
  6. Morgan, N. & Bourlard, H. Generalization and Parameter Estimation in Feedforward Nets: Some Experiments in Proceedings of the 2nd International Conference on Neural Information Processing Systems, NIPS (ed. Touretzky, D.) 630–637 (MIT Press, Cambridge, MA, USA, 1989).
  7. Mondada, F. et al. The e-puck, a robot designed for education in engineering in Proceedings of the 9th Conference on Autonomous Robot Systems and Competitions (eds. Gonçalves, P., Torres, P. & Alves, C.) 59–65 (Instituto Politécnico de Castelo Branco, Castelo Branco, Portugal, 2009).
  8. Lunacek, M & Whitley, L. D. The Dispersion Metric and the CMA Evolution Strategy in Proceedings of the 8th annual conference on Genetic and Evolutionary Computation, GECCO. (ACM: New York, NY, USA, 2006) 477–484.

Cited by

See all 53

Metadata

Authors
Hasselmann, K., Ligot, A., Ruddick, J., & Birattari, M.
Affiliations
Université Libre de Bruxelles (BE)
Corresponding author
Birattari, Mauro
Venue
Nature Communications
Published
2021-07-16
Volume
12
Issue
1
Pages
4345-4345
ISSN
2041-1723 (ISSN-L)
OpenAlex
W3178918501
Cited by
53
References
62
Citations by year
2026: 10 · 2025: 11 · 2024: 14 · 2023: 5 · 2022: 8 · 2021: 5
FWCI
3.42
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature Communications (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • Dépôt institutionnel de l'Université libre de Bruxelles (Université Libre de Bruxelles) (submittedVersion, other-oa)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • European Commission (681872, HORIZON2020)
  • Fonds De La Recherche Scientifique - FNRS
Topics
  • Physical Sciences › Engineering › Mechanical Engineering › Modular Robots and Swarm Intelligence
  • Physical Sciences › Computer Science › Artificial Intelligence › Evolutionary Algorithms and Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Metaheuristic Optimization Algorithms Research
Keywords
neuroevolution, swarm robotics, reality gap, sim-to-real transfer, collective behavior, neural network controller, evolutionary algorithm comparison

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Empirical assessment and comparison of neuro-evolutionary methods for the automatic off-line design of robot swarms. https://app.cochat.ai/papers/10.1038/s41467-021-24642-3
The paper
Hasselmann, K., Ligot, A., Ruddick, J., & Birattari, M. (2021). Empirical assessment and comparison of neuro-evolutionary methods for the automatic off-line design of robot swarms. Nature Communications. https://doi.org/10.1038/s41467-021-24642-3

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.