Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 113 of the 113 references it cites were found in a registry or an index; 16 of the 49 sources its claims cite were checked for support.

Emergent behaviour and neural dynamics in artificial agents tracking odour plumes

Open access
Singh, Satpreet H., van Breugel, Floris, Rao, Rajesh P. N., Brunton, Bingni W. · 2023 · Nature Machine Intelligence

Abstract

Tracking an odour plume to locate its source under variable wind and plume statistics is a complex task. Flying insects routinely accomplish such tracking, often over long distances, in pursuit of food or mates. Several aspects of this remarkable behaviour and its underlying neural circuitry have been studied experimentally. Here we take a complementary in silico approach to develop an integrated understanding of their behaviour and neural computations. Specifically, we train artificial recurrent neural network agents using deep reinforcement learning to locate the source of simulated odour plumes that mimic features of plumes in a turbulent flow. Interestingly, the agents’ emergent behaviours resemble those of flying insects, and the recurrent neural networks learn to compute task-relevant variables with distinct dynamic structures in population activity. Our analyses put forward a testable behavioural hypothesis for tracking plumes in changing wind direction, and we provide key intuitions for memory requirements and neural dynamics in odour plume tracking.

Overview

CoChat Summary

TL;DR

The paper trains recurrent neural network agents with deep reinforcement learning to track simulated turbulent odour plumes and finds emergent behaviours and neural representations resembling those of flying insects. It reports that agents track the plume centreline rather than current wind direction, use short-timescale memory for constant wind but longer memory for switching winds, and exhibit low-dimensional population dynamics organized into distinct regimes.

Key findings

  • DRL-trained RNN agents successfully localize plume sources across constant, switch-once, switch-many, sparse and sparser plume configurations; their behaviour decomposes into three modules—tracking, lost and recovering—based on time since last odour encounter (tracking encounters under 0.5 s; recovering about 0.5 s; lost typically over 1 s).
  • Agents track the plume centreline rather than the current wind direction: course-direction distributions align better with the centreline than with wind, and in switch-once plumes the peak is closer to ±180° relative to centreline versus ≈45° relative to wind; this trend holds across all five RNN agents.
  • RNN population activity is low dimensional, with the first five to eight principal components explaining 90% of variance in the 64-dimensional population activity (Fig. 4g shows first five PCs). The networks represent head direction, time since last odour, an exponentially weighted moving average of odour concentration (window ≈0.3 s, R² = 0.91) and of odour encounters (window ≈1.9 s, R² = 0.59); classifiers using these represented features plus instantaneous sensory inputs are 10–18% more accurate than instantaneous-only classifiers and 26–51% more accurate than a majority-class classifier, with time since last encounter always among the top two most important features.
  • Neural dynamics organize into overlapping regimes rather than fixed points: tracking shows funnel-like structures, lost shows quasiperiodic limit cycles, and recovering shows an amorphous transition region; the same approximate structures emerge in four of five RNN agents. Training produces unstable eigenvalues outside the unit circle, with at least one strictly real eigenvalue larger than unity for all five agents; the bulk of stimulus integration timescales lie within about 12 time steps (≈0.5 s), and RNNs outperform feedforward MLPs on every plume tracking task, with the largest gains on switching-wind and sparse tasks.

Methods

  • Plume simulation: a particle-based two-dimensional model with a spatially homogeneous wind field (0.5 m s⁻¹, configurable direction), an odour source at the origin emitting puffs as a Poisson process, radial diffusion, downwind advection and random cross-wind perturbation. The arena spans [−2 m, +10 m] × [−5 m, +5 m] (120 m²), simulated at 100 iterations per second. Wind configurations include constant, switch-once (one 45° anticlockwise switch), switch-many (switches about every 3 s), sparse (0.4-fold puff birth rate) and sparser (0.5-fold radial diffusion plus sparse).
  • Agents and training: actor-critic networks with a 64-unit tanh RNN receiving three-dimensional egocentric inputs (wind x, y and odour concentration) and producing continuous turn (max ±6.25π rad s⁻¹) and forward-movement (max 2.5 m s⁻¹) actions; actor and critic heads are two-layer MLPs. Agents are trained with PPO; 14 independently initialized networks are trained per architecture (RNNs and MLPs with 2, 4, 6, 8, 10 or 12 time steps of sensory history). Evaluation uses 240 episodes per agent (15 initial locations, two initial simulation timestamps, eight initial head directions) across constant, switch-once and switch-many configurations; only the top five seeds per architecture are analysed.
  • Analyses: PCA on RNN population activity; random forest classifiers predicting discretized actions, with permutation importance scores over N = 30 randomized permutations; linear regression to determine moving-average window sizes for odour concentration and encounters; eigenvalue spectra of the recurrence matrix and stimulus integration timescales from the recurrence Jacobian; and performance comparison between RNNs and feedforward MLPs.

Limitations

  • Stated: the plume simulator is computationally efficient but approximate and does not capture some real-plume aspects such as filamentous nature or variation of whiff duration and frequency with distance from source. The study uses vanilla recurrent units with no biomechanical body model, no spiking neurons and no biologically inspired connectivity constraints; future work could add spiking networks, connectome-derived constraints, multiple antennae, biomechanical bodies, multitask training, other sensory modalities and 3D virtual reality.
  • Stated: most results describe one agent chosen at random from the top five performers of 14 trained agents, with some analyses across five selected agents; notable performance variability was observed. Behaviour-module thresholds were determined by visual inspection, and classifier-based importance estimates are approximate due to action discretization and correlations between features.
  • Stated: current reverse-engineering methods that characterize RNNs using discrete dynamical features such as fixed points are not applicable to the continuous and amorphous dynamical structures encountered here; new methods are needed for comparing multiple agents at the behavioural level and for understanding training-induced unstable RNN connectivity eigenmodes.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s42256-022-00599-w

Cites

113 of 113 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

12 Valid · 1 Not valid · 18 Not checked · 104 Own results and interpretations

77 more claims

Create a free account to see all 135 claims and to verify the 2 references still open.

See all claims — free account
  • Probably not a substantial claim. 31
  • It may state two separate claims as one. 29

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

2 of 2

Citations

Cites

Show all 90 references

Research works printed but not linked: 7

  1. Kanitscheider, I. & Fiete, I. Training recurrent networks to generate hypotheses about how the brain solves hard navigation problems. In Proc. 31st International Conference on Neural Information Processing Systems (eds von Luxburg, U. et al.) 4532–4541 (Curran Associates Inc., 2017).
  2. Sutton, R. S. & Barto, A. G. Reinforcement Learning: An Introduction (MIT Press, 2018).
  3. Konda, V. R. & Tsitsiklis, J. N. Actor–critic algorithms. Adv. Neural Inf. Process. Syst. 12, 1008–1014 (1999).
  4. Baker, T. C. Upwind flight and casting flight: complementary phasic and tonic systems used for location of sex pheromone sources by male moth. In Proc. 10th International Symposium on Olfaction and Taste (ed. Doving, K. B.) 18–25 (1990).
  5. Stachenfeld, K. et al. Learned simulators for turbulence. In International Conference on Learning Representations (2021).
  6. Duncker, L., Driscoll, L., Shenoy, K. V., Sahani, M. & Sussillo, D. Organizing recurrent network dynamics by task-computation to enable continual learning. In Proc. 34th International Conference on Neural Information Processing Systems (eds Larochelle, H. et al.) 14387–14397 (Curran Associates Inc., 2020).
  7. Henaff, M., Szlam, A. & Le Cun, Y. Recurrent orthogonal networks and long-memory tasks. In Proc. 33rd International Conference on Machine Learning (eds Balcan, M. F. & Weinberger, K. Q.) 2034–2042 (JMLR, 2016).

Cited by

See all 61

Metadata

Authors
Singh, S. H., van Breugel, F., Rao, R. P. N., & Brunton, B. W.
Affiliations
University of Washington (US)
Corresponding author
Singh, Satpreet H.
Venue
Nature Machine Intelligence
Published
2023-01-25
Volume
5
Issue
1
Pages
58-70
ISSN
2522-5839 (ISSN-L)
OpenAlex
W4317952808
Cited by
61
References
90
Citations by year
2026: 12 · 2025: 25 · 2024: 17 · 2023: 7
FWCI
7.38
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature Machine Intelligence (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion, cc-by)
Funders
  • United States Department of Defense | United States Air Force | AFMC | Air Force Research Laboratory (FA8651-20-1-0002, FA9550-21-0122)
  • United States Department of Defense | Defense Advanced Research Projects Agency (HR001120C0021)
  • National Science Foundation (EEC-1028725)
  • United States Department of Defense | United States Air Force | AFMC | Air Force Office of Scientific Research (FA9550-19-1-0386, FA9550-18-1-0114)
  • Washington Research Foundation
Topics
  • Life Sciences › Neuroscience › Cellular and Molecular Neuroscience › Neurobiology and Insect Physiology Research
  • Life Sciences › Agricultural and Biological Sciences › Insect Science › Insect Pheromone Research and Control
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
Keywords
artificial agents, recurrent neural networks, deep reinforcement learning, emergent behavior, neural dynamics, population activity, turbulent flow, wind direction change, insect navigation, memory requirements

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Emergent behaviour and neural dynamics in artificial agents tracking odour plumes. https://app.cochat.ai/papers/10.1038/s42256-022-00599-w
The paper
Singh, S. H., van Breugel, F., Rao, R. P. N., & Brunton, B. W. (2023). Emergent behaviour and neural dynamics in artificial agents tracking odour plumes. Nature Machine Intelligence. https://doi.org/10.1038/s42256-022-00599-w

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.