CoChat
Sign Up
Cited by 19Insect Pheromone Research and ControlDiffusion and Search DynamicsReinforcement Learning in Robotics

Optimal policies for Bayesian olfactory search in turbulent flows

Open access
Heinonen, R. A., Biferale, L., Celani, A., Vergassola, M.·2023·Physical Review E
DOI 10.1103/physreve.107.055105PMC PMC12165250
Open this paper in CoChat — free accountOpen PDF

Abstract

In many practical scenarios, a flying insect must search for the source of an emitted cue which is advected by the atmospheric wind. On the macroscopic scales of interest, turbulence tends to mix the cue into patches of relatively high concentration over a background of very low concentration, so that the insect will detect the cue only intermittently and cannot rely on chemotactic strategies which simply climb the concentration gradient. In this work we cast this search problem in the language of a partially observable Markov decision process and use the Perseus algorithm to compute strategies that are near-optimal with respect to the arrival time. We test the computed strategies on a large two-dimensional grid, present the resulting trajectories and arrival time statistics, and compare these to the corresponding results for several heuristic strategies, including (space-aware) infotaxis, Thompson sampling, and QMDP. We find that the near-optimal policy found by our implementation of Perseus outperforms all heuristics we test by several measures. We use the near-optimal policy to study how the search difficulty depends on the starting location. We also discuss the choice of initial belief and the robustness of the policies to changes in the environment. Finally, we present a detailed and pedagogical discussion about the implementation of the Perseus algorithm, including the benefits-and pitfalls-of employing a reward-shaping function.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1103/physreve.107.055105

Cites

50 of 50 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • By employing Pontryagin's minimum principle on the probability density function space, the solution satisfying the optimality condition δJ/δu = 0 is given by Eq. (S4).

    “By employing Pontryagin's minimum principle on the probability density function space [1-9], the solution that satisfies the optimality condition δ ˜ J/δ ˜ u = 0 is given by”

    From Theoretical Analysis of Resource-Induced Phase Transitions in Estimation Strategies (2026) · cites this paper as [6]

    The citation is attached to the use of Pontryagin’s minimum principle on probability-density-function space, not specifically to the citing paper’s Eq. (S4). The cited paper’s Methods (POMDP setup and POMDP solution; page unavailable) describe a Bellman equation and the Perseus value-iteration algorithm. Searches for Pontryagin’s principle and the stated variational optimality condition found no passage addressing that method. (LLM, conf=0.76)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

1 of the 51 references couldn’t be matched to a known paper.

Research works printed but not linked: 6

  1. [28]Kuwana Y, Nagasawa S, Shimoyama I, and Kanzaki R, Synthesis of the pheromone-oriented behaviour of silkworm moths by a mobile robot with moth antennae as pheromone sensors, Biosens. Bioelectron 14, 195 (1999).
  2. [33]Smoluchowski M. v., Versuch einer mathematischen Theorie der Koagulationskinetik kolloider Lösungen, Z. Phys. Chem 92, 129 (1918).
  3. [37]Cassandra AR, Exact and approximate algorithms for partially observable Markov decision processes, Ph.D. thesis, Brown University, 1998.
  4. [44]Osband I, Van Roy B, Russo DJ, and Wen Z, Deep exploration via randomized value functions, J. Mach. Learn. Res 20, 1 (2019).
  5. [50]Shani G, Brafman RI, and Shimony SE, Prioritizing point-based POMDP solvers, in European Conference on Machine Learning, edited by Fürnkranz J, Scheffer T, and Spiliopoulou M (Springer, Berlin, Heidelberg, 2006), pp. 389– 400.
  6. [53]Smith T and Simmons R, Point-based POMDP algorithms: Improved analysis and implementation, in UAI’05 Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence, edited by Bacchus F and Jaakkola T (Arlington AUAI Press, 2005), pp. 542– 549.

Cited by

Sorted by citations

Related

Authors
Heinonen, R. A., Biferale, L., Celani, A., & Vergassola, M.
Affiliations
  • University of Rome Tor Vergata (IT)
  • The Abdus Salam International Centre for Theoretical Physics (ICTP) (IT)
  • Centre National de la Recherche Scientifique (FR)
ORCID
Venue
Physical Review E
Publisher
American Physical Society (APS)
Published
2023-05-16
Volume
107
Issue
5
Pages
055105-055105
ISSN
  • 2470-0045 (ISSN-L)
  • 2470-0053
DOI
10.1103/physreve.107.055105
PMC
PMC12165250
OpenAlex
W4376640537
Cited by
19
References
51
Citations by year
2026: 5 · 2025: 7 · 2024: 5 · 2023: 2
FWCI
3.77
Type
journal article
Language
en
Open access
Licence
link.aps.org/licenses/aps-default-license (vor)
Locations
  • Physical review. E (publishedVersion)
  • PubMed (publishedVersion)
  • Europe PMC (PubMed Central) (acceptedVersion)
  • PubMed Central (acceptedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • HAL (submittedVersion, other-oa)
  • Cineca Institutional Research Information System (Tor Vergata University) (submittedVersion)
  • Institutional Research Information System University of Turin (University of Turin) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • European Research Council (882340)
  • Horizon 2020 (882340)
  • H2020 Marie Skłodowska-Curie Actions (956457)
Topics
  • Life Sciences › Agricultural and Biological Sciences › Insect Science › Insect Pheromone Research and Control
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Diffusion and Search Dynamics
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
Keywords
olfactory search, insect navigation, turbulent advection, partially observable Markov decision processes, infotaxis, Thompson sampling, reward shaping, time of arrival, initial beliefs, search difficulty
MeSH terms
Algorithms, Animals, Bayes Theorem, Insecta, Smell, Policy

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review