Cited by 56Fire effects on ecosystemsFire Detection and Safety SystemsReinforcement Learning in Robotics

Using Spatial Reinforcement Learning to Build Forest Wildfire Dynamics Models From Satellite Images

Open access
Ganapathi Subramanian, Sriram, Crowley, Mark·2018·Frontiers in ICT

Abstract

Machine learning algorithms have increased tremendously in power in recent years but have yet to be fully utilized in many ecology and sustainable resource management domains such as wildlife reserve design, forest fire management and invasive species spread. One thing these domains have in common is that they contain dynamics that can be characterized as a Spatially Spreading Process (SSP) which requires many parameters to be set precisely to model the dynamics, spread rates and directional biases of the elements which are spreading. We present related work in Artificial Intelligence and Machine Learning for SSP sustainability domains including forest wildfire prediction. We then introduce a novel approach for learning in SSP domains using Reinforcement Learning (RL) where fire is the agent at any cell in the landscape and the set of actions the fire can take from a location at any point in time includes spreading North, South, East, West or not spreading. This approach inverts the usual RL setup since the dynamics of the corresponding Markov Decision Process (MDP) is a known function for immediate wildfire spread. Meanwhile, we learn an agent policy for a predictive model of the dynamics of a complex spatially-spreading process. Rewards are provided for correctly classifying which cells are on fire or not compared to satellite and other related data. We examine the behaviour of five RL algorithms on this problem: Value Iteration, Policy Iteration, Q-Learning, Monte Carlo Tree Search and Asynchronous Advantage Actor-Critic (A3C). We compare to a Gaussian process based supervised learning approach and discuss the relation of our approach to manually constructed, state-of-the-art methods from forest wildfire modelling. We also discuss the relation of our approach to manually constructed, state-of-the-art methods from forest wildfire modelling. We validate our approach with satellite image data of two massive wildfire events in Northern Alberta, Canada; the Fort McMurray fire of 2016 and the Richardson fire of 2011. The results show that we can learn predictive, agent-based policies as models of spatial dynamics using RL on readily available satellite images that other methods and have many additional advantages in terms of generalizability and interpretability.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3389/fict.2018.00006

Cites

31 of 31 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • RL is used to predict the spatial progression of fire, which helps firefighters decide when to let a fire burn and when to stop it.

    “In [266,267], RL is used to predict the spatial progression of fire.This helps firefighters decide when to let a fire burn and when to stop it [355].”

    From Tackling climate change with machine learning (2023) · cites this paper as [267]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

1 of the 32 references couldn’t be matched to a known paper.

Research works printed but not linked: 5

  1. WF12157 · DOI · Paper page
  2. Mcgregor, S., Houtman, R., Buckingham, H., Montgomery, C., Metoyer, R., and Dietterich, T. G. (2016). "Fast simulation for computational sustainability sequential decision making problems, " in Proceedings of the 4th International Conference on Computational Sustainability, New York, 5-7.
  3. Rasmussen, C. E., and Williams, C. K. (2006). Gaussian Processes for Machine Learning, Vol. 1. Cambridge: MIT Press.
  4. Subramanian, S. G., and Crowley, M. (2017). "Learning forest wildfire dynam- ics from satellite images using reinforcement learning, " in Conference on Reinforcement Learning and Decision Making (Ann Arbor, MI).
  5. Watkins, C. (1989). Learning from Delayed Rewards. PhD thesis, King's College, Cambridge, UK.

Cited by

Sorted by citations

Authors
Ganapathi Subramanian, S., & Crowley, M.
Affiliations
University of Waterloo (CA)
Corresponding author
Crowley, Mark
Venue
Frontiers in ICT
Published
2018-04-19
Volume
5
ISSN
2297-198X (ISSN-L)
OpenAlex
W2800336546
Cited by
56
References
32
Citations by year
2026: 2 · 2025: 6 · 2024: 8 · 2023: 9 · 2022: 12 · 2021: 12 · 2020: 4 · 2019: 3
FWCI
2.78
Type
journal article
Language
en
Open access
Locations
Frontiers in ICT (publishedVersion, cc-by)
Funders
  • Mitacs
  • University of Waterloo
Topics
  • Physical Sciences › Environmental Science › Global and Planetary Change › Fire effects on ecosystems
  • Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Fire Detection and Safety Systems
  • Physical Sciences › Computer Science › Artificial Intelligence › Reinforcement Learning in Robotics
Keywords
wildfire spread prediction, satellite imagery, Markov decision process, wildfire management, Gaussian process comparison

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review