Cited by 746Remote-Sensing Image ClassificationAdvanced Neural Network ApplicationsRemote Sensing and Land Use

Comprehensive survey of deep learning in remote sensing: theories, tools, and challenges for the community

Open access
Ball, John E., Anderson, Derek T., Chan, Chee Seng·2017·Journal of Applied Remote Sensing

Abstract

In recent years, deep learning (DL), a rebranding of neural networks (NNs), has risen to the top in numerous areas, namely computer vision (CV), speech recognition, and natural language processing. Whereas remote sensing (RS) possesses a number of unique challenges, primarily related to sensors and applications, inevitably RS draws from many of the same theories as CV, e.g., statistics, fusion, and machine learning, to name a few. This means that the RS community should not only be aware of advancements such as DL, but also be leading researchers in this area. Herein, we provide the most comprehensive survey of state-of-the-art RS DL research. We also review recent new developments in the DL field that can be used in DL for RS. Namely, we focus on theories, tools, and challenges for the RS community. Specifically, we focus on unsolved challenges and opportunities as they relate to (i) inadequate data sets, (ii) human-understandable solutions for modeling physical phenomena, (iii) big data, (iv) nontraditional heterogeneous data sources, (v) DL architectures and learning algorithms for spectral, spatial, and temporal data, (vi) transfer learning, (vii) an improved theoretical understanding of DL systems, (viii) high barriers to entry, and (ix) training and optimizing the DL.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1117/1.jrs.11.042609

Cites

289 of 289 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The first DL-related special issue appeared in the SPIE Journal of Applied Remote Sensing, followed by the 'Deep Learning for Remotely Sensed Data' special issue of the ISPRS Journal of Photogrammetry and Remote Sensing in 2018.

    “Another notable finding was that the first DL-related special issue appeared in the SPIE Journal of Applied Remote Sensing (Ball et al., 2017), followed by the “Deep Learning for Remotely Sensed Data” special issue of the ISPRS Journal of Photogrammetry and Remote Sensing in 2018.”

    From Deep learning in remote sensing applications: A meta-analysis and review (2019) · cites this paper as Ball et al., 2017

    The paper’s journal footer (Vol. 11, no. 4, 2017; reproduced under Conclusions) establishes that this deep-learning survey appeared in the Journal of Applied Remote Sensing. Our copy does not establish that it appeared in a special issue or that the issue was the first DL-related one (searched: “special issue” and “special section”). (LLM, conf=0.76)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Part of this paper’s reference list could not be split into single entries.

Flagged first, then by citations

Research works printed but not linked: 33

  1. G. Jianya et al., "A review of multi-temporal remote sensing data change detection algo- rithms," Int. Arch. Photogramm. Remote Sens. Spatial Inf. Sci. 37(B7), 757-762 (2008).
  2. N. Keshava, "A survey of spectral unmixing algorithms," Lincoln Lab. J. 14(1), 55-78 (2003).
  3. M. D. Zeiler, G. W. Taylor, and R. Fergus, "Adaptive deconvolutional networks for mid and high level feature learning," in 2011 Int. Conf. on Computer Vision, pp. 2018-2025 (2011).
  4. M. D. Zeiler and R. Fergus, "Visualizing and understanding convolutional networks," in European Conf. on Computer Vision, pp. 818-833, Springer (2014).
  5. I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning, MIT Press, Cambridge, Massachusetts (2016).
  6. D. Arpit et al., "Why regularized auto-encoders learn sparse representation?," in Int. Conf. on Machine Learning, pp. 136-144 (2016).
  7. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet classification with deep con- volutional neural networks," in Advances in Neural Information Processing Systems, G. Tesauro, D. Touretzky, and T. Leed, Eds., pp. 1097-1105 (2012).
  8. R. Salakhutdinov and G. Hinton, "Deep Boltzmann machines," in Proc. of the Twelfth Int. Conf. on Artificial Intelligence and Statistics (AISTATS'09), Vol. 5, pp. 448-455 (2009).
  9. Q. V. Le, "A tutorial on deep learning part 2: autoencoders, convolutional neural networks and recurrent neural networks," Google Brain, pp. 1-20 (2015).
  10. O. Sharir and A. Shashua, "On the expressive power of overlapping operations of deep networks," arXiv:1703.02065 (2017). · arXiv
  11. T. Tieleman and G. Hinton, "Lecture 6.5-rmsprop: divide the gradient by a running average of its recent magnitude," COURSERA: Neural Networks Mach. Learn. 4(2), 26-31 (2012).
  12. R. R. Salakhutdinov and G. E. Hinton, "A better way to pretrain deep Boltzmann machines," in Advances in Neural Information Processing Systems, F. Pereira et al., Eds., pp. 2447-2455, Curran Associates, Inc. (2012).
  13. R. Pascanu, T. Mikolov, and Y. Bengio, "On the difficulty of training recurrent neural networks," in Int. Conf. on Machine Learning, pp. 1310-1318 (2013).
  14. J. Martens and I. Sutskever, "Learning recurrent neural networks with Hessian-free optimization," in Proc. of the 28th Int. Conf. on Machine Learning (ICML 2011), pp. 1033-1040 (2011).
  15. Y. Bengio, A. C. Courville, and P. Vincent, "Unsupervised feature learning and deep learning: a review and new perspectives," CoRR, abs/1206.55381 (2012).
  16. R. Raina, A. Madhavan, and A. Y. Ng, "Large-scale deep unsupervised learning using graphics processors," in Proc. of the 26th Annual Int. Conf. on Machine Learning (ICML 2009), pp. 873-880, ACM, New York (2009).
  17. D. Scherer, A. Müller, and S. Behnke, "Evaluation of pooling operations in convolutional architectures for object recognition," in Proc. of the 20th Int. Conf. on Artificial Neural Networks: Part III (ICANN 2010), pp. 92-101, Springer-Verlag, Berlin, Heidelberg (2010).
  18. J. Dean et al., "Large scale distributed deep networks," in Proc. of the 25th Int. Conf. on Neural Information Processing Systems (NIPS 2012), pp. 1223-1231, Curran Associates Inc., USA (2012).
  19. R. K. Srivastava, K. Greff, and J. Schmidhuber, "Highway networks," CoRR abs/ 1505.00387 (2015).
  20. J. G. Zilly et al., "Recurrent highway networks," CoRR abs/1607.03474 (2016). · arXiv
  21. J. Wan et al., "Deep learning for content-based image retrieval: a comprehensive study," in Proc. of the 22nd ACM Int. Conf. on Multimedia, pp. 157-166 (2014).
  22. Y. Jia et al., "Caffe: convolutional architecture for fast feature embedding," in ACM Int. Conf. on Multimedia, pp. 675-678 (2014).
  23. M. Abadi et al., "TensorFlow: a system for large-scale machine learning," in Proc. of the 12th USENIX Symp. on Operating Systems Design and Implementation (OSDI), Savannah, Georgia (2016).
  24. A. Vedaldi and K. Lenc, "MatConvNet: convolutional neural networks for MATLAB," in Proc. of the 23rd ACM Int. Conf. on Multimedia, pp. 689-692, ACM (2015).
  25. A. Handa et al., "GVNN: neural network library for geometric computer vision," in Computer Vision Workshop (ECCV 2016), pp. 67-82, Springer International (2016).
  26. R. Al-Rfou et al., "Theano: a Python framework for fast computation of mathematical expressions," arXiv:1605.02688 (2016). · arXiv
  27. M. Baccouche et al., "Sequential deep learning for human action recognition," in Int. Workshop on Human Behavior Understanding, pp. 29-39, Springer (2011).
  28. D. Tran et al., "Learning spatiotemporal features with 3D convolutional networks," in IEEE Int. Conf. on Computer Vision (ICCV 2015), pp. 4489-4497 (2015).
  29. L. Wang et al., "Temporal segment networks: towards good practices for deep action recognition," in European Conf. on Computer Vision, pp. 20-36 (2016).
  30. S. Levine et al., "Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection," arXiv:1603.02199 (2016). · arXiv
  31. John E. Ball is an assistant professor of electrical and computer engineering at Mississippi State University (MSU), USA. He received his PhD from MSU in 2007. He is a codirector of the Sensor Analysis and Intelligence Laboratory (SAIL) at MSU. He has authored more than 45 articles, and 22 technical and reports. His research interests are deep learning, remote sensing, and signal/image processing. He is an associate editor for the SPIE Journal of Applied Remote Sensing.
  32. Ball, Anderson, and Chan: Comprehensive survey of deep learning in remote sensing: theories. . . Journal of Applied Remote Sensing 042609-53 Oct-Dec 2017 • Vol. 11(4) Downloaded From: https://www.spiedigitallibrary.org/journals/Journal-of-Applied-Remote-Sensing on 21 Jul 2023 Terms of Use: https://www.spiedigitallibrary.org/terms-of-use
  33. Derek T. Anderson received his PhD from the University of Missouri in 2010. He is an asso- ciate professor and the Robert D. Guyton Chair in ECE at Mississippi State University, with the Naval Research Laboratory, and codirects the Sensor Analysis and Intelligence Laboratory. His research includes data/information fusion for machine learning and automated decision making in computer vision. He has published more than 110 articles, and he is an AE for IEEE TFS and program cochair of FUZZ-IEEE 2019.

Cited by

Sorted by citations

Authors
Ball, J. E., Anderson, D. T., & Chan, C. S.
Affiliations
  • Mississippi State University (US)
  • National University of Singapore (SG)
Venue
Journal of Applied Remote Sensing
Published
2017-09-23
Volume
11
Issue
04
Pages
1-54
ISSN
1931-3195 (ISSN-L)
OpenAlex
W2751694392
Cited by
746
References
289
Citations by year
2026: 51 · 2025: 85 · 2024: 70 · 2023: 107 · 2022: 94 · 2021: 119 · 2020: 106 · 2019: 80 · 2018: 34
FWCI
49.88
Type
journal article
Language
en
Open access
Locations
  • Journal of Applied Remote Sensing (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Engineering › Media Technology › Remote-Sensing Image Classification
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Earth and Planetary Sciences › Atmospheric Science › Remote Sensing and Land Use
Keywords
deep learning, remote sensing, artificial neural networks, heterogeneous data sources, transfer learning, big data, physical modeling, barriers to entry

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review