CoChat
Sign Up
PreprintCited by 5,137Advanced Neural Network ApplicationsAdvanced Image and Video Retrieval TechniquesDomain Adaptation and Few-Shot Learning

Spatial transformer networks

Open access
Jaderberg, Max, Simonyan, Karen, Zisserman, Andrew, Kavukcuoglu, Koray·2015·arXiv
DOI 10.48550/arxiv.1506.02025arXiv 1506.02025
Open this paper in CoChat — free accountOpen PDF

Abstract

Convolutional Neural Networks define an exceptionally powerful class of models, but are still limited by the lack of ability to be spatially invariant to the input data in a computationally and parameter efficient manner. In this work we introduce a new learnable module, the Spatial Transformer, which explicitly allows the spatial manipulation of data within the network. This differentiable module can be inserted into existing convolutional architectures, giving neural networks the ability to actively spatially transform feature maps, conditional on the feature map itself, without any extra training supervision or modification to the optimisation process. We show that the use of spatial transformers results in models which learn invariance to translation, scale, rotation and more generic warping, resulting in state-of-the-art performance on several benchmarks, and for a number of classes of transformations.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1506.02025

Cites

6 of 6 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • STN can be inserted anywhere in various networks to execute a spatial transform on an input feature map because it is a completely differentiable module.

    “STN can be inserted anywhere in various networks to execute a spatial transform on an input feature map because it is a completely differentiable module.”

    From A review of deep learning-based deformable medical image registration (2022) · cites this paper as [80]

  • Spatial transformer networks (STNs) introduced in 2015 are one significant advancement beneficial in this era.

    “The spatial transformer networks (STNs) (80) introduced in 2015 is one significant advancement that is beneficial in this era.”

    From A review of deep learning-based deformable medical image registration (2022) · cites this paper as [80]

  • RTNs are a form of learned feed-forward attention inspired by Spatial Transformer Networks (STNs).

    “RTNs are a form of learned feed-forward attention inspired by Spatial Transformer Networks (STNs) [51] which have worked well for computer vision problems.”

    From An Introduction to Deep Learning for the Physical Layer (2017) · cites this paper as [51]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 33

  1. [1][1] J. Ba, V. Mnih, and K. Kavukcuoglu. Multiple object recognition with visual attention. ICLR , 2015. -
  2. [2][2] F. Bookstein. Principal warps : Thin-plate splines and the decomposition of deformations. IEEE PAMI , 1989. -
  3. [3][3] S. Branson, G. Van Horn, S. Belongie, and P. Perona. Bird species categorization using pose normalized deep convolutional nets. BMVC. , 2014. -
  4. [4][4] J. Bruna and S. Mallat. Invariant scattering convolution networks. IEEE PAMI , 35(8):1872-1886, 2013. -
  5. [5][5] M. Cimpoi, S. Maji, and A. Vedaldi. Deep filter banks for texture recognition and segmentation. In CVPR , 2015. -
  6. [6][6] T. S. Cohen and M. Welling. Transformation properties of learned visual representations. ICLR , 2015. -
  7. [8][8] J. D. Foley, A. Van Dam, S. K. Feiner, J. F. Hughes, and R. L. Phillips. Introduction to computer graphics , volume 55. Addison-Wesley Reading, 1994. -
  8. [9][9] B. J. Frey and N. Jojic. Fast, large-scale transformation-invariant clustering. In NIPS , 2001. -
  9. [10][10] R. Gens and P. M. Domingos. Deep symmetry networks. In NIPS , 2014. -
  10. [11][11] R. B. Girshick, J. Donahue, T. Darrell, and J. Malik. Rich feature hierarchies for accurate object detection and semantic segmentation. In CVPR , 2014. -
  11. [12][12] G. Gkioxari, R. Girshick, and J. Malik. Contextual action recognition with r* cnn. arXiv:1505.01197 , 2015. - · arXiv
  12. [14][14] K. Gregor, I. Danihelka, A. Graves, and D. Wierstra. Draw: A recurrent neural network for image generation. ICML , 2015. -
  13. [15][15] G. E. Hinton. A parallel computation that assigns canonical object-based frames of reference. In IJCAI , 1981. -
  14. [17][17] G. E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. Improving neural networks by preventing co-adaptation of feature detectors. CoRR , abs/1207.0580, 2012. - · arXiv
  15. [19][19] M. Jaderberg, K. Simonyan, A. Vedaldi, and A. Zisserman. Synthetic data and artificial neural networks for natural scene text recognition. NIPS DLW , 2014. -
  16. [20][20] A. Kanazawa, A. Sharma, and D. Jacobs. Locally scale-invariant convolutional neural networks. In NIPS , 2014. -
  17. [21][21] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE , 86(11):2278-2324, 1998. -
  18. [22][22] K. Lenc and A. Vedaldi. Understanding image representations by measuring their equivariance and equivalence. CVPR , 2015. -
  19. [23][23] T. Lin, A. RoyChowdhury, and S. Maji. Bilinear CNN models for fine-grained visual recognition. arXiv:1504.07889 , 2015. - · arXiv
  20. [24][24] J. Long, E. Shelhamer, and T. Darrell. Fully convolutional networks for semantic segmentation. In CVPR , 2015. -
  21. [25][25] Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, and A. Y. Ng. Reading digits in natural images with unsupervised feature learning. In NIPS DLW , 2011. -
  22. [26][26] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein, et al. Imagenet large scale visual recognition challenge. arXiv:1409.0575 , 2014. - · arXiv
  23. [27][27] J. Schmidhuber and R. Huber. Learning to generate artificial fovea trajectories for target detection. International Journal of Neural Systems , 2(01n02):125-134, 1991. -
  24. [29][29] P. Sermanet, A. Frome, and E. Real. Attention for fine-grained categorization. arXiv:1412.7054 , 2014. - · arXiv
  25. [30][30] M. Simon and E. Rodner. Neural activation constellations: Unsupervised part model discovery with convolutional networks. arXiv:1504.08289 , 2015. - · arXiv
  26. [31][31] K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. ICLR , 2015. -
  27. [32][32] Karen Simonyan and Andrew Zisserman. Two-stream convolutional networks for action recognition in videos. In NIPS , pages 568-576, 2014. -
  28. [33][33] K. Sohn and H. Lee. Learning invariant representations with local transformations. arXiv:1206.6418 , 2012. - · arXiv
  29. [34][34] M. F. Stollenga, J. Masci, F. Gomez, and J. Schmidhuber. Deep networks with internal selective attention through feedback connections. In NIPS , 2014. -
  30. [35][35] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. Going deeper with convolutions. CVPR , 2015. -
  31. [36][36] T. Tieleman. Optimizing Neural Networks that Generate Images . PhD thesis, University of Toronto, 2014. -
  32. [37][37] J. J. Tompson, A. Jain, Y. LeCun, and C. Bregler. Joint training of a convolutional network and a graphical model for human pose estimation. In NIPS , pages 1799-1807, 2014. -
  33. [40][40] Ning Zhang, Jeff Donahue, Ross Girshick, and Trevor Darrell. Part-based r-cnns for fine-grained category detection. In Computer Vision-ECCV 2014 , pages 834-849. Springer, 2014.

Cited by

Sorted by citations

Authors
Jaderberg, M., Simonyan, K., Zisserman, A., & Kavukcuoglu, K.
Affiliations
Google DeepMind (United Kingdom) (GB)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2015
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.1506.02025
arXiv
1506.02025
OpenAlex
W2951005624
Cited by
5,137
References
6
Citations by year
2026: 18 · 2025: 320 · 2024: 727 · 2023: 774 · 2022: 733 · 2021: 790 · 2020: 597 · 2019: 630 · 2018: 308 · 2017: 182 · 2016: 48 · 2015: 9
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
Keywords
spatial transformer network, spatial transformation, spatial invariance, translation invariance, scale invariance, rotation invariance, convolutional neural networks, parameter efficiency

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review