PreprintCited by 246Parallel Computing and Optimization TechniquesAdvanced Neural Network ApplicationsDigital Filter Design and Implementation

Fast Convolutional Nets With fbfft: A GPU Performance Evaluation

Open access
Vasilache, Nicolas, Johnson, Jeff, Mathieu, Michael, Chintala, Soumith, Piantino, Serkan, LeCun, Yann·2014·arXiv

Abstract

We examine the performance profile of Convolutional Neural Network training on the current generation of NVIDIA Graphics Processing Units. We introduce two new Fast Fourier Transform convolution implementations: one based on NVIDIA's cuFFT library, and another based on a Facebook authored FFT implementation, fbfft, that provides significant speedups over cuFFT (over 1.5x) for whole CNNs. Both of these convolution implementations are available in open source, and are faster than NVIDIA's cuDNN implementation for many common convolutional layers (up to 23.5x for some synthetic kernel configurations). We discuss different performance regimes of convolutions, comparing areas where straightforward time domain convolutions outperform Fourier frequency domain convolutions. Details on algorithmic applications of NVIDIA GPU hardware specifics in the implementation of fbfft are also provided.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1412.7580

Cites

13 of 14 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The convolution calculation can be accelerated by using the Fast Fourier Transform and the Inverse FFT.

    “The above calculation can be accelerated by using the Fast Fourier Transform (FFT) and the Inverse FFT (IFFT) [160-163].”

    From Object Detection in 20 Years: A Survey (2023) · cites this paper as [162]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 8

  1. Bergstra, James, Breuleux, Olivier, Bastien, Fr´ ed´ eric, Lamblin, Pascal, Pascanu, Razvan, Desjardins, Guillaume, Turian, Joseph, Warde-Farley, David, and Bengio, Yoshua. Theano: a CPU and GPU math expression compiler. In Proceedings of the Python for Scientific Computing Conference (SciPy) , June 2010. Oral Presentation.
  2. Bluestein, Leo I. A linear filtering approach to the computation of discrete Fourier transform. Audio and Electroacoustics, IEEE Transactions on , 18(4):451-455, December 1970. ISSN 0018-9278.
  3. Brosch, Tom and Tam, Roger C. Efficient training of convolutional deep belief networks in the frequency domain for application to high-resolution 2d and 3d images. Neural Computation , 27 (1):211-227, 2015. doi: 10.1162/NECO a 00682. URL http://dx.doi.org/10.1162/ NECO\_a\_00682 . · DOI
  4. Chetlur, Sharan, Woolley, Cliff, Vandermersch, Philippe, Cohen, Jonathan, Tran, John, Catanzaro, Bryan, and Shelhamer, Evan. cudnn: Efficient primitives for deep learning. CoRR , abs/1410.0759, 2014. URL http://arxiv.org/abs/1410.0759 . · arXiv
  5. Krizhevsky, Alex, Sutskever, Ilya, and Hinton, Geoffrey E. Imagenet classification with deep convolutional neural networks. In Pereira, F., Burges, C.J.C., Bottou, L., and Weinberger, K.Q. (eds.), Advances in Neural Information Processing Systems 25 , pp. 1097-1105. Curran Associates, Inc., 2012. URL http://papers.nips.cc/paper/4824-imagenetclassification-with-deep-convolutional-neural-networks.pdf .
  6. Lavin, Andrew. maxdnn: An efficient convolution kernel for deep learning with maxwell gpus. CoRR , abs/1501.06633, 2015. URL http://arxiv.org/abs/1501.06633 . · arXiv
  7. Sermanet, Pierre, Eigen, David, Zhang, Xiang, Mathieu, Michael, Fergus, Rob, and LeCun, Yann. Overfeat: Integrated recognition, localization and detection using convolutional networks. In International Conference on Learning Representations (ICLR 2014) . CBLS, April 2014. URL http://openreview.net/document/d332e77d-459a-4af8-b3ed-55ba .
  8. Volkov, V. Better performance at lower occupancy. In GPU Technology Conference , 2010. URL http://www.cs.berkeley.edu/˜ volkov/volkov10-GTC.pdf .

Cited by

Sorted by citations

Authors
Vasilache, N., Johnson, J., Mathieu, M., Chintala, S., Piantino, S., & LeCun, Y.
Affiliations
Meta (United States) (US)
Venue
arXiv
Publisher
arXiv
Published
2014
ISSN
2331-8422 (ISSN-L)
OpenAlex
W1789336918
Cited by
246
References
14
Citations by year
2025: 2 · 2024: 9 · 2023: 14 · 2022: 13 · 2021: 33 · 2020: 22 · 2019: 34 · 2018: 35 · 2017: 40 · 2016: 32 · 2015: 12
Type
posted content
Language
en
Open access
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
Nvidia
Topics
  • Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Signal Processing › Digital Filter Design and Implementation
Keywords
fast convolution, cuFFT, cuDNN, convolutional neural network training, GPU performance, NVIDIA GPUs

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review