The DOI matches its DataCite record; 13 of the 14 references it cites were found in a registry or an index, and 1 are not checked yet.
Fast Convolutional Nets With fbfft: A GPU Performance Evaluation
Open accessAbstract
We examine the performance profile of Convolutional Neural Network training on the current generation of NVIDIA Graphics Processing Units. We introduce two new Fast Fourier Transform convolution implementations: one based on NVIDIA's cuFFT library, and another based on a Facebook authored FFT implementation, fbfft, that provides significant speedups over cuFFT (over 1.5x) for whole CNNs. Both of these convolution implementations are available in open source, and are faster than NVIDIA's cuDNN implementation for many common convolutional layers (up to 23.5x for some synthetic kernel configurations). We discuss different performance regimes of convolutions, comparing areas where straightforward time domain convolutions outperform Fourier frequency domain convolutions. Details on algorithmic applications of NVIDIA GPU hardware specifics in the implementation of fbfft are also provided.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“The above calculation can be accelerated by using the Fast Fourier Transform (FFT) and the Inverse FFT (IFFT) [160-163].”
From Object Detection in 20 Years: A Survey (2023) · cites this paper as [162]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
An algorithm for the machine calculation of complex Fourier series
Cooley, J. W., & Tukey, J. W. (1965) · Mathematics of Computation · doi:10.1090/s0025-5718-1965-0178586-1 · Cited by 13,154
Natural Language Processing (almost) from ScratchPreprint
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. P. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1103.0398 · Cited by 6,923
Caffe: Convolutional Architecture for Fast Feature EmbeddingPreprint
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1408.5093 · Cited by 5,299
A bridging model for parallel computation
Valiant, L. (1990) · CACM · doi:10.1145/79173.79181 · Cited by 4,318
A Bridging Model for Parallel Computation
Valiant, L. G. (1990) · ACM eBooks · doi:10.1145/3749740.3749753 · Cited by 3,697
Torch7: A Matlab-like Environment for Machine Learning
Collobert, R., Kavukcuoglu, K., & Farabet, C. (2011) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 1,241
Understanding digital signal processing
Lyons, R. G. (1996) · Cited by 1,017
(2013) · doi:10.1145/2491956.2462176 · Cited by 885
Irigoin, F., & Triolet, R. (1988) · ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL) · doi:10.1145/73560.73588 · Cited by 565
Parallel Computing Experiences with CUDA
Garland, M., LeGrand, S., Nickolls, J. R., Anderson, J. A., Hardwick, J., Morton, S., Phillips, E. H., Zhang, Y., & Volkov, V. M. (2008) · IEEE Micro · doi:10.1109/mm.2008.57 · Cited by 545
Research works printed but not linked: 8
- Bergstra, James, Breuleux, Olivier, Bastien, Fr´ ed´ eric, Lamblin, Pascal, Pascanu, Razvan, Desjardins, Guillaume, Turian, Joseph, Warde-Farley, David, and Bengio, Yoshua. Theano: a CPU and GPU math expression compiler. In Proceedings of the Python for Scientific Computing Conference (SciPy) , June 2010. Oral Presentation.
- Bluestein, Leo I. A linear filtering approach to the computation of discrete Fourier transform. Audio and Electroacoustics, IEEE Transactions on , 18(4):451-455, December 1970. ISSN 0018-9278.
- Brosch, Tom and Tam, Roger C. Efficient training of convolutional deep belief networks in the frequency domain for application to high-resolution 2d and 3d images. Neural Computation , 27 (1):211-227, 2015. doi: 10.1162/NECO a 00682. URL http://dx.doi.org/10.1162/ NECO\_a\_00682 . · DOI
- Chetlur, Sharan, Woolley, Cliff, Vandermersch, Philippe, Cohen, Jonathan, Tran, John, Catanzaro, Bryan, and Shelhamer, Evan. cudnn: Efficient primitives for deep learning. CoRR , abs/1410.0759, 2014. URL http://arxiv.org/abs/1410.0759 . · arXiv
- Krizhevsky, Alex, Sutskever, Ilya, and Hinton, Geoffrey E. Imagenet classification with deep convolutional neural networks. In Pereira, F., Burges, C.J.C., Bottou, L., and Weinberger, K.Q. (eds.), Advances in Neural Information Processing Systems 25 , pp. 1097-1105. Curran Associates, Inc., 2012. URL http://papers.nips.cc/paper/4824-imagenetclassification-with-deep-convolutional-neural-networks.pdf .
- Lavin, Andrew. maxdnn: An efficient convolution kernel for deep learning with maxwell gpus. CoRR , abs/1501.06633, 2015. URL http://arxiv.org/abs/1501.06633 . · arXiv
- Sermanet, Pierre, Eigen, David, Zhang, Xiang, Mathieu, Michael, Fergus, Rob, and LeCun, Yann. Overfeat: Integrated recognition, localization and detection using convolutional networks. In International Conference on Learning Representations (ICLR 2014) . CBLS, April 2014. URL http://openreview.net/document/d332e77d-459a-4af8-b3ed-55ba .
- Volkov, V. Better performance at lower occupancy. In GPU Technology Conference , 2010. URL http://www.cs.berkeley.edu/˜ volkov/volkov10-GTC.pdf .
Cited by
Sorted by citations
Items 201–245 of 245
Evaluating the Merits of Ranking in Structured Network Pruning
Sharma, K., Ramakrishnan, N., Prakash, A., Kei, L. S., & Srikanthan, T. (2020) · IEEE International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs47774.2020.00183 · Cited by 2
Rizvi, S. M., Rahman, A. A.-H. A., Khalil-Hani, M., & Ayat, S. O. (2021) · Indonesian Journal of Electrical Engineering and Informatics (IJEEI) · doi:10.52549/ijeei.v9i1.2737 · Cited by 5
A Structure to Effectively Prepare the Data for Sliding Window in Deep Learning
Wang, B., & Li, M. (2021) · IEEE International Conference on Signal and Image Processing (ICSIP) · doi:10.1109/icsip52628.2021.9688916 · Cited by 2
Curia, F. (2022) · Journal of Quality in Health Care & Economics · doi:10.23880/jqhe-16000310 · Cited by 1
Differential Image-based Fast and Compatible Convolutional Layers for Multi-core Processors
Hong, S., & Park, D. (2023) · International Conference on Artificial Intelligence in Information and Communication (ICAIIC) · doi:10.1109/icaiic57133.2023.10066972 · Cited by 2
Deep actionlet proposals for driver's behavior monitoring
Ulhaq, A., He, J., & Zhang, Y. (2017) · International Conference on Image and Vision Computing New Zealand (IVCNZ) · doi:10.1109/ivcnz.2017.8402447 · Cited by 2
ZNNi - Maximizing the Inference Throughput of 3D Convolutional Networks on Multi-Core CPUs and GPUsPreprint
Zlateski, A., Lee, K., & Seung, H. S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.05688 · Cited by 2
Wang, H., Lin, J. T., & Wang, Z. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.13388 · Cited by 3
Convolution Smooth: A Post-Training Quantization Method for Convolutional Neural Networks
Chen, Y., & Wang, Z. (2025) · IEEE Access · doi:10.1109/access.2025.3558956 · Cited by 1
Robust Hypersphere-based Weight Imprinting for Few-Shot Learning
Passalis, N., Iosifidis, A., Gabbouj, M., & Tefas, A. (2020) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco47968.2020.9287340 · Cited by 1
CubeGen: Code Generation for Accelerated GEMM-Based Convolution with Tiling
Phaosawasdi, A., Rodrigues, C. I., Chen, L. F., & Wu, P. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-72789-5_11 · Cited by 1
Hartley Stochastic Computing For Convolutional Neural Networks
Mozafari, S. H., Clark, J. J., Gross, W. J., & Meyer, B. H. (2021) · SiPS ... design and implementation - IEEE Workshop on Signal Processing Systems · doi:10.1109/sips52927.2021.00049 · Cited by 1
A Fine-grained Optimization to Winograd Convolution Based on Micro-architectural Features of CPU
Chen, X., Chen, Z., Lu, Y., & Huang, D. (2021) · IEEE International Symposium on Parallel and Distributed Processing with Applications (ISPA) · doi:10.1109/ispa-bdcloud-socialcom-sustaincom52081.2021.00165 · Cited by 1
Research on compression method of yolov5 model based on channel pruning
Bie, Y., Yang, R., Zhang, X., Li, Y., & Gu, Y. (2022) · China International SAR Symposium (CISS) · doi:10.1109/ciss57580.2022.9971442 · Cited by 1
Hyper-convolutions via implicit kernels for medical image analysis
Ma, T., Wang, A. Q., Dalca, A. V., & Sabuncu, M. R. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102796 · Cited by 1
Jin, H., Qamar, S., Zheng, R., & Ahmad, P. (2018) · Journal of Intelligent & Fuzzy Systems · doi:10.3233/jifs-171329 · Cited by 1
Data Analyst Platform Using Open Source Based Deep Learning Engine
Kim, A., & Lee, J. (2018) · International Conference on Platform Technology and Service · doi:10.1109/platcon.2018.8472770 · Cited by 1
Performance Analysis of Different Convolution Algorithms in GPU Environment
Xu, R., Ma, S., & Guo, Y. (2018) · IEEE International Conference on Networking, Architecture and Storage (NAS) · doi:10.1109/nas.2018.8515695 · Cited by 1
Fourier Transform Layer for Fast Foreground Segmentation in Samples’ Images of Tissue Biopsies
Żak, J., Siemion, K., Roszkowiak, Ł., & Korzyńska, A. (2021) · Lecture notes in networks and systems · doi:10.1007/978-3-030-83704-4_12 · Cited by 1
Kernel-Segregated Transpose Convolution Operation
Tida, V. S., Chilukoti, S. V., Hsu, S., & Hei, X. (2023) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2023.840 · Cited by 1
Improving the efficiency of deep convolutional networks
Peemen, M. (2017) · TU/e Research Portal · Cited by 1
Auto Deep Compression by Reinforcement Learning Based Actor-Critic StructurePreprint
Hakkak, H. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.02886 · Cited by 1
Faster Convergence & Generalization in DNNsPreprint
Singh, G., & Shawe-Taylor, J. S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.11414 · Cited by 2
See far with TPNET: a Tile Processor and a CNN SymbiosisPreprint
Filippov, A., & Dzhimiev, O. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.08032 · Cited by 1
Benchmarking, autotuning and crowdtuning OpenCL programs using the Collective Knowledge framework
Lokhmotov, A., & Fursin, G. (2016) · International Workshop on OpenCL · doi:10.1145/2909437.2909460
Optimizing Memory Efficiency for Many-Core Architecture.
Li, C. (2016) · NCSU Libraries Repository (North Carolina State University Libraries)
Application case study—machine learning
Ginsburg, B. (2017) · Elsevier eBooks · doi:10.1016/b978-0-12-811986-0.00016-9 · Cited by 1
Test-cost-sensitive Convolutional Neural Networks with Expert Branches
Naghibi, M., Anvari, R., Forghani, A., & Bidgoli, B. M. (2019) · Signal & Image Processing An International Journal · doi:10.5121/sipij.2019.10502
Scalable Profiling and Visualization for Characterizing Microbiomes
Valdes, C. (2020) · FIU - Digital Commons (Florida International University) · doi:10.25148/etd.fidc008938
Accelerated Object Detection for Autonomous Driving in CVIS Based on Pruned CNN Architecture
Wang, C., Qian, L., Ding, L., Yang, F., Zhi, C., & Xu, Y. (2021) · Communications in computer and information science · doi:10.1007/978-981-16-1194-0_11
FPGA Based Deep Learning Accelerator for RF Applications: A Design Framework
den Boer, H. (2021)
Accelerating Convolutional Neural Network Using Discrete Orthogonal TransformsPreprint
Reis, E., & Benlamri, R. (2021) · doi:10.36227/techrxiv.14593686
Performance Analysis of DFT and FFT Algorithms on Modern GPUsPreprint
Davuluru, V. S. P., Hettiarachchi, D. L. N., & Balster, E. J. (2022) · doi:10.36227/techrxiv.20264424.v1
Performance Analysis of DFT and FFT Algorithms on Modern GPUsPreprint
Davuluru, V. S. P., Hettiarachchi, D. L. N., & Balster, E. J. (2022) · doi:10.36227/techrxiv.20264424
Evaluating FFT-Based Convolutions on Skin Diseases Dataset
Aboulmira, A., Hrimech, H., & Lachgar, M. (2023) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-3-031-27762-7_39
MLPs: Efficient Training of MiniGo on Large-scale Heterogeneous Computing System
Qiao, P., He, Z., Li, R., Jiang, J., Dou, Y., & Li, D. (2023) · IEEE International Conference on Parallel and Distributed Systems (ICPADS) · doi:10.1109/icpads56603.2022.00068
INTEGRATING AI HARDWARE IN ACADEMIC TEACHING: EXPERIENCES AND SCOPE FROM BRANDENBURG AND BAVARIA
Xiong, Z., Stober, D., Krstic, M. D., Korup, O., Arango-Carmona, M. I., Li, H., & Werner, M. (2023) · ISPRS annals of the photogrammetry, remote sensing and spatial information sciences · doi:10.5194/isprs-annals-x-5-w1-2023-75-2023
Training Acceleration of Frequency Domain CNNs Using Activation Compression
Mozafari, S. H., Clark, J. J., Gross, W. J., & Meyer, B. H. (2023) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas46773.2023.10181694
Noise-Tolerant Self-Embedded LSTM for Seismic Event Classification
Canário, J. P., & Rios, R. A. (2023) · Machine learning for signal processing ... · doi:10.1109/mlsp55844.2023.10285940
Bokade, R., & Gharat, N. (2024) · International Research Journal on Advanced Engineering Hub (IRJAEH) · doi:10.47392/irjaeh.2024.0073
Optimizing RGB Convolution on Cortex-M7 MCU: Approaches for Performance Improvement
Lee, S., Sung, J., & Kwon, J.-W. (2024) · IEEE Access · doi:10.1109/access.2024.3439586
Data infrastructure for connected transport systems
Luckow, A., & Kennedy, K. (2024) · Elsevier eBooks · doi:10.1016/b978-0-443-13878-2.00007-2
Fast and Accurate DNN Performance Estimation across Diverse Hardware Platforms
Kakrannaya, V. V., Rai, S. B., Sivasubramaniam, A., & Zhu, T. (2024) · Proceedings - International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems · doi:10.1109/mascots64422.2024.10786578
SSFFT: Energy-Efficient Selective Scaling for Fast Fourier Transform in Embedded GPUs
Yang, D., Jeon, J., Gil, M., Kim, J. H., Kim, S., Koo, G., Yoon, M. K., & Oh, Y. (2025) · ACM SIGPLAN/SIGBED Conference on Languages, Compilers, and Tools for Embedded Systems (LCTES) · doi:10.1145/3735452.3735529
Metadata
- Authors
- Vasilache, N., Johnson, J., Mathieu, M., Chintala, S., Piantino, S., & LeCun, Y.
- Affiliations
- Meta (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2014
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1412.7580
- OpenAlex
- W1789336918
- Cited by
- 246
- References
- 14
- Citations by year
- 2025: 2 · 2024: 9 · 2023: 14 · 2022: 13 · 2021: 33 · 2020: 22 · 2019: 34 · 2018: 35 · 2017: 40 · 2016: 32 · 2015: 12
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Nvidia
- Topics
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Signal Processing › Digital Filter Design and Implementation
- Keywords
- fast convolution, cuFFT, cuDNN, convolutional neural network training, GPU performance, NVIDIA GPUs
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
