Fast Convolutional Nets With fbfft: A GPU Performance Evaluation
Open accessAbstract
We examine the performance profile of Convolutional Neural Network training on the current generation of NVIDIA Graphics Processing Units. We introduce two new Fast Fourier Transform convolution implementations: one based on NVIDIA's cuFFT library, and another based on a Facebook authored FFT implementation, fbfft, that provides significant speedups over cuFFT (over 1.5x) for whole CNNs. Both of these convolution implementations are available in open source, and are faster than NVIDIA's cuDNN implementation for many common convolutional layers (up to 23.5x for some synthetic kernel configurations). We discuss different performance regimes of convolutions, comparing areas where straightforward time domain convolutions outperform Fourier frequency domain convolutions. Details on algorithmic applications of NVIDIA GPU hardware specifics in the implementation of fbfft are also provided.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The convolution calculation can be accelerated by using the Fast Fourier Transform and the Inverse FFT.
“The above calculation can be accelerated by using the Fast Fourier Transform (FFT) and the Inverse FFT (IFFT) [160-163].”
From Object Detection in 20 Years: A Survey (2023) · cites this paper as [162]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
An algorithm for the machine calculation of complex Fourier series
Cooley, J. W., & Tukey, J. W. (1965) · Mathematics of Computation · doi:10.1090/s0025-5718-1965-0178586-1 · Cited by 13,154 · see citing works
Natural Language Processing (almost) from ScratchPreprint
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. P. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1103.0398 · Cited by 6,882 · see citing works
Caffe: Convolutional Architecture for Fast Feature EmbeddingPreprint
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1408.5093 · Cited by 4,749 · see citing works
A bridging model for parallel computation
Valiant, L. (1990) · CACM · doi:10.1145/79173.79181 · Cited by 4,318
A Bridging Model for Parallel Computation
Valiant, L. G. (1990) · ACM eBooks · doi:10.1145/3749740.3749753 · Cited by 3,697 · see citing works
Torch7: A Matlab-like Environment for Machine Learning
Collobert, R., Kavukcuoglu, K., & Farabet, C. (2011) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 1,241 · see citing works
Understanding digital signal processing
Lyons, R. G. (1996) · Cited by 1,017 · see citing works
(2013) · doi:10.1145/2491956.2462176 · Cited by 885 · see citing works
Irigoin, F., & Triolet, R. (1988) · ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL) · doi:10.1145/73560.73588 · Cited by 565 · see citing works
Parallel Computing Experiences with CUDA
Garland, M., LeGrand, S., Nickolls, J. R., Anderson, J. A., Hardwick, J., Morton, S., Phillips, E. H., Zhang, Y., & Volkov, V. M. (2008) · IEEE Micro · doi:10.1109/mm.2008.57 · Cited by 545 · see citing works
Fast Training of Convolutional Networks through FFTs
Mathieu, M., Henaff, M., & LeCun, Y. (2013) · arXiv · doi:10.48550/arxiv.1312.5851 · Cited by 273 · see citing works
Self-citationWang, Q. J., & Chung, Y.-W. (2013) · Springer eBooks · doi:10.1007/978-0-387-92897-5_100464 · Cited by 231 · see citing works
A Family of High-Performance Matrix Multiplication Algorithms
Gunnels, J. A., Gustavson, F. G., Henry, G., & Geijn, R. A. (2006) · Lecture notes in computer science · doi:10.1007/11558958_30 · Cited by 52 · see citing works
Speeding up Nek5000 with autotuning and specialization
Shin, J., Hall, M. W., Chame, J., Chen, C., Fischer, P., & Hovland, P. (2010) · International Conference on Supercomputing · doi:10.1145/1810085.1810120 · Cited by 44 · see citing works
Flagged first, then by citations
Research works printed but not linked: 8
- Bergstra, James, Breuleux, Olivier, Bastien, Fr´ ed´ eric, Lamblin, Pascal, Pascanu, Razvan, Desjardins, Guillaume, Turian, Joseph, Warde-Farley, David, and Bengio, Yoshua. Theano: a CPU and GPU math expression compiler. In Proceedings of the Python for Scientific Computing Conference (SciPy) , June 2010. Oral Presentation.
- Bluestein, Leo I. A linear filtering approach to the computation of discrete Fourier transform. Audio and Electroacoustics, IEEE Transactions on , 18(4):451-455, December 1970. ISSN 0018-9278.
- Brosch, Tom and Tam, Roger C. Efficient training of convolutional deep belief networks in the frequency domain for application to high-resolution 2d and 3d images. Neural Computation , 27 (1):211-227, 2015. doi: 10.1162/NECO a 00682. URL http://dx.doi.org/10.1162/ NECO\_a\_00682 . · DOI
- Chetlur, Sharan, Woolley, Cliff, Vandermersch, Philippe, Cohen, Jonathan, Tran, John, Catanzaro, Bryan, and Shelhamer, Evan. cudnn: Efficient primitives for deep learning. CoRR , abs/1410.0759, 2014. URL http://arxiv.org/abs/1410.0759 . · arXiv
- Krizhevsky, Alex, Sutskever, Ilya, and Hinton, Geoffrey E. Imagenet classification with deep convolutional neural networks. In Pereira, F., Burges, C.J.C., Bottou, L., and Weinberger, K.Q. (eds.), Advances in Neural Information Processing Systems 25 , pp. 1097-1105. Curran Associates, Inc., 2012. URL http://papers.nips.cc/paper/4824-imagenetclassification-with-deep-convolutional-neural-networks.pdf .
- Lavin, Andrew. maxdnn: An efficient convolution kernel for deep learning with maxwell gpus. CoRR , abs/1501.06633, 2015. URL http://arxiv.org/abs/1501.06633 . · arXiv
- Sermanet, Pierre, Eigen, David, Zhang, Xiang, Mathieu, Michael, Fergus, Rob, and LeCun, Yann. Overfeat: Integrated recognition, localization and detection using convolutional networks. In International Conference on Learning Representations (ICLR 2014) . CBLS, April 2014. URL http://openreview.net/document/d332e77d-459a-4af8-b3ed-55ba .
- Volkov, V. Better performance at lower occupancy. In GPU Technology Conference , 2010. URL http://www.cs.berkeley.edu/˜ volkov/volkov10-GTC.pdf .
Cited by
Sorted by citations
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
Zhang, X., Zhou, X., Lin, M., & Sun, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00716 · Cited by 9,982 · see citing works
Recent advances in convolutional neural networks
Gu, J., Wang, Z., Kuen, J., Ma, L., Shahroudy, A., Shuai, B., Liu, T., Wang, X., Wang, G., Cai, J., et al. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2017.10.013 · Cited by 6,576 · see citing works
Object Detection in 20 Years: A Survey
Zou, Z., Chen, K., Shi, Z., Guo, Y., & Ye, J. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2023.3238524 · Cited by 3,689 · see citing works
“The above calculation can be accelerated by using the fast Fourier transform (FFT) and the inverse FFT (IFFT) [160], [161], [162], [163].”
Channel Pruning for Accelerating Very Deep Neural Networks
He, Y., Zhang, X., & Sun, J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.155 · Cited by 2,647 · see citing works
Pouyanfar, S., Sadiq, S. B., Yan, Y., Tian, H., Tao, Y., Reyes, M. P., Shyu, M., Chen, S., & Iyengar, S. S. (2018) · ACM Computing Surveys · doi:10.1145/3234150 · Cited by 1,435 · see citing works
AMC: AutoML for Model Compression and Acceleration on Mobile Devices
He, Y., Lin, J., Liu, Z., Wang, H., Li, L.-J., & Han, S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_48 · Cited by 1,400 · see citing works
Fast Algorithms for Convolutional Neural Networks
Lavin, A., & Gray, S. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.435 · Cited by 937 · see citing works
Accelerating Very Deep Convolutional Networks for Classification and Detection
Zhang, X., Zou, J., He, K., & Sun, J. Z. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2502579 · Cited by 847 · see citing works
DeepChain: Auditable and Privacy-Preserving Deep Learning with Blockchain-based Incentive
Weng, J., Weng, J., Zhang, J., Li, M., Zhang, Y., & Luo, W. (2019) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2019.2952332 · Cited by 610 · see citing works
Scaling Local Self-Attention for Parameter Efficient Visual Backbones
Vaswani, A., Ramachandran, P., Srinivas, A., Parmar, N., Hechtman, B. A., & Shlens, J. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.01270 · Cited by 483 · see citing works
Fast ConvNets Using Group-Wise Brain Damage
Lebedev, V., & Lempitsky, V. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.280 · Cited by 397 · see citing works
Exploring Sparsity in Image Super-Resolution for Efficient Inference
Wang, L., Dong, X., Wang, Y., Ying, X., Lin, Z., An, W., & Guo, Y. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00488 · Cited by 340 · see citing works
FireCaffe: Near-Linear Acceleration of Deep Neural Network Training on Compute Clusters
Iandola, F., Moskewicz, M. W., Ashraf, K. M., & Keutzer, K. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.284 · Cited by 285 · see citing works
Yu, J., Lukefahr, A., Palframan, D. J., Dasika, G. S., Das, R., & Mahlke, S. A. (2017) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1145/3079856.3080215 · Cited by 269 · see citing works
Tensor Comprehensions: Framework-Agnostic High-Performance Machine Learning AbstractionsPreprint
Vasilache, N., Zinenko, O., Theodoridis, T., Goyal, P., DeVito, Z., Moses, W. S., Verdoolaege, S., Adams, A., & Cohen, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.04730 · Cited by 250 · see citing works
Self-citationCentripetal SGD for Pruning Very Deep Convolutional Networks With Complicated Structure
Ding, X., Ding, G., Guo, Y., & Han, J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00508 · Cited by 198 · see citing works
Toward Compact ConvNets via Structure-Sparsity Regularized Filter Pruning
Lin, S., Ji, R., Li, Y., Deng, C. D., & Li, X. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2906563 · Cited by 181 · see citing works
Channel-level acceleration of deep face representations
Polyak, A., & Wolf, L. (2015) · IEEE Access · doi:10.1109/access.2015.2494536 · Cited by 171 · see citing works
Villa, O., Stephenson, M., Nellans, D. W., & Keckler, S. W. (2019) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1145/3352460.3358307 · Cited by 169 · see citing works
Deep learning in the automotive industry: Applications and tools
Luckow, A., Cook, M., Ashcraft, N., Weill, E., Djerekarov, E., & Vorster, B. (2016) · 2016 IEEE International Conference on Big Data (Big Data) · doi:10.1109/bigdata.2016.7841045 · Cited by 164 · see citing works
fpgaConvNet: Mapping regular and irregular convolutional neural networks on FPGAs
Venieris, S. I., & Bouganis, C.-S. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2844093 · Cited by 161 · see citing works
Accelerating Convolutional Neural Network With FFT on Embedded Hardware
Abtahi, T., Shea, C., Kulkarni, A., & Mohsenin, T. (2018) · IEEE Transactions on Very Large Scale Integration (VLSI) Systems · doi:10.1109/tvlsi.2018.2825145 · Cited by 150 · see citing works
FCNN: Fourier Convolutional Neural Networks
Pratt, H., Williams, B., Coenen, F., & Zheng, Y. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-71249-9_47 · Cited by 138 · see citing works
Performance Analysis of GPU-Based Convolutional Neural Networks
Li, X., Zhang, G., Huang, H., Wang, Z., & Zheng, W. (2016) · Proceedings of the International Conference on Parallel Processing · doi:10.1109/icpp.2016.15 · Cited by 142 · see citing works
Compressing Convolutional Neural Networks in the Frequency Domain
Chen, W., Wilson, J. T., Tyree, S., Weinberger, K. Q., & Chen, Y. (2016) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2939672.2939839 · Cited by 130 · see citing works
Regularization of deep neural networks with spectral dropout
Khan, S. H., Hayat, M., & Porikli, F. (2018) · Neural Networks · doi:10.1016/j.neunet.2018.09.009 · Cited by 131 · see citing works
Performance analysis of CNN frameworks for GPUs
Kim, H., Nam, H., Jung, W., & Lee, J. (2017) · IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS) · doi:10.1109/ispass.2017.7975270 · Cited by 114 · see citing works
Ooi, B. C., Tan, K., Wang, S., Wang, W., Cai, Q., Chen, G., Gao, J., Luo, Z., Tung, A. K. H., Wang, Y., et al. (2015) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2733373.2807410 · Cited by 113 · see citing works
Optimizing Memory Efficiency for Deep Convolutional Neural Networks on GPUs
Li, C., Yang, Y., Feng, M., Chakradhar, S., & Zhou, H. (2016) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1109/sc.2016.53 · Cited by 104 · see citing works
swDNN: A Library for Accelerating Deep Learning Applications on Sunway TaihuLight
Fang, J., Fu, H., Zhao, W., Chen, B., Zheng, W., & Yang, G. (2017) · IEEE International Parallel and Distributed Processing Symposium (IPDPS) · doi:10.1109/ipdps.2017.20 · Cited by 90 · see citing works
Comparative Study of Deep Learning Software FrameworksPreprint
Bahrampour, S., Ramakrishnan, N., Schott, L., & Shah, M. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06435 · Cited by 89 · see citing works
PID Controller-Based Stochastic Optimization Acceleration for Deep Neural Networks
Wang, H., Luo, Y., An, W., Sun, Q., Xu, J., & Lei Zhang, A. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2963066 · Cited by 90 · see citing works
Facial Expression Recognition Using Frequency Neural Network
Tang, Y. D., Zhang, X., Hu, X., Wang, S., & Wang, H. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.3037467 · Cited by 89 · see citing works
Reduction of Multiplications in Convolutional Neural Networks
Ali, M., Yin, B., Kunar, A., Sheikh, A. M., & Bilal, H. (2020) · Chinese Control Conference (CCC) · doi:10.23919/ccc50068.2020.9188843 · Cited by 86 · see citing works
Optimizing Depthwise Separable Convolution Operations on GPUs
Lu, G., Zhang, W., & Wang, Z. (2021) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2021.3084813 · Cited by 83 · see citing works
Network Sketching: Exploiting Binary Structure in Deep CNNs
Guo, Y., Yao, A., Zhao, H., & Chen, Y. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.430 · Cited by 81 · see citing works
Performance Evaluation of cuDNN Convolution Algorithms on NVIDIA Volta GPUs
Jordà, M., Valero‐Lara, P., & Peña, A. J. (2019) · IEEE Access · doi:10.1109/access.2019.2918851 · Cited by 78 · see citing works
MEC: Memory-efficient Convolution for Deep Neural NetworkPreprint
Cho, M., & Brand, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.06873 · Cited by 78 · see citing works
Model Compression Using Progressive Channel Pruning
Guo, J., Zhang, W., Ouyang, W., & Xu, D. (2020) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2020.2996231 · Cited by 74 · see citing works
Persistent RNNs: stashing recurrent weights on-chip
Diamos, G., Sengupta, S., Catanzaro, B., Chrzanowski, M., Coates, A., Elsen, E., Engel, J., Hannun, A., & Satheesh, S. (2016) · International Conference on Machine Learning · Cited by 71 · see citing works
Ko, J. H., Mudassar, B. A., Na, T., & Mukhopadhyay, S. (2017) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1145/3061639.3062228 · Cited by 71 · see citing works
A Survey on the Optimization of Neural Network Accelerators for Micro-AI On-Device Inference
Mazumder, A. N., Meng, J., Rashid, H.-A., Kallakuri, U., Zhang, X., Seo, J.-S., & Mohsenin, T. (2021) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2021.3129415 · Cited by 67 · see citing works
A comparative study of open source deep learning frameworks
Shatnawi, A. M., Al-Bdour, G., Al-Qurran, R., & Al‐Ayyoub, M. (2018) · International Conference on Information and Communication Systems (ICICS) · doi:10.1109/iacs.2018.8355444 · Cited by 59 · see citing works
HPTT: a high-performance tensor transposition C++ library
Springer, P., Su, T., & Bientinesi, P. (2017) · ACM SIGPLAN International Workshop on Libraries, Languages and Compilers for Array Programming (ARRAY) · doi:10.1145/3091966.3091968 · Cited by 54 · see citing works
Structured Probabilistic Pruning for Convolutional Neural Network AccelerationPreprint
Wang, H., Zhang, Q., Wang, Y., & Hu, H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.06994 · Cited by 54 · see citing works
Shah, N., Chaudhari, P., & Varghese, K. (2018) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2815085 · Cited by 51 · see citing works
Optimizing N-dimensional, winograd-based convolution for manycore CPUs
Jia, Z., Zlateski, A., Durand, F., & Li, K. (2018) · ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming · doi:10.1145/3178487.3178496 · Cited by 52 · see citing works
A Winograd-Based Integrated Photonics Accelerator for Convolutional Neural Networks
Mehrabian, A., Miscuglio, M., Alkabani, Y., Sorger, V. J., & El‐Ghazawi, T. (2019) · IEEE Journal of Selected Topics in Quantum Electronics · doi:10.1109/jstqe.2019.2957443 · Cited by 53 · see citing works
Machine learning on FPGAs to face the IoT revolution
Zhang, X., Ramachandran, A., Zhuge, C., He, D., Zuo, W., Cheng, Z., Rupnow, K., & Chen, D. (2017) · International Conference on Computer Aided Design · doi:10.5555/3199700.3199810 · Cited by 50 · see citing works
High Performance Zero-Memory Overhead Direct ConvolutionsPreprint
Zhang, J., Franchetti, F., & Low, T. M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.10170 · Cited by 50 · see citing works
Cavigelli, L., Degen, P., & Luca Benini, C. (2017) · International Conference on Distributed Smart Cameras · doi:10.1145/3131885.3131906 · Cited by 48 · see citing works
Hadjis, S., Abuzaid, F., Zhang, C., & Ré, C. (2015) · doi:10.1145/2799562.2799641 · Cited by 47 · see citing works
Zlateski, A., Lee, K., & Seung, H. S. (2016) · IEEE International Parallel and Distributed Processing Symposium (IPDPS) · doi:10.1109/ipdps.2016.119 · Cited by 45 · see citing works
Ravikumar, A., Sriraman, H., Saketh, P. M. S., Lokesh, S., & Karanam, A. (2022) · PeerJ Computer Science · doi:10.7717/peerj-cs.909 · Cited by 44 · see citing works
Face Recognition with Hybrid Efficient Convolution Algorithms on FPGAs
Zhuge, C., Liu, X., Zhang, X., Gummadi, S., Xiong, J., & Chen, D. (2018) · ACM Great Lakes Symposium on VLSI (GLSVLSI) · doi:10.1145/3194554.3194597 · Cited by 44 · see citing works
Zero and data reuse-aware fast convolution for deep neural networks on GPU
Park, H., Kim, D., Ahn, J., & Yoo, S. (2016) · International Conference on Hardware/Software Codesign and System Synthesis · doi:10.1145/2968456.2968476 · Cited by 43 · see citing works
The Next 700 Accelerated Layers
Vasilache, N., Zinenko, O., Theodoridis, T., Goyal, P., DeVito, Z., Moses, W. S., Verdoolaege, S., Adams, A., & Cohen, A. (2019) · ACM Transactions on Architecture and Code Optimization · doi:10.1145/3355606 · Cited by 43 · see citing works
Self-citationSAR-Optical Image Matching by Integrating Siamese U-Net With FFT Correlation
Fang, Y., Hu, J. P., Du, C., Liu, Z.-B., & Zhang, L. (2021) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2021.3100531 · Cited by 41 · see citing works
2PFPCE: Two-Phase Filter Pruning Based on Conditional EntropyPreprint
Min, C., Wang, A., Chen, Y., Xu, W., & Chen, X. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.02220 · Cited by 38 · see citing works
Fractional Fourier Transform Meets Transformer Encoder
Şahi̇nuç, F., & Koç, A. (2022) · IEEE Signal Processing Letters · doi:10.1109/lsp.2022.3217975 · Cited by 36 · see citing works
Deep Convolutional Neural Network on iOS Mobile Devices
Chen, C.-F., Lee, G. G., Sritapan, V., & Lin, C. (2016) · SiPS ... design and implementation - IEEE Workshop on Signal Processing Systems · doi:10.1109/sips.2016.31 · Cited by 33 · see citing works
Accelerating convolutional neural network with FFT on tiny cores
Abtahi, T., Kulkarni, A., & Mohsenin, T. (2017) · IEEE International Symposium on Circuits and Systems proceedings · doi:10.1109/iscas.2017.8050588 · Cited by 33 · see citing works
FeatherCNN: Fast Inference Computation with TensorGEMM on ARM Architectures
Lan, H., Meng, J., Hundt, C., Schmidt, B., Deng, M., Wang, X., Liu, W., Qiao, Y., & Feng, S. (2019) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2019.2939785 · Cited by 33 · see citing works
Accelerating Fourier and Number Theoretic Transforms using Tensor Cores and Warp Shuffles
Durrani, S., Chughtai, M. S., Hidayetoğlu, M., Tahir, R., Dakkak, A., Rauchwerger, L., Zaffar, F., & Hwu, W. (2021) · International Conference on Parallel Architectures and Compilation Techniques (PACT) · doi:10.1109/pact52795.2021.00032 · Cited by 33 · see citing works
clCaffe: OpenCL Accelerated Caffe for Convolutional Neural Networks
Bottleson, J., Kim, S., Andrews, J., Bindu, P., Murthy, D. N., & Jin, J. (2016) · IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum · doi:10.1109/ipdpsw.2016.182 · Cited by 32 · see citing works
Fast Branch Convolutional Neural Network for Traffic Sign Recognition
Hu, W., Zhuo, Q., Zhang, C., & Li, J. (2017) · IEEE Intelligent Transportation Systems Magazine · doi:10.1109/mits.2017.2709780 · Cited by 32 · see citing works
Machine learning on FPGAs to face the IoT revolution
Zhang, X., Ramachandran, A., Zhuge, C., He, D., Zuo, W., Cheng, Z., Rupnow, K., & Chen, D. (2017) · International Conference on Computer Aided Design · doi:10.1109/iccad.2017.8203862 · Cited by 32 · see citing works
Scene Categorization with Spectral Features
Khan, S. H., Hayat, M., & Porikli, F. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.601 · Cited by 32 · see citing works
DWM: A Decomposable Winograd Method for Convolution Acceleration
Huang, D., Zhang, X., Zhang, R., Zhi, T., He, D., Guo, J., Liu, C., Guo, Q., Du, Z., Liu, S., et al. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5838 · Cited by 31 · see citing works
The Indirect Convolution AlgorithmPreprint
Dukhan, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.02129 · Cited by 31 · see citing works
Deep insight: Convolutional neural network and its applications for COVID-19 prognosis
Khanday, N. Y., & Sofi, S. A. (2021) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2021.102814 · Cited by 31 · see citing works
Compressing Convolutional Neural NetworksPreprint
Chen, W., Wilson, J. T., Tyree, S., Weinberger, K. Q., & Chen, Y. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.04449 · Cited by 30 · see citing works
Deep Tensor Convolution on MulticoresPreprint
Budden, D. M., Matveev, A., Santurkar, S., Chaudhuri, S. R., & Shavit, N. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.06565 · Cited by 30 · see citing works
Deep compare: A study on using convolutional neural networks to compare image patches
Zagoruyko, S., & Nikos Komodakis, A. (2017) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2017.10.007 · Cited by 30 · see citing works
High-Performance Generalized Tensor Operations
Gareev, R. A., Grosser, T., & Kruse, M. (2018) · ACM Transactions on Architecture and Code Optimization · doi:10.1145/3235029 · Cited by 28 · see citing works
Taxonomy, state-of-the-art, challenges and applications of visual understanding: A review
Khanday, N. Y., & Sofi, S. A. (2021) · Computer Science Review · doi:10.1016/j.cosrev.2021.100374 · Cited by 28 · see citing works
From Algorithm to Hardware: A Survey on Efficient and Safe Deployment of Deep Neural Networks
Geng, X., Wang, Z., Chen, C., Xu, Q., Xu, K., Jin, C., Gupta, M., Yang, X., Chen, Z., Sabry Aly, M. M., et al. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3394494 · Cited by 26 · see citing works
WooKong: A Ubiquitous Accelerator for Recommendation Algorithms with Custom Instruction Sets on FPGA
Wang, C., Gong, L., Ma, X., Li, X., & Zhou, X. (2020) · IEEE Transactions on Computers · doi:10.1109/tc.2020.2988209 · Cited by 26 · see citing works
Convolution Neural Networks: A Comparative Study for Image Classification
Darapaneni, N., Krishnamurthy, B., & Paduri, A. R. (2020) · IEEE International Conference on Industrial and Information Systems (ICIIS) · doi:10.1109/iciis51140.2020.9342667 · Cited by 26 · see citing works
Exploiting Non-Local Priors via Self-Convolution for Highly-Efficient Image Restoration
Guo, L., Zha, Z., Ravishankar, S., & Wen, B. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3140918 · Cited by 26 · see citing works
Compact Convolutional Neural Network Cascade for Face DetectionPreprint
Kalinowski, I., & Spitsyn, V. (2015) · arXiv.org · doi:10.48550/arxiv.1508.01292 · Cited by 37 · see citing works
Light YOLO for High-Speed Gesture Recognition
Ni, Z., Chen, J., Sang, N., Gao, C., & Liu, L. (2018) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2018.8451766 · Cited by 25 · see citing works
A Fast Algorithm for Convolutional Neural Networks Using Tile-based Fast Fourier Transforms
Lin, J., & Yao, Y. (2019) · Neural Processing Letters · doi:10.1007/s11063-019-09981-z · Cited by 24 · see citing works
Tang, X., Peng, J., Zhong, B., Li, J., & Yan, Z. (2021) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2021.106110 · Cited by 25 · see citing works
Accelerating Sparse CNN Inference on GPUs with Performance-Aware Weight Pruning
Rumi, M. A., Ma, X., Wang, Y., & Jiang, P. (2020) · International Conference on Parallel Architectures and Compilation Techniques (PACT) · doi:10.1145/3410463.3414648 · Cited by 23 · see citing works
The anatomy of efficient FFT and winograd convolutions on modern CPUs
Zlateski, A., Jia, Z., Li, K., & Durand, F. (2019) · International Conference on Supercomputing · doi:10.1145/3330345.3330382 · Cited by 23 · see citing works
Optimizing Convolutional Neural Networks on the Sunway TaihuLight Supercomputer
Zhao, W., Fu, H., Fang, J., Zheng, W., Gan, L., & Yang, G. (2018) · ACM Transactions on Architecture and Code Optimization · doi:10.1145/3177885 · Cited by 22 · see citing works
Machine learning on FPGAs to face the IoT revolution
Zhang, X., Ramachandran, A., Zhuge, C., He, D., Zuo, W., Cheng, Z., Rupnow, K., & Chen, D. (2017) · International Conference on Computer Aided Design · doi:10.1109/iccad.2017.8203875 · Cited by 22 · see citing works
Convolution Accelerator Designs Using Fast Algorithms
Zhao, Y., Wang, D., & Wang, L. (2019) · Algorithms · doi:10.3390/a12050112 · Cited by 22 · see citing works
Machine Learning Enabled Spectrum Sharing in Dense LTE-U/Wi-Fi Coexistence Scenarios
Dziedzic, A., Sathya, V., Rochman, M. I., Ghosh, M., & Krishnan, S. (2020) · IEEE Open Journal of Vehicular Technology · doi:10.1109/ojvt.2020.2981519 · Cited by 21 · see citing works
Enabling Efficient Fast Convolution Algorithms on GPUs via MegaKernels
Jia, L., Liang, Y. E., Li, X., Lu, L., & Yan, S. (2020) · IEEE Transactions on Computers · doi:10.1109/tc.2020.2973144 · Cited by 20 · see citing works
ZNNi: maximizing the inference throughput of 3D convolutional networks on CPUs and GPUs
Zlateski, A., Lee, K., & Seung, H. S. (2016) · IEEE International Conference on High Performance Computing, Data, and Analytics · doi:10.5555/3014904.3015002 · Cited by 20 · see citing works
Distributed Training Large-Scale Deep Architectures
Zou, S.-X., Chen, C.-Y., Wu, J.-L., Chou, C., Tsao, C.-C., Tung, K.-C., Lin, T., Sung, C.-L., & Chang, E. Y. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-69179-4_2 · Cited by 20 · see citing works
Speed-Up of Object Detection Neural Network with GPU
Fukagai, T., Maeda, K., Tanabe, S., Shirahata, K., Tomita, Y., Ike, A., & Nakagawa, A. (2018) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2018.8451814 · Cited by 20 · see citing works
TTC: a tensor transposition compiler for multiple architectures
Springer, P., Sankaran, A., & Bientinesi, P. (2016) · ACM SIGPLAN International Workshop on Libraries, Languages and Compilers for Array Programming (ARRAY) · doi:10.1145/2935323.2935328 · Cited by 19 · see citing works
A Faster Algorithm for Reducing the Computational Complexity of Convolutional Neural Networks
Zhao, Y., Wang, D., Wang, L., & Liu, P. (2018) · Algorithms · doi:10.3390/a11100159 · Cited by 19 · see citing works
Tschopp, F., Martel, J., Turaga, S. C., Cook, M., & Funke, J. (2016) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2016.7493487 · Cited by 18 · see citing works
A spectrum-domain instance segmentation model for casting defects
Lin, J., Ma, L., & Yao, Y. (2021) · Integrated Computer-Aided Engineering · doi:10.3233/ica-210666 · Cited by 18 · see citing works
A performance analysis framework for exploiting GPU microarchitectural capability
Zhou, K., Tan, G., Zhang, X., Wang, C., & Sun, N. (2017) · International Conference on Supercomputing · doi:10.1145/3079079.3079083 · Cited by 19 · see citing works
Hyperdrive: A Multi-Chip Systolically Scalable Binary-Weight CNN Inference Engine
Andri, R., Cavigelli, L., Rossi, D., & Luca Benini, C. (2019) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2019.2905654 · Cited by 18 · see citing works
- Authors
- Vasilache, N., Johnson, J., Mathieu, M., Chintala, S., Piantino, S., & LeCun, Y.
- Affiliations
- Meta (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2014
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1412.7580
- OpenAlex
- W1789336918
- Cited by
- 246
- References
- 14
- Citations by year
- 2025: 2 · 2024: 9 · 2023: 14 · 2022: 13 · 2021: 33 · 2020: 22 · 2019: 34 · 2018: 35 · 2017: 40 · 2016: 32 · 2015: 12
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Nvidia
- Topics
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Signal Processing › Digital Filter Design and Implementation
- Keywords
- fast convolution, cuFFT, cuDNN, convolutional neural network training, GPU performance, NVIDIA GPUs
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review