Speeding up Convolutional Neural Networks with Low Rank Expansions
Abstract
The focus of this paper is speeding up the application of convolutional neural networks. While delivering impressive results across a range of computer vision and machine learning tasks, these networks are computationally demanding, limiting their deployability. Convolutional layers generally consume the bulk of the processing time, and so in this work we present two simple schemes for drastically speeding up these layers. This is achieved by exploiting cross-channel or filter redundancy to construct a low rank basis of filters that are rank-1 in the spatial domain. Our methods are architecture agnostic, and can be easily applied to existing CPU and GPU convolutional frameworks for tuneable speedup performance. We demonstrate this with a real world network designed for scene text character recognition [15], showing a possible 2.5× speedup with no loss in accuracy, and 4.5× speedup with less than 1% drop in accuracy, still achieving state-of-the-art on standard benchmarks.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In light-weight models, element-wise operations occupy a considerable amount of time, especially on GPU.
“As shown in Figure 2, in light-weight models like [15,14], element-wise operations occupy considerable amount of time, especially on GPU.”
From ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design (2018) · cites this paper as [14]
MobileNet v2 uses an inverted bottleneck structure that violates G1, and it uses depthwise convolutions and ReLUs on thick feature maps, violating G4.
“MobileNet v2 [14] uses an inverted bottleneck structure that violates G1 . It uses depthwise convolutions and ReLUs on 'thick' feature maps. This violates G4 .”
From ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design (2018) · cites this paper as [14]
Recent advances in light-weight neural network architectures are mostly based on the metric of FLOPs and do not consider the proposed properties.
“Recent advances in light-weight neural network architectures [15,13,14,9,11,10,12] are mostly based on the metric of FLOPs and do not consider these properties above.”
From ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design (2018) · cites this paper as [14]
FLOPs is an indirect metric and is usually not equivalent to the direct metric such as speed or latency.
“However, FLOPs is an indirect metric. It is an approximation of, but usually not equivalent to the direct metric that we really care about, such as speed or latency.”
From ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design (2018) · cites this paper as [14]
As reported in [14], MobileNet v2 of 500+ MFLOPs has comparable accuracy with ShuffleNet v2 (25.3% vs. 25.1% top-1 error).
“As reported in [14], MobileNet v2 of 500+ MFLOPs has comparable accuracy with the counterpart ShuffleNet v2 (25.3% vs. 25.1% top-1 error)”
From ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design (2018) · cites this paper as [14]
All networks in comparison have four levels of computational complexity, about 40, 140, 300, and 500+ MFLOPs.
“Following the common practice [15,13,14], all networks in comparison have four levels of computational complexity, i.e. about 40, 140, 300 and 500+ MFLOPs.”
From ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design (2018) · cites this paper as [14]
No citation lists for this paper
No citation lists for this paper yet — checked again daily.
References 38 · Cited by 1,164
- Authors
- Jaderberg, M., Vedaldi, A., & Zisserman, A.
- Affiliations
- University of Oxford (GB)
- Venue
- Proceedings of the British Machine Vision Conference 2014
- Publisher
- British Machine Vision Association
- Published
- 2014
- Pages
- 88.1-88.13
- OpenAlex
- W1996901117
- Cited by
- 1,164
- References
- 38
- Citations by year
- 2026: 34 · 2025: 64 · 2024: 66 · 2023: 100 · 2022: 137 · 2021: 171 · 2020: 169 · 2019: 151 · 2018: 129 · 2017: 70 · 2016: 52 · 2015: 18 · 2014: 3
- FWCI
- 25.64
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- British Machine Vision Conference (BMVC) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Keywords
- convolutional neural networks, filter redundancy, computational acceleration, deployability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
