The DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index.
Dynamic Convolution: Attention Over Convolution Kernels
Open accessAbstract
Light-weight convolutional neural networks (CNNs) suffer performance degradation as their low computational budgets constrain both the depth (number of convolution layers) and the width (number of channels) of CNNs, resulting in limited representation capability. To address this issue, we present Dynamic Convolution, a new design that increases model complexity without increasing the network depth or width. Instead of using a single convolution kernel per layer, dynamic convolution aggregates multiple parallel convolution kernels dynamically based upon their attentions, which are input dependent. Assembling multiple kernels is not only computationally efficient due to the small kernel size, but also has more representation power since these kernels are aggregated in a non-linear way via attention. By simply using dynamic convolution for the state-of-the-art architecture MobileNetV3-Small, the top-1 accuracy of ImageNet classification is boosted by 2.9% with only 4% additional FLOPs and 2.9 AP gain is achieved on COCO keypoint detection.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00474 · Cited by 28,755
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015
(2010) · International Conference on Machine Learning · Cited by 13,995
Squeeze-and-Excitation Networks
Hu, J., Shen, L., Albanie, S., Sun, G., & Wu, E. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2913372 · Cited by 12,157
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
Zhang, X., Zhou, X., Lin, M., & Sun, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00716 · Cited by 9,982
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
Zhang, X., Zhou, X., Lin, M., & Sun, J. (2017) · arXiv · doi:10.48550/arxiv.1707.01083 · Cited by 9,982
Howard, A., Sandler, M., Chen, B., Wang, W., Chen, L.-C., Tan, M., Chu, G., Vasudevan, V., Zhu, Y., Pang, R., et al. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00140 · Cited by 9,708
ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
Ma, N., Zhang, X., Zheng, H.-T., & Sun, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01264-9_8 · Cited by 7,356 · cited 4×
“Efficient CNNs: Recently, designing efficient CNN architectures [14, 11, 25, 10, 40, 22] has been an active research area.”
BackgroundInfluentialLearning Transferable Architectures for Scalable Image Recognition
Zoph, B., Vasudevan, V., Shlens, J., & Le, Q. V. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00907 · Cited by 6,249
SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
Iandola, F. N., Han, S., Moskewicz, M. W., Ashraf, K., Dally, W. J., & Keutzer, K. (2016) · arXiv · doi:10.48550/arxiv.1602.07360 · Cited by 6,201 · cited 2×
“Efficient CNNs: Recently, designing efficient CNN architectures [14, 11, 25, 10, 40, 22] has been an active research area.”
BackgroundMethodDeep High-Resolution Representation Learning for Human Pose Estimation
Sun, K., Xiao, B., Liu, D., & Wang, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00584 · Cited by 5,984
mixup: Beyond Empirical Risk Minimization
Zhang, H., Cisse, M., Dauphin, Y. N., & Lopez-Paz, D. (2017) · arXiv · doi:10.48550/arxiv.1710.09412 · Cited by 5,831
Han, S., Mao, H., & Dally, W. J. (2015) · arXiv · doi:10.48550/arxiv.1510.00149 · Cited by 4,662
Neural Architecture Search with Reinforcement Learning
Zoph, B., & Le, Q. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.01578 · Cited by 3,974
MnasNet: Platform-Aware Neural Architecture Search for Mobile
Tan, M., Chen, B., Pang, R., Vasudevan, V., Sandler, M., Howard, A., & Le, Q. V. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00293 · Cited by 3,416
DARTS: Differentiable architecture search
Liu, H., Simonyan, K., & Yang, Y. (2018) · arXiv · doi:10.48550/arxiv.1806.09055 · Cited by 3,248
Learning Efficient Convolutional Networks through Network Slimming
Liu, Z., Li, J., Shen, Z., Huang, G., Yan, S., & Zhang, C. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.298 · Cited by 2,761
Self-citationBinaryConnect: Training Deep Neural Networks with binary weights during propagations
Courbariaux, M., Bengio, Y., & David, J.-P. (2015) · arXiv · doi:10.48550/arxiv.1511.00363 · Cited by 2,180
What is the best multi-stage architecture for object recognition?
Jarrett, K., Kavukcuoglu, K., Ranzato, M. A., & LeCun, Y. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459469 · Cited by 2,153 · cited 1×
“ReLU [24, 16]).”
Simple Baselines for Human Pose Estimation and Tracking
Xiao, B., Wu, H., & Wei, Y. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01231-1_29 · Cited by 2,149
AMC: AutoML for Model Compression and Acceleration on Mobile Devices
He, Y., Lin, J., Liu, Z., Wang, H., Li, L.-J., & Han, S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_48 · Cited by 1,400
Self-citationFBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search
Wu, B., Keutzer, K., Dai, X., Zhang, P., Wang, Y., Sun, F., Wu, Y., Tian, Y., Vajda, P., & Jia, Y. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01099 · Cited by 1,261
Self-citationHAQ: Hardware-Aware Automated Quantization With Mixed Precision
Wang, K., Liu, Z., Lin, Y., Lin, J., & Han, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00881 · Cited by 978
Self-citationACNet: Strengthening the Kernel Skeletons for Powerful CNN via Asymmetric Convolution Blocks
Ding, X., Guo, Y., Ding, G., & Han, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00200 · Cited by 912
Zhu, C., Han, S., Mao, H., & Dally, W. J. (2016) · arXiv · doi:10.48550/arxiv.1612.01064 · Cited by 865
Once-for-All: Train One Network and Specialize it for Efficient Deployment
Cai, H., Gan, C., Wang, T., Zhang, Z., & Han, S. (2019) · arXiv · doi:10.48550/arxiv.1908.09791 · Cited by 845
Single Path One-Shot Neural Architecture Search with Uniform Sampling
Guo, Z., Zhang, X., Mu, H., Heng, W., Liu, Z., Wei, Y., & Sun, J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58517-4_32 · Cited by 813
Self-citationMixup: Beyond empirical risk minimization
Zhang, H. (2024) · arXiv (Cornell University) · doi:10.57702/dcy1c3gw · Cited by 773
SNAS: Stochastic Neural Architecture Search
Xie, S., Zheng, H., Liu, C., & Lin, L. (2018) · arXiv · doi:10.48550/arxiv.1812.09926 · Cited by 731
LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks
Zhang, D., Yang, J., Ye, D., & Hua, G. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01237-3_23 · Cited by 693
SkipNet: Learning Dynamic Routing in Convolutional Networks
Wang, X., Yu, F., Dou, Z.-Y., Darrell, T., & Gonzalez, J. E. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01261-8_25 · Cited by 625
Lin, J., Rao, Y., Lu, J., & Zhou, J. (2017) · neural information processing systems · Cited by 597
BlockDrop: Dynamic Inference Paths in Residual Networks
Wu, Z., Nagarajan, T., Kumar, A., Rennie, S., Davis, L. S., Grauman, K., & Feris, R. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00919 · Cited by 465
Yang, J., Shen, X., Xing, J., Tian, X., Li, H., Deng, B., Huang, J., & Hua, X.-S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00748 · Cited by 387
Shift: A Zero FLOP, Zero Parameter Alternative to Spatial Convolutions
Wu, B., Wan, A., Yue, X., Jin, P., Zhao, S., Golmant, N., Gholaminejad, A., Gonzalez, J., & Keutzer, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00951 · Cited by 381
Regularized Evolution for Image Classifier Architecture Search
Real, E., Aggarwal, A., Huang, Y., & Le, Q. V. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33014780 · Cited by 328
Yu, J., Yang, L., Xu, N., Yang, J., & Huang, T. S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.08928 · Cited by 216
Multi-Scale Dense Networks for Resource Efficient Image Classification
Huang, G. M., Chen, D., Li, T., Wu, F., van der Maaten, L., & Weinberger, K. Q. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.09844 · Cited by 208
Self-citationCondConv: Conditionally Parameterized Convolutions for Efficient Inference
Yang, B., Bender, G., Le, Q. V., & Ngiam, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.04971 · Cited by 208
Dynamic Deep Neural Networks: Optimizing Accuracy-Efficiency Trade-Offs by Selective Execution
Liu, L., & Deng, J. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11630 · Cited by 176
Research works printed but not linked: 2
- iandola, 2016, Squeezenet: Alexnet-level accuracy with 50x fewer parameters and <1mb model size, CoRR abs/1602 07360
- cai, 0, ProxylessNAS: Direct neural architecture search on target task and hardware, International Conference on Learning Representations
Cited by
Attention mechanisms in computer vision: A survey
Guo, M.-H., Xu, T.-X., Liu, J.-J., Liu, Z.-N., Jiang, P.-T., Mu, T.-J., Zhang, S.-H., Martin, R. R., Cheng, M.-M., & Hu, S.-M. (2022) · Computational Visual Media · doi:10.1007/s41095-022-0271-y · Cited by 2,566
“[116] proposed dynamic convolution, a novel operator design that increases representational power with negligible additional computational cost and does not change the width or depth of the network in parallel with CondConv [174].”
BackgroundSelf-citationGoing deeper with Image Transformers
Touvron, H., Cord, M., Sablayrolles, A., Synnaeve, G., & Jegou, H. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00010 · Cited by 1,259
DetectoRS: Detecting Objects with Recursive Feature Pyramid and Switchable Atrous Convolution
Qiao, S., Chen, L.-C., & Yuille, A. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01008 · Cited by 1,245
“Conditional Convolution Conditional convolutional networks adopt dynamic kernels, widths, or depths, e.g ., [16,39,43,48,74,77].”
BackgroundConditional DETR for Fast Training Convergence
Meng, D., Chen, X., Fan, Z., Zeng, G., Li, H., Yuan, Y., Sun, L., & Wang, J. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00363 · Cited by 1,055
Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognition
Chen, Y., Zhang, Z., Yuan, C., Li, B., Deng, Y., & Hu, W. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01311 · Cited by 1,039
Self-citation
Metadata
- Authors
- Chen, Y., Dai, X., Liu, M., Chen, D., Yuan, L., & Liu, Z.
- Venue
- 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2020-06
- Pages
- 11027-11036
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W3034421924
- Cited by
- 1,477
- References
- 45
- FWCI
- 44.36
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.