The DOI matches its Crossref record; 60 of the 60 references it cites were found in a registry or an index.
Rewrite the Stars
Abstract
Recent studies have drawn attention to the untapped potential of the “star operation” (element-wise multiplication) in network design. While intuitive explanations abound, the foundational rationale behind its application remains largely unexplored. Our study attempts to reveal the star operation's ability of mapping inputs into high-dimensional, non-linear feature spaces-akin to kernel tricks-without widening the network. We further introduce StarNet, a simple yet pow-erful prototype, demonstrating impressive performance and low latency under compact network structure and efficient budget. Like stars in the sky, the star operation appears unremarkable but holds a vast universe of potential. Our work encourages further exploration across tasks, with codes available at https://github.com/ma-xu/Rewrite-the-Stars.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00474 · Cited by 28,755
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741 · cited 1×
“Commonly employed activations like ReLU and GELU, however, are subject to certain drawbacks such as ‘mean shift’ [4, 22, 24] and information loss [46], among others.”
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Tan, M., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1905.11946 · Cited by 17,072
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839
Decoupled Weight Decay Regularization
Loshchilov, I., & Hutter, F. (2017) · arXiv · doi:10.48550/arxiv.1711.05101 · Cited by 12,522
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
Zhang, X., Zhou, X., Lin, M., & Sun, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00716 · Cited by 9,982
ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices
Zhang, X., Zhou, X., Lin, M., & Sun, J. (2017) · arXiv · doi:10.48550/arxiv.1707.01083 · Cited by 9,982
Liu, Z., Mao, H., Wu, C.-Y., Feichtenhofer, C., Darrell, T., & Xie, S. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01167 · Cited by 8,494
ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
Ma, N., Zhang, X., Zheng, H.-T., & Sun, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01264-9_8 · Cited by 7,356
Kernel Methods for Pattern Analysis
Shawe-Taylor, J., & Cristianini, N. (2004) · Cambridge University Press eBooks · doi:10.1017/cbo9780511809682 · Cited by 6,587
mixup: Beyond Empirical Risk Minimization
Zhang, H., Cisse, M., Dauphin, Y. N., & Lopez-Paz, D. (2017) · arXiv · doi:10.48550/arxiv.1710.09412 · Cited by 5,831
CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features
Yun, S., Han, D., Chun, S., Oh, S. J., Yoo, Y., & Choe, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00612 · Cited by 5,400
SGDR: Stochastic Gradient Descent with Warm Restarts
Loshchilov, I., & Hutter, F. (2016) · arXiv · doi:10.48550/arxiv.1608.03983 · Cited by 5,309
GhostNet: More Features From Cheap Operations
Han, K., Wang, Y., Tian, Q., Guo, J., Xu, C., & Xu, C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00165 · Cited by 5,222
Self-citationGaussian Error Linear Units (GELUs)
Hendrycks, D., & Gimpel, K. (2016) · arXiv · doi:10.48550/arxiv.1606.08415 · Cited by 3,537
MnasNet: Platform-Aware Neural Architecture Search for Mobile
Tan, M., Chen, B., Pang, R., Vasudevan, V., Sandler, M., Howard, A., & Le, Q. V. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00293 · Cited by 3,416
Run, Don't Walk: Chasing Higher FLOPS for Faster Neural Networks
Chen, J., Kao, S.-H., He, H., Zhuo, W., Wen, S., Lee, C.-H., & Chan, S.-H. G. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01157 · Cited by 2,343
Deep Networks with Stochastic Depth
Huang, G., Sun, Y., Liu, Z., Sedra, D., & Weinberger, K. Q. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_39 · Cited by 2,104
Kernel methods in machine learning
Hofmann, T., Schölkopf, B., & Smola, A. J. (2008) · The Annals of Statistics · doi:10.1214/009053607000000677 · Cited by 1,651
Mamba: Linear-Time Sequence Modeling with Selective State Spaces
Gu, A., & Dao, T. (2023) · arXiv · doi:10.48550/arxiv.2312.00752 · Cited by 1,579 · cited 1×
“Star operation exhibits promising performance and efficiency across various research fields, including Natural Language Processing ( i.e. , Monarch Mixer [16], Mamba [17], Hyena Hierarchy [44], 45, 60], with “ ?”
Dynamic Convolution: Attention Over Convolution Kernels
Chen, Y., Dai, X., Liu, M., Chen, D., Yuan, L., & Liu, Z. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01104 · Cited by 1,477
Self-citationGoing deeper with Image Transformers
Touvron, H., Cord, M., Sablayrolles, A., Synnaeve, G., & Jegou, H. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00010 · Cited by 1,259
Pattern Recognition and Machine Learning
(1971) · Springer eBooks · doi:10.1007/978-1-4615-7566-5 · Cited by 1,144
EfficientNetV2: Smaller Models and Faster Training
Tan, M., & Le, Q. V. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.00298 · Cited by 1,098
Guo, M.-H., Lu, C.-Z., Liu, Z.-N., Cheng, M.-M., & Hu, S.-M. (2023) · Computational Visual Media · doi:10.1007/s41095-023-0364-2 · Cited by 1,073
Dynamic Head: Unifying Object Detection Heads with Attentions
Dai, X., Chen, Y., Xiao, B., Chen, D., Liu, M., Yuan, L., & Zhang, L. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00729 · Cited by 1,044
Self-citationWhen does label smoothing help?
Müller, R., Kornblith, S., & Hinton, G. (2019) · arXiv · doi:10.48550/arxiv.1906.02629 · Cited by 859
Mobile-Former: Bridging MobileNet and Transformer
Chen, Y., Dai, X., Chen, D., Liu, M., Dong, X., Yuan, L., & Liu, Z. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00520 · Cited by 731
Self-citationResidual Networks Behave Like Ensembles of Relatively Shallow Networks
Veit, A., Wilber, M., & Belongie, S. (2016) · arXiv · doi:10.48550/arxiv.1605.06431 · Cited by 705
Gather-Excite: Exploiting Feature Context in Convolutional Neural Networks
Hu, J., Shen, L., Albanie, S., Sun, G., & Vedaldi, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.12348 · Cited by 462
MobileOne: An Improved One millisecond Mobile Backbone
Vasu, P. K. A., Gabriel, J., Zhu, J., Tuzel, O., & Ranjan, A. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00764 · Cited by 394
Rethinking Bottleneck Structure for Efficient Mobile Network Design
Zhou, D., Hou, Q., Chen, Y., Feng, J., & Yan, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58580-8_40 · Cited by 292
EdgeViTs: Competing Light-Weight CNNs on Mobile Devices with Vision Transformers
Pan, J., Bulat, A., Tan, F., Zhu, X., Dudziak, L., Li, H., Tzimiropoulos, G., & Martinez, B. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-20083-0_18 · Cited by 266
Yang, J., Li, C., Dai, X., Yuan, L., & Gao, J. (2022) · arXiv · doi:10.48550/arxiv.2203.11926 · Cited by 265
Self-citationHigh-Performance Large-Scale Image Recognition Without Normalization
Brock, A., De, S., Smith, S. L., & Simonyan, K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.06171 · Cited by 245
Conv2Former: A Simple Transformer-Style ConvNet for Visual Recognition
Hou, Q., Lu, C.-Z., Cheng, M.-M., & Feng, J. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3401450 · Cited by 239
HorNet: Efficient High-Order Spatial Interactions with Recursive Gated Convolutions
Rao, Y., Zhao, W., Tang, Y., Zhou, J., Lim, S. N., & Lu, J. (2022) · neural information processing systems · doi:10.52202/068431-0752 · Cited by 217
Scale-Aware Modulation Meet Transformer
Lin, W., Wu, Z., Chen, J., Huang, J., & Jin, L. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00553 · Cited by 208
AdderNet: Do We Really Need Multiplications in Deep Learning?
Chen, H., Wang, Y., Xu, C., Shi, B., Xu, C., Tian, Q., & Xu, C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00154 · Cited by 199
Self-citationMogaNet: Multi-order Gated Aggregation NetworkPreprint
Li, S., Wang, Z., Liu, Z., Tan, C., Lin, H., Wu, D., Chen, Z., Zheng, J., & Li, S. Z. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2211.03295 · Cited by 193
Yang, J., Li, C., Dai, X., & Gao, J. (2022) · neural information processing systems · doi:10.52202/068431-0304 · Cited by 193
Self-citationChen, Y., Dai, X., Liu, M., Chen, D., Yuan, L., & Liu, Z. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58529-7_21 · Cited by 186
Self-citationFasterViT: Fast Vision Transformers with Hierarchical AttentionPreprint
Hatamizadeh, A., Heinrich, G., Yin, H., Tao, A., Álvarez, J. M., Kautz, J., & Molchanov, P. A. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.06189 · Cited by 186
DeepShift: Towards Multiplication-Less Neural Networks
Elhoushi, M., Chen, Z., Shafiq, F., Tian, Y. H., & Li, J. Y. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00268 · Cited by 117
Hyena Hierarchy: Towards Larger Convolutional Language Models
Poli, M., Massaroli, S., Nguyen, E., Fu, D. Y., Dao, T., Baccus, S., Bengio, Y., Ermon, S., & Ré, C. (2023) · arXiv · doi:10.48550/arxiv.2302.10866 · Cited by 96
Monarch Mixer: A Simple Sub-Quadratic GEMM-Based Architecture
Fu, D. Y., Arora, S., Grogan, J., Johnson, I., Eyuboglu, E. S., Thomas, A. W., Spector, B., Poli, M., Rudra, A., & Ré, C. (2023) · neural information processing systems · doi:10.52202/075280-3390 · Cited by 79
Video-FocalNets: Spatio-Temporal Focal Modulation for Video Action Recognition
Wasim, S. T., Khattak, M. U., Naseer, M., Khan, S. H., Shah, M. A., & Khan, F. S. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01267 · Cited by 64
SPANet: Frequency-balancing Token Mixer using Spectral Pooling Aggregation Modulation
Yun, G., Yoo, J., Kim, K., Lee, J., & Kim, D. H. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00562 · Cited by 39
Poly-NL: Linear Complexity Non-local Layers With 3rd Order Polynomials
Babiloni, F., Marras, I., Kokkinos, F., Deng, J., Chrysos, G. G., & Zafeiriou, S. P. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01035 · Cited by 28
Metadata
- Authors
- Ma, X., Dai, X., Bai, Y., Wang, Y., & Fu, Y.
- Venue
- 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2024-06-16
- Pages
- 5694-5703
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4402727675
- Cited by
- 620
- References
- 60
- FWCI
- 135.37
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.