RaScaNet: Learning Tiny Models by Raster-Scanning Images
Abstract
Deploying deep convolutional neural networks on ultra-low power systems is challenging due to the extremely limited resources. Especially, the memory becomes a bottleneck as the systems put a hard limit on the size of on-chip memory. Because peak memory explosion in the lower layers is critical even in tiny models, the size of an input image should be reduced with sacrifice in accuracy. To overcome this drawback, we propose a novel Raster-Scanning Network, named RaScaNet, inspired by raster-scanning in image sensors. RaScaNet reads only a few rows of pixels at a time using a convolutional neural network and then sequentially learns the representation of the whole image using a recurrent neural network. The proposed method operates on an ultra-low power system without input size reduction; it requires 15.9–24.3× smaller peak memory and 5.3–12.9× smaller weight memory than the state-of-the-art tiny models. Moreover, RaScaNet fully exploits on-chip SRAM and cache memory of the system as the sum of the peak memory and the weight memory does not exceed 60 KB, improving the power efficiency of the system. In our experiments, we demonstrate the binary classification performance of RaScaNet on Visual Wake Words and Pascal VOC datasets.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Yoo et al. (2021) show that peak memory explosion can be minimized in embedded systems by using RaScaNet for image processing applications for the TinyML ecosystem.
“A study shows that peak memory explosion can be minimized in embedded systems by using the Raster-Scanning-Network (RaS-caNet) for image processing applications for the TinyML ecosystem (Yoo et al., 2021).”
From A review on TinyML: State-of-the-art and prospects (2022) · cites this paper as Yoo et al., 2021
RaScaNet needs 15.9-24.3x lesser peak memory and 5.3-12.9x less weight memory than existing tiny models.
“The study shows that the network needs 15.9-24.3xlesser peak memory.”
From A review on TinyML: State-of-the-art and prospects (2022) · cites this paper as Yoo et al., 2021
RaScaNet captures on-chip SRAM and needs a maximum of 60 KB of capacity.
“The on-chip SRAM is captured by this network that needs a maximum of 60 KB of capacity.”
From A review on TinyML: State-of-the-art and prospects (2022) · cites this paper as Yoo et al., 2021
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460 · see citing works
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667 · see citing works
MobileNetV2: Inverted Residuals and Linear Bottlenecks
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., & Chen, L.-C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00474 · Cited by 28,755 · see citing works
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105 · see citing works
CBAM: Convolutional Block Attention Module
Woo, S., Park, J., Lee, J.-Y., & Kweon, I. S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_1 · Cited by 25,853 · see citing works
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · see citing works
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,890 · see citing works
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 21,793 · see citing works
Xception: Deep Learning with Depthwise Separable Convolutions
Chollet, F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.195 · Cited by 20,380 · see citing works
The pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599 · see citing works
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015 · see citing works
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Tan, M., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1905.11946 · Cited by 17,072 · see citing works
Decoupled Weight Decay Regularization
Loshchilov, I., & Hutter, F. (2017) · arXiv · doi:10.48550/arxiv.1711.05101 · Cited by 12,522 · see citing works
Squeeze-and-Excitation Networks
Hu, J., Shen, L., Albanie, S., Sun, G., & Wu, E. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2913372 · Cited by 12,157 · see citing works
Howard, A., Sandler, M., Chen, B., Wang, W., Chen, L.-C., Tan, M., Chu, G., Vasudevan, V., Zhu, Y., Pang, R., et al. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00140 · Cited by 9,708 · see citing works
Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
Zhang, K., Zuo, W., Chen, Y., Meng, D., & Zhang, L. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2662206 · Cited by 8,983 · see citing works
MnasNet: Platform-Aware Neural Architecture Search for Mobile
Tan, M., Chen, B., Pang, R., Vasudevan, V., Sandler, M., Howard, A., & Le, Q. V. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00293 · Cited by 3,416 · see citing works
DARTS: Differentiable architecture search
Liu, H., Simonyan, K., & Yang, Y. (2018) · arXiv · doi:10.48550/arxiv.1806.09055 · Cited by 3,248 · see citing works
Learning Efficient Convolutional Networks through Network Slimming
Liu, Z., Li, J., Shen, Z., Huang, G., Yan, S., & Zhang, C. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.298 · Cited by 2,761 · see citing works
Progressive Neural Architecture Search
Liu, C., Zoph, B., Neumann, M., Shlens, J., Wei, H., Li, L.-J., Fei-Fei, L., Yuille, A., Huang, J., & Murphy, K. P. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01246-5_2 · Cited by 1,946 · see citing works
DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients
Zhou, S., Wu, Y., Ni, Z., Zhou, X., Wen, H., & Zou, Y. (2016) · arXiv · doi:10.48550/arxiv.1606.06160 · Cited by 1,690 · see citing works
ProxylessNAS: Direct Neural Architecture Search on Target Task and HardwarePreprint
Cai, H., Zhu, L., & Han, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.00332 · Cited by 1,652 · see citing works
(2018) · International Conference on Machine Learning · Cited by 1,465 · see citing works
AMC: AutoML for Model Compression and Acceleration on Mobile Devices
He, Y., Lin, J., Liu, Z., Wang, H., Li, L.-J., & Han, S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_48 · Cited by 1,400 · see citing works
Rethinking the Value of Network PruningPreprint
Liu, Z., Sun, M., Zhou, T., Huang, G. M., & Darrell, T. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.05270 · Cited by 1,225 · see citing works
Efficient Neural Architecture Search via Parameter SharingPreprint
Pham, H., Guan, M. Y., Zoph, B., Le, Q. V., & Dean, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.03268 · Cited by 1,082 · see citing works
PACT: Parameterized Clipping Activation for Quantized Neural Networks
Choi, J., Wang, Z., Venkataramani, S., Chuang, P. I.-J., Srinivasan, V., & Gopalakrishnan, K. (2018) · arXiv · doi:10.48550/arxiv.1805.06085 · Cited by 703 · see citing works
MCUNet: Tiny Deep Learning on IoT DevicesPreprint
Lin, J., Chen, W.-M., Lin, Y., Cohn, J., Gan, C., & Han, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.10319 · Cited by 425 · see citing works
Learning to Quantize Deep Networks by Optimizing Quantization Intervals With Task Loss
Jung, S., Son, C., Lee, S., Son, J., Han, J.-J., Kwak, Y., Hwang, S. J., & Choi, C. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00448 · Cited by 385 · see citing works
Self-citationBenchmarking TinyML Systems: Challenges and Direction
Banbury, C. R., Reddi, V. J., Lam, M., Fu, W., Fazel, A., Holleman, J., Huang, X., Hurtado, R., Kanter, D., Lokhmotov, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2003.04821 · Cited by 216 · see citing works
Visual Wake Words DatasetPreprint
Chowdhery, A., Warden, P., Shlens, J., Howard, A., & Rhodes, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.05721 · Cited by 131 · see citing works
SpArSe: Sparse Architecture Search for CNNs on Resource-Constrained MicrocontrollersPreprint
Fedorov, I., Adams, R. P., Mattina, M., & Whatmough, P. N. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.12107 · Cited by 130 · see citing works
Glance and Focus: a Dynamic Approach to Reducing Spatial Redundancy in Image ClassificationPreprint
Wang, Y., Lv, K., Huang, R., Song, S., Yang, L., & Huang, G. M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.05300 · Cited by 126 · see citing works
IoTNet: An Efficient and Accurate Convolutional Neural Network for IoT Devices
Lawrence, T., & Li, Z. (2019) · Sensors · doi:10.3390/s19245541 · Cited by 52 · see citing works
RNNPool: Efficient Non-linear Pooling for RAM Constrained InferencePreprint
Saha, O., Kusupati, A., Simhadri, H. V., Varma, M., & Jain, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.11921 · Cited by 38 · see citing works
Flagged first, then by citations
2 of the 40 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
A review on TinyML: State-of-the-art and prospects
Ray, P. P. (2022) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2021.11.019 · Cited by 460 · see citing works
“A study shows that peak memory explosion can be minimized in embedded systems by using the Raster-Scanning-Network (RaS-caNet) for image processing applications for the TinyML ecosystem (Yoo et al., 2021).”
Machine Learning for Microcontroller-Class Hardware: A Review
Saha, S. S., Sandha, S. S., & Srivastava, M. B. (2022) · IEEE Sensors Journal · doi:10.1109/jsen.2022.3210773 · Cited by 275 · see citing works
Hou, X., Guan, Y., & Han, T. (2022) · ACM Annual International Conference on Mobile Computing and Networking (MobiCom) · doi:10.1145/3495243.3560528 · Cited by 16 · see citing works
Deep Technologies Using Big Data in: Energy and Waste Management
Verma, J. (2023) · Advanced technologies and societal change · doi:10.1007/978-981-19-5723-9_2 · Cited by 11 · see citing works
Vu, T. P., Luong, V. S., & Le, M. (2024) · Nondestructive Testing And Evaluation · doi:10.1080/10589759.2024.2360052 · Cited by 9 · see citing works
On-Chip Data Reduction and Object Detection for a Feature-Extractable CMOS Image Sensor
Morikaku, Y., Ujiie, R., Morikawa, D., Shima, H., Yoshida, K., & Okura, S. (2024) · Electronics · doi:10.3390/electronics13214295 · Cited by 6 · see citing works
FedSNN: Training Slimmable Neural Network With Federated Learning in Edge Computing
Xu, Y., Liao, Y., Xu, H., Wang, Z., Wang, L., Liu, J., & Qian, C. (2024) · IEEE Transactions on Networking · doi:10.1109/tnet.2024.3487582 · Cited by 4 · see citing works
Protection of the Information System of the Printing Enterprise from Cyber Threats
Durnyak, B., Shepita, P., & Tupychak, L. (2023) · International Workshop on Intelligent Information Technologies & Systems of Information Security (IntelITSIS) · doi:10.55056/ceur-ws.org/vol-3373/paper31.pdf · Cited by 1 · see citing works
Synthesis and Modeling of Systems with Combined Fuzzy P + I - Regulators
Durnyak, B., Луцків, М. М., Shepita, P., Sheketa, V., Pasieka, N., & Pasieka, M. (2023) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-3-031-36115-9_18 · Cited by 1 · see citing works
Durnyak, B., Neroda, T. V., Shepita, P., Tupychak, L., Pasieka, N., & Romanyshyn, Y. (2023) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-3-031-36115-9_19 · Cited by 1 · see citing works
Improved You Only Look Once Model for UAVs/Ships Relative Attitude Detection
Qiu, J., Yu, F., Xu, F., Wang, J., & Chen, X. (2024) · Lecture notes in electrical engineering · doi:10.1007/978-981-97-8658-9_21
CNN-RNN-Based Image Recognition for On-Chip Processing in a Feature-Extractable CMOS Image Sensor
Osuka, Y., Morikaku, Y., Iegaki, R., Ujiie, R., Shima, H., Yoshida, K., & Okura, S. (2025) · The Journal of The Institute of Image Information and Television Engineers · doi:10.3169/itej.79.660
Simulation of a Hybrid Operation of the System with a Simplified Version of the Fuzzy PD-Controller
Durnyak, B., Луцків, М. М., & Shepita, P. (2023) · International Workshop on Intelligent Information Technologies & Systems of Information Security (IntelITSIS) · doi:10.55056/ceur-ws.org/vol-3373/paper46.pdf
Fuzzy Model of Raster Transformation of Square Elements
Durnyak, B., Луцків, М. М., & Shepita, P. (2022) · International Workshop on Intelligent Information Technologies & Systems of Information Security (IntelITSIS) · doi:10.55056/ceur-ws.org/vol-3156/paper8.pdf
- Authors
- Yoo, J., Lee, D., Son, C., Jung, S., Yoo, B., Choi, C., Han, J.-J., & Han, B.
- Affiliations
- Seoul National University (KR)
- Samsung (South Korea) (KR)
- Venue
- 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2021-06
- Pages
- 13668-13677
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W3181580200
- Cited by
- 14
- References
- 40
- Citations by year
- 2025: 1 · 2024: 4 · 2023: 5 · 2022: 3 · 2021: 1
- FWCI
- 1.06
- Type
- conference paper
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Advanced Memory and Neural Computing
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Low-power high-performance VLSI design
- Keywords
- ultra-low-power systems, memory bottleneck, recurrent neural networks, convolutional neural networks, on-chip SRAM, cache memory, image classification, Pascal VOC
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
