Toward Attention-Based TinyML: A Heterogeneous Accelerated Architecture and Automated Deployment Flow
Open accessAbstract
One of the challenges for Tiny Machine Learning (tinyML) is keeping up with the evolution of Machine Learning models from Convolutional Neural Networks to Transformers. We address this by leveraging a heterogeneous architectural template coupling RISC-V processors with hardwired accelerators supported by an automated deployment flow. We demonstrate Attention-based models in a tinyML power envelope with an octacore cluster coupled with an accelerator for quantized Attention. Our deployment flow enables end-to-end 8-bit Transformer inference, achieving leading-edge energy efficiency and throughput of 2960 GOp/J and 154GOp/s (0.65 V, 22nm FD-SOI technology).
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
This TinyML flow enables low-latency local inference without continuous cloud dependency.
“This flow enables low-latency local inference [34] without continuous cloud dependency.”
From Explainable AI and Multi-Agent Systems for Energy Management in IoT-Edge Environments: A State of the Art Review (2026) · cites this paper as [34]
Pruning, quantization, and knowledge distillation shrink parameter counts and reduce runtime while preserving most predictive performance, though over-compression risks degrading behavior during rare or extreme events.
“Pruning, quantization, and knowledge distillation shrink parameter counts and reduce runtime while preserving most predictive performance, though over-compression risks degrading behavior during rare or extreme events [33,34].”
From Explainable AI and Multi-Agent Systems for Energy Management in IoT-Edge Environments: A State of the Art Review (2026) · cites this paper as [34]
Cites
Robust Speech Recognition via Large-Scale Weak Supervision
Radford, A., Kim, J. W., Xu, T., Brockman, G., McLeavey, C., & Sutskever, I. (2022) · arXiv · doi:10.48550/arxiv.2212.04356 · Cited by 1,953 · see citing works
MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devices
Sun, Z., Yu, H., Song, X., Liu, R., Yang, Y., & Zhou, D. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.195 · Cited by 757 · see citing works
A Comprehensive Survey on TinyML
Abadade, Y., Temouden, A., Bamoumen, H., Benamar, N., Chtouki, Y., & Hafid, A. S. (2023) · IEEE Access · doi:10.1109/access.2023.3294111 · Cited by 366 · cited 3× · see citing works
“Although numerous model-specific accelerators have been proposed in recent years [1], designing SoCs that can integrate these accelerators while remaining adaptable to evolving AI models remains an open challenge, particularly under tight memory constraints.”
InfluentialI-BERT: Integer-only BERT Quantization
Kim, S., Gholami, A., Yao, Z., Mahoney, M. W., & Keutzer, K. (2021) · arXiv · doi:10.48550/arxiv.2101.01321 · Cited by 186 · see citing works
RISC-V Instruction Set Architecture Extensions: A Survey
Cui, E., Li, T., & Qian, W. (2023) · IEEE Access · doi:10.1109/access.2023.3246491 · Cited by 157 · see citing works
Banbury, C., Reddi, V. J., Torelli, P., Holleman, J., Jeffries, N., Kiraly, C., Montino, P., Kanter, D., Ahmed, S., Pau, D., et al. (2021) · arXiv · doi:10.48550/arxiv.2106.07597 · Cited by 121 · see citing works
An Analysis of Accelerator Coupling in Heterogeneous Architectures
Cota, E. G., Mantovani, P., Guglielmo, G. D., & Carloni, L. P. (2015) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1145/2744769.2744794 · Cited by 101 · see citing works
Zaruba, F., Schuiki, F., Hoefler, T., & Luca Benini, C. (2020) · IEEE Transactions on Computers · doi:10.1109/tc.2020.3027900 · Cited by 94 · see citing works
Scherer, M., Macan, L., Jung, V. J. B., Wiese, P., Bompani, L., Burrello, A., Conti, F., & Benini, L. (2024) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2024.3443718 · Cited by 26 · see citing works
Self-citationITA: An Energy-Efficient Attention and Softmax Accelerator for Quantized Transformers
İslamoğlu, G., Scherer, M., Paulin, G., Fischer, T., Jung, V. J. B., Garofalo, A., & Luca Benini, C. (2023) · International Symposium on Low Power Electronics and Design (ISLPED) · doi:10.1109/islped58423.2023.10244348 · Cited by 26 · see citing works
Self-citation
Flagged first, then by citations
2 of the 12 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Advancing TinyML in IoT: A Holistic System-Level Perspective for Resource-Constrained AI
Ortiz, L. A. P., Soliz, I. F. M., & Balarezo, V. K. G. (2025) · Future Internet · doi:10.3390/fi17060257 · Cited by 33 · see citing works
Álvarez-López, C., González-Briones, A., & Li, T. (2026) · Electronics · doi:10.3390/electronics15020385 · Cited by 23 · see citing works
Efficient Detection of Microplastics on Edge Devices With Tailored Compiler for TinyML Applications
Cerioli, A., Petrosino, L., Sasso, D., Laroche, C., Piechowiak, T., Pezzarossa, L., Merone, M., Vollero, L., & Sabatini, A. (2025) · IEEE Access · doi:10.1109/access.2025.3567816 · Cited by 10 · see citing works
A Flexible Template for Edge Generative AI With High-Accuracy Accelerated Softmax and GELU
Belano, A., Tortorella, Y., Garofalo, A., Luca Benini, C., Rossi, D., & Conti, F. (2025) · IEEE Journal on Emerging and Selected Topics in Circuits and Systems · doi:10.1109/jetcas.2025.3562734 · Cited by 9 · see citing works
Self-citationSinigaglia, M., Kiamarzi, A., Bertuletti, M., Ghionda, L., Orlandi, M., Tedeschi, R., Giampietro, A. D., Tortorella, Y., Bertaccini, L., Benatti, S., et al. (2025) · IEEE Transactions on Circuits and Systems I Regular Papers · doi:10.1109/tcsi.2025.3580939 · Cited by 3 · see citing works
Self-citationThroughput Maximization for Transformer Inference on Processing Near-Memory Architectures
Ge, M., Zhong, Y., Chen, S., & Kang, Y. (2025) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2025.3574235 · Cited by 3 · see citing works
Lin, C.-T., Moon, S., Huang, P. X., & Seok, M. (2025) · doi:10.1109/esserc66193.2025.11214062 · Cited by 2 · see citing works
Instruction Frequency-Based ALU Simplification on RISC-V for TinyML
Asai, Y., Pang, W. L., & Goh, H. H. (2025) · IEEE International Conference on Recent Advances and Innovations in Engineering (ICRAIE) · doi:10.1109/icraie65839.2025.11239153
Gupta, R., Sanghvi, V., & Shabista, S. (2026) · IEEE Consumer Communications and Networking Conference · doi:10.1109/ccnc65079.2026.11366556
Exploring Efficient Circuits and Systems for Acceleration of TinyML Applications
Alzoubi, E., & Masadeh, M. (2025) · International Conference on Decision Aid Sciences and Applications (DASA) · doi:10.1109/dasa68193.2025.11498729
- Authors
- Wiese, P., İslamoğlu, G., Scherer, M., Macan, L., Jung, V. J.-B., Burrello, A., Conti, F., & Benini, L.
- Affiliations
- ETH Zurich (CH)
- University of Bologna (IT)
- Politecnico di Torino (IT)
- Department of Information Technology and Electrical Engineering (D-ITET), ETH Zïrich, Zïrich, Switzerland
- ORCID
- Venue
- IEEE Design & Test
- Published
- 2025-10
- Volume
- 42
- Issue
- 5
- Pages
- 63-72
- ISSN
- 2168-2356 (ISSN-L)
- 2168-2364
- OpenAlex
- W4406172504
- Cited by
- 9
- References
- 12
- Citations by year
- 2026: 1 · 2025: 8
- FWCI
- 6.20
- Type
- journal article
- Language
- en
- Open access
- green
- hdl.handle.net
- Licence
- Locations
- IEEE Design and Test (publishedVersion)
- Repository for Publications and Research Data (ETH Zurich) (acceptedVersion, other-oa)
- arXiv (Cornell University) (submittedVersion, cc-by)
- PORTO@Iris (submittedVersion, other-oa)
- Archivio istituzionale della ricerca (Alma Mater Studiorum Università di Bologna) (submittedVersion, other-oa)
- arXiv (Cornell University) (cc-by)
- ETH Zürich Research Collection
- Funders
- Key Digital Technologies Joint Undertaking (101095947)
- HORIZON EUROPE Digital, Industry and Space (101070374, 101070634)
- Topics
- Physical Sciences › Computer Science › Hardware and Architecture › Embedded Systems Design Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Engineering › Electrical and Electronic Engineering › Low-power high-performance VLSI design
- Keywords
- TinyML, attention-based models, transformer inference, RISC-V, heterogeneous architectures, hardware accelerators, energy efficiency, FDSOI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
