Rethinking Generalization in American Sign Language Prediction for Edge Devices with Extremely Low Memory Footprint
Open accessThe DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Abstract
Due to the boom in technical compute in the last few years, the world has seen massive advances in artificially intelligent systems solving diverse real-world problems. But a major roadblock in the ubiquitous acceptance of these models is their enormous computational complexity and memory footprint. Hence efficient architectures and training techniques are required for deployment on extremely low resource inference endpoints. This paper proposes an architecture for detection of alphabets in American Sign Language on an ARM Cortex-M7 microcontroller having just 496 KB of framebuffer RAM. Leveraging parameter quantization is a common technique that might cause varying drops in test accuracy. This paper proposes using interpolation as augmentation amongst other techniques as an efficient method of reducing this drop, which also helps the model generalize well to previously unseen noisy data. The proposed model is about 185 KB post-quantization and inference speed is 20 frames per second.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In (Paul et al., 2020), an American sign language prediction aware architecture is presented.”
From A review on TinyML: State-of-the-art and prospects (2022) · cites this paper as Paul et al., 2020
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Authors in [28] proposed a model to detect the American Sign language (ASL) Alphabet and transcribed to text and speech in real time on tiny wearable IoT devices.”
From TinyML: Enabling of Inference Deep Learning Models on Ultra-Low-Power IoT Edge Devices for AI Applications (2022) · cites this paper as [28]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The outcomes after deploying the model on the device showed a model with five interpolation strategies outperforming another model with 98.84% accuracy and 20 frames per second (FPS), whilst the accuracy of the model with basic augmentation techniques was 95.24%.”
From TinyML: Enabling of Inference Deep Learning Models on Ultra-Low-Power IoT Edge Devices for AI Applications (2022) · cites this paper as [28]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“A generalization is aimed to predict for ASL fingerspelling with 98.84% test accuracy.”
From A review on TinyML: State-of-the-art and prospects (2022) · cites this paper as Paul et al., 2020
The cited source says what the claim attributes to it.
In the citing paper
“The generalizability of 74.58% is achieved in this study and can be exploited at any ARM MCUs.”
From A review on TinyML: State-of-the-art and prospects (2022) · cites this paper as Paul et al., 2020
2 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Understanding deep learning requires rethinking generalization
Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O. (2016) · arXiv · doi:10.48550/arxiv.1611.03530 · Cited by 2,838 · cited 2×
“Some other interesting problems that must be tackled for efficient ASL recognition are high generalization error in machine learning, where the network is limited to a tiny size and cannot memorize the data but has to learn meaningful representations [18], and effects of color constancy errors like white balance distu…”
BackgroundSpelling it out: Real-time ASL fingerspelling recognition
Pugeault, N., & Bowden, R. (2011) · 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops) · doi:10.1109/iccvw.2011.6130290 · Cited by 415 · cited 1×
“Various feature extraction methods have been proposed and implemented like Scale-invariant Feature Transform (SIFT) [2][3], Wavelet Moments [4], Histogram of Oriented Gradients (HOG) [5] and Gabor Filters [6].”
MethodAmerican Sign Language alphabet recognition using Microsoft Kinect
Dong, C., Leu, M. C., & Yin, Z. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2015.7301347 · Cited by 240
American Sign Language Recognition Using Leap Motion Controller with Machine Learning Approach
Chong, T.-W., & Lee, B.-G. (2018) · Sensors · doi:10.3390/s18103554 · Cited by 161 · cited 1×
“However, most edge devices right now can barely be expected to have Kinect sensors like in [1], or Leap Motion Controllers like in [9][10], while being affordable, and cybergloves like in [11] are impractical outside a laboratory setting.”
BackgroundReBNet: Residual Binarized Neural Network
Ghasemzadeh, M., Samragh, M., & Koushanfar, F. (2018) · 2018 IEEE 26th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM) · doi:10.1109/fccm.2018.00018 · Cited by 132 · cited 1×
“Recent advances in Neural Network design and performance analysis in ARM Cortex-M CPUs [15], and newer architecture improvements like residual binarized networks [16] and memory-optimal direct convolutions have ushered in newer apostles of efficiency in CNNs with TinyML [17].”
BackgroundGesture Recognition of RGB and RGB-D Static Images Using Convolutional Neural Networks
Crespo, R. A. G., Verdú, E., Khari, M., & Garg, A. K. (2019) · International Journal of Interactive Multimedia and Artificial Intelligence · doi:10.9781/ijimai.2019.09.002 · Cited by 117 · cited 1×
“Classifiers like Artificial Neural Networks (ANNs) [7], Supports Vector Machines (SVMs) [8], and Decision Trees (DTs) [9] have been shown to be robust enough for ASL recognition.”
BackgroundAfifi, M., & Brown, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00033 · Cited by 116 · cited 1×
“Some other interesting problems that must be tackled for efficient ASL recognition are high generalization error in machine learning, where the network is limited to a tiny size and cannot memorize the data but has to learn meaningful representations [18], and effects of color constancy errors like white balance distu…”
BackgroundSign Language Translation Using Deep Convolutional Neural Networks
Abiyev, R. H., Arslan, M., & Idoko, J. B. (2020) · KSII Transactions on Internet and Information Systems · doi:10.3837/tiis.2020.02.009 · Cited by 98 · cited 1×
“Classifiers like Artificial Neural Networks (ANNs) [7], Supports Vector Machines (SVMs) [8], and Decision Trees (DTs) [9] have been shown to be robust enough for ASL recognition.”
BackgroundÖz, C., & Leu, M. C. (2005) · Lecture notes in computer science · doi:10.1007/11427445_25 · Cited by 43 · cited 1×
“However, most edge devices right now can barely be expected to have Kinect sensors like in [1], or Leap Motion Controllers like in [9][10], while being affordable, and cybergloves like in [11] are impractical outside a laboratory setting.”
BackgroundReal Time Hand Gesture Recognition Using SiftPreprint
Gurjal, P. S., & Kunnur, K. S. (2012) · viXra · Cited by 30
Research works printed but not linked: 3
- [14] V. Bheda and D. Radpour, 'Using Deep Convolutional Networks for Gesture Recognition in American Sign Language ,' 2017. -
- [15] L. Lai, N. Suda, and V. Chandra, 'CMSIS -NN: Efficient Neural Network Kernels for Arm CortexM CPUs,' Jan. 2018, [Online]. Available: http://arxiv.org/abs/1801.06601. - · arXiv
- [2] N. H. Dardas and N. D. Georganas, 'Real -Time Hand Gesture Detection and Recognition.pdf,' vol. 60, no. 11, pp. 35923607, 2011. -
Cited by
A review on TinyML: State-of-the-art and prospects
Ray, P. P. (2022) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2021.11.019 · Cited by 460
“In (Paul et al., 2020), an American sign language prediction aware architecture is presented.”
Machine Learning for Microcontroller-Class Hardware: A Review
Saha, S. S., Sandha, S. S., & Srivastava, M. (2022) · IEEE Sensors Journal · doi:10.1109/jsen.2022.3210773 · Cited by 253
TinyML for Ultra-Low Power AI and Large Scale IoT Deployments: A Systematic Review
Schizas, N., Karras, A., Karras, C., & Sioutas, S. (2022) · Future Internet · doi:10.3390/fi14120363 · Cited by 215
Tiny Machine Learning: Progress and Futures [Feature]
Lin, J., Zhu, L., Chen, W.-M., Wang, W.-C., & Han, S. (2023) · IEEE Circuits and Systems Magazine · doi:10.1109/mcas.2023.3302182 · Cited by 186
“TinyML is also capable of predicting and recognizing sign languages [28]. with the capability to perform anomaly detection to reduce human efforts [32, 33, 34].”
Alajlan, N. N., & Ibrahim, D. M. (2022) · Micromachines · doi:10.3390/mi13060851 · Cited by 224
“Images Handwritten digits [38] Sign MNIST dataset from Kaggle [39] Kaggle ASL dataset (26 classes) [40] ASL dataset created by authors [28] Kaggle ASL Alphabet test set.”
Method
Metadata
- Authors
- Paul, A. J., Mohan, P., & Sehgal, S.
- Affiliations
- SRM Institute of Science and Technology (IN)
- Venue
- 2020 IEEE Recent Advances in Intelligent Computational Systems (RAICS)
- Publisher
- IEEE
- Published
- 2020-12-03
- Pages
- 147-152
- OpenAlex
- W3107838996
- Cited by
- 32
- References
- 15
- Citations by year
- 2026: 1 · 2025: 2 · 2024: 8 · 2023: 5 · 2022: 9 · 2021: 7
- FWCI
- 3.84
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Recent Advances in Intelligent Computational Systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Hand Gesture Recognition Systems
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Keywords
- American Sign Language, sign language recognition, edge inference, low memory footprint, parameter quantization, model generalization, noisy data robustness, lightweight neural networks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Rethinking Generalization in American Sign Language Prediction for Edge Devices with Extremely Low Memory Footprint. https://app.cochat.ai/papers/10.1109/raics51191.2020.9332480
- The paper
- Paul, A. J., Mohan, P., & Sehgal, S. (2020). Rethinking Generalization in American Sign Language Prediction for Edge Devices with Extremely Low Memory Footprint. 2020 IEEE Recent Advances in Intelligent Computational Systems (RAICS). https://doi.org/10.1109/raics51191.2020.9332480
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.