Domain-Adaptive Online Active Learning for Real-Time Intelligent Video Analytics on Edge Devices
Open accessThe DOI matches its Crossref record; 39 of the 46 references it cites were found in a registry or an index.
Abstract
Deep learning (DL) for intelligent video analytics is increasingly pervasive in various application domains, ranging from Healthcare to Industry 5.0. A significant trend involves deploying DL models on edge devices with limited resources. Techniques, such as pruning, quantization, and early exit, have demonstrated the feasibility of real-time inference at the edge by compressing and optimizing deep neural networks (DNNs). However, adapting pretrained models to new and dynamic scenarios remains a significant challenge. While solutions like domain adaptation, active learning (AL), and teacher-student knowledge distillation (KD) contribute to addressing this challenge, they often rely on cloud or well-equipped computing platforms for fine tuning. In this study, we propose a framework for domain-adaptive online AL of DNN models tailored for intelligent video analytics on resource-constrained devices. Our framework employs a KD approach where both teacher and student models are deployed on the edge device. To determine when to retrain the student DNN model without ground-truth or cloud-based teacher inference, our model utilizes singular value decomposition of input data. It implements the identification of key data frames and efficient retraining of the student through the teacher execution at the edge, aiming to prevent model overfitting. We evaluate the framework through two case studies: 1) human pose estimation and 2) car object detection, both implemented on an NVIDIA Jetson NX device.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566 · cited 1×
“…pose estimation (HPE) and 2) automotive OD, implemented using standard and widely used platforms (TRTPose [13], OpenPose [14], YOLO [15], single shot multibox detector (SSD) [16]) employing DNN architectures, such as ResNet [17], DenseNet [18], and Darknet [19], on an NVIDIA Jetson NX edge device.”
MethodMicrosoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460 · cited 3×
“Our evaluation encompasses both standard datasets (COCO [20], H3.”
MethodSSD: Single Shot MultiBox Detector
Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., & Berg, A. C. (2015) · arXiv · doi:10.48550/arxiv.1512.02325 · Cited by 22,443
SSD: Single Shot MultiBox Detector
Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., & Berg, A. C. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_2 · Cited by 22,128
The pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599 · cited 2×
“6M [21], VOC [22], LaSOT [23]) and a real-world scenario (i.e., an HPE dataset collected on a smart manufacturing line) for quantitative assessment.”
BackgroundMethodDistilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · cited 1×
“A promising approach is the knowledge distillation (KD) [11], which relies on predictions made by a heavier and more accurate model (referred to as the teacher ) to transfer knowledge to a light-weight model (referred to as the student ).”
BackgroundYOLOv7: Trainable Bag-of-Freebies Sets New State-of-the-Art for Real-Time Object Detectors
Wang, C.-Y., Bochkovskiy, A., & Liao, H.-Y. M. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.00721 · Cited by 13,198 · cited 3×
“We adopted state of the art models for OD, such as YOLO [15] and SSD [16], which provide high speed and accuracy for real-time detection tasks.”
MethodHoward, A., Sandler, M., Chen, B., Wang, W., Chen, L.-C., Tan, M., Chu, G., Vasudevan, V., Zhu, Y., Pang, R., et al. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00140 · Cited by 9,708 · cited 1×
“Examples are model compression techniques, such as neural network pruning [5], quantization [6], compact network design [7], as well as strategies like early exit [8].”
MethodOpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields
Cao, Z., Hidalgo, G., Simon, T., Wei, S.-E., & Sheikh, Y. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2929257 · Cited by 5,536 · cited 3×
“It employs CNNs and platforms, such as TRTPose [13] and OpenPose [14].”
MethodThe Approximation of One Matrix by Another of Lower Rank
Eckart, C., & Young, G. (1936) · Psychometrika · doi:10.1007/bf02288367 · Cited by 3,800 · cited 1×
“According to the Eckart-Young Theorem [42], the best-possible low-rank approximation of matrix M obtained by minimizing the Frobenius norm of the difference between M and the approximated matrix ˜ M is achieved through the truncated SVD of M .”
MethodHuman3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments
Ionescu, C., Papava, D., Olaru, V., & Sminchisescu, C. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.248 · Cited by 3,653 · cited 2×
“6M [21], VOC [22], LaSOT [23]) and a real-world scenario (i.e., an HPE dataset collected on a smart manufacturing line) for quantitative assessment.”
BackgroundSimple Baselines for Human Pose Estimation and Tracking
Xiao, B., Wu, H., & Wei, Y. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01231-1_29 · Cited by 2,149 · cited 1×
“…pose estimation (HPE) and 2) automotive OD, implemented using standard and widely used platforms (TRTPose [13], OpenPose [14], YOLO [15], single shot multibox detector (SSD) [16]) employing DNN architectures, such as ResNet [17], DenseNet [18], and Darknet [19], on an NVIDIA Jetson NX edge device.”
MethodThe singular value decomposition: Its computation and some applications
Klema, V., & Laub, A. (1980) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1980.1102314 · Cited by 1,631 · cited 1×
“The matrix M can be decomposed into three submatrices using the SVD method [41], as expressed in where U and V are orthonormal matrices of dimensions R n × n and R 2 J × 2 J , respectively, and they contain information about the spatial and temporal properties among the samples.”
MethodBranchyNet: Fast inference via early exiting from deep neural networks
Teerapittayanon, S., McDanel, B., & Kung, H. T. (2016) · 2016 23rd International Conference on Pattern Recognition (ICPR) · doi:10.1109/icpr.2016.7900006 · Cited by 1,326 · cited 1×
“Examples are model compression techniques, such as neural network pruning [5], quantization [6], compact network design [7], as well as strategies like early exit [8].”
MethodKhan, W. Z., Ahmed, E., Hakak, S., Yaqoob, I., & Ahmed, A. (2019) · Future Generation Computer Systems · doi:10.1016/j.future.2019.02.050 · Cited by 1,157 · cited 1×
“The deployment of convolutional neural network (CNN)-based DL models directly on resource-constrained edge devices [4] has gained considerable attention and various solutions have been proposed to achieve lightweight, efficient and accurate inference at the edge.”
BackgroundLearning Loss for Active Learning
Yoo, D., & Kweon, I. S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00018 · Cited by 740 · cited 1×
“Yoo and Kweon [27] developed a loss prediction model that learns how to limit the loss defined in the target model.”
BackgroundEnergy-based Out-of-distribution Detection
Liu, W., Wang, X., Owens, J. D., & Li, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.03759 · Cited by 412
LaSOT: A High-quality Large-scale Single Object Tracking Benchmark
Fan, H., Bai, H., Lin, L., Yang, F., Chu, P., Deng, G., Yu, S., Harshit, Huang, M., Liu, J., et al. (2021) · International Journal of Computer Vision · doi:10.1007/s11263-020-01387-y · Cited by 360 · cited 2×
“To achieve this, we utilized sequences containing cars from the LaSOT [23] dataset as the test sequence for our OAL.”
BackgroundMethodPerformance animation from low-dimensional control signals
Chai, J., & Hodgins, J. K. (2005) · ACM Transactions on Graphics · doi:10.1145/1073204.1073248 · Cited by 342
EagleEye: Fast Sub-net Evaluation for Efficient Neural Network Pruning
Li, B., Wu, B., Su, J., & Wang, G. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58536-5_38 · Cited by 163 · cited 1×
“Examples are model compression techniques, such as neural network pruning [5], quantization [6], compact network design [7], as well as strategies like early exit [8].”
MethodA Survey on Green Deep LearningPreprint
Xu, J., Zhou, W., Fu, Z., Zhou, H., & Li, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.05193 · Cited by 159
A Survey on Spatio-temporal Data Analytics Systems
Alam, M. M., Torgo, L., & Bifet, A. (2022) · ACM Computing Surveys · doi:10.1145/3507904 · Cited by 140 · cited 2×
“When applied to a sequence of streaming video, DNNs generate spatio-temporal patterns that are essential for tasks, such as motion analysis, quality enhancement, object detection (OD), and action recognition [3].”
MethodOnline Model Distillation for Efficient Video Inference
Mullapudi, R. T., Chen, S., Zhang, K., Ramanan, D., & Fatahalian, K. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00367 · Cited by 96 · cited 6×
“The blue line, on the other hand, represents the OAL model inspired by [12].”
BackgroundResultInfluentialReal-Time Video Inference on Edge Devices via Adaptive Model Streaming
Khani, M., Hamadanian, P., Nasr-Esfahany, A., & Alizadeh, M. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00453 · Cited by 83 · cited 1×
“Khani et al. [33] proposed Adaptive Model Streaming, an edge–cloud method to boost lightweight model performance for real-time video inference.”
Method3-D Canonical Pose Estimation and Abnormal Gait Recognition With a Single RGB-D Camera
Guo, Y., Deligianni, F., Gu, X., & Yang, G. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2928775 · Cited by 77 · cited 1×
“Deep neural networks (DNNs) are particularly adapt at extracting spatial patterns from singular video frames.”
BackgroundSingular Value Decomposition (SVD)
Brunton, S. L., & Kutz, J. N. (2019) · Cambridge University Press eBooks · doi:10.1017/9781108380690.002 · Cited by 76
Mehlin, V., Schacht, S., & Lanquillon, C. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.01980 · Cited by 69 · cited 1×
“Since the training process is computationally heavy and resource-intensive, researchers have employed techniques to improve training efficiency [9], including the use of active learning (AL).”
BackgroundActive Learning for Human Pose Estimation
Liu, B., & Ferrari, V. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.468 · Cited by 67 · cited 1×
“Liu and Ferrari [26] proposed an AL framework to reduce the amount of annotations required in a large unlabeled dataset.”
BackgroundMultilayer Joint Gait-Pose Manifolds for Human Gait Motion Modeling
Ding, M., & Fan, G. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2014.2373393 · Cited by 60 · cited 1×
“Given the inherent connectivity of body segments through physical articulations and the extensive use of motion synergies by the motor cortex in human motion [37], [38], [39], it is possible to efficiently represent sample sequences in lower dimensions.”
BackgroundSparsity-Inducing Binarized Neural Networks
Wang, P., He, X., Li, G., Zhao, T., & Cheng, J. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6900 · Cited by 49 · cited 1×
“Examples are model compression techniques, such as neural network pruning [5], quantization [6], compact network design [7], as well as strategies like early exit [8].”
MethodMotion Capture Data Completion and Denoising by Singular Value Thresholding
Lai, R. Y. Q., Yuen, P. C., & Lee, K. K. W. (2011) · HKBU Institutional Repository (Hong Kong Baptist University) · doi:10.2312/eg2011/short/045-048 · Cited by 48
Compressed Singular Value Decomposition for Image and Video Processing
Erichson, N. B., Brunton, S. L., & Kutz, J. N. (2017) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2017.222 · Cited by 35 · cited 1×
“One common application in video processing is low-rank approximation for image compression [36].”
BackgroundOnline Domain Adaptation for Semantic Segmentation in Ever-Changing Conditions
Panagiotakopoulos, T., Dovesi, P. L., Härenstam-Nielsen, L., & Poggi, M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19830-4_8 · Cited by 29
Stream-Based Active Distillation for Scalable Model Deployment
Manjah, D., Cacciarelli, D., Benkedadra, M., Standaert, B., de Hertaing, G. R., Macq, B. M. M., Galland, S., & Vleeschouwer, C. D. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00528 · Cited by 17 · cited 1×
“Manjah et al. [32] introduced a real-time OAL framework aimed at fine-tuning a lightweight model for OD in videos.”
BackgroundSimilarity of human motion: congruity between perception and data
Miura, K., Furukawa, H., & Shoji, M. (2006) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.2006.384875 · Cited by 13 · cited 1×
“Given the inherent connectivity of body segments through physical articulations and the extensive use of motion synergies by the motor cortex in human motion [37], [38], [39], it is possible to efficiently represent sample sequences in lower dimensions.”
BackgroundLim, J., Lee, J., Lee, C., Kim, G.-W., Cha, Y., Sim, J., & Rhim, S. (2021) · IEEE International Conference on Intelligence and Safety for Robotics (ISR) · doi:10.1109/isr50024.2021.9419504 · Cited by 12 · cited 1×
“industrial environments [2].”
Lu, P.-Y., Cheng, Y.-J., Li, C., & Lin, H.-T. (2023) · Transactions on machine learning research · doi:10.48550/arxiv.2306.08954 · Cited by 11 · cited 1×
“Lu et al. [25] proposed a performance benchmark of AL strategies for binary classification.”
BackgroundDeep Active Learning For Human Pose Estimation Via Consistency Weighted Core-Set Approach
Zhang, W., Guo, Z., Zhi, R., & Baofeng, W. (2021) · Proceedings - International Conference on Image Processing · doi:10.1109/icip42928.2021.9506630 · Cited by 8 · cited 1×
“Zhang et al. [28] proposed a query strategy that selects frames balancing between information representativeness and uncertainty.”
BackgroundSpatio-temporal analysis and comparison of 3D videos
Cammarasana, S., & Patanè, G. (2022) · The Visual Computer · doi:10.1007/s00371-022-02409-1 · Cited by 5 · cited 1×
“Spatio-temporal models [34] represent a distinct class of models designed to process spatio-temporal data, such as those used in HPE and OD.”
Background
Research works printed but not linked: 4
- 2020, NVIDIA AI IoT
- Redmon, 2016, DarkNet: Open source neural networks in C.
- Shukla, VL4Pose: Active learning through out-of-distribution detection for pose estimation, 1
- Cacciarelli, 2023, A survey on online active learning, arXiv:2302.08893 · arXiv
Cited by
A survey on multi-modal and weakly supervised approaches for robust anomaly detection in video data
Barbosa, R. Z., Oliveira, H. S., & Tavares, J. M. R. S. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103388 · Cited by 12
Computational Relevance of Model Pruning and Quantization for Low-Powered AI
Mandal, A., Hasan, N., Jain, A., Mohanty, J. R., & Sharma, V. (2025) · International Conference on Multidisciplinary Research and Innovations in Engineering (MRIE) · doi:10.1109/mrie66930.2025.11156676
Data-Efficient Stream-Based Active Distillation For Scalable Edge Model Deployment
Manjah, D., Bary, T., Gérin, B., Macq, B. M. M., & Vleeschouwer, C. D. (2025) · IEEE International Conference on Image Processing Workshops (ICIPW) · doi:10.1109/icipw68931.2025.11385953
“Future work could investigate alternative initial sampling schemes, e.g., the methods of [10, 28] to broaden the range of starting points.”
Metadata
- Authors
- Boldo, M., De Marchi, M., Martini, E., Aldegheri, S., & Bombieri, N.
- Affiliations
- University of Verona (IT)
- Department of Engineering for Innovation Medicine, University of Verona, Verona, Italy
- Venue
- IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
- Published
- 2024-11
- Volume
- 43
- Issue
- 11
- Pages
- 4105-4116
- ISSN
- 0278-0070 (ISSN-L)
- 1937-4151
- OpenAlex
- W4404103113
- Semantic Scholar
- 9e5d0c9ce6472f52e515f2507f8c8cef14224c34
- Cited by
- 4
- References
- 46
- Citations by year
- 2026: 1 · 2025: 3
- FWCI
- 0.59
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Keywords
- domain adaptation, online active learning, edge computing, intelligent video analytics, knowledge distillation, teacher-student models, singular value decomposition, human pose estimation, resource-constrained devices, real-time inference
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.