The DOI matches its Crossref record; 24 of the 25 references it cites were found in a registry or an index.
Chameleon: scalable adaptation of video analytics
Abstract
Applying deep convolutional neural networks (NN) to video data at scale poses a substantial systems challenge, as improving inference accuracy often requires a prohibitive cost in computational resources. While it is promising to balance resource and accuracy by selecting a suitable NN configuration (e.g., the resolution and frame rate of the input video), one must also address the significant dynamics of the NN configuration's impact on video analytics accuracy. We present Chameleon, a controller that dynamically picks the best configurations for existing NN-based video analytics pipelines. The key challenge in Chameleon is that in theory, adapting configurations frequently can reduce resource consumption with little degradation in accuracy, but searching a large space of configurations periodically incurs an overwhelming resource overhead that negates the gains of adaptation. The insight behind Chameleon is that the underlying characteristics (e.g., the velocity and sizes of objects) that affect the best configuration have enough temporal and spatial correlation to allow the search cost to be amortized over time and across multiple video feeds. For example, using the video feeds of five traffic cameras, we demonstrate that compared to a baseline that picks a single optimal configuration offline, Chameleon can achieve 20-50% higher accuracy with the same amount of resources, or achieve the same accuracy with only 30--50% of the resources (a 2-3X speedup).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 1×
“, [17, 18]) which have led to a continuous stream of increasingly accurate models for object detection and classification.”
BackgroundHe, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105
The pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015
Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning
Szegedy, C., Ioffe, S., Vanhoucke, V., & Alemi, A. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11231 · Cited by 4,636
Speed/Accuracy Trade-Offs for Modern Convolutional Object Detectors
Huang, J., Rathod, V., Sun, C., Zhu, M., Korattikara, A., Fathi, A., Fischer, I., Wojna, Z., Song, Y., Guadarrama, S., et al. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.351 · Cited by 2,743 · cited 1×
“The inference models are implemented in Tensorflow and are pretrained on standard image datasets [23], and the switching of video frame rate and resolution is done by FFmpeg [10].”
Method(1993) · Choice Reviews Online · doi:10.5860/choice.31-0986 · Cited by 1,540
Analysis of Thompson Sampling for the multi-armed bandit problem
Agrawal, S., & Goyal, N. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1111.1797 · Cited by 819 · cited 1×
“al [19] use Thompson sampling [11] to optimize the layout of a multi-component web page; they use greedy hill climbing to select the next layout, similar to Chameleon, but Chameleon exploits more independence structure and monotonicity.”
MethodReal-Time Video Analytics: The Killer App for Edge Computing
Ananthanarayanan, G., Bahl, P., Bodik, P., Chintalapudi, K., Philipose, M., Ravindranath, L., & Sinha, S. (2017) · Computer · doi:10.1109/mc.2017.3641638 · Cited by 496 · cited 2×
“As we move to the scenario of edge camera analytics [13], such a separation would be hard to achieve.”
BackgroundSelf-citation
Research works printed but not linked: 2
- FFmpeg. http://ffmpeg.org/ , 2000 --2018. FFmpeg. http://ffmpeg.org/, 2000--2018.
- W. E. Hoover and M. Rockville . Algorithms for confidence circles and ellipses . Citeseer , 1984 . W. E. Hoover and M. Rockville. Algorithms for confidence circles and ellipses. Citeseer, 1984.
Cited by
Edge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing
Zhou, Z., Chen, X., Li, E., Zeng, L., Luo, K., & Zhang, J. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2918951 · Cited by 2,455
Deep Learning With Edge Computing: A Review
Chen, J., & Ran, X. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2921977 · Cited by 1,540
“[120] M. S. Riazi, C. Weinert, O. Tkachenko, E. M. Songhori, T. Schneider, and F. Koushanfar, “Chameleon: A hybrid secure computation framework for machine learning applications,” in Proc.”
MethodInfluentialConvergence of Edge Computing and Deep Learning: A Comprehensive Survey
Wang, X., Han, Y., Leung, V. C. M., Niyato, D., Yan, X., & Chen, X. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2970550 · Cited by 1,498
“In order to adjust the configuration of the input video stream (such as frame resolution and sampling rate) online with low cost, Chameleon [138] greatly saves the cost of searching the best model configuration by leveraging temporal and spatial correlations of the video inputs, and allows the cost”
BackgroundMethodInfluentialEdge Assisted Real-time Object Detection for Mobile Augmented Reality
Liu, L., Li, H., & Gruteser, M. (2019) · The 25th Annual International Conference on Mobile Computing and Networking · doi:10.1145/3300061.3300116 · Cited by 552
“VideoStorm [45] and Chameleon [46] achieve higher accuracy video analytics with the same amount of computational resources on the cloud by adapting the video configurations.”
BackgroundVideoEdge: Processing Camera Streams using Hierarchical Clusters
Hung, C.-C., Ananthanarayanan, G., Bodik, P., Golubchik, L., Yu, M., Bahl, P., & Philipose, M. (2018) · 2018 IEEE/ACM Symposium on Edge Computing (SEC) · doi:10.1109/sec.2018.00016 · Cited by 287
“Chameleon [35] is the recent video analytics work for continuously adjusting DNN configurations to optimize accuracy or reduce resources costs based on the temporal and spatial correlation among the video frames.”
BackgroundSelf-citationReducto: On-Camera Filtering for Resource-Efficient Real-Time Video Analytics
Li, Y., Padmanabhan, A., Zhao, P., Wang, Y., Xu, G. H., & Netravali, R. (2020) · Proceedings of the Annual conference of the ACM Special Interest Group on Data Communication on the applications, technologies, architectures, and protocols for computer communication · doi:10.1145/3387514.3405874 · Cited by 269
“We also compared Reducto with two systems that improve the efficiency of real-time video analytics pipelines, CloudSeg [57] and Chameleon [41].”
BackgroundMethodInfluentialEdge Intelligence in Intelligent Transportation Systems: A Survey
Gong, T., Zhu, L., Yu, F. R., & Tang, T. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3275741 · Cited by 256
Server-Driven Video Streaming for Deep Learning Inference
Du, K., Pervaiz, A., Yuan, X., Chowdhery, A., Zhang, Q., Hoffmann, H., & Jiang, J. (2020) · Proceedings of the Annual conference of the ACM Special Interest Group on Data Communication on the applications, technologies, architectures, and protocols for computer communication · doi:10.1145/3387514.3405887 · Cited by 243
Self-citationJoint Configuration Adaptation and Bandwidth Allocation for Edge-based Real-time Video Analytics
Wang, C., Zhang, S., Chen, Y., Qian, Z., Wu, J., & Xiao, M. (2020) · IEEE INFOCOM 2020 - IEEE Conference on Computer Communications · doi:10.1109/infocom41043.2020.9155524 · Cited by 226
“It has been experimentally observed in [10] that frame resolution and frame sampling rate independently impact accuracy, allowing us to model the accuracy of the configuration of uk in time slot t as k( ∑N i=0 x i k,tri)φ t k(fk,t).”
InfluentialKang, D. D., Bailis, P. D., & Zaharia, M. (2019) · Proceedings of the VLDB Endowment · doi:10.14778/3372716.3372725 · Cited by 208
“taipei, night-street, amsterdam, and archie are widely used in video analytics systems [9,37,40,46,70] and we collected two other streams.”
BackgroundMethodScaling Video Analytics on Constrained Edge NodesPreprint
Canel, C., Kim, T., Zhou, G., Li, C., Lim, H., Andersen, D. G., Kaminsky, M., & Dulloor, S. R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.13536 · Cited by 206
“Chameleon (Jiang et al., 2018b) dynamically adjusts a video processing pipeline’s hyperparameters as the content in the scene changes, using temporal and spatial (i.e., across nearby cameras) correlations to prune the optimization search space.”
BackgroundMethodEdge computing enabled video segmentation for real-time traffic monitoring in internet of vehicles
Wan, S., Ding, S., & Chen, C. (2022) · Pattern Recognition · doi:10.1016/j.patcog.2021.108146 · Cited by 194
Elf: accelerate high-resolution mobile deep vision with content-aware parallel offloading
Zhang, W., He, Z., Liu, L., Jia, Z., Liu, Y., Gruteser, M., Raychaudhuri, D., & Zhang, Y. (2021) · Proceedings of the 27th Annual International Conference on Mobile Computing and Networking · doi:10.1145/3447993.3448628 · Cited by 172
“Chameleon [76] dynamically selects the best configurations for existing NN-based video analytics to save computing resources by up to 10×.”
BackgroundEdge Video Analytics for Public Safety: A Review
Zhang, Q., Sun, H., Wu, X., & Zhong, H. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2925910 · Cited by 156
“[146] designed a controller, Chameleon, which dynamically sets the best configurations for deep convolutional neural network-based video analytic pipelines.”
BackgroundThe computing continuum: From IoT to the cloud
Al-Dulaimy, A., Jansen, M. F., Johansson, B., Trivedi, A., Iosup, A., Ashjaei, M., Galletta, A., Kimovski, D., Prodan, R., Tserpes, K., et al. (2024) · Internet of Things · doi:10.1016/j.iot.2024.101272 · Cited by 151
Distream: scaling live video analytics with workload-adaptive distributed edge intelligence
Zeng, X., Fang, B., Shen, H., & Zhang, M. (2020) · ACM Conference on Embedded Networked Sensor Systems (SenSys) · doi:10.1145/3384419.3430721 · Cited by 135
Edge Video Analytics: A Survey on Applications, Systems and Enabling Techniques
Xu, R., Razavi, S., & Zheng, R. (2023) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2023.3323091 · Cited by 134
“To ensure comprehensive coverage of research papers on EVA in the scope identified, we first collect papers using the tool Collected Papers , starting from several top-cited representative papers: [26], [27], [28], [29], [30].”
BackgroundMethodInfluentialEnergy-Aware Inference Offloading for DNN-Driven Applications in Mobile Edge Clouds
Xu, Z., Zhao, L., Liang, W., RANA, O. F., Zhou, P., Xia, Q., Xu, W., & Wu, G. (2020) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2020.3032443 · Cited by 128
“Neural networks and deep neural networks computations are gaining increasingly popularity to enable AI in mobile devices [3], [15], [21], [22], [24], [58].”
BackgroundLlama: A Heterogeneous & Serverless Framework for Auto-Tuning Video Analytics Pipelines
Romero, F., Zhao, M., Yadwadkar, N. J., & Kozyrakis, C. (2021) · ACM Symposium on Cloud Computing (SoCC) · doi:10.1145/3472883.3486972 · Cited by 128
DeepSlicing: Collaborative and Adaptive CNN Inference With Low Latency
Zhang, S., Zhang, S., Qian, Z., Wu, J., Jin, Y., & Lu, S. (2021) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2021.3058532 · Cited by 127
“, image classification [1], [2], [3], object detection [4], [5], [6] and video analytics [7], [8], [9].”
BackgroundArtificial Intelligence of Things: A Survey
Siam, M. S. I., Ahn, H., Liu, L., Alam, S., Shen, H., Cao, Z., Shroff, N. B., Krishnamachari, B., Srivastava, M. B., & Zhang, M. (2024) · ACM Transactions on Sensor Networks · doi:10.1145/3690639 · Cited by 123
Ekya: Continuous Learning of Video Analytics Models on Edge Compute ServersPreprint
Bhardwaj, R., Xia, Z., Ananthanarayanan, G., Jiang, J., Karianakis, N., Shu, Y., Hsieh, K., Bahl, V., & Stoica, I. G. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.10557 · Cited by 122
“Prior work has made dramatic advancements in developing profilers to efficiently obtain the resource-accuracy relationship for inference configurations [36].”
BackgroundMethodInfluentialSelf-citationZhao, K., Zhou, Z., Chen, X., Zhou, R., Zhang, X., Yu, S., & Wu, D. (2022) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2022.3189186 · Cited by 114
“In addition tomodel selection, application configuration adaption [18] is alsowidely applied to flexibly navigate the resource-latency-accuracy tradeoff in edge computing paradigm.”
MethodEdge Coordinated Query Configuration for Low-Latency and Accurate Video Analytics
Yang, P., Lyu, F., Wu, W. R., Zhang, N., Yu, L., & Shen, X. (2019) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2019.2949347 · Cited by 104
“Another line of research focuses on optimizing the configuration of neural networks, including Chameleon [21] and MCDNN [22], to reduce resource consumption, while maintaining accuracy.”
BackgroundEdge-assisted Online On-device Object Detection for Real-time Video Analytics
Hanyao, M., Jin, Y., Qian, Z., Zhang, S., & Lu, S. (2021) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom42981.2021.9488741 · Cited by 103
Edge Assisted Mobile Semantic Visual SLAM
Xu, J., Cao, H., Li, D., Huang, K., Qian, C., Shangguan, L., & Yang, Z. (2020) · IEEE INFOCOM 2020 - IEEE Conference on Computer Communications · doi:10.1109/infocom41043.2020.9155438 · Cited by 101
“VideoStrom [52] and Chameleon [53] achieve higher accuracy video analytics with the same amount of computational resources on the cloud by adapting the video configurations.”
BackgroundSurveilEdge: Real-time Video Query based on Collaborative Cloud-Edge Deep Learning
Wang, S., Yang, S., & Zhao, C. (2020) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom41043.2020.9155284 · Cited by 98
Adaptive Configuration Selection and Bandwidth Allocation for Edge-Based Video Analytics
Zhang, S., Wang, C., Jin, Y., Wu, J., Qian, Z., Xiao, M., & Lu, S. (2021) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2021.3106937 · Cited by 97
“A detected object is identified as true positive when its bounding box has the same label and it has sufficient spatial overlap with the corresponding ground truth [14].”
BackgroundMethodResultInfluentialOnline Model Distillation for Efficient Video Inference
Mullapudi, R. T., Chen, S., Zhang, K., Ramanan, D., & Fatahalian, K. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00367 · Cited by 96
“) Unlike other datasets for efficient inference, which consist of streams from fixed-viewpoint cameras such as traffic cameras [19], we capture a diverse array of challenges: from fixed-viewpoint cameras, to constantly moving and zooming television cameras, and hand-held and egocentric video.”
BackgroundFlexDNN: Input-Adaptive On-Device Deep Learning for Efficient Mobile Vision
Fang, B., Zeng, X., Zhang, F., Xu, H. F., & Zhang, M. (2020) · IEEE/ACM Symposium on Edge Computing (SEC) · doi:10.1109/sec50012.2020.00014 · Cited by 90
“Based on this observation, input-adaptive video analytics systems such as Chameleon [15] have recently emerged.”
BackgroundMethodInfluentialAutoML for Video Analytics with Edge Computing
Galanopoulos, A., Ayala‐Romero, J. A., Leith, D. J., & Iosifidis, G. (2021) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom42981.2021.9488704 · Cited by 90
“These interesting works either do not offer optimality bounds [10], [34] or consider asymptotic-only performance [35], assume known models [12], [34], [35] or convex functions [12].”
BackgroundInfluentialA Contemporary Survey on Live Video Streaming from a Computation-Driven Perspective
Dao, N., Tran, A.-T., Tu, N. H., Thanh, T. T., Bao, V. N. Q., & Cho, S. (2022) · ACM Computing Surveys · doi:10.1145/3519552 · Cited by 89
“Subsequently, the authors of [58] and [96] proposed MCDNN and Chameleon novel algorithms based on deep neural network (DNN) utilization, respectively, to frequently adapt configurations for LVS applications.”
BackgroundMethodDNN Surgery: Accelerating DNN Inference on the Edge Through Layer Partitioning
Liang, H., Sang, Q., Hu, C., Cheng, D., Zhou, X., Wang, D., Bao, W., & Wang, Y. (2023) · IEEE Transactions on Cloud Computing · doi:10.1109/tcc.2023.3258982 · Cited by 87
“The research question is how a cloud can support many DNN inference requests/queries with minimal cluster resource [15], [50].”
BackgroundThe Internet of Federated Things (IoFT)
Kontar, R. A., Shi, N., Yue, X., Chung, S., Byon, E., Chowdhury, M., Jin, J. J., Kontar, W., Masoud, N., Nouiehed, M., et al. (2021) · IEEE Access · doi:10.1109/access.2021.3127448 · Cited by 86
“Video Analytics: Cameras deployed for traffic control and surveillance continuously record and analyze large volumes of recorded video using video analytics [105, 118, 323], which has been made possible by recent advances in computer vision.”
MethodCASVA: Configuration-Adaptive Streaming for Live Video Analytics
Zhang, M., Wang, F., & Liu, J. (2022) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom48880.2022.9796875 · Cited by 86
ApproxDet: content and contention-aware approximate object detection for mobiles
Xu, R., Zhang, C.-L., Wang, P., Lee, J., Mitra, S., Chaterji, S., Li, Y., & Bagchi, S. (2020) · ACM Conference on Embedded Networked Sensor Systems (SenSys) · doi:10.1145/3384419.3431159 · Cited by 81
“ion. Approximate Video Analytics. Our work shares similar ideas to several recent works in video analytics in the systems community. For example, server-side solutions like VideoStorm [69], Chameleon [32], and Focus [25] exploit various configurations and DNN models to optimize video analytics queries, but they also r…”
Morphling: Fast, Near-Optimal Auto-Configuration for Cloud-Native Model Serving
Wang, L.-P., Yang, L.-Y., Yu, Y.-H., Wang, W., Li, B., Sun, X., He, J., & Zhang, L.-P. (2021) · ACM Symposium on Cloud Computing · doi:10.1145/3472883.3486987 · Cited by 71
VStore: A Data Store for Analytics on Large Videos
Xu, T., Botelho, L. M., & Lin, F. X. (2019) · European Conference on Computer Systems · doi:10.1145/3302424.3303971 · Cited by 81
Blockchain-based Collaborative Edge Intelligence for Trustworthy and Real-Time Video Surveillance
Zhang, M., Cao, J., Sahni, Y., Chen, Q., Jiang, S., & Yang, L. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3203397 · Cited by 78
Towards scalable edge-native applications
Wang, J., Feng, Z., George, S., Iyengar, R., Pillai, P. S., & Satyanarayanan, M. (2019) · ACM/IEEE Symposium on Edge Computing (SEC) · doi:10.1145/3318216.3363308 · Cited by 75
“Others have investigated exploiting multiple deep models with accuracy and resource tradeoffs [11, 16].”
BackgroundSensor-Augmented Neural Adaptive Bitrate Video Streaming on UAVs
Xiao, X., Wang, W., Chen, T., Cao, Y., Jiang, T., & Zhang, Q. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2945167 · Cited by 73
“[22] leverage the correlations of video content to dynamically pick the best video configurations.”
BackgroundEdgeDuet: Tiling Small Object Detection for Edge Assisted Autonomous Mobile Vision
Wang, X., Yang, Z., Wu, J., Zhao, Y., & Zhou, Z. (2021) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom42981.2021.9488843 · Cited by 72
“Therefore we use the outputs (cars and pedestrians) of a YOLOv3-spp (2560x2560) model to re-annotate these videos and treat them as the ground truth, as with other recent studies on video analytics [15], [37], [38].”
MethodODIN: automated drift detection and recovery in video analytics
Suprem, A., Arulraj, J. J. P., Pu, C., & Ferreira, J. E. (2020) · Proceedings of the VLDB Endowment · doi:10.14778/3407790.3407837 · Cited by 68
“For instance, in case of a traffic surveillance dataset, it uses an expensive, more accurate model for object detection under high traffic conditions and a slower, less accurate model otherwise [14].”
BackgroundMethodVisor: Privacy-Preserving Video Analytics as a Cloud ServicePreprint
Poddar, R., Ananthanarayanan, G., Setty, S., Volos, S., & Popa, R. A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.09628 · Cited by 68
“The choice of pipeline modules is application dependent [36, 44] and Visor targets confidentiality for all pipeline modules, their different combinations, and vision CNNs.”
BackgroundMethodInfluentialSelf-citationVideo Analytics - Killer App for Edge Computing
Ananthanarayanan, G., Bahl, V., Cox, L. P., Crown, A., Nogbahi, S., & Shu, Y. (2019) · International Conference on Mobile Systems, Applications, and Services · doi:10.1145/3307334.3328589 · Cited by 67
Self-citationVaBUS: Edge-Cloud Real-Time Video Analytics via Background Understanding and Subtraction
Wang, H., Li, Q., Sun, H., Chen, Z., Hao, Y., Peng, J., Yuan, Z., Fu, J., & Jiang, Y. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2022.3221995 · Cited by 66
“…optimization techniques including model distillation [26], splitting [27], sharing [28] and cascading [29], adaptively tuning a set of control knobs to constantly meet accuracy and latency requirements [7], [30], [31], optimized information pruning techniques [11], [12], [13], [14], etc.”
MethodALERT: Accurate Learning for Energy and TimelinessPreprint
Wan, C., Santriaji, M. H., Rogers, E. A., Hoffmann, H., Maire, M., & Lu, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.00119 · Cited by 66
“Image classification on video might have deadlines from 1 second down to the camera latency (e.g., 1/60 seconds) [47].”
BackgroundJellyfish: Timely Inference Serving for Dynamic Edge Networks
Nigade, V., Bauszat, P., Bal, H. E., & Wang, L. (2022) · IEEE Real-Time Systems Symposium (RTSS) · doi:10.1109/rtss55097.2022.00032 · Cited by 65
“Recent works such as VideoStorm [42], AWStream [23], Chameleon [43], DeepDeci-sion [24], JCAB [34], DDS [26], and SPINN [30] have proposed adaptive solutions for networked video analytics.”
BackgroundMethodDecomposable Intelligence on Cloud-Edge IoT Framework for Live Video Analytics
Zhang, Y., Liu, J.-H., Wang, C.-Y., & Wei, H. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2997091 · Cited by 63
“The configuration selection impacts resource consumption, latency, and accuracy of the video analytics [13].”
BackgroundContinuous, Real-Time Object Detection on Mobile Devices without Offloading
Liu, M., Ding, X., & Du, W. (2020) · IEEE International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs47774.2020.00085 · Cited by 63
“We use the detection result from the previous frame [33].”
BackgroundMethodInfluentialModel Assertions for Monitoring and Improving ML ModelsPreprint
Kang, D. D., Raghavan, D., Bailis, P. D., & Zaharia, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.01668 · Cited by 63
“Many modern, academic video analytics systems use an object detection method (Kang et al., 2017; 2019; Hsieh et al., 2018; Jiang et al., 2018; Xu et al., 2019; Canel et al., 2019) trained on MS-COCO (Lin et al., 2014), a corpus of still images.”
BackgroundMethodA wearable obstacle avoidance device for visually impaired individuals with cross-modal learning
Gao, Y., Wu, D., Song, J., Zhang, X., Hou, B., Liu, H., Liao, J., & Zhou, L. (2025) · Nature Communications · doi:10.1038/s41467-025-58085-x · Cited by 62
“Recent research on video analytics tasks (e.g., object detection, semantic segmentation) has argued that discarding frames or setting lower resolution for pixels irrelevant to target tasks may improve compression ef fi ciency without impacting task performance 35 – 37 .”
AccMPEG: Optimizing Video Encoding for Video AnalyticsPreprint
Du, K., Zhang, Q., Arapin, A., Wang, H., Xia, Z., & Jiang, J. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2204.12534 · Cited by 62
“, 2018) or downsize the quality/framerate of an entire video segment (Xu et al., 2019; Jiang et al., 2018; Zhang et al., 2017; Haris et al., 2018), offload inference of RoI bounding boxes (Zhang et al.”
BackgroundMethodSelf-citationTowards cloud-edge collaborative online video analytics with fine-grained serverless pipelines
Zhang, M., Wang, F., Zhu, Y., Liu, J., & Wang, Z. (2021) · ACM Multimedia Systems Conference (MMSys) · doi:10.1145/3458305.3463377 · Cited by 60
“To further quantify video content dynamics, we define the average objects per frame (AOPF) metric and the average unique objects per frame (AUOPF) metric as follows: We present the variations of these two metrics over time in Figure 3a and Figure 3b ( 𝑇 is set to 1 minute in these two figures).”
BackgroundMethodApproximate query service on autonomous IoT cameras
Xu, M., Zhang, X., Liu, Y., Huang, G., Liu, X., & Lin, F. X. (2020) · International Conference on Mobile Systems, Applications, and Services · doi:10.1145/3386901.3388948 · Cited by 60
“gn goal of AutCam is to best use limited energy for answering queries with high confidence, i.e., with both bounded and low errors. This problem is never addressed in existing video analytics systems [56, 59, 60, 98, 99, 103] to our knowledge. 3.1 Query On a camera, AutCam runs a query to continuously count objects in…”
BackgroundInfluentialLiu, S., Wang, T., Li, J., Sun, D., Srivastava, M. B., & Abdelzaher, T. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548033 · Cited by 59
“, resolution, frame rate) designed for human perception in the adaptation [4, 6, 24, 27, 47, 56, 57].”
BackgroundΣακάς, Δ. Π., Reklitis, D. P., Giannakopoulos, N. T., & Trivellas, P. (2023) · European Research on Management and Business Economics · doi:10.1016/j.iedeen.2023.100221 · Cited by 57
ePerceptive: energy reactive embedded intelligence for batteryless sensors
Montanari, A., Sharma, M., Jenkus, D., Alloulah, M., Qendro, L., & Kawsar, F. (2020) · ACM Conference on Embedded Networked Sensor Systems (SenSys) · doi:10.1145/3384419.3430782 · Cited by 56
“Similarly, in Chameleon, different knobs like frame rate, resolution, type of inference model is tuned based on temporal and spatial information for video processing to minimise computation [26].”
BackgroundFiGO: Fine-Grained Query Optimization in Video Analytics
Cao, J., Sarkar, K., Hadidi, R., Arulraj, J. J. P., & Kim, H. (2022) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/3514221.3517857 · Cited by 55
Energy-Efficient Task Offloading of Edge-Aided Maritime UAV Systems
Li, H., Wu, S., Jiao, J., Lin, X., Zhang, N., & Zhang, Q. (2022) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2022.3205127 · Cited by 54
Are Machine Learning Cloud APIs Used Correctly?
Wan, C., Liu, S., Hoffmann, H., Maire, M., & Lu, S. (2021) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse43902.2021.00024 · Cited by 54
“…API result to return within a couple of seconds to maintain good software interactivity [64], [65]; in addition, some applications (11 out of 360) process streaming data, audio, video, and others, from a sensor, and hence have to finish each API call in less than one second [66] to avoid data loss.”
BackgroundEdge-Based Video Analytics: A SurveyPreprint
Hu, M., Luo, Z., Pasdar, A., Lee, Y. C., Zhou, Y., & Wu, D. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.14329 · Cited by 54
“Collaborative Intelligence Chameleon [100] Mobile Collaboration Use temporal and spatial correlation of cross-camera to picks the best configurations for existing NN-based video analytics pipelines × × 20-50% higher accuracy with the same amount of resources, or the same accuracy with 30-50% of the resources”
BackgroundInFi: end-to-end learnable input filter for resource-efficient mobile-centric inference
Yuan, M., Zhang, L., He, F., Tong, X., & Li, X. (2022) · ACM Annual International Conference on Mobile Computing and Networking (MobiCom) · doi:10.1145/3495243.3517016 · Cited by 53
Edge-Assisted On-Device Model Update for Video Analytics in Adverse Environments
Kong, Y., Yang, P., & Cheng, Y. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612585 · Cited by 53
“Empowered by the advancement of deep neural networks (DNNs) and the increasing prevalence of cameras, video analytics has become an indispensable component in various applications, such as traffic monitoring [17, 18, 37, 48], autonomous driving [1, 7, 14, 25], intelligent industry [8, 16], etc.”
BackgroundCost-Efficient Continuous Edge Learning for Artificial Intelligence of Things
Jia, L., Zhou, Z., Xu, F., & Jin, H. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3104089 · Cited by 52
EC²Detect: Real-Time Online Video Object Detection in Edge-Cloud Collaborative IoT
Guo, S., Zhao, C., Wang, G., Yang, J., & Yang, S. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3173685 · Cited by 51
“industry [2], [3] are concentrating on intelligent video ana-”
BackgroundInterference-Aware Scheduling for Inference Serving
Mendoza, D., Romero, F., Li, Q., Yadwadkar, N. J., & Kozyrakis, C. (2021) · Workshop on Machine Learning and Systems (EuroMLSys) · doi:10.1145/3437984.3458837 · Cited by 51
“Applications across various domains, such as healthcare [16], security [22], and analytics [17, 27, 33] are now relying on the inference process of the machine learning lifecycle (which accounts for over 90% of infrastructure cost on AWS [5]).”
MethodEmpower Vision Applications with LoRA LMM
Mi, L., Wang, W., Tu, W., He, Q., Kong, R., Fang, X., Dong, Y., Zhang, Y., Li, Y., Li, M., et al. (2025) · European Conference on Computer Systems · doi:10.1145/3689031.3717472 · Cited by 50
Low-Latency Edge Video Analytics for On-Road Perception of Autonomous Ground Vehicles
Lin, J., Yang, P., Zhang, N., Lyu, F., Chen, X., & Yu, L. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3181986 · Cited by 50
RESPIRE: Reducing Spatial–Temporal Redundancy for Efficient Edge-Based Industrial Video Analytics
Dai, X., Yang, P., Zhang, X., Dai, Z., & Yu, L. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3162598 · Cited by 48
“Directly reducing the computing cost of video data with satisfied analytic accuracy is essential in large-scale real-time camera clusters [5].”
BackgroundMethodApplication-Aware IoT Camera Virtualization for Video Analytics Edge Computing
Jang, S. Y., Lee, Y., Shin, B., & Lee, D. (2018) · IEEE/ACM Symposium on Edge Computing (SEC) · doi:10.1109/sec.2018.00017 · Cited by 48
“Chameleon [27] exploits deep convolution neural network (NN) for efficient video analytics.”
BackgroundEdgeKE: An On-Demand Deep Learning IoT System for Cognitive Big Data on Industrial Edge Devices
Fang, W. W., Feng, X., Ding, Y., Xiong, N. N., & Leung, V. C. M. (2020) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2020.3044930 · Cited by 48
“It is necessary to pick the most efficient one among them for saving computation costs [25].”
BackgroundInfluentialNeural Networks Meet Physical Networks: Distributed Inference Between Edge Devices and the Cloud
Chinchali, S. P., Cidon, E., Pergament, E., Chu, T., & Katti, S. (2018) · Proceedings of the 17th ACM Workshop on Hot Topics in Networks · doi:10.1145/3286062.3286070 · Cited by 46
Serverless Data Science - Are We There Yet? A Case Study of Model Serving
Wu, Y., Dinh, T. T. A., Hu, G., Zhang, M., Chee, Y. M., & Ooi, B. C. (2022) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/3514221.3517905 · Cited by 45
TASTI: Semantic Indexes for Machine Learning-based Queries over Unstructured Data
Kang, D. D., Guibas, J., Bailis, P. D., Hashimoto, T., & Zaharia, M. (2022) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/3514221.3517897 · Cited by 44
INFaaS: A Model-less and Managed Inference Serving SystemPreprint
Romero, F., Li, Q., Yadwadkar, N. J., & Kozyrakis, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.13348 · Cited by 44
“The number of applications relying on inference from Machine Learning (ML) models, such as video analytics [82], is already large [14,45,47,65,73] and expected to keep growing.”
BackgroundAccelerating approximate aggregation queries with expensive predicates
Kang, D. D., Guibas, J., Bailis, P. D., Hashimoto, T., Sun, Y., & Zaharia, M. (2021) · Proceedings of the VLDB Endowment · doi:10.14778/3476249.3476285 · Cited by 43
“We used the night-street (also known as jackson ) and taipei video datasets, which are commonly used for video analytics evaluation [8, 31–33 , SELECT AVG(count_cars(frame)) FROM video WHERE count_cars(frame) > 0 which computes the average number of cars in the video, conditioning on cars present.”
MethodUAV-Assisted Target Tracking and Computation Offloading in USV-Based MEC Networks
Wang, Z., Du, J., Jiang, C., Ren, Y., & Zhang, X.-P. S. (2024) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2024.3396121 · Cited by 43
“This relationship is often modeled as a convex function, typically based on Gaussian fitting of data [27]: where γ 1 , γ 2 , and γ 3 are empirical fitting parameters.”
BackgroundA Decade of Video Analytics at Edge: Training, Deployment, Orchestration, and Platforms
Bai, T., Zhao, H., Huang, L., Wang, Z., Kim, D. I., & Nallanathan, A. (2025) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2025.3563377 · Cited by 43
“To mimic this issue, Chameleon [140] was proposed to periodically search an exponentially large configuration space to find the optimal video configuration and reuse the configuration across related video queries by exploiting both temporal and spatial correlation among video feeds, which efficiently…”
ApproxNet: Content and Contention-Aware Video Object Classification System for Embedded Clients
Xu, R., Kumar, R., Wang, P., Bai, P., Meghanath, G., Chaterji, S., Mitra, S., & Bagchi, S. (2021) · ACM Transactions on Sensor Networks · doi:10.1145/3463530 · Cited by 43
“VideoStorm [83], Chameleon [34], and Focus [24] exploited various configurations and DNN models to handle video analytics queries in a situation-tailored manner.”
BackgroundLarge-scale Video Analytics with Cloud–Edge Collaborative Continuous Learning
Nan, Y., Jiang, S., & Li, M. (2023) · ACM Transactions on Sensor Networks · doi:10.1145/3624478 · Cited by 42
“Such an assumption has been adopted and proven valid in existing works [20].”
BackgroundEdgeVision: Towards Collaborative Video Analytics on Distributed Edges for Performance Maximization
Gao, G., Dong, Y., Wang, R., & Zhou, X. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3385678 · Cited by 42
“To calculate the recognition accuracy under different video resolutions, we compare it with the ground truth obtained from the recognition results of the most accurate model under the original resolution, following the approach consistent with prior works [2], [21], [49].”
BackgroundResultCross-Camera Inference on the Constrained Edge
Li, J., Liu, L., Xu, H. L., Wu, S., & Xue, C. J. (2023) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom53939.2023.10229045 · Cited by 41
“First, DNN-based inference is inherently compute-intensive, especially for object detection models with complex convolutional layers [19, 26, 37].”
BackgroundMethodRavindran, A. (2023) · IoT · doi:10.3390/iot4040021 · Cited by 42
“Chameleon [67] is another work that employs this technique.”
MethodThe SPEC-RG Reference Architecture for The Compute Continuum
Jansen, M., Al-Dulaimy, A., Papadopoulos, A. V., Trivedi, A., & Iosup, A. (2023) · IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid) · doi:10.1109/ccgrid57682.2023.00051 · Cited by 41
“Once for all [61], Videostorm [62], Chameleon [63], and Mainstream [64] While model inference is much less compute and storage intensive than model training, it still requires the use of specialized deep learning frameworks like Neurosurgeon [19] (P3) and models like MobileNet [65] to be deployable on constrained edge…”
BackgroundCONVINCE: Collaborative Cross-Camera Video Analytics at the Edge
Pasandi, H. B., & Nadeem, T. (2020) · IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops) · doi:10.1109/percomworkshops48775.2020.9156251 · Cited by 40
TASM: A Tile-Based Storage Manager for Video Analytics
Daum, M., Haynes, B. M., He, D., Mazumdar, A., & Bałazińska, M. (2021) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde51399.2021.00156 · Cited by 40
“or for specific properties of the videos being queried [25].”
BackgroundScheduling Massive Camera Streams to Optimize Large-Scale Live Video Analytics
Rong, C., Wang, J. H., Liu, J., Wang, J., Li, F., & Huang, X. (2021) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2021.3125359 · Cited by 40
OTIF: Efficient Tracker Pre-processing over Large Video Datasets
Bastani, F., & Madden, S. R. (2022) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/3514221.3517835 · Cited by 40
“NoScope [11], Chameleon [9], and CaTDet [17] propose methods to accelerate the extraction of object detections and object tracks from video.”
MethodInfluentialGEMEL: Model Merging for Memory-Efficient, Real-Time Video Analytics at the EdgePreprint
Padmanabhan, A., Agarwal, N., Iyer, A. P., Ananthanarayanan, G., Shu, Y., Karianakis, N., Xu, G. H., & Netravali, R. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2201.07705 · Cited by 40
“As in existing pipelines [16,58,70], users register inference tasks (or “queries”) at GEMEL’s cloud component by providing a DNN, and specifying the input video feed(s) to run on as well as the required accuracy for the results.”
BackgroundMethodInfluentialSelf-citationEdge Video Analytics With Adaptive Information Gathering: A Deep Reinforcement Learning Approach
Wang, S., Bi, S., & Zhang, Y. A. (2023) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2023.3237202 · Cited by 39
“communication resource allocation [12].”
A Utility-Based Optimization Framework for Edge Service Entity Caching
Liang, Y., Ge, J., Zhang, S., Wu, J., Tang, Z. J., & Luo, B. (2019) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2019.2915218 · Cited by 38
“Chameleon [12] utilized temporal persistence of top-k configurations and spatial similarities to minimize the resource consumption for video analysis.”
MethodYuan, Z., Azzino, T., Yu, H., Lyu, Y., Pei, H., Boldini, A., Mezzavilla, M., Beheshti, M., Porfiri, M., Hudson, T. E., et al. (2022) · IEEE Access · doi:10.1109/access.2022.3157876 · Cited by 38
“[39] propose methods to adapt frame size and frame rate as well as detection models to meet a target detection accuracy based on video content.”
MethodAccDecoder: Accelerated Decoding for Neural-enhanced Video Analytics
Yuan, T., Mi, L., Wang, W., Dai, H., & Fu, X. (2023) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom53939.2023.10228933 · Cited by 38
“, autonomous driving) is built on it; second, we seek to keep consistent with prior video analytics work [5], [58]–[60] to allow a straightforward performance comparison.”
ResultVideo Analytics with Zero-streaming CamerasPreprint
Xu, M., Xu, T., Liu, Y., & Lin, F. X. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.12342 · Cited by 37
“[68] does so for network transport; Chameleon [55] and VideoStorm [50, 90] do so with video formats.”
OsmoticGate: Adaptive Edge-Based Real-Time Video Analytics for the Internet of Things
Qian, B., Wen, Z., Tang, J., Yuan, Y., Zomaya, A. Y., & Ranjan, R. K. (2022) · IEEE Transactions on Computers · doi:10.1109/tc.2022.3193630 · Cited by 36
“[10], [11], [12], [13] have been conducted on scheduling and configuring the video analytics jobs (queries).”
BackgroundCuttlefish: Neural Configuration Adaptation for Video Analysis in Live Augmented Reality
Chen, N., Quan, S., Zhang, S., Qian, Z., Jin, Y., Wu, J., Li, W., & Lu, S. (2020) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2020.3035044 · Cited by 35
“Jiang et al. [36] presented Chameleon to dynamically pick the best configuration for existing NN-based video analytics pipelines.”
BackgroundClownfish: Edge and Cloud Symbiosis for Video Stream Analytics
Nigade, V., Wang, L., & Bal, H. E. (2020) · IEEE/ACM Symposium on Edge Computing (SEC) · doi:10.1109/sec50012.2020.00012 · Cited by 35
“Current practices for DL-based video stream analytics are either cloud- [6], [7] or edge-only [3], [13], but none of them are a silver bullet.”
BackgroundVSS: A Storage System for Video Analytics
Haynes, B. M., Daum, M., He, D., Mazumdar, A., Bałazińska, M., Cheung, A., & Ceze, L. (2021) · ACM SIGMOD International Conference on Management of Data (SIGMOD) · doi:10.1145/3448016.3459242 · Cited by 35
“, the SMOL system rescales video to various resolutions [28] and Chameleon dynamically adjusts input resolution [24]).”
BackgroundInfluentialLow-latency speculative inference on distributed multi-modal data streams
Li, T., Huang, J., Risinger, E., & Ganesan, D. (2021) · International Conference on Mobile Systems, Applications, and Services · doi:10.1145/3458864.3467884 · Cited by 35
“One is based on frame sampling, and the other two are profiling methods [39, 84].”
BackgroundMethodInfluential
Metadata
- Authors
- Jiang, J., Ananthanarayanan, G., Bodik, P., Sen, S., & Stoica, I.
- Affiliations
- University of Chicago (US)
- Microsoft (United States) (US)
- Microsoft Research
- University of California, Berkeley (US)
- Venue
- Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication
- Publisher
- ACM
- Published
- 2018-08-07
- Pages
- 253-266
- OpenAlex
- W2887117815
- Cited by
- 467
- References
- 25
- Citations by year
- 2026: 33 · 2025: 58 · 2024: 69 · 2023: 74 · 2022: 70 · 2021: 69 · 2020: 50 · 2019: 38 · 2018: 6
- FWCI
- 12.28
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM SIGCOMM Conference (SIGCOMM) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Surveillance and Tracking Methods
- Physical Sciences › Computer Science › Information Systems › Advanced Database Systems and Queries
- Keywords
- video analysis, deep convolutional neural networks, object detection, chameleons
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.