Chameleon: scalable adaptation of video analytics
The DOI matches its Crossref record; 24 of the 25 references it cites were found in a registry or an index.
Abstract
Applying deep convolutional neural networks (NN) to video data at scale poses a substantial systems challenge, as improving inference accuracy often requires a prohibitive cost in computational resources. While it is promising to balance resource and accuracy by selecting a suitable NN configuration (e.g., the resolution and frame rate of the input video), one must also address the significant dynamics of the NN configuration's impact on video analytics accuracy. We present Chameleon, a controller that dynamically picks the best configurations for existing NN-based video analytics pipelines. The key challenge in Chameleon is that in theory, adapting configurations frequently can reduce resource consumption with little degradation in accuracy, but searching a large space of configurations periodically incurs an overwhelming resource overhead that negates the gains of adaptation. The insight behind Chameleon is that the underlying characteristics (e.g., the velocity and sizes of objects) that affect the best configuration have enough temporal and spatial correlation to allow the search cost to be amortized over time and across multiple video feeds. For example, using the video feeds of five traffic cameras, we demonstrate that compared to a baseline that picks a single optimal configuration offline, Chameleon can achieve 20-50% higher accuracy with the same amount of resources, or achieve the same accuracy with only 30--50% of the resources (a 2-3X speedup).
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 1×
“, [17, 18]) which have led to a continuous stream of increasingly accurate models for object detection and classification.”
BackgroundHe, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105
The pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599
MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
Howard, A. G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1704.04861 · Cited by 19,015
Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning
Szegedy, C., Ioffe, S., Vanhoucke, V., & Alemi, A. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.11231 · Cited by 4,636
Speed/Accuracy Trade-Offs for Modern Convolutional Object Detectors
Huang, J., Rathod, V., Sun, C., Zhu, M., Korattikara, A., Fathi, A., Fischer, I., Wojna, Z., Song, Y., Guadarrama, S., et al. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.351 · Cited by 2,743 · cited 1×
“The inference models are implemented in Tensorflow and are pretrained on standard image datasets [23], and the switching of video frame rate and resolution is done by FFmpeg [10].”
Method(1993) · Choice Reviews Online · doi:10.5860/choice.31-0986 · Cited by 1,540
Analysis of Thompson Sampling for the multi-armed bandit problem
Agrawal, S., & Goyal, N. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1111.1797 · Cited by 819 · cited 1×
“al [19] use Thompson sampling [11] to optimize the layout of a multi-component web page; they use greedy hill climbing to select the next layout, similar to Chameleon, but Chameleon exploits more independence structure and monotonicity.”
MethodErnest: efficient performance prediction for large-scale advanced analytics
Venkataraman, S., Yang, Z., Franklin, M. J. M., Recht, B., & Stoica, I. G. (2016) · Networked Systems Design and Implementation · Cited by 559
Self-citationReal-Time Video Analytics: The Killer App for Edge Computing
Ananthanarayanan, G., Bahl, P., Bodik, P., Chintalapudi, K., Philipose, M., Ravindranath, L., & Sinha, S. (2017) · Computer · doi:10.1109/mc.2017.3641638 · Cited by 496 · cited 2×
“As we move to the scenario of edge camera analytics [13], such a separation would be hard to achieve.”
BackgroundSelf-citationBayesian Approach to Global Optimization: Theory and Applications
Mockus, J. (1989) · Mathematics and Its Applications · doi:10.1007/978-94-009-0909-0 · Cited by 406
Cherrypick: adaptively unearthing the best cloud configurations for big data analytics
Alipourfard, O., Liu, H. H., Chen, J., Venkataraman, S., Yu, M., & Zhang, M. (2017) · Networked Systems Design and Implementation · Cited by 374
Han, S., Shen, H., Philipose, M., Agarwal, S., Wolman, A., & Krishnamurthy, A. (2016) · Proceedings of the 14th Annual International Conference on Mobile Systems, Applications, and Services · doi:10.1145/2906388.2906396 · Cited by 367 · cited 3×
“While prior video analytics systems [16, 32, 33] profile the processing pipeline to minimize cost, they only do so once, at the beginning of the video.”
BackgroundNoScope: optimizing neural network queries over video at scale
Kang, D., Emmons, J., Abuzaid, F., Bailis, P., & Zaharia, M. (2017) · Proceedings of the VLDB Endowment · doi:10.14778/3137628.3137664 · Cited by 367 · cited 3×
“Instead, we use an expensive golden configuration to provide the “ground truth”, following prior work [22, 24, 32].”
BackgroundMethod(2017) · Networked Systems Design and Implementation · Cited by 329
Self-citationThe Bayesian approach to global optimization
Mockus, J. (2005) · Lecture Notes in Control and Information Sciences · doi:10.1007/bfb0006170 · Cited by 283 · cited 1×
“Ernest [31] uses optimal experimental design [28] to optimize the VM configuration of a job, while Cherrypick [12] uses Bayesian optimization [27] to find an optimal cloud configuration for general applications.”
MethodThe Design and Implementation of a Wireless Video Surveillance System
Zhang, T., Chowdhery, A., Bahl, P. (., Jamieson, K., & Banerjee, S. (2015) · Proceedings of the 21st Annual International Conference on Mobile Computing and Networking · doi:10.1145/2789168.2790123 · Cited by 272 · cited 1×
“While prior video analytics systems [16, 32, 33] profile the processing pipeline to minimize cost, they only do so once, at the beginning of the video.”
BackgroundBestConfig: tapping the performance potential of systems via automatic configuration tuning
Zhu, Y., Liu, J., Guo, M., Bao, Y., Ma, W., Liu, Z., Song, K., & Yang, Y. (2017) · Proceedings of the 2017 Symposium on Cloud Computing · doi:10.1145/3127479.3128605 · Cited by 239 · cited 1×
“Several recent systems have also faced an exponentially large configuration search space in their problem domains [12, 19, 31, 34].”
BackgroundEfficient Contextual Bandits in Non-stationary WorldsPreprint
Luo, H., Wei, C.-Y., Agarwal, A., & Langford, J. C. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.01799 · Cited by 187 · cited 1×
“, [26]), but these are too inefficient at present.”
BackgroundAn Efficient Bandit Algorithm for Realtime Multivariate Optimization
Hill, D. N., Nassif, H., Liu, Y., Iyer, A., & Vishwanathan, S. V. N. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098184 · Cited by 113 · cited 5×
“al [19] use Thompson sampling [11] to optimize the layout of a multi-component web page; they use greedy hill climbing to select the next layout, similar to Chameleon, but Chameleon exploits more independence structure and monotonicity.”
BackgroundMethodInfluentialFocus: Querying Large Video Datasets with Low Latency and Low Cost
Hsieh, K., Ananthanarayanan, G., Bodík, P., Bahl, P., Philipose, M., Gibbons, P. B., & Mutlu, O. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.03493 · Cited by 93 · cited 2×
“Instead, we use an expensive golden configuration to provide the “ground truth”, following prior work [22, 24, 32].”
BackgroundMethodSelf-citationFast Video Classification via Adaptive Cascading of Deep Models
Shen, H., Han, S., Philipose, M., & Krishnamurthy, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.236 · Cited by 66 · cited 1×
“An exception is [29] which retrains the NN model to detect the set of popular objects as it changes over time.”
BackgroundVideo Analytics towards Vision Zero
Loewenherz, F., Bahl, V., & Wang, Y. (2017) · ITE journal · Cited by 43
Research works printed but not linked: 2
- FFmpeg. http://ffmpeg.org/ , 2000 --2018. FFmpeg. http://ffmpeg.org/, 2000--2018.
- W. E. Hoover and M. Rockville . Algorithms for confidence circles and ellipses . Citeseer , 1984 . W. E. Hoover and M. Rockville. Algorithms for confidence circles and ellipses. Citeseer, 1984.
Cited by
Edge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing
Zhou, Z., Chen, X., Li, E., Zeng, L., Luo, K., & Zhang, J. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2918951 · Cited by 2,455
Deep Learning With Edge Computing: A Review
Chen, J., & Ran, X. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2921977 · Cited by 1,540
“[120] M. S. Riazi, C. Weinert, O. Tkachenko, E. M. Songhori, T. Schneider, and F. Koushanfar, “Chameleon: A hybrid secure computation framework for machine learning applications,” in Proc.”
MethodInfluentialConvergence of Edge Computing and Deep Learning: A Comprehensive Survey
Wang, X., Han, Y., Leung, V. C. M., Niyato, D., Yan, X., & Chen, X. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2970550 · Cited by 1,498
“In order to adjust the configuration of the input video stream (such as frame resolution and sampling rate) online with low cost, Chameleon [138] greatly saves the cost of searching the best model configuration by leveraging temporal and spatial correlations of the video inputs, and allows the cost”
BackgroundMethodInfluentialEdge Assisted Real-time Object Detection for Mobile Augmented Reality
Liu, L., Li, H., & Gruteser, M. (2019) · The 25th Annual International Conference on Mobile Computing and Networking · doi:10.1145/3300061.3300116 · Cited by 552
“VideoStorm [45] and Chameleon [46] achieve higher accuracy video analytics with the same amount of computational resources on the cloud by adapting the video configurations.”
BackgroundVideoEdge: Processing Camera Streams using Hierarchical Clusters
Hung, C.-C., Ananthanarayanan, G., Bodik, P., Golubchik, L., Yu, M., Bahl, P., & Philipose, M. (2018) · 2018 IEEE/ACM Symposium on Edge Computing (SEC) · doi:10.1109/sec.2018.00016 · Cited by 287
“Chameleon [35] is the recent video analytics work for continuously adjusting DNN configurations to optimize accuracy or reduce resources costs based on the temporal and spatial correlation among the video frames.”
BackgroundSelf-citation
Metadata
- Authors
- Jiang, J., Ananthanarayanan, G., Bodik, P., Sen, S., & Stoica, I.
- Affiliations
- University of Chicago (US)
- Microsoft (United States) (US)
- Microsoft Research
- University of California, Berkeley (US)
- Venue
- Proceedings of the 2018 Conference of the ACM Special Interest Group on Data Communication
- Publisher
- ACM
- Published
- 2018-08-07
- Pages
- 253-266
- OpenAlex
- W2887117815
- Cited by
- 467
- References
- 25
- Citations by year
- 2026: 33 · 2025: 58 · 2024: 69 · 2023: 74 · 2022: 70 · 2021: 69 · 2020: 50 · 2019: 38 · 2018: 6
- FWCI
- 12.28
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM SIGCOMM Conference (SIGCOMM) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Surveillance and Tracking Methods
- Physical Sciences › Computer Science › Information Systems › Advanced Database Systems and Queries
- Keywords
- video analysis, deep convolutional neural networks, object detection, chameleons
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.