FFS-VA: A Fast Filtering System for Large-scale Video Analytics
The DOI matches its Crossref record; 43 of the 43 references it cites were found in a registry or an index.
Abstract
Surveillance video cameras are ubiquitous around us. Full-feature object-detection models such as YOLOv2 can automatically analyze surveillance videos in real-time with high accuracy while consuming huge computational resources. Directly applying these models for practical scenarios with large-scale deployed cameras requires prohibitively expensive computation. This, however, is both wasteful and unnecessary considering the fact that the concerned anomalous events occur rarely among these massive volumes of video streams. Therefore, in this paper, we propose a Fast Filtering System for Video Analytics (FFS-VA), a pipelined multi-stage video analyzing system, to make video analytics much cost-effective. FFS-VA is designed to filter out vast but non-target-object frames by two prepositive stream-specialized filters and a small full-function tiny-YOLO model, to drastically reduce the number of video frames arriving at the full-feature model in the back-end. FFS-VA presents a global feedback-queue mechanism to balance the processing rates of different filters in both intra-stream and inter-stream processes. FFS-VA also designs a dynamic batch technique to achieve an adjustable trade-off between throughput and latency. FFS-VA reasonably distributes all tasks on CPUs and GPUs to fully exploit the underlying hardware resources. We implement a FFS-VA prototype and evaluate FFS-VA against the state-of-the-art YOLOv2 under the same hardware and representative video workloads. The experimental results show that under a 10% target-object occurrence rate on two GPUs, FFS-VA can support up to 30 concurrent video streams (7x more than YOLOv2) in the online case, and obtain 3x speedup when offline analyzing a stream, with an accuracy loss of less than 2%.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
(2016) · Cited by 31,590
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105
YOLO9000: Better, Faster, Stronger
Redmon, J., & Farhadi, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.690 · Cited by 19,892
The pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599 · cited 1×
“In 2014, R-CNN [30] first achieves 53.7% mean average precision (mAP) on PASCAL VOC 2010 [22].”
MethodInfluentialRapid object detection using a boosted cascade of simple features
Viola, P., & Jones, M. (2005) · Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001 · doi:10.1109/cvpr.2001.990517 · Cited by 18,300
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2389824 · Cited by 11,971
R-FCN: Object Detection via Region-based Fully Convolutional Networks
Dai, J., Li, Y., He, K., & Sun, J. Z. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.06409 · Cited by 3,149
OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks
Sermanet, P., Eigen, D., Zhang, X., Mathieu, M., Fergus, R., & LeCun, Y. (2013) · arXiv · doi:10.48550/arxiv.1312.6229 · Cited by 2,680
Robust object tracking with online multiple instance learning
Babenko, B., Yang, M.-H., & Belongie, S. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.226 · Cited by 2,007 · cited 1×
“, vehicle counting [12], license plate detection [6], cars or pedestrians tracking [3]).”
BackgroundEIE: efficient inference engine on compressed deep neural network
Han, S., Liu, X., Mao, H., Pu, J., Pedram, A., Horowitz, M. A., & Dally, W. J. (2016) · ACM SIGARCH Computer Architecture News · doi:10.1145/3007787.3001163 · Cited by 1,937
A convolutional neural network cascade for face detection
Li, H., Lin, Z., Shen, X., Brandt, J., & Hua, G. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299170 · Cited by 1,390
Deep Convolutional Network Cascade for Facial Point Detection
Sun, Y., Wang, X., & Tang, X. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.446 · Cited by 1,382
EIE: Efficient Inference Engine on Compressed Deep Neural Network
Han, S., Liu, X., Mao, H., Pu, J., Pedram, A., Horowitz, M. A., & Dally, W. J. (2016) · 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca.2016.30 · Cited by 1,077
Delving Deeper into Convolutional Networks for Learning Video RepresentationsPreprint
Ballas, N., Yao, L., Pal, C., & Courville, A. C. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06432 · Cited by 873
Comparative Study of CNN and RNN for Natural Language Processing
Yin, W., Kann, K., Yu, M., & Schütze, H. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.01923 · Cited by 861
A Review of Computer Vision Techniques for the Analysis of Urban Traffic
Buch, N., Velastin, S. A., & Orwell, J. (2011) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2011.2119372 · Cited by 653
License Plate Recognition From Still Images and Video Sequences: A Survey
Anagnostopoulos, C.-N. E., Anagnostopoulos, I. E., Psoroulas, I. D., Loumos, V., & Kayafas, E. (2008) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2008.922938 · Cited by 645
A survey of video processing techniques for traffic applications
Kastrinaki, V, Zervakis, M., & Kalaitzakis, K. (2003) · Image and Vision Computing · doi:10.1016/s0262-8856(03)00004-0 · Cited by 527
NoScope: optimizing neural network queries over video at scale
Kang, D., Emmons, J., Abuzaid, F., Bailis, P., & Zaharia, M. (2017) · Proceedings of the VLDB Endowment · doi:10.14778/3137628.3137664 · Cited by 367 · cited 3×
“Using SNM for rapid image recognition in this case can ensure the accuracy to be over 95% [7].”
MethodInfluential(2017) · Networked Systems Design and Implementation · Cited by 329
Learning Complexity-Aware Cascades for Deep Pedestrian Detection
Cai, Z., Saberian, M., & Vasconcelos, N. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.384 · Cited by 326
Girshick, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.08083 · Cited by 276
Wang, M., Fang, E. X., & Liu, H. (2017) · Mathematical Programming · doi:10.1007/s10107-016-1017-3 · Cited by 178
Locality constraint distance metric learning for traffic congestion detection
Wang, Q., Wan, J., & Yuan, Y. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2017.03.030 · Cited by 137
Efficient region-based motion segmentation for a video monitoring system
Kim, J. B., & Kim, H. J. (2002) · Pattern Recognition Letters · doi:10.1016/s0167-8655(02)00194-0 · Cited by 131
Video processing techniques for traffic flow monitoring: A survey
Tian, B., Yao, Q., Gu, Y., Wang, K., & Li, Y. (2011) · 2011 14th International IEEE Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2011.6083125 · Cited by 106 · cited 1×
“Video monitoring involves many tasks [34], including vehicle tracking [4], object detection [13] and so on.”
BackgroundOptasia: A Relational Platform for Efficient Large-Scale Video Analytics
Lu, Y., Chowdhery, A., & Kandula, S. (2016) · ACM Symposium on Cloud Computing (SoCC) · doi:10.1145/2987550.2987564 · Cited by 96
Predicting Video Saliency with Object-to-Motion CNN and Two-layer Convolutional LSTMPreprint
Jiang, L. W., Xu, M., & Wang, Z. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.06316 · Cited by 96
Learning Complexity-Aware Cascades for Deep Pedestrian DetectionPreprint
Cai, Z., Saberian, M. J., & Vasconcelos, N. M. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.05348 · Cited by 55
Papert, S. A. (1966) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 44
Calculation of traffic flow breakdown probability to optimize link throughput
Wang, H., Rudy, K., Li, J., et al. (2010) · Applied Mathematical Modelling · doi:10.1016/j.apm.2010.02.027 · Cited by 42
Self-citationA scalable and flexible framework for smart video surveillance
Nazare, A. C., & Schwartz, W. R. (2016) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2015.10.014 · Cited by 38
A Traffic Congestion Estimation Approach from Video Using Time-Spatial Imagery
Li, L., Chen, L., Huang, X., et al. (2008) · International Conference on Intelligent Networks and Intelligent Systems · doi:10.1109/icinis.2008.182 · Cited by 37
Vehicles detection in Traffic Flow
Mo, G., & Zhang, S. (2010) · International Conference on Natural Computation (ICNC) · doi:10.1109/icnc.2010.5583178 · Cited by 32
Efficient graph-based search for object detection
Wei, H., Yang, C.-Z., & Yu, Q. (2017) · Information Sciences · doi:10.1016/j.ins.2016.12.039 · Cited by 23
Graca, C., Falcão, G., Figueiredo, I. N., & Kumar, S. (2015) · Journal of Real-Time Image Processing · doi:10.1007/s11554-015-0517-3 · Cited by 16
Detection of abnormal traffic video images based on high-dimensional fuzzy geometry
Huang, L., Li, Z., & Wang, B. (2017) · Automatic Control and Computer Sciences · doi:10.3103/s014641161703004x · Cited by 6
ARMS, N. (1966) · Elsevier eBooks · doi:10.1016/b978-0-08-011563-4.50015-2 · Cited by 2
Research works printed but not linked: 1
- Zhicheng Wang et al. 2017. A Novel Fire Detection Approach Based on CNN-SVM Using Tensorflow . In Proceedings of ICIC. 682--693 . Zhicheng Wang et al. 2017. A Novel Fire Detection Approach Based on CNN-SVM Using Tensorflow. In Proceedings of ICIC. 682--693.
Cited by
Edge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing
Zhou, Z., Chen, X., Li, E., Zeng, L., Luo, K., & Zhang, J. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2918951 · Cited by 2,455
“Available: https://arxiv.org/abs/1811.01268 [100] C. Zhang, Q. Cao, H. Jiang, W. Zhang, J. Li, and J. Yao, “FFS-VA: A fast filtering system for large-scale video analytics,” in Proc.”
BackgroundInfluentialConvergence of Edge Computing and Deep Learning: A Comprehensive Survey
Wang, X., Han, Y., Leung, V. C. M., Niyato, D., Yan, X., & Chen, X. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2970550 · Cited by 1,498
“Concerning object detection, FFSVA uses two prepositive stream-specialized filters and a small full-function tiny-YOLO model to filter out vast but non-target-object frames [137].”
Method6G networks: Beyond Shannon towards semantic and goal-oriented communications
Calvanese Strinati, E., & Barbarossa, S. (2021) · Computer Networks · doi:10.1016/j.comnet.2021.107930 · Cited by 525
Metadata
- Authors
- Zhang, C., Cao, Q., Jiang, H., Zhang, W., Li, J., & Yao, J.
- Affiliations
- Wuhan National Laboratory for Optoelectronics (CN)
- The University of Texas at Arlington (US)
- Venue
- Proceedings of the 47th International Conference on Parallel Processing
- Publisher
- ACM
- Published
- 2018-08-13
- Pages
- 1-10
- ISSN
- 0190-3918 (ISSN-L)
- OpenAlex
- W2885918706
- Cited by
- 16
- References
- 43
- Citations by year
- 2024: 2 · 2023: 2 · 2022: 4 · 2021: 4 · 2020: 2 · 2019: 2
- FWCI
- 0.60
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Proceedings of the International Conference on Parallel Processing (publishedVersion)
- Funders
- National Science Foundation
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Surveillance and Tracking Methods
- Physical Sciences › Computer Science › Artificial Intelligence › Anomaly Detection Techniques and Applications
- Keywords
- object detection, Tiny-YOLO, frame filtering, feedback queues
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.