Video Object Detection Base on RGB and Optical Flow Analysis
Abstract
Although the image object detection technology developed quickly, the object detection in videos has special conditions such as motion blur, video defocus, part occlusion, rare posture, and needs to keep the target detection result consistent in time. It cannot apply image object detection models directly on videos. In this paper, we proposed the video object detection method based on the YOLO-v3 together with the FlowNet 2.0 optical flow extraction network to make full use of the information of the consistent video frames. The proposed Flow-guided Partial Warp method operates at the feature map level to make the detection result more accurate. In order to solve the motion blur influence of background, the optical flow compression by bilinear interpolation can minimize the influence. Meanwhile, the proposed method also uses the frame skipping to avoid the repeated use of convolutional neural networks in frames with little change, which reduces the consumption of computing resources and speeds up the running of the model. In addition, this paper also attempts to use the autonomous vehicles simulator as the input data of the video target detection algorithm. Finally, after testing in some typical scenarios, the detection results of most scenes are obviously improved after using optical flow guidance and aggregation feature maps.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Experimental results show that using depth information, optical flow information, and LiDAR point clouds in addition to conventional RGB data can improve the performance of one-stage based detection systems.
“According to experimental results, it has been shown that the use of depth information [183, 184], optical flow information [185], and LiDAR point clouds [186] in addition to conventional RGB data can improve the performance of one-stage based detection systems.”
From A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets (2022) · cites this paper as [185]
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works
(2016) · Cited by 31,590 · see citing works
A Multi-Modal Distributed Real-Time IoT System for Urban Traffic Control (Invited Paper)
Khanam, Z., Achari, V. P. S., Boukhennoufa, I., et al. (2024) · Durham Research Online (Durham University) · doi:10.4230/oasics.ng-res.2024.2 · Cited by 14,048 · see citing works
Flownet: Learning optical flow with convolutional networks
Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., Smagt, P. V. D., Cremers, D., & Brox, T. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.316 · Cited by 4,313 · see citing works
FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks
Ilg, E., Mayer, N., Saikia, T., Keuper, M., Dosovitskiy, A., & Brox, T. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.179 · Cited by 3,177 · see citing works
Pyramidal implementation of the Lucas Kanade feature tracker description of the algorithm
Bouguet, J.-Y. (1999) · Cited by 1,557 · see citing works
Flow-Guided Feature Aggregation for Video Object Detection
Zhu, X., Wang, Y., Dai, J., Yuan, L., & Wei, Y. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.52 · Cited by 729 · see citing works
Self-citationDeep Feature Flow for Video Recognition
Zhu, X., Xiong, Y., Dai, J., Yuan, L., & Wei, Y. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.441 · Cited by 678 · see citing works
T-CNN: Tubelets With Convolutional Neural Networks for Object Detection From Videos
Kang, K., Li, H., Yan, J., Zeng, X., Yang, B., Xiao, T., Zhang, C., Wang, Z., Wang, R., Wang, X., et al. (2018) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2017.2736553 · Cited by 554 · see citing works
NoScope: optimizing neural network queries over video at scale
Kang, D., Emmons, J., Abuzaid, F., Bailis, P., & Zaharia, M. (2017) · Proceedings of the VLDB Endowment · doi:10.14778/3137628.3137664 · Cited by 367 · see citing works
Generative Neural Networks for Anomaly Detection in Crowded Scenes
Wang, T., Qiao, M., Lin, Z., Li, C., Snoussi, H., Liu, Z., & Choi, C. (2018) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2018.2878538 · Cited by 164 · see citing works
Self-citationSimpleFlow: A Non‐iterative, Sublinear Optical Flow Algorithm
Tao, M. W., Bai, J., Kohli, P., & Paris, S. (2012) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2012.03013.x · Cited by 139 · see citing works
A reinforcement learning approach for UAV target searching and tracking
Wang, T., Qin, R., Chen, Y., et al. (2018) · Multimedia Tools and Applications · doi:10.1007/s11042-018-5739-5 · Cited by 107 · see citing works
Self-citationAbnormal event detection based on analysis of movement information of video sequence
Wang, T., Qiao, M., Deng, Y., et al. (2017) · Optik · doi:10.1016/j.ijleo.2017.07.064 · Cited by 73 · see citing works
Self-citationMotion Blur Rendering: State of the Art
Navarro, F., Arbeloa, F. J. S., & Gutiérrez, D. (2011) · Computer Graphics Forum · doi:10.1111/j.1467-8659.2010.01840.x · Cited by 57 · see citing works
NoScope: Optimizing Neural Network Queries over Video at ScalePreprint
Kang, D. D., Emmons, J., Abuzaid, F., Bailis, P. D., & Zaharia, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.02529 · Cited by 47 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Bayoudh, K., Knani, R., Hamdaoui, F., & Mtibaa, A. (2022) · The Visual Computer · doi:10.1007/s00371-021-02166-7 · Cited by 456 · see citing works
- Authors
- Zhang, S., Wang, T., Wang, C., Wang, Y., Shan, G., & Snoussi, H.
- Affiliations
- Beihang University (CN)
- Shenyang Aerospace University (CN)
- Centre National de la Recherche Scientifique (FR)
- Venue
- 2019 2nd China Symposium on Cognitive Computing and Hybrid Intelligence (CCHI)
- Publisher
- IEEE
- Published
- 2019-09
- Pages
- 280-284
- OpenAlex
- W2986415772
- Cited by
- 7
- References
- 16
- Citations by year
- 2025: 1 · 2024: 1 · 2023: 1 · 2022: 3 · 2021: 1
- FWCI
- 0.33
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- China Symposium on Cognitive Computing and Hybrid Intelligence (CCHI) (publishedVersion)
- HAL (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Surveillance and Tracking Methods
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
- Keywords
- video object detection, optical flow, YOLOv3, motion blur, occlusion, frame skipping, bilinear interpolation, real-time detection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
