The DOI matches its Crossref record; 51 of the 53 references it cites were found in a registry or an index.
Flow-Guided Feature Aggregation for Video Object Detection
Open accessAbstract
Extending state-of-the-art object detectors from image to video is challenging. The accuracy of detection suffers from degenerated object appearances in videos, e.g., motion blur, video defocus, rare poses, etc. Existing work attempts to exploit temporal information on box level, but such methods are not trained end-to-end. We present flow-guided feature aggregation, an accurate and end-to-end learning framework for video object detection. It leverages temporal coherence on feature level instead. It improves the per-frame features by aggregation of nearby features along the motion paths, and thus improves the video recognition accuracy. Our method significantly improves upon strong singleframe baselines in ImageNet VID [33], especially for more challenging fast moving objects. Our framework is principled, and on par with the best engineered systems winning the ImageNet VID challenges 2016, without additional bells-and-whistles. The code would be released.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Indeed, overlooking these factors results in a decrease in the object detection performance in videos.”
From A Systematic Review of Drone Based Road Traffic Monitoring System (2022) · cites this paper as [31]
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 11×
“The ResNet-50, ResNet-101, and the Aligned-Inception-ResNet models are all pre-trained on ImageNet classification.”
BackgroundMethodInfluentialVery Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148 · cited 3×
“State-of-the-art methods for general object detection [10, 30, 26, 5] are mainly based on deep CNNs [22, 36, 40, 14].”
BackgroundMethodImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 3×
“State-of-the-art methods for general object detection [10, 30, 26, 5] are mainly based on deep CNNs [22, 36, 40, 14].”
BackgroundMethodFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472 · cited 5×
“In [11], a multistage pipeline called Regions with Convolutional Neural Networks (R-CNN) is proposed for training deep CNN to classify region proposals for object detection.”
BackgroundMethodInfluentialFaster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562 · cited 3×
“State-of-the-art methods for general object detection [10, 30, 26, 5] are mainly based on deep CNNs [22, 36, 40, 14].”
BackgroundMethodImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · cited 12×
“Our approach is evaluated on the large-scale ImageNet VID dataset [33].”
MethodInfluential(2014) · Cited by 33,419
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684 · cited 3×
“Recent years have witnessed significant progress in object detection [11].”
BackgroundMethod(2016) · Cited by 31,590
Flagged first, then by citations
2 of the 53 references couldn’t be matched to a known paper.
Research works printed but not linked: 2
- yang, 2016, Efficient object detection from videos
- tran, 2016, Deep end2end voxe12voxel prediction, CVPR Workshop
Cited by
Sorted by citations
Simple Baselines for Human Pose Estimation and Tracking
Xiao, B., Wu, H., & Wei, Y. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01231-1_29 · Cited by 2,278
“d occlusion introduced by video frames. As shown in Fig. 2(c), the detector misses the left black person due to fast motion. Temporal information is often leveraged to generate more robust detections [36,35]. We propose to generate boxes for the processing frame from nearby frames using temporal information expressed…”
BackgroundSelf-citationTSM: Temporal Shift Module for Efficient Video Understanding
Lin, J., Gan, C., & Han, S. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00718 · Cited by 2,209
“We also compare to a strong baseline FGFA [60] that uses optical flow to aggregate the temporal information from 21 frames (past 10 frames and future 10 frames) for offline video detection.”
MethodZhou, X., Koltun, V., & Krähenbühl, P. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58548-8_28 · Cited by 1,428
“posals using bounding box regression. In video object detection, Kang et al. [16,17] feed stacked consecutive frames into the network and do object detection for a whole video segment. And Zhu et al. [59] use flow to warp intermediate features from previous frames to accelerate inference. Our method belongs to this cat…”
MethodVideo Enhancement with Task-Oriented Flow
Xue, T., Chen, B., Wu, J., Wei, D., & Freeman, W. T. (2019) · International Journal of Computer Vision · doi:10.1007/s11263-018-01144-2 · Cited by 1,401
“…(Liu et al 2017), light-field interpolation (Wang et al 2017), novel view synthesis (Zhou et al 2016), eye gaze manipulation (Ganin et al 2016), object detection (Zhu et al 2017), denoising (Wen et al 2017), and super-resolution (Caballero et al 2017; Tao et al 2017; Makansi et al 2017).”
A Survey of Deep Learning-Based Object Detection
Jiao, L., Zhang, F., Liu, F., Yang, S., Li, L., Feng, Z., & Qu, R. (2019) · IEEE Access · doi:10.1109/access.2019.2939201 · Cited by 1,454
“Video detectors [266][267][268][269][270][271][272][273][274][275] exploit temporal contexts to meet this challenge.”
BackgroundDetection and tracking meet drones challenge
Zhu, P., Wen, L., Du, D., Bian, X., Fan, H., Hu, Q., & Ling, H. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3119563 · Cited by 1,336
Video Enhancement with Task-Oriented FlowPreprint
Xue, T., Chen, B., Wu, J., Wei, D., & Freeman, W. T. (2019) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 1,163
A Comparative Analysis of Object Detection Metrics with a Companion Open-Source Toolkit
Padilla, R., Passos, W. L., Dias, T. L. B., Netto, S. L., & da Silva, E. A. B. (2021) · Electronics · doi:10.3390/electronics10030279 · Cited by 735
“Many works in the literature use the mAP as a metric to evaluate the performance of object detection models in video sequences [92–98].”
BackgroundComputer Vision for Autonomous Vehicles
Janai, J., Güney, F., Behl, A., & Geiger, A. (2020) · Foundations and Trends® in Computer Graphics and Vision · doi:10.1561/0600000079 · Cited by 707
BasicVSR: The Search for Essential Components in Video Super-Resolution and Beyond
Chan, K. C. K., Wang, X., Yu, K., Dong, C., & Loy, C. C. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00491 · Cited by 626
“In this work, we conduct extensive analysis and present a simple, strong and versatile baseline, BasicVSR, which can serve as a back-bone with abundant flexibilities in design.”
MethodA review of object detection based on deep learning
Xiao, Y., Tian, Z., Yu, J., Zhang, Y., Liu, S., Du, S., & Lan, X. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-020-08976-6 · Cited by 584
“achieved good results by using optical flow to improve the speed and accuracy of video recognition [292, 293].”
MethodYang, L., Fan, Y., & Xu, N. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00529 · Cited by 537
“While object identity information are often utilized for improve robustness of detection algorithms [9, 36, 33], the evaluation metric is limited to per-frame detection and does not require joint object detection and tracking.”
BackgroundA comprehensive review of object detection with deep learning
Kaur, R., & Singh, S. (2023) · Digital Signal Processing · doi:10.1016/j.dsp.2022.103812 · Cited by 539
An Empirical Study of Spatial Attention Mechanisms in Deep Networks
Zhu, X., Cheng, D., Zhang, Z., Lin, S., & Dai, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00679 · Cited by 659
“Motivated by their success in NLP tasks [2, 28, 14, 6, 26, 30, 31, 36], attention mechanisms have also been employed in computer vision applications such as relational reasoning among objects [3, 33], image captioning [41], image generation [45, 42], image recognition [20, 38, 17, 22, 46, 13], and video recognition [4…”
BackgroundSelf-citationTransTrack: Multiple Object Tracking with TransformerPreprint
Sun, P., Cao, J., Jiang, Y., Zhang, R., Xie, E., Yuan, Z., Wang, C., & Luo, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.15460 · Cited by 781
“In video object detection, FGFA [49] uses optical flow to warp intermediate features from the previous frames to accelerate inference.”
MethodDeep learning for unmanned aerial vehicle-based object detection and tracking: a survey
Wu, X., Li, W., Hong, D., Tao, R., & Du, Q. (2022) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2021.3115137 · Cited by 421
Self-Correction for Human Parsing
Li, P., Xu, Y., Wei, Y., & Yang, Y. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3048039 · Cited by 452
Self-citationMemory Enhanced Global-Local Aggregation for Video Object Detection
Chen, Y., Cao, Y., Hu, H., & Wang, L. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01035 · Cited by 366
Computer vision algorithms and hardware implementations: A survey
Feng, X., Jiang, Y., Yang, X., Du, M., & Li, X. S. (2019) · Integration · doi:10.1016/j.vlsi.2019.07.005 · Cited by 380
Channel Attention Is All You Need for Video Frame Interpolation
Choi, M., Kim, H., Han, B., Xu, N., & Lee, K. M. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6693 · Cited by 350
“…classi-fication (Carreira and Zisserman 2017; Simonyan and Zis-serman 2014a; Xie et al. 2018), object detection and segmentation (Cheng et al. 2017; Zhu et al. 2018; 2017), video prediction (Li et al. 2018; Reda et al. 2018), and others (Chen et al. 2017; Lai et al. 2018)—frequently depend on…”
BackgroundOcclusion Aware Unsupervised Learning of Optical Flow
Wang, Y., Yang, Y., Yang, Z., Zhao, L., Wang, P., & Xu, W. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00513 · Cited by 331
“With the accurate optical flow prediction, one could estimate the 3D structure of a scene [18], segment moving objects based on motion cues [38], track objects in a complicated environment [11], and build important visual cues for many high level vision tasks such as video action recognition [45] and video object dete…”
Self-citationEnd-to-End Flow Correlation Tracking with Spatial-Temporal Attention
Zhu, Z. D., Wu, W., Zou, W., & Yan, J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00064 · Cited by 322
“DFF [47] and FGFA [46] utilize flow information to speed up vision recognition (segmentation and video detection) and upgrade performance, respectively.”
BackgroundTowards High Performance Video Object Detection
Zhu, X., Dai, J., Yuan, L., & Wei, Y. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00753 · Cited by 285
“[36] improves feature quality by adaptive aggregation, which increases computation.”
BackgroundMethodInfluentialSelf-citationSequence Level Semantics Aggregation for Video Object Detection
Wu, H., Chen, Y., Wang, N., & Zhang, Z. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00931 · Cited by 282
New Generation Deep Learning for Video Object Detection: A Survey
Jiao, L., Zhang, R., Liu, F., Yang, S., Hou, B., Li, L., & Tang, X. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3053249 · Cited by 274
Object Detection in Video with Spatiotemporal Sampling Networks
Bertasius, G., Torresani, L., & Shi, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01258-8_21 · Cited by 258
“For example, we note that it would take about four days to train an optical flow CNN of FGFA [24] from scratch and then four additional days to train FGFA [24] for video object detection, making it eight days of total training time.”
BackgroundMethodInfluentialLi, D., Wang, R., Xie, C., Liu, L., Zhang, J., Li, R., Wang, F., Zhou, M., & Liu, W. (2020) · Sensors · doi:10.3390/s20030578 · Cited by 259
“…video detection is di ff erent from still-image detection in many ways, and often remains challenges such as video defocus, motion blur and part occlusion [29] The video frames of rice diseases and pests symptoms which contain video defocus, motion blur and part occlusion are shown in Figure 2.”
BackgroundRelation Distillation Networks for Video Object Detection
Deng, J., Pan, Y., Yao, T., Zhou, W., Li, H., & Mei, T. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00712 · Cited by 245
“Following the common protocols in [8, 46, 49, 54], we utilize both ImageNet VID and ImageNet object detection (DET) dataset to train our RDN.”
BackgroundMethodInfluentialCross View Fusion for 3D Human Pose Estimation
Qiu, H., Wang, C., Wang, J., Wang, N., & Zeng, W. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00444 · Cited by 241
“For example, in [34], Zhu et al . propose to warp the features of the neighboring frames (in a video sequence) to the current frame according to optical flow in order to robustly detect the objects.”
Instance-Aware, Context-Focused, and Memory-Efficient Weakly Supervised Object Detection
Ren, Z., Yu, Z., Yang, X., Liu, M.-Y., Lee, Y. J., Schwing, A. G., & Kautz, J. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01061 · Cited by 237
EEMEFN: Low-Light Image Enhancement via Edge-Enhanced Multi-Exposure Fusion Network
Zhu, M., Pan, P., Chen, W., & Yang, Y. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.7013 · Cited by 234
“The low-quality images may significantly reduce the performance of computer vision methods (Zhu et al. 2017; Long, Shelhamer, and Darrell 2015), which rely”
BackgroundRetinaTrack: Online Single Stage Joint Detection and Tracking
Lu, Z., Rathod, V., Votel, R., & Huang, J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01468 · Cited by 254
Transferable Adversarial Attacks for Image and Video Object Detection
Wei, X., Liang, S., Chen, N., & Cao, X. (2019) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/134 · Cited by 274
“Compared with the image scenario, video object detection considers the temporal interactions between frames [31, 32].”
BackgroundMethodInfluentialRelationTrack: Relation-Aware Multiple Object Tracking With Decoupled Representation
Yu, E., Li, Z., Han, S., & Wang, H. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3150169 · Cited by 219
“Many attempts have been conducted to overcome the disadvantages brought by implementing two separate sub-models [2], [32], [33], [34], [35].”
BackgroundFully Motion-Aware Network for Video Object Detection
Wang, S., Zhou, Y., Yan, J., & Deng, Z. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01261-8_33 · Cited by 213
“As contrast, FGFA [30] attempts to leverage temporal coherence on feature level by aggregating features of nearby frames along the motion paths.”
BackgroundMethodInfluentialTransVOD: End-to-End Video Object Detection With Spatial-Temporal Transformers
Zhou, Q., Li, X., He, L., Yang, Y., Cheng, G., Tong, Y., Ma, L., & Tao, D. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3223955 · Cited by 230
Video Object Detection with an Aligned Spatial-Temporal Memory
Xiao, F., & Lee, Y. J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01237-3_30 · Cited by 203
“However, [51,50] use optical flow, which needs to be computed either externally e.”
BackgroundMethodInfluentialSong, T., Sun, L., Xie, D., Sun, H., & Pu, S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01234-2_33 · Cited by 193
“We also list the result of our TLL combined with FGFA [27].”
BackgroundMethodResultInfluentialTF-Blender: Temporal Feature Blender for Video Object Detection
Cui, Y., Yan, L., Cao, Z., & Liu, D. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00803 · Cited by 194
“Albeit being superior over the prior efforts, FGFA [42] and SELSA [36] encounter several obstacles from achieving optimal performance: 1) They focus on modeling the global relation for every neighboring frame while ignoring the preservation of the local spatial information for aggregation; 2) They…”
BackgroundMethodInfluentialFlow Guided Recurrent Neural Encoder for Video Salient Object Detection
Li, G., Xie, Y., Wei, T., Wang, K., & Lin, L. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00342 · Cited by 189
“Inspired by [45, 44], we incorporate optical flow to enable feature warping across frames and compensate for the change caused by object motion.”
BackgroundMethodMAST: A Memory-Augmented Self-Supervised Tracker
Lai, Z., Lu, E., & Xie, W. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00651 · Cited by 179
“Memory models have been used for many applications, including reading comprehension [51], summarization [50], tracking[69], video understanding[7], and image and video captioning [70, 74].”
MethodMeMViT: Memory-Augmented Multiscale Vision Transformer for Efficient Long-Term Video Recognition
Wu, C.-Y., Li, Y., Mangalam, K., Fan, H., Xiong, B., Malik, J., & Feichtenhofer, C. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01322 · Cited by 201
“, [57]) to parse a video frame-by-frame, to consider longer-term context, most existing works use causal convolutions [6, 10, 36], RNNs [17,46], or feature fusion [8,84].”
BackgroundAction-Stage Emphasized Spatiotemporal VLAD for Video Action Recognition
Tu, Z., Li, H., Zhang, D., Dauwels, J., Li, B., & Yuan, J. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2890749 · Cited by 168
“In each ActionS segment, after locating the key frame feature map fS Kk , we apply the flow-guided warping strategy of [45] to select the discriminative local features by comparing the similarity between fS Kk and the warped feature maps of other frames.”
BackgroundMethodKim, D., Woo, S., Lee, J., & Kweon, I. S. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00988 · Cited by 166
“It combines several existing tasks: video object segmentation [1,3,4,27,30,35,36,39] and video object detection [9,42,43], and aims at simultaneous detection, segmentation, and tracking of instances in videos.”
MethodClassifying, Segmenting, and Tracking Object Instances in Video with Mask Propagation
Bertasius, G., & Torresani, L. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00976 · Cited by 166
“Most modern video object detection systems [46, 5, 40, 13] implement some form of spatiotemporal feature alignment for improving object detection accuracy in individual video frames.”
BackgroundTAO: A Large-Scale Benchmark for Tracking Any Object
Dave, A., Khurana, T., Tokmakov, P., Schmid, C., & Ramanan, D. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58558-7_26 · Cited by 185
Mobile Video Object Detection with Temporally-Aware Feature Maps
Zhu, M., & Liu, M. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00596 · Cited by 199
“Flowguided Feature Aggregation [42] improves detection accuracy by warping and averaging features from nearby frames with adaptive weighting.”
BackgroundMoNet: Deep Motion Exploitation for Video Object Segmentation
Xiao, H., Feng, J., Lin, G., Liu, Y., & Zhang, M. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00125 · Cited by 151
“Inspired by the success in video object detection [37, 38], we propose to align the features from adjacent frames It−m and It+k to It by exploiting motion cues, before combining them.”
MethodTowards An End-to-End Framework for Flow-Guided Video Inpainting
Li, Z., Lu, C.-Z., Qin, J., Guo, C., & Cheng, M. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01704 · Cited by 167
“The motion information across frames well assists many video-related tasks, such as video understanding [3,31], video segmentation [11,47], video object detection [65], depth estimation [18,36], video super-resolution [4, 57], frame interpolation [22, 27], etc.”
BackgroundAnnotating Objects and Relations in User-Generated Videos
Shang, X., Di, D., Xiao, J., Cao, Y., Yang, X., & Chua, T. (2019) · International Conference on Multimedia Retrieval · doi:10.1145/3323873.3325056 · Cited by 149
3D Object Detection From Images for Autonomous Driving: A Survey
Ma, X., Ouyang, W., Simonelli, A., & Ricci, E. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3346386 · Cited by 164
“On the other hand, lots of works had proved the effectiveness of using video data in many tasks, including 2D detection [195], [196], depth estimation [197], [198], and LiDAR-based 3D detection [199], [200].”
3D Object Detection from Images for Autonomous Driving: A Survey
Ma, X., Ouyang, W., Simonelli, A., & Ricci, E. (2022) · arXiv · doi:10.48550/arxiv.2202.02980 · Cited by 147
Kinematic 3D Object Detection in Monocular Video
Brazil, G., Pons-Moll, G., Liu, X., & Schiele, B. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58592-1_9 · Cited by 188
“A common trend in video-based detection is to benefit the accuracy-efficiency trade-off via reducing the frame redundancy [1,26,42,47,48].”
BackgroundA Review of Video Object Detection: Datasets, Metrics and Methods
Zhu, H., Wei, H., Li, B., Yuan, X., & Kehtarnavaz, N. (2020) · Applied Sciences · doi:10.3390/app10217834 · Cited by 163
“Similar 366 to the FGFA method in [22], it relies on the idea that detection on a single frame would have 367 difficulties dealing with noise sources such as motion blur and video defocus.”
BackgroundMethodInfluentialEnable Deep Learning on Mobile Devices: Methods, Systems, and Applications
Cai, H., Lin, J., Lin, Y., Liu, Z., Tang, H., Wang, H., Zhu, L., & Han, S. (2022) · ACM Transactions on Design Automation of Electronic Systems · doi:10.1145/3486618 · Cited by 162
Vision Meets Drones: Past, Present and FuturePreprint
Zhu, P., Wen, L., Du, D., Bian, X., Hu, Q., & Ling, H. (2020) · arXiv (Cornell University) · Cited by 145
VisDrone-VID2019: The Vision Meets Drone Object Detection in Video Challenge Results
Zhu, P., Du, D., Wen, L., Bian, X., Ling, H., Hu, Q., Peng, T., Zheng, J., Wang, X., Zhang, Y., et al. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00031 · Cited by 146
“On the other hand, there are 4 more methods ( i.e. , DetKITSY (A.5), EODST++ (A.7), Libra-HBR (A.11) and Sniper+ (A.12)) that outperforms or are on par with the best baseline method FGFA [43].”
BackgroundMethodInfluentialSelf-citationA Systematic Review of Drone Based Road Traffic Monitoring System
Bisio, I., Garibotto, C., Haleem, H., Lavagetto, F., & Sciarrone, A. (2022) · IEEE Access · doi:10.1109/access.2022.3207282 · Cited by 139
“…the following concepts must be taken into account: the spatial and temporal correlation of frames to overcome feature extraction redundancies, and the effect of motion blur, occlusion, and posture changes, which can contribute to the low quality of some individual frames in a video sequence [31].”
BackgroundContext R-CNN: Long Term Temporal Context for Per-Camera Object Detection
Beery, S., Wu, G., Rathod, V., Votel, R., & Huang, J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01309 · Cited by 140
“Leading methods have used pixel level flow (or flow-like concepts) to aggregate features [7, 56–58] or used correlation [13] to densely relate features at the current timestep to an adjacent timestep.”
MethodObject Detection Using Deep Learning Methods in Traffic Scenarios
Boukerche, A., & Hou, Z. (2021) · ACM Computing Surveys · doi:10.1145/3434398 · Cited by 137
“Research using optical flow to predict changes between frames include References [53, 141, 165, 167, 168], where References [167, 168] are two basis and the others are improvements or extensions of them.”
BackgroundMethodEnd-to-End United Video Dehazing and Detection
Li, B., Peng, X., Wang, Z., Xu, J., & Feng, D. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12287 · Cited by 127
“In outdoor surveillance or autonomous driving, object detection from video (Kang et al. 2016; Tripathi et al. 2016; Zhu et al. 2017) is widely desirable, whose performance is known to heavily suffer from the existence of haze.”
BackgroundDoTA: Unsupervised Detection of Traffic Anomaly in Driving Videos
Yao, Y., Wang, X., Xu, M., Pu, Z., Wang, Y., Atkins, E. M., & Crandall, D. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3150763 · Cited by 139
“Other methods [61], [62], [63], [64], [65], [66] use local feature aggregation for detection in a”
MethodSelf-citationEfficient Uncertainty Estimation for Semantic Segmentation in Videos
Huang, P., Hsu, W., Chiu, C.-Y., Wu, T., & Sun, M. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01246-5_32 · Cited by 123
“To enhance the reference feature map, they utilize optical flow network the work of Zhu et al.[35] to estimate the motions between nearby frames and the reference frame.”
MethodFlowFormer++: Masked Cost Volume Autoencoding for Pretraining Optical Flow Estimation
Shi, X., Huang, Z., Li, D., Zhang, M., Cheung, K. C., See, S. C. W., Qin, H., Dai, J., & Li, H. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00160 · Cited by 137
Self-citationTemporal ROI Align for Video Object Recognition
Gong, T., Chen, K., Wang, X., Chu, Q., Zhu, F., Lin, D., Yu, N., & Feng, H. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i2.16234 · Cited by 118
Optimizing Video Object Detection via a Scale-Time Lattice
Chen, K., Wang, J., Yang, S., Zhang, X., Xiong, Y., Loy, C. C., & Lin, D. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00815 · Cited by 120
“Temporal propagation The idea of temporal propagation is previously explored in the video object detection literatures [16, 32, 33].”
BackgroundMethodInfluentialVideoINR: Learning Video Implicit Neural Representation for Continuous Space-Time Super-Resolution
Chen, Z., Chen, Y., Liu, J., Xu, X., Goel, V., Wang, Z., Shi, H., & Wang, X. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00209 · Cited by 117
“This introduces challenges in recognition systems such as video object detection [53], and we are still struggling at learning to recognize motion and actions from discrete frames [4, 12].”
Real-time face & eye tracking and blink detection using event cameras
Ryan, C., O’Sullivan, B., Elrasad, A., Cahill, A., Lemley, J., Kielty, P., Posch, C., & Perot, E. (2021) · Neural Networks · doi:10.1016/j.neunet.2021.03.019 · Cited by 117
“In contrast, feature-level methods integrate image features (Broad et al., 2018; Liu & Zhu, 2018; Zhu et al., 2017).”
MethodObject Detection in Videos by High Quality Object Linking
Tang, P., Wang, C., Wang, X., Liu, W., Zeng, W., & Wang, J. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2910529 · Cited by 116
“We evaluate the performance on slow, medium, and fast ones which are formed according to their speed as done in [18].”
BackgroundMethodInfluentialEnd-to-End Video Object Detection with Spatial-Temporal Transformers
He, L., Zhou, Q., Li, X., Niu, L., Cheng, G., Li, X., Liu, W., Tong, Y., Ma, L., & Zhang, L. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475285 · Cited by 115
“Since the ground truth of the official testing set is not publicly available, we follow the widely adopted setting in previous work [11, 50, 53, 60] where we train our models using a combination of ImageNet VID and DET datasets [40] and measure the performance on the validation set using the mean…”
BackgroundMethodInfluentialRecent Advances in Object Detection in the Age of Deep Convolutional Neural NetworksPreprint
Agarwal, S., du Terrail, J. O., & Jurie, F. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.03193 · Cited by 145
“Deep feature flow [485, 486] ran the convolutional subnetwork only on sparse key frames, propagated deep feature maps to other frames via a flow field.”
BackgroundMethodInfluentialObject Guided External Memory Network for Video Object Detection
Deng, H., Yang, H., Song, T., Zhang, Z., Xue, Z., Ma, R., Robertson, N. M., & Guan, H. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00678 · Cited by 110
“For a fair comparison with the previous work [45, 44, 35, 2, 43], we”
BackgroundMethodResultInfluentialSpSequenceNet: Semantic Segmentation Network on 4D Point Clouds
Shi, H., Lin, G., Wang, H., Hung, T.-Y., & Wang, Z. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00463 · Cited by 106
“Optic flow methods [22, 29] use the nearest pixel from two different frames to generate local optic flow and achieve significant performance.”
MethodSchindler, F., & Steinhage, V. (2021) · Ecological Informatics · doi:10.1016/j.ecoinf.2021.101215 · Cited by 103
A Spatial-Temporal Feature-Based Detection Framework for Infrared Dim Small Target
Du, J., Lu, H., Zhang, L., Hu, M., Chen, S. Q., Deng, Y., Shen, X., & Zhang, Y. (2021) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2021.3117131 · Cited by 103
“In flow-guided feature aggregation (FGFA) [32], a framework for VOD is presented.”
BackgroundMethodProgressive Sparse Local Attention for Video Object Detection
Guo, C., Fan, B., Gu, J., Zhang, Q., Xiang, S., Véronique Prinet, A., & Pan, C. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00401 · Cited by 106
“To boost the performance, MANet [35] and FGFA [42] use the optical flow estimated by FlowNet [8] to aggregate the feature of multiple nearby frames.”
BackgroundMethodLeveraging Long-Range Temporal Relationships Between Proposals for Video Object Detection
Shvets, M., Liu, W., & Berg, A. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00985 · Cited by 102
“Current state-of-the-art methods for video object detection all leverage pixel-level features [1, 3, 19, 26].”
BackgroundMethodInfluentialVideo object detection for autonomous driving: Motion-aid feature calibration
Liu, D., Cui, Y., Chen, Y., Zhang, J., & Fan, B. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.05.027 · Cited by 95
YOLOV: Making Still Image Object Detectors Great at Video Object Detection
Shi, Y., Wang, N., & Guo, X. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i2.25320 · Cited by 101
Mobile Video Object Detection with Temporally-Aware Feature MapsPreprint
Liu, M., & Zhu, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.06368 · Cited by 92
“Flowguided Feature Aggregation [42] improves detection accuracy by warping and averaging features from nearby frames with adaptive weighting.”
BackgroundMultiNet++: Multi-Stream Feature Aggregation and Geometric Loss Strategy for Multi-Task Learning
Chennupati, S., Sistu, G., Yogamani, S. K., & Rawashdeh, S. A. (2019) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00159 · Cited by 103
“These features can be extracted from different CNNs operating on different input data [61, 37] or from a CNN operating on different resolutions of input [24].”
BackgroundMining Inter-Video Proposal Relations for Video Object Detection
Han, M., Wang, Y., Chang, X., & Qiao, Y. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58589-1_26 · Cited by 89
Self-citationEVA²: Exploiting Temporal Redundancy in Live Computer Vision
Buckler, M., Bedoukian, P., Jayasuriya, S., & Sampson, A. (2018) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1109/isca.2018.00051 · Cited by 94
“(2) DFF uses a fixed key frame rate calibrated to the “worst-case scenario” for scene motion.”
BackgroundMethodInfluentialDogfight: Detecting Drones from Drones Videos
Ashraf, M. W., Sultani, W., & Shah, M. A. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00699 · Cited by 91
“, [48] proposed to include the feature representation of each frame by aggregation of nearby frames features using optical flow.”
BackgroundMethodResultInfluentialMPPNet: Multi-frame Feature Intertwining with Proxy Points for 3D Temporal Object Detection
Chen, X., Shi, S., Zhu, B., Cheung, K. C., Xu, H., & Li, H. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-20074-8_39 · Cited by 89
Chen, S., Ji, L., Zhu, J., et al. (2024) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2024.3350024 · Cited by 86
“For the data-driven methods such as 3DCNN [48], DFF [57], FGDA [58], RDN [59], SELSA [16], MEGA [18], TROI [20], ACM [10], TSDF [47], RISTD [49], STFBD [19], ISNet [42], UIUNet [43], SANet [60], Authorized licensed use limited to the terms of the applicable license agreement with IEEE.”
MethodKFNet: Learning Temporal Camera Relocalization Using Kalman Filtering
Zhou, L., Luo, Z., Shen, T., Zhang, J., Zhen, M., Yao, Y., Fang, T., & Quan, L. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00497 · Cited by 84
“Although the flow-guided warping is employed, TPooler and SWeight only achieve marginal improvements over SCoordNet compared with KFNet, which justifies the advantage of the Kalman filtering system.”
BackgroundInfluentialTSM: Temporal Shift Module for Efficient and Scalable Video Understanding on Edge Devices
Lin, J., Gan, C., Wang, K., & Han, S. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3029799 · Cited by 88
“Other methods on video object detection [71] fuses information along temporal dimension after the 2D feature is extracted by the backbone, which results in high latency, and also loses low-level temporal cues.”
InfluentialVideoFlow: Exploiting Temporal Cues for Multi-frame Optical Flow Estimation
Shi, X., Huang, Z., Bian, W., Li, D., Zhang, M., Cheung, K. C., See, S. C. W., Qin, H., Dai, J., & Li, H. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01146 · Cited by 91
“It is widely adopted to tackle various downstream video problems, including video restoration [32, 66, 12, 34, 5, 50, 72, 36], video object detection [78, 61, 79], video synthesis [69, 18, 19, 63, 62] and action recognition [58, 47, 76], providing valuable information on motion and cross-frame correspondence.”
BackgroundSelf-citationVisDrone-VDT2018: The Vision Meets Drone Video Detection and Tracking Challenge Results
Zhu, P., Wen, L., Du, D., Bian, X., Ling, H., Hu, Q., Wu, H., Nie, Q., Cheng, H., Liu, C., et al. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-11021-5_29 · Cited by 80
Self-citationRekavandi, A. M., Xu, L., Boussaïd, F., Seghouane, A., Hoefs, S., & Bennamoun, M. (2025) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2025.3530678 · Cited by 119
Fast Video Object Segmentation With Temporal Aggregation Network and Dynamic Template Matching
Huang, X., Xu, J., Tai, Y., & Tang, C. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00890 · Cited by 76
“However, most of them require optical flows to obtain pixel-wise alignment [64, 63] which can be quite computationally extensive.”
BackgroundTemporally Identity-Aware SSD With Attentional LSTM
Chen, X., Yu, J., & Wu, Z. (2019) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2019.2894261 · Cited by 79
Beyond Short-Term Snippet: Video Relation Detection With Spatio-Temporal Global Context
Liu, C., Yang, J., Xu, K., Gong, G., & Mu, Y. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01085 · Cited by 75
“Feature-level methods [55, 56, 57] utilizes temporal information via flow-guided feature propagation from previous frames.”
MethodLearning Where to Focus for Efficient Video Object Detection
Jiang, Z., Liu, Y., Yang, C., Liu, J., Gao, P., Zhang, Q., Xiang, S., & Pan, C. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58517-4_2 · Cited by 72
“To improve detection accuracy, different methods [4, 31, 37, 41] have been proposed to aggregate features across frames.”
BackgroundMethodInfluentialAutomated rip current detection with region based convolutional neural networks
de Silva, A., Mori, I., Dusek, G., Davis, J., & Pang, A. (2021) · Coastal Engineering · doi:10.1016/j.coastaleng.2021.103859 · Cited by 74
“These methods have been extended to detect objects in videos [26, 35, 54].”
Fast Object Detection in Compressed Video
Wang, S., Alibaba Group, Lu, H., & Deng, Z. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00720 · Cited by 71
“Since frames may suffer from imaging-related degradations, most previous works [53, 44, 48, 21, 22] focus on improving frame-wise detection results.”
BackgroundMethodInfluentialJoint Anchor-Feature Refinement for Real-Time Accurate Object Detection in Images and Videos
Chen, X., Yu, J., Kong, S., Wu, Z., & Wen, L. (2020) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2020.2980876 · Cited by 70
“To the best of our knowledge, main ideas of temporal detection include i) post-processing [7], ii) tracking-based location [8], [9], iii) feature aggregation with motion information [8], [10]– [13], iv) RNN-based feature propagation [12], [14]–[16], and v) batch-frame processing (i.”
BackgroundA Multitask Benchmark Dataset for Satellite Video: Object Detection, Tracking, and Segmentation
Li, S., Zhou, Z., Zhao, M., Yang, J. B., Guo, W., Lv, Y., Kou, L., Wang, H., & Gu, Y. (2023) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2023.3278075 · Cited by 80
“For the video object detection experiments, we selected four representative video object detection methods, namely, DFF [63], FGFA [50], SELSA [64], and Temporal ROI Align [65].”
MethodReal-time Object Detection for Streaming Perception
Yang, J., Liu, S., Li, Z., Li, X., & Sun, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00531 · Cited by 72
“Some recent methods [1, 7, 11, 62] employ attention mechanism, optical flow, and tracking method, aiming to aggregate rich features for the complex video variation, e.”
Method
Metadata
- Authors
- Zhu, X., Wang, Y., Dai, J., Yuan, L., & Wei, Y.
- Affiliations
- University of Science and Technology of China (CN)
- Microsoft (United States) (US)
- Venue
- 2017 IEEE International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2017-10
- Pages
- 408-417
- OpenAlex
- W2964286567
- Cited by
- 729
- References
- 53
- Citations by year
- 2026: 48 · 2025: 57 · 2024: 86 · 2023: 91 · 2022: 102 · 2021: 123 · 2020: 96 · 2019: 75 · 2018: 47 · 2017: 4
- FWCI
- 17.47
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Surveillance and Tracking Methods
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Visual Attention and Saliency Detection
- Keywords
- video object detection, optical flow, temporal feature aggregation, movement paths, end-to-end learning, motion blur, fast-moving objects
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
