Video Self-Stitching Graph Network for Temporal Action Localization
The DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Abstract
Temporal action localization (TAL) in videos is a challenging task, especially due to the large variation in action temporal scales. Short actions usually occupy a major proportion in the datasets, but tend to have the lowest performance. In this paper, we confront the challenge of short actions and propose a multi-level cross-scale solution dubbed as video self-stitching graph network (VSGN). We have two key components in VSGN: video self-stitching (VSS) and cross-scale graph pyramid network (xGPN). In VSS, we focus on a short period of a video and magnify it along the temporal dimension to obtain a larger scale. We stitch the original clip and its magnified counterpart in one input sequence to take advantage of the complementary properties of both scales. The xGPN component further exploits the cross-scale correlations by a pyramid of cross-scale graph networks, each containing a hybrid module to aggregate features from across scales as well as within the same scale. Our VSGN not only enhances the feature representations, but also generates more positive anchors for short actions and more short training samples. Experiments demonstrate that VSGN obviously improves the localization performance of short actions as well as achieving the state-of-the-art overall performance on THUMOS-14 and ActivityNet-v1.3. VSGN code is available at https://github.com/coolbay/VSGN.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Feature Pyramid Networks for Object Detection
Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.106 · Cited by 30,939
YOLOv4: Optimal Speed and Accuracy of Object Detection
Bochkovskiy, A., Wang, C.-Y., & Liao, H.-Y. M. (2020) · arXiv · doi:10.48550/arxiv.2004.10934 · Cited by 10,307
Path Aggregation Network for Instance Segmentation
Liu, S., Qi, L., Qin, H., Shi, J., & Jia, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00913 · Cited by 10,011
EfficientDet: Scalable and Efficient Object Detection
Tan, M., Pang, R., & Le, Q. V. (2019) · arXiv · Cited by 9,841
Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset
Carreira, J., & Zisserman, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.502 · Cited by 9,570
Focal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2858826 · Cited by 9,556 · cited 1×
“esponding ground-truth actions, respectively. To represent the distance, we adopt the generalized intersection-over-union (GIoU) [29] and adapt it to the temporal domain. For L cls, we use focal loss [22] between the predicted classification scores and the ground-truth categories. L scris computed the same way as the T…”
MethodDynamic Graph CNN for Learning on Point Clouds
Wang, Y., Sun, Y., Liu, Z., Sarma, S. E., Bronstein, M. M., & Solomon, J. M. (2019) · ACM Transactions on Graphics · doi:10.1145/3326362 · Cited by 7,199
FCOS: Fully Convolutional One-Stage Object Detection
Tian, Z., Shen, C., Chen, H., & He, T. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00972 · Cited by 7,013
Generalized Intersection Over Union: A Metric and a Loss for Bounding Box Regression
Rezatofighi, H., Tsoi, N., Gwak, J., Sadeghian, A., Reid, I., & Savarese, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00075 · Cited by 6,612 · cited 1×
“To represent the distance, we adopt the generalized intersection-over-union (GIoU) [29] and adapt it to the temporal domain.”
MethodUcf101: A dataset of 101 human actions classes from videos in the wild
Soomro, K., Zamir, A. R., & Shah, M. (2012) · arXiv · doi:10.48550/arxiv.1212.0402 · Cited by 4,575
A Closer Look at Spatiotemporal Convolutions for Action Recognition
Tran, D., Wang, H., Torresani, L., Ray, J., LeCun, Y., & Paluri, M. (2017) · arXiv · doi:10.48550/arxiv.1711.11248 · Cited by 3,664 · cited 1×
“The blue curve areas are the original video; the red curves show the ground-truth actions, which have confidence scores 1.0 within their boundaries; the green segments are our predicted actions with their start/end time, confidence scores, and predicted labels.”
BackgroundInfluentialWu, Y., & He, K. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-019-01198-w · Cited by 3,601 · cited 1×
“M loc and M cls each contain 4 blocks of Conv1d (3, 1), group normalization (GN) [38] and ReLU layers, followed by one Conv1d (1, 1) to generate the location offsets and the classification scores for each anchor segment [41].”
MethodActivityNet: A large-scale video benchmark for human activity understanding
Heilbron, F. C., Escorcia, V., Ghanem, B., & Niebles, J. C. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298698 · Cited by 2,578 · cited 3×
“3 [5].”
Self-citationSoft-NMS — Improving Object Detection with One Line of Code
Bodla, N., Singh, B., Chellappa, R., & Davis, L. S. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.593 · Cited by 2,274 · cited 1×
“In post-processing, we apply soft-NMS [4] to suppress redundant predictions, and keep 100 predictions for final evaluation.”
MethodNAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detection
Ghiasi, G., Lin, T.-Y., & Le, Q. V. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00720 · Cited by 2,099
Temporal Action Detection with Structured Segment Networks
Zhao, Y., Xiong, Y., Wang, L., Wu, Z., Tang, X., & Lin, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.317 · Cited by 892
BMN: Boundary-Matching Network for Temporal Action Proposal Generation
Lin, T., Liu, X., Li, X., Ding, E., & Wen, S. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00399 · Cited by 694 · cited 3×
“3 dataset [20, 19, 42, 2, 46], but since these methods are better suited for a small scale (e.g., 100 frames), most videos are actually down -scaled rather than up -scaled.”
BackgroundMethodSOD-MTGAN: Small Object Detection via Multi-Task Generative Adversarial Network
Bai, Y., Zhang, Y., Ding, M., & Ghanem, B. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01261-8_13 · Cited by 487 · cited 1×
“Multiple methods have been proposed to deal with small objects specifically [3, 28] or object scale variation in general [4, 21] in images.”
BackgroundSelf-citationGaussian Temporal Awareness Networks for Action Localization
Long, F., Yao, T., Qiu, Z., Tian, X., Luo, J., & Mei, T. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00043 · Cited by 380
Chen, T., Wang, Z., Li, L., Li, D., Li, Z., Chang, X., Bian, P., Liang, G., Wang, Q., & Xie, T. (2025) · Schloss Dagstuhl – Leibniz-Zentrum für Informatik · doi:10.4230/lipics.ecoop.2025.6
Cited by
Ego4D: Around the World in 3,000 Hours of Egocentric Video
Grauman, K., Westbury, A., Byrne, E., Chavis, Z., Furnari, A., Girdhar, R., Hamburger, J., Jiang, H., Liu, M., Liu, X., et al. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01842 · Cited by 630
Self-citationActionFormer: Localizing Moments of Actions with Transformers
Zhang, C.-L., Wu, J., & Li, Y. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19772-7_29 · Cited by 435
(2023) · doi:10.1109/cvpr52729.2023.01808 · Cited by 226
“The second category directly predicts the distance between an instant and its corresponding boundary based on the instant-level feature [20,33,47,49].”
BackgroundRefineTAD: Learning Proposal-free Refinement for Temporal Action Detection
Feng, Y., Zhang, Z., Quan, R., Wang, L., & Qin, J. (2023) · Proceedings of the 31st ACM International Conference on Multimedia · doi:10.1145/3581783.3611872 · Cited by 8
Online Temporal Action Localization with Memory-Augmented Transformer
Song, Y., Kim, D., Cho, M., & Kwak, S. (2025) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-72655-2_5 · Cited by 3
Metadata
- Authors
- Zhao, C., Thabet, A., & Ghanem, B.
- Venue
- 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2021-10
- Pages
- 13638-13647
- OpenAlex
- W3202076256
- Cited by
- 151
- References
- 20
- FWCI
- 9.36
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.