The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
Local-Global Video-Text Interactions for Temporal Grounding
Open accessAbstract
This paper addresses the problem of text-to-video temporal grounding, which aims to identify the time interval in a video semantically relevant to a text query. We tackle this problem using a novel regression-based model that learns to extract a collection of mid-level features for semantic phrases in a text query, which corresponds to important semantic entities described in the query (e.g., actors, objects, and actions), and reflect bi-modal interactions between the linguistic features of the query and the visual features of the video in multiple levels. The proposed method effectively predicts the target time interval by exploiting contextual information from local to global during bi-modal interactions. Through in-depth ablation studies, we find out that incorporating both local and global context in video and text interactions is crucial to the accurate grounding. Our experiment shows that the proposed method outperforms the state of the arts on Charades-STA and ActivityNet Captions datasets by large margins, 7.44% and 4.61% points at Recall@tIoU=0.5 metric, respectively.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
(2016) · Cited by 31,590
Wang, X., Girshick, R., Gupta, A., & He, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00813 · Cited by 11,596
Learning Spatiotemporal Features with 3D Convolutional Networks
Tran, D., Bourdev, L., Fergus, R., Torresani, L., & Paluri, M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.510 · Cited by 9,593
Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset
Carreira, J., & Zisserman, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.502 · Cited by 9,570
SlowFast Networks for Video Recognition
Feichtenhofer, C., Fan, H., Malik, J., & He, K. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00630 · Cited by 4,007
ActivityNet: A large-scale video benchmark for human activity understanding
Heilbron, F. C., Escorcia, V., Ghanem, B., & Niebles, J. C. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298698 · Cited by 2,578
A Structured Self-attentive Sentence Embedding
Lin, Z., Feng, M., Santos, C. N. D., Yu, M., Xiang, B., Zhou, B., & Bengio, Y. (2017) · arXiv · doi:10.48550/arxiv.1703.03130 · Cited by 1,743 · cited 1×
“Thus, we adopt a regularization term introduced in [19] to enforce query attention weights to be distinct along different steps:”
MethodDense-Captioning Events in Videos
Krishna, R., Hata, K., Ren, F., Fei-Fei, L., & Niebles, J. C. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.83 · Cited by 1,251
Temporal Action Localization in Untrimmed Videos via Multi-stage CNNs
Shou, Z., Wang, D., & Chang, S.-F. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.119 · Cited by 968
Localizing Moments in Video with Natural Language
Hendricks, L. A., Wang, O., Shechtman, E., Sivic, J., Darrell, T., & Russell, B. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.618 · Cited by 958
Temporal Action Detection with Structured Segment Networks
Zhao, Y., Xiong, Y., Wang, L., Wu, Z., Tang, X., & Lin, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.317 · Cited by 892
TALL: Temporal Activity Localization via Language Query
Gao, J., Sun, C., Yang, Z., & Nevatia, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.563 · Cited by 842
R-C3D: Region Convolutional 3D Network for Temporal Activity Detection
Xu, H., Das, A., & Saenko, K. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.617 · Cited by 783
Hadamard Product for Low-rank Bilinear PoolingPreprint
Kim, J.-H., On, K.-W., Lim, W., Kim, J., Ha, J.-W., & Zhang, B. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.04325 · Cited by 780
End-to-End Learning of Action Detection from Frame Glimpses in Videos
Yeung, S., Russakovsky, O., Mori, G., & Fei-Fei, L. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.293 · Cited by 585 · cited 2×
“Third, there exist some approaches [18, 34] based on single-shot detection like SSD [21] for fast inference.”
BackgroundMethodShou, Z., Chan, J., Zareian, A., Miyazawa, K., & Chang, S.-F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.155 · Cited by 574
Graph Convolutional Networks for Temporal Action Localization
Zeng, R., Huang, W., Gan, C., Tan, M., Rong, Y., Zhao, P., & Huang, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00719 · Cited by 551
TVQA: Localized, Compositional Video Question Answering
Lei, J., Yu, L., Bansal, M., & Berg, T. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1167 · Cited by 541 · cited 2×
“Furthermore, with the recent advances of deep learning on top of large-scale datasets [2, 16, 17, 23], research on video content understanding is moving towards multi-modal problems (e.”
BackgroundSingle Shot Temporal Action Detection
Lin, T., Zhao, X., & Shou, Z. (2017) · Proceedings of the 25th ACM international conference on Multimedia · doi:10.1145/3123266.3123343 · Cited by 385
Yuan, Y., Mei, T., & Zhu, W. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33019159 · Cited by 335
MAN: Moment Alignment Network for Natural Language Moment Retrieval via Iterative Graph Adjustment
Zhang, D., Dai, X., Wang, X., Wang, Y.-F., & Davis, L. S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00134 · Cited by 331
Temporally Grounding Natural Sentence in Video
Chen, J., Chen, X., Ma, L., Jie, Z., & Chua, T.-S. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1015 · Cited by 328
Attentive Moment Retrieval in Videos
Liu, M., Wang, X., Nie, L., He, X., Chen, B., & Chua, T.-S. (2018) · The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval · doi:10.1145/3209978.3210003 · Cited by 307
Multilevel Language and Vision Integration for Text-to-Clip Retrieval
Xu, H., He, K., Plummer, B. A., Sigal, L., Sclaroff, S., & Saenko, K. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33019062 · Cited by 307
Dynamic Graph Attention for Referring Expression Comprehension
Yang, S., Li, G., & Yu, Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00474 · Cited by 233
Streamlined Dense Video Captioning
Mun, J., Yang, L., Ren, Z., Xu, N., & Han, B. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00675 · Cited by 213
Self-citationLanguage-Driven Temporal Activity Localization: A Semantic Matching Reinforcement Learning Model
Wang, W., Huang, Y., & Wang, L. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00042 · Cited by 198
Semantic Proposal for Activity Localization in Videos via Sentence Query
Chen, S., & Jiang, Y.-G. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33018199 · Cited by 187
Rodríguez-Opazo, C., Marrese-Taylor, E., Saleh, F. S., Li, H., & Gould, S. J. (2020) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv45572.2020.9093328 · Cited by 186
MAC: Mining Activity Concepts for Language-Based Temporal Localization
Ge, R., Gao, J., Chen, K., & Nevatia, R. (2019) · 2019 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2019.00032 · Cited by 175
ExCL: Extractive Clip Localization Using Natural Language Descriptions
Ghosh, S., Agarwal, A., Parekh, Z., & Hauptmann, A. G. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1198 · Cited by 162
He, D., Zhao, X., Huang, J., Li, F., Liu, X., & Wen, S. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33018393 · Cited by 158
Temporal Activity Detection in Untrimmed Videos with Recurrent Neural Networks
Montes, A., Salvador, A., Pascual, S., & Giró-i-Nieto, X. (2016) · UPCommons institutional repository (Universitat Politècnica de Catalunya) · doi:10.48550/arxiv.1608.08128 · Cited by 98
MarioQA: Answering Questions by Watching Gameplay Videos
Mun, J., Seo, P. H., Jung, I., & Han, B. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.312 · Cited by 95
Self-citationTripping through time: Efficient Localization of Activities in Videos
Hahn, M., Kadav, A., Rehg, J. M., & Graf, H. P. (2020) · British Machine Vision Conference (BMVC) · doi:10.5244/c.34.135 · Cited by 89
ExCL: Extractive Clip Localization Using Natural Language DescriptionsPreprint
Ghosh, S., Agarwal, A., Parekh, Z., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.02755 · Cited by 77
Cited by
Neighbor-view Enhanced Model for Vision and Language Navigation
An, D., Qi, Y., Huang, Y., Wu, Q., Wang, L., & Tan, T. (2021) · Proceedings of the 29th ACM International Conference on Multimedia · doi:10.1145/3474085.3475282 · Cited by 74
Mindstorms in Natural Language-Based Societies of Mind
Zhuge, M., Liu, H., Faccio, F., Ashley, D. R., Csordás, R., Gopalakrishnan, A., Hamdi, A., Hammoud, H. A. A. K., Herrmann, V., Irie, K., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.17066 · Cited by 24
Mindstorms in Natural Language-Based Societies of Mind
Zhuge, M., Liu, H., Faccio, F., Ashley, D. R., Csordás, R., Gopalakrishnan, A., Hamdi, A., Hammoud, H. A. A. K., Herrmann, V., Irie, K., et al. (2025) · Computational Visual Media · doi:10.26599/cvm.2025.9450460 · Cited by 24
Metadata
- Authors
- Mun, J., Cho, M., & Han, B.
- Venue
- 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2020-06
- Pages
- 10807-10816
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W3035339529
- Cited by
- 303
- References
- 39
- FWCI
- 12.57
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.