The DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
Learning 2D Temporal Adjacent Networks for Moment Localization with Natural Language
Open accessAbstract
We address the problem of retrieving a specific moment from an untrimmed video by a query sentence. This is a challenging problem because a target moment may take place in relations to other temporal moments in the untrimmed video. Existing methods cannot tackle this challenge well since they consider temporal moments individually and neglect the temporal dependencies. In this paper, we model the temporal relations between video moments by a two-dimensional map, where one dimension indicates the starting time of a moment and the other indicates the end time. This 2D temporal map can cover diverse video moments with different lengths, while representing their adjacent relations. Based on the 2D map, we propose a Temporal Adjacent Network (2D-TAN), a single-shot framework for moment localization. It is capable of encoding the adjacent temporal relation, while learning discriminative features for matching video moments with referring expressions. We evaluate the proposed 2D-TAN on three challenging benchmarks, i.e., Charades-STA, ActivityNet Captions, and TACoS, where our 2D-TAN outperforms the state-of-the-art.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Learning Spatiotemporal Features with 3D Convolutional Networks
Tran, D., Bourdev, L., Fergus, R., Torresani, L., & Paluri, M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.510 · Cited by 9,593
Learning Temporal Regularity in Video Sequences
Hasan, M., Choi, J., Neumann, J., Roy-Chowdhury, A. K., & Davis, L. S. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.86 · Cited by 1,397
Dense-Captioning Events in Videos
Krishna, R., Hata, K., Ren, F., Fei-Fei, L., & Niebles, J. C. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.83 · Cited by 1,251
Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding
Sigurdsson, G. A., Varol, G., Wang, X., Farhadi, A., Laptev, I., & Gupta, A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_31 · Cited by 1,013
Temporal Action Detection with Structured Segment Networks
Zhao, Y., Xiong, Y., Wang, L., Wu, Z., Tang, X., & Lin, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.317 · Cited by 892
TALL: Temporal Activity Localization via Language Query
Gao, J., Sun, C., Yang, Z., & Nevatia, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.563 · Cited by 842
TVSum: Summarizing web videos using titles
Song, Y., Vallmitjana, J., Stent, A., & Jaimes, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299154 · Cited by 623
TVQA: Localized, Compositional Video Question Answering
Lei, J., Yu, L., Bansal, M., & Berg, T. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1167 · Cited by 541
Grounding Action Descriptions in Videos
Regneri, M., Rohrbach, M., Wetzel, D., Thater, S., Schiele, B., & Pinkal, M. (2013) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00207 · Cited by 467
Yuan, Y., Mei, T., & Zhu, W. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33019159 · Cited by 335
MAN: Moment Alignment Network for Natural Language Moment Retrieval via Iterative Graph Adjustment
Zhang, D., Dai, X., Wang, X., Wang, Y.-F., & Davis, L. S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00134 · Cited by 331
Temporally Grounding Natural Sentence in Video
Chen, J., Chen, X., Ma, L., Jie, Z., & Chua, T.-S. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1015 · Cited by 328
Multilevel Language and Vision Integration for Text-to-Clip Retrieval
Xu, H., He, K., Plummer, B. A., Sigal, L., Sclaroff, S., & Saenko, K. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33019062 · Cited by 307
Cross-Modal Interaction Networks for Query-Based Moment Retrieval in Videos
Zhang, Z., Lin, Z., Zhao, Z., & Xiao, Z. (2019) · Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3331184.3331235 · Cited by 234
Language-Driven Temporal Activity Localization: A Semantic Matching Reinforcement Learning Model
Wang, W., Huang, Y., & Wang, L. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00042 · Cited by 198
Semantic Proposal for Activity Localization in Videos via Sentence Query
Chen, S., & Jiang, Y.-G. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33018199 · Cited by 187
Script Data for Attribute-Based Recognition of Composite Activities
Rohrbach, M., Regneri, M., Andriluka, M., Amin, S., Pinkal, M., & Schiele, B. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33718-5_11 · Cited by 185
Localizing Moments in Video with Temporal Language
Hendricks, L. A., Wang, O., Shechtman, E., Šivic, J., Darrell, T., & Russell, B. C. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1168 · Cited by 183
MAC: Mining Activity Concepts for Language-Based Temporal Localization
Ge, R., Gao, J., Chen, K., & Nevatia, R. (2019) · 2019 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2019.00032 · Cited by 175
Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding
Sigurdsson, G. A., Varol, G., Wang, X., Farhadi, A., Ivan Laptev, A., & Gupta, A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1604.01753 · Cited by 174
Temporal Action Detection with Structured Segment NetworksPreprint
Zhao, Y., Xiong, Y., Wang, L., Wu, Z., Tang, X., & Lin, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.06228 · Cited by 123
Multi-modal Circulant Fusion for Video-to-Language and Backward
Wu, A., & Han, Y. (2018) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/143 · Cited by 99
Cross-Modal Video Moment Retrieval with Spatial and Language-Temporal Attention
Jiang, B., Huang, X., Yang, C., & Yuan, J. (2019) · International Conference on Multimedia Retrieval · doi:10.1145/3323873.3325019 · Cited by 90
Tripping through time: Efficient Localization of Activities in Videos
Hahn, M., Kadav, A., Rehg, J. M., & Graf, H. P. (2020) · British Machine Vision Conference (BMVC) · doi:10.5244/c.34.135 · Cited by 89
Temporal Modular Networks for Retrieving Complex Compositional Activities in Videos
Liu, B., Yeung, S. Y., Chou, E., Huang, D.-A., Fei-Fei, L., & Niebles, J. C. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01219-9_34 · Cited by 84
Find and Focus: Retrieve and Localize Video Events with Natural Language Queries
Shao, D., Xiong, Y., Zhao, Y., Huang, Q., Qiao, Y., & Lin, D. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01240-3_13 · Cited by 83
A Dataset for Telling the Stories of Social Media Videos
Gella, S., Lewis, M., & Rohrbach, M. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1117 · Cited by 69
Exploiting Temporal Relationships in Video Moment Localization with Natural Language
Zhang, S., Su, J., & Luo, J. (2019) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3343031.3350879 · Cited by 69
Self-citationVAL: Visual-Attention Action Localizer
Song, X., & Han, Y. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00767-6_32 · Cited by 31
Cited by
VideoAgent: A Memory-augmented Multimodal Agent for Video Understanding
Fan, Y., Ma, X., Wu, R., Du, Y., Li, J., Gao, Z., & Li, Q. (2024) · arXiv · doi:10.48550/arxiv.2403.11481 · Cited by 60
Mindstorms in Natural Language-Based Societies of Mind
Zhuge, M., Liu, H., Faccio, F., Ashley, D. R., Csordás, R., Gopalakrishnan, A., Hamdi, A., Hammoud, H. A. A. K., Herrmann, V., Irie, K., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.17066 · Cited by 24
Mindstorms in Natural Language-Based Societies of Mind
Zhuge, M., Liu, H., Faccio, F., Ashley, D. R., Csordás, R., Gopalakrishnan, A., Hamdi, A., Hammoud, H. A. A. K., Herrmann, V., Irie, K., et al. (2025) · Computational Visual Media · doi:10.26599/cvm.2025.9450460 · Cited by 24
Metadata
- Authors
- Zhang, S., Peng, H., Fu, J., & Luo, J.
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2020-04-03
- Volume
- 34
- Issue
- 07
- Pages
- 12870-12877
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W2997429269
- Cited by
- 482
- References
- 34
- FWCI
- 19.30
- Type
- conference paper
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.