The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
TALL: Temporal Activity Localization via Language Query
Abstract
This paper focuses on temporal localization of actions in untrimmed videos. Existing methods typically train classifiers for a pre-defined list of actions and apply them in a sliding window fashion. However, activities in the wild consist of a wide combination of actors, actions and objects; it is difficult to design a proper activity list that meets users' needs. We propose to localize activities by natural language queries. Temporal Activity Localization via Language (TALL) is challenging as it requires: (1) suitable design of text and video representations to allow cross-modal matching of actions and language queries; (2) ability to locate actions accurately given features from sliding windows of limited granularity. We propose a novel Cross-modal Temporal Regression Localizer (CTRL) to jointly model text query and video clips, output alignment scores and action boundary regression results for candidate clips. Lor evaluation, we adopt TaCoS dataset, and build a new dataset for this task on top of Charades by adding sentence temporal annotations, called Charades-STA. We also build complex sentence queries in Charades-STA for test. Experimental results show that CTRL outperforms previous methods significantly on both datasets.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Learning Spatiotemporal Features with 3D Convolutional Networks
Tran, D., Bourdev, L., Fergus, R., Torresani, L., & Paluri, M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.510 · Cited by 9,593
The Stanford CoreNLP Natural Language Processing Toolkit
Manning, C., Surdeanu, M., Bauer, J., Finkel, J., Bethard, S., & McClosky, D. (2014) · Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations · doi:10.3115/v1/p14-5010 · Cited by 7,037
Cited by
Localizing Moments in Video with Natural Language
Hendricks, L. A., Wang, O., Shechtman, E., Sivic, J., Darrell, T., & Russell, B. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.618 · Cited by 958
Less is More: CLIPBERT for Video-and-Language Learning via Sparse Sampling
Lei, J., Li, L., Zhou, L., Gan, Z., Berg, T. L., Bansal, M., & Liu, J. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00725 · Cited by 589
TVQA: Localized, Compositional Video Question Answering
Lei, J., Yu, L., Bansal, M., & Berg, T. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1167 · Cited by 541
“Most recently, moment localization was proposed in (Hendricks et al., 2017; Gao et al., 2017), where the goal is to localize a short moment from a long video sequence given a query description.”
BackgroundLearning 2D Temporal Adjacent Networks for Moment Localization with Natural Language
Zhang, S., Peng, H., Fu, J., & Luo, J. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6984 · Cited by 482
HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training
Li, L., Chen, Y.-C., Cheng, Y., Gan, Z., Yu, L., & Liu, J. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.161 · Cited by 408
Yuan, Y., Mei, T., & Zhu, W. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33019159 · Cited by 335
MAN: Moment Alignment Network for Natural Language Moment Retrieval via Iterative Graph Adjustment
Zhang, D., Dai, X., Wang, X., Wang, Y.-F., & Davis, L. S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00134 · Cited by 331
Temporally Grounding Natural Sentence in Video
Chen, J., Chen, X., Ma, L., Jie, Z., & Chua, T.-S. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1015 · Cited by 328
VideoMamba: State Space Model for Efficient Video Understanding
Li, K., Li, X., Wang, Y., He, Y., Wang, Y., Wang, L., & Qiao, Y. (2025) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-73347-5_14 · Cited by 312
Attentive Moment Retrieval in Videos
Liu, M., Wang, X., Nie, L., He, X., Chen, B., & Chua, T.-S. (2018) · The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval · doi:10.1145/3209978.3210003 · Cited by 307
Multilevel Language and Vision Integration for Text-to-Clip Retrieval
Xu, H., He, K., Plummer, B. A., Sigal, L., Sclaroff, S., & Saenko, K. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33019062 · Cited by 307
Local-Global Video-Text Interactions for Temporal Grounding
Mun, J., Cho, M., & Han, B. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01082 · Cited by 303
MVBench: A Comprehensive Multi-modal Video Understanding Benchmark
Li, K., Wang, Y., He, Y., Li, Y., Wang, Y., Liu, Y., Wang, Z., Xu, J., Chen, G., Lou, P., et al. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52733.2024.02095 · Cited by 300
Span-based Localizing Network for Natural Language Video Localization
Zhang, H., Sun, A., Jing, W., & Zhou, J. T. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.585 · Cited by 297
Dense Regression Network for Video Grounding
Zeng, R., Xu, H., Huang, W., Chen, P., Tan, M., & Gan, C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01030 · Cited by 286
TVR: A Large-Scale Dataset for Video-Subtitle Moment Retrieval
Lei, J., Yu, L., Berg, T. L., & Bansal, M. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58589-1_27 · Cited by 255
A Survey on Deep Learning Technique for Video Segmentation
Zhou, T., Porikli, F., Crandall, D. J., Van Gool, L., & Wang, W. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3225573 · Cited by 255
Cross-modal Moment Localization in Videos
Liu, M., Wang, X., Nie, L., Tian, Q., Chen, B., & Chua, T.-S. (2018) · Proceedings of the 26th ACM international conference on Multimedia · doi:10.1145/3240508.3240549 · Cited by 253
Target Adaptive Context Aggregation for Video Scene Graph Generation
Teng, Y., Wang, L., Li, Z., & Wu, G. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01343 · Cited by 74
A survey of transformer-based multimodal pre-trained modals
Han, X., Wang, Y.-T., Feng, J.-L., Deng, C., Chen, Z.-H., Huang, Y.-A., Su, H., Hu, L., & Hu, P.-W. (2023) · Neurocomputing · doi:10.1016/j.neucom.2022.09.136 · Cited by 59
Aligning Subtitles in Sign Language Videos
Bull, H., Afouras, T., Varol, G., Albanie, S., Momeni, L., & Zisserman, A. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01135 · Cited by 30
Mindstorms in Natural Language-Based Societies of Mind
Zhuge, M., Liu, H., Faccio, F., Ashley, D. R., Csordás, R., Gopalakrishnan, A., Hamdi, A., Hammoud, H. A. A. K., Herrmann, V., Irie, K., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.17066 · Cited by 24
Mindstorms in Natural Language-Based Societies of Mind
Zhuge, M., Liu, H., Faccio, F., Ashley, D. R., Csordás, R., Gopalakrishnan, A., Hamdi, A., Hammoud, H. A. A. K., Herrmann, V., Irie, K., et al. (2025) · Computational Visual Media · doi:10.26599/cvm.2025.9450460 · Cited by 24
Bai, S., Cai, Y., Chen, R., Chen, K., Chen, X., Cheng, Z., Deng, L., Ding, W., Gao, C., Ge, C., et al. (2025) · arXiv · doi:10.48550/arxiv.2511.21631 · Cited by 27
“…Understanding: We use seven benchmarks to evaluate the model’s video understanding capabilities: VideoMME (Fu et al., 2024a), MVBench (Li et al., 2024b), VideoMMMU (Hu et al., 2025), MMVU (Zhao et al., 2025), LVBench (Wang et al., 2024d), MLVU (Zhou et al., 2024), Charades-STA (Gao et al., 2017).”
InfluentialSelf-citationToward Long Video Understanding via Fine-Detailed Video Story Generation
You, Z., Wen, Z., Chen, Y., Li, X., Zeng, R., Wang, Y., & Tan, M. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3514820 · Cited by 11
Metadata
- Authors
- Gao, J., Sun, C., Yang, Z., & Nevatia, R.
- Venue
- 2017 IEEE International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2017-10
- Pages
- 5277-5285
- OpenAlex
- W2964089981
- Cited by
- 842
- References
- 38
- FWCI
- 10.19
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.