The DOI matches its Crossref record; 60 of the 60 references it cites were found in a registry or an index.
MAN: Moment Alignment Network for Natural Language Moment Retrieval via Iterative Graph Adjustment
Open accessAbstract
This research strives for natural language moment retrieval in long, untrimmed video streams. The problem is not trivial especially when a video contains multiple moments of interests and the language describes complex temporal dependencies, which often happens in real scenarios. We identify two crucial challenges: semantic misalignment and structural misalignment. However, existing approaches treat different moments separately and do not explicitly model complex moment-wise temporal relations. In this paper, we present Moment Alignment Network (MAN), a novel framework that unifies the candidate moment encoding and temporal structural reasoning in a single-shot feed-forward network. MAN naturally assigns candidate moment representations aligned with language semantics over different temporal locations and scales. Most importantly, we propose to explicitly model moment-wise temporal relations as a structured graph and devise an iterative graph adjustment network to jointly learn the best structure in an end-to-end manner. We evaluate the proposed approach on two challenging public benchmarks DiDeMo and Charades-STA, where our MAN significantly outperforms the state-of-the-art by a large margin.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
(2016) · Cited by 31,590
TensorFlow: A system for large-scale machine learning
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08695 · Cited by 11,822
Wang, X., Girshick, R., Gupta, A., & He, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00813 · Cited by 11,596
Self-citationLearning Spatiotemporal Features with 3D Convolutional Networks
Tran, D., Bourdev, L., Fergus, R., Torresani, L., & Paluri, M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.510 · Cited by 9,593
Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset
Carreira, J., & Zisserman, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.502 · Cited by 9,570
Two-Stream Convolutional Networks for Action Recognition in Videos
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1406.2199 · Cited by 6,756
Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition
Yan, S., Xiong, Y., & Lin, D. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12328 · Cited by 5,410
A Closer Look at Spatiotemporal Convolutions for Action Recognition
Tran, D., Wang, H., Torresani, L., Ray, J., LeCun, Y., & Paluri, M. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00675 · Cited by 3,516
Convolutional Two-Stream Network Fusion for Video Action Recognition
Feichtenhofer, C., Pinz, A., & Zisserman, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.213 · Cited by 2,684
Learning Spatio-Temporal Representation with Pseudo-3D Residual Networks
Qiu, Z., Yao, T., & Mei, T. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.590 · Cited by 1,874
Relation Networks for Object Detection
Hu, H., Gu, J., Zhang, Z., Dai, J., & Wei, Y. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00378 · Cited by 1,524
A simple neural network module for relational reasoning
Santoro, A., Raposo, D., Barrett, D. G. T., Malinowski, M., Pascanu, R., Battaglia, P., & Lillicrap, T. (2017) · arXiv · doi:10.48550/arxiv.1706.01427 · Cited by 1,281
Image retrieval using scene graphs
Johnson, J., Krishna, R., Stark, M., Li, L.-J., Shamma, D. A., Bernstein, M. S., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298990 · Cited by 1,101
Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding
Sigurdsson, G. A., Varol, G., Wang, X., Farhadi, A., Laptev, I., & Gupta, A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_31 · Cited by 1,013
Self-citationTemporal Action Localization in Untrimmed Videos via Multi-stage CNNs
Shou, Z., Wang, D., & Chang, S.-F. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.119 · Cited by 968
Localizing Moments in Video with Natural Language
Hendricks, L. A., Wang, O., Shechtman, E., Sivic, J., Darrell, T., & Russell, B. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.618 · Cited by 958
Temporal Action Detection with Structured Segment Networks
Zhao, Y., Xiong, Y., Wang, L., Wu, Z., Tang, X., & Lin, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.317 · Cited by 892
Graph R-CNN for Scene Graph Generation
Yang, J., Lu, J., Lee, S., Batra, D., & Parikh, D. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01246-5_41 · Cited by 851
TALL: Temporal Activity Localization via Language Query
Gao, J., Sun, C., Yang, Z., & Nevatia, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.563 · Cited by 842
R-C3D: Region Convolutional 3D Network for Temporal Activity Detection
Xu, H., Das, A., & Saenko, K. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.617 · Cited by 783
Rethinking the Faster R-CNN Architecture for Temporal Action Localization
Chao, Y.-W., Vijayanarasimhan, S., Seybold, B., Ross, D. A., Deng, J., & Sukthankar, R. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00124 · Cited by 725
Detecting and Recognizing Human-Object Interactions
Gkioxari, G., Girshick, R., Dollar, P., & He, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00872 · Cited by 602
Shou, Z., Chan, J., Zareian, A., Miyazawa, K., & Chang, S.-F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.155 · Cited by 574
Modeling mutual context of object and human pose in human-object interaction activities
Yao, B., & Fei-Fei, L. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5540235 · Cited by 573
Observing Human-Object Interactions: Using Spatial and Functional Compatibility for Recognition
Gupta, A., Kembhavi, A., & Davis, L. S. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.83 · Cited by 502
Self-citationDetecting Visual Relationships with Deep Relational Networks
Dai, B., Zhang, Y., & Lin, D. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.352 · Cited by 502
Semantic Segmentation with Second-Order Pooling
Carreira, J., Caseiro, R., Batista, J., & Sminchisescu, C. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33786-4_32 · Cited by 458
Single Shot Temporal Action Detection
Lin, T., Zhao, X., & Shou, Z. (2017) · Proceedings of the 25th ACM international conference on Multimedia · doi:10.1145/3123266.3123343 · Cited by 385
Action and Event Recognition with Fisher Vectors on a Compact Feature Set
Oneata, D., Verbeek, J., & Schmid, C. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.228 · Cited by 376
Modeling Relationships in Referential Expressions with Compositional Modular Networks
Hu, R., Rohrbach, M., Andreas, J., Darrell, T., & Saenko, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.470 · Cited by 375
Yao, J., Fidler, S., & Urtasun, R. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247739 · Cited by 365
Towards Good Practices for Very Deep Two-Stream ConvNets
Wang, L., Xiong, Y., Wang, Z., & Qiao, Y. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1507.02159 · Cited by 340
Temporally Grounding Natural Sentence in Video
Chen, J., Chen, X., Ma, L., Jie, Z., & Chua, T.-S. (2018) · Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-1015 · Cited by 328
The More You Know: Using Knowledge Graphs for Image Classification
Marino, K., Salakhutdinov, R., & Gupta, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.10 · Cited by 315
Multilevel Language and Vision Integration for Text-to-Clip Retrieval
Xu, H., He, K., Plummer, B. A., Sigal, L., Sclaroff, S., & Saenko, K. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33019062 · Cited by 307
Temporal Context Network for Activity Localization in Videos
Dai, X., Singh, B., Zhang, G., Davis, L. S., & Chen, Y. Q. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.610 · Cited by 271
Self-citationVideo Captioning via Hierarchical Reinforcement Learning
Wang, X., Chen, W., Wu, J., Wang, Y.-F., & Wang, W. Y. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00443 · Cited by 262
Self-citationAction Localization with Tubelets from Motion
Jain, M., Van Gemert, J., Jegou, H., Bouthemy, P., & Snoek, C. G. M. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.100 · Cited by 246
End-to-End, Single-Stream Temporal Action Detection in Untrimmed Videos
Buch, S., Escorcia, V., Ghanem, B., & Niebles, J. C. (2017) · Procedings of the British Machine Vision Conference 2017 · doi:10.5244/c.31.93 · Cited by 240 · cited 2×
“To better localize temporal boundaries, a large body of work incorporated deep networks into the detection framework and obtained improved performance [5, 28, 10, 41, 2, 61, 42, 53, 59].”
BackgroundTemporal Localization of Actions with Actoms
Gaidon, A., Harchaoui, Z., & Schmid, C. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.65 · Cited by 217
Tracking by Natural Language Specification
Li, Z., Tao, R., Gavves, E., Snoek, C. G. M., & Smeulders, A. W. M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.777 · Cited by 177
Weakly-Supervised Learning of Visual Relations
Peyre, J., Laptev, I., Schmid, C., & Sivic, J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.554 · Cited by 172
Localizing Moments in Video with Temporal Language
Hendricks, L. A., Wang, O., Shechtman, E., Šivic, J., Darrell, T., & Russell, B. C. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1168 · Cited by 158
Attend and Interact: Higher-Order Object Interactions for Video Understanding
Ma, C., Kadav, A., Melvin, I., Kira, Z., AlRegib, G., & Graf, H. P. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00710 · Cited by 137
Combining the Right Features for Complex Event Recognition
Tang, K., Yao, B., Fei-Fei, L., & Koller, D. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.335 · Cited by 84
FASON: First and Second Order Information Fusion Network for Texture Recognition
Dai, X., Ng, J. Y.-H., & Davis, L. S. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.646 · Cited by 83
Self-citationProgressively Parsing Interactional Objects for Fine Grained Action Detection
Ni, B., Yang, X., & Gao, S. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.116 · Cited by 69
Temporal Modular Networks for Retrieving Complex Compositional Activities in Videos
Liu, B., Yeung, S. Y., Chou, E., Huang, D.-A., Fei-Fei, L., & Niebles, J. C. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01219-9_34 · Cited by 67
Mettes, P., van Gemert, J. C., Cappallo, S., Mensink, T., & Snoek, C. G. M. (2015) · International Conference on Multimedia Retrieval · doi:10.1145/2671188.2749404 · Cited by 53
S3D: Single Shot multi-Span Detector via Fully 3D Convolutional NetworksPreprint
Zhang, D., Dai, X., Wang, X., & Wang, Y.-F. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.08069 · Cited by 46
Self-citationDynamic Temporal Pyramid Network: A Closer Look at Multi-scale Modeling for Activity Detection
Zhang, D., Dai, X., & Wang, Y.-F. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-20870-7_44 · Cited by 45
Self-citationLearning to Compose Topic-Aware Mixture of Experts for Zero-Shot Video Captioning
Wang, X. E., Wu, J., Zhang, D., Su, Y., & Wang, W. Y. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33018965 · Cited by 36
Self-citationTAN: Temporal Aggregation Network for Dense Multi-Label Action Recognition
Dai, X., Singh, B., Ng, J. Y.-H., & Davis, L. S. (2019) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2019.00022 · Cited by 22
Self-citationChen, T., Wang, Z., Li, L., Li, D., Li, Z., Chang, X., Bian, P., Liang, G., Wang, Q., & Xie, T. (2025) · Schloss Dagstuhl – Leibniz-Zentrum für Informatik · doi:10.4230/lipics.ecoop.2025.6
Cited by
Wang, X., Huang, Q., Celikyilmaz, A., Gao, J., Shen, D., Wang, Y.-F., Wang, W. Y., & Zhang, L. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00679 · Cited by 522
Self-citationVaTeX: A Large-Scale, High-Quality Multilingual Dataset for Video-and-Language Research
Wang, X., Wu, J., Chen, J., Li, L., Wang, Y.-F., & Wang, W. Y. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00468 · Cited by 486
Self-citationLearning 2D Temporal Adjacent Networks for Moment Localization with Natural Language
Zhang, S., Peng, H., Fu, J., & Luo, J. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6984 · Cited by 482
Local-Global Video-Text Interactions for Temporal Grounding
Mun, J., Cho, M., & Han, B. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01082 · Cited by 303
Dense Regression Network for Video Grounding
Zeng, R., Xu, H., Huang, W., Chen, P., Tan, M., & Gan, C. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01030 · Cited by 286
Metadata
- Authors
- Zhang, D., Dai, X., Wang, X., Wang, Y.-F., & Davis, L. S.
- Venue
- 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2019-06
- Pages
- 1247-1257
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2962869524
- Cited by
- 331
- References
- 60
- FWCI
- 15.43
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.