The DOI matches its Crossref record; 63 of the 63 references it cites were found in a registry or an index.
Dense-Captioning Events in Videos
Open accessAbstract
Most natural videos contain numerous events. For example, in a video of a “man playing a piano”, the video might also contain “another man dancing” or “a crowd clapping”. We introduce the task of dense-captioning events, which involves both detecting and describing events in a video. We propose a new model that is able to identify all events in a single pass of the video while simultaneously describing the detected events with natural language. Our model introduces a variant of an existing proposal module that is designed to capture both short as well as long events that span minutes. To capture the dependencies between the events in a video, our model introduces a new captioning module that uses contextual information from past and future events to jointly describe all events. We also introduce ActivityNet Captions, a large-scale benchmark for dense-captioning events. ActivityNet Captions contains 20k videos amounting to 849 video hours with 100k total descriptions, each with its unique start and end time. Finally, we report performances of our model for dense-captioning events, video retrieval and localization.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vision meets robotics: The KITTI dataset
Geiger, A., Lenz, P., Stiller, C., & Urtasun, R. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913491297 · Cited by 9,913
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248
Large-Scale Video Classification with Convolutional Neural Networks
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.223 · Cited by 6,354
Self-citation3D Convolutional Neural Networks for Human Action Recognition
Ji, S., Xu, W., Yang, M., & Yu, K. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.59 · Cited by 6,235
Recurrent neural network based language model
Mikolov, T., Karafiát, M., Burget, L., Černocký, J., & Khudanpur, S. (2010) · Interspeech 2010 · doi:10.21437/interspeech.2010-343 · Cited by 5,390
Long-term recurrent convolutional networks for visual recognition and description
Donahue, J., Hendricks, L. A., Guadarrama, S., Rohrbach, M., Venugopalan, S., Darrell, T., & Saenko, K. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298878 · Cited by 5,228
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
Krishna, R., Zhu, Y., Groth, O., Johnson, J., Hata, K., Kravitz, J., Chen, S., Kalantidis, Y., Li, L.-J., Shamma, D. A., et al. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-016-0981-7 · Cited by 5,107
Self-citationDeep visual-semantic alignments for generating image descriptions
Karpathy, A., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298932 · Cited by 4,902
Self-citationUcf101: A dataset of 101 human actions classes from videos in the wild
Soomro, K., Zamir, A. R., & Shah, M. (2012) · arXiv · doi:10.48550/arxiv.1212.0402 · Cited by 4,575
Hmdb: a large video database for human motion recognition
Kuehne, H., Jhuang, H., Garrote, E., Poggio, T., & Serre, T. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126543 · Cited by 3,965
Social LSTM: Human Trajectory Prediction in Crowded Spaces
Alahi, A., Goel, K., Ramanathan, V., Robicquet, A., Fei-Fei, L., & Savarese, S. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.110 · Cited by 3,483
Self-citationLearning realistic human actions from movies
Laptev, I., Marszalek, M., Schmid, C., & Rozenfeld, B. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587756 · Cited by 3,444
Recognizing human actions: A local SVM approach
Schüldt, C., Laptev, I., & Barbara Caputo, B. (2004) · Cited by 2,908
ActivityNet: A large-scale video benchmark for human activity understanding
Heilbron, F. C., Escorcia, V., Ghanem, B., & Niebles, J. C. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298698 · Cited by 2,578
Self-citationMSR-VTT: A Large Video Description Dataset for Bridging Video and Language
Xu, J., Mei, T., Yao, T., & Rui, Y. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.571 · Cited by 1,759
Sequence to Sequence -- Video to Text
Venugopalan, S., Rohrbach, M., Donahue, J., Mooney, R., Darrell, T., & Saenko, K. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.515 · Cited by 1,352
Recognizing human action in time-sequential images using hidden Markov model
Yamato, J., Ohya, J., & Ishii, K. (2003) · Proceedings 1992 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.1992.223161 · Cited by 1,273
Large-scale Video Classification with Convolutional Neural Networks
Karpathy, A., Toderici, G. D., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014) · Cited by 1,266
Self-citationMarszalek, M., Laptev, I., & Schmid, C. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206557 · Cited by 1,168
DenseCap: Fully Convolutional Localization Networks for Dense Captioning
Johnson, J., Karpathy, A., & Fei-Fei, L. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.494 · Cited by 1,137
Self-citationDescribing Videos by Exploiting Temporal Structure
Yao, L., Torabi, A., Cho, K., Ballas, N., Pal, C., Larochelle, H., & Courville, A. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.512 · Cited by 1,035
Translating Videos to Natural Language Using Deep Recurrent Neural Networks
Venugopalan, S., Xu, H., Donahue, J., Rohrbach, M., Mooney, R., & Saenko, K. (2015) · Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.3115/v1/n15-1173 · Cited by 959
Discovering Event Structure in Continuous Narrative Perception and Memory
Baldassano, C., Chen, J., Zadbood, A., Pillow, J. W., Hasson, U., & Norman, K. A. (2017) · Neuron · doi:10.1016/j.neuron.2017.06.041 · Cited by 952
Collecting Highly Parallel Data for Paraphrase Evaluation
Chen, D. J., & Dolan, W. B. (2011) · Cited by 898
Modeling Temporal Structure of Decomposable Motion Segments for Activity Classification
Niebles, J. C., Chen, C.-W., & Fei-Fei, L. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15552-9_29 · Cited by 700
Self-citationTVSum: Summarizing web videos using titles
Song, Y., Vallmitjana, J., Stent, A., & Jaimes, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299154 · Cited by 623
Video Paragraph Captioning Using Hierarchical Recurrent Neural Networks
Yu, H., Wang, J., Huang, Z., Yang, Y., & Xu, W. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.496 · Cited by 564
Jointly Modeling Embedding and Translation to Bridge Video and Language
Pan, Y., Mei, T., Yao, T., Li, H., & Rui, Y. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.497 · Cited by 563
Gkioxari, G., & Malik, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298676 · Cited by 528
A database for fine grained activity detection of cooking activities
Rohrbach, M., Amin, S., Andriluka, M., & Schiele, B. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247801 · Cited by 521
Grounding Action Descriptions in Videos
Regneri, M., Rohrbach, M., Wetzel, D., Thater, S., Schiele, B., & Pinkal, M. (2013) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00207 · Cited by 467
Detecting Irregularities in Images and in Video
Boiman, O., & Irani, M. (2007) · International Journal of Computer Vision · doi:10.1007/s11263-006-0009-9 · Cited by 439
DAPs: Deep Action Proposals for Action Understanding
Escorcia, V., Caba Heilbron, F., Niebles, J. C., & Ghanem, B. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46487-9_47 · Cited by 400
Self-citationVideo summarization by learning submodular mixtures of objectives
Gygli, M., Grabner, H., & Van Gool, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298928 · Cited by 397
A Hierarchical Approach for Generating Descriptive Image Paragraphs
Krause, J., Johnson, J., Krishna, R., & Fei-Fei, L. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.356 · Cited by 364
Self-citationDescribing Videos by Exploiting Temporal StructurePreprint
Yao, L., Torabi, A., Cho, K., Ballas, N., Pal, C., Larochelle, H., & Courville, A. C. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1502.08029 · Cited by 310
Xu, R., Xiong, C., Chen, W., & Corso, J. (2015) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v29i1.9512 · Cited by 298
Das, P., Xu, C., Doell, R. F., & Corso, J. J. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.340 · Cited by 282
Highlight Detection with Pairwise Deep Ranking for First-Person Video Summarization
Yao, T., Mei, T., & Rui, Y. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.112 · Cited by 276
Fast Temporal Activity Proposals for Efficient Detection of Human Actions in Untrimmed Videos
Heilbron, F. C., Niebles, J. C., & Ghanem, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.211 · Cited by 276
Self-citationAutomatic annotation of human actions in video
Duchenne, O., Laptev, I., Sivic, J., Bach, F., & Ponce, J. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459279 · Cited by 275
We Are Dynamo: Overcoming Stalling and Friction in Collective Action for Crowd Workers
Salehi, N., Irani, L. C., Bernstein, M. S., Alkhatib, A., Ogbe, E., Milland, K., & Clickhappier (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702508 · Cited by 274
Multi-task deep visual-semantic embedding for video thumbnail selection
Liu, W., Mei, T., Zhang, Y., Che, C., & Luo, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298994 · Cited by 255
Spatiotemporal Deformable Part Models for Action Detection
Tian, Y., Sukthankar, R., & Shah, M. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.341 · Cited by 240
Sequence to Sequence -- Video to TextPreprint
Venugopalan, S., Rohrbach, M., Donahue, J., Mooney, R. J., Darrell, T., & Saenko, K. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.00487 · Cited by 225
Temporal Localization of Actions with Actoms
Gaidon, A., Harchaoui, Z., & Schmid, C. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.65 · Cited by 217
Coherent Multi-sentence Video Description with Variable Level of Detail
Rohrbach, A., Rohrbach, M., Qiu, W., Friedrich, A., Pinkal, M., & Schiele, B. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-11752-2_15 · Cited by 206
Hollywood in Homes: Crowdsourcing Data Collection for Activity Understanding
Sigurdsson, G. A., Varol, G., Wang, X., Farhadi, A., Ivan Laptev, A., & Gupta, A. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1604.01753 · Cited by 174
Unsupervised Extraction of Video Highlights via Robust Recurrent Auto-Encoders
Yang, H., Wang, B., Lin, S., Wipf, D., Guo, M., & Guo, B. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.526 · Cited by 172
Dense Captioning with Joint Inference and Visual Context
Yang, L., Tang, K., Yang, J., & Li, L.-J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.214 · Cited by 167
Parsing Videos of Actions with Segmental Grammars
Pirsiavash, H., & Ramanan, D. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.85 · Cited by 161
Using Descriptive Video Services to Create a Large Data Source for Video Annotation ResearchPreprint
Torabi, A., Pal, C., Larochelle, H., & Courville, A. C. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1503.01070 · Cited by 149
Schematic storyboarding for video visualization and editing
Goldman, D. B., Curless, B., Salesin, D., & Seitz, S. M. (2006) · ACM Transactions on Graphics · doi:10.1145/1141911.1141967 · Cited by 148
Schematic storyboarding for video visualization and editing
Goldman, D. B., Curless, B., Salesin, D., & Seitz, S. M. (2006) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/1179352.1141967 · Cited by 138
Slow and Steady Feature Analysis: Higher Order Temporal Coherence in Video
Jayaraman, D., & Grauman, K. L. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.418 · Cited by 136
Video Action Detection with Relational Dynamic-Poselets
Wang, L., Qiao, Y., & Tang, X. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10602-1_37 · Cited by 124
The Long-Short Story of Movie Description
Rohrbach, A., Rohrbach, M., & Schiele, B. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24947-6_17 · Cited by 119
A discriminative key pose sequence model for recognizing human interactions
Vahdat, A., Gao, B., Ranjbar, M., & Mori, G. (2011) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2011.6130458 · Cited by 89
Learning Joint Representations of Videos and Sentences with Web Image Search
Otani, M., Nakashima, Y., Rahtu, E., Heikkilä, J., & Yokoya, N. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46604-0_46 · Cited by 81
Multiple Granularity Analysis for Fine-Grained Action Detection
Ni, B.-B., Paramathayalan, V. R., & Moulin, P. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.102 · Cited by 79
A Multi-scale Multiple Instance Video Description NetworkPreprint
Xu, H., Venugopalan, S., Ramanishka, V., Rohrbach, M., & Saenko, K. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.05914 · Cited by 71
Efficient Action Localization with Approximately Normalized Fisher Vectors
Oneaţă, D., Verbeek, J. J., & Schmid, C. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.326 · Cited by 67
A Dataset for Movie DescriptionPreprint
Rohrbach, A., Rohrbach, M., Tandon, N., & Schiele, B. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1501.02530 · Cited by 58
Cited by
Transformers in Vision: A Survey
Khan, S., Naseer, M., Hayat, M., Zamir, S. W., Khan, F. S., & Shah, M. (2022) · ACM Computing Surveys · doi:10.1145/3505244 · Cited by 3,595
The “Something Something” Video Database for Learning and Evaluating Visual Common Sense
Goyal, R., Kahou, S. E., Michalski, V., Materzynska, J., Westphal, S., Kim, H., Haenel, V., Fruend, I., Yianilos, P., Mueller-Freitag, M., et al. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.622 · Cited by 1,471
Multimodal learning with transformers: A survey
Xu, P., Zhu, X., & Clifton, D. A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3275156 · Cited by 1,072
HellaSwag: Can a Machine Really Finish Your Sentence?
Zellers, R., Holtzman, A., Bisk, Y., Farhadi, A., & Choi, Y. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1472 · Cited by 830
Multi-modal Transformer for Video Retrieval
Gabeur, V., Sun, C., Alahari, K., & Schmid, C. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58548-8_13 · Cited by 615
Metadata
- Authors
- Krishna, R., Hata, K., Ren, F., Fei-Fei, L., & Niebles, J. C.
- Venue
- 2017 IEEE International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2017-10
- Pages
- 706-715
- OpenAlex
- W2610163825
- Cited by
- 1,251
- References
- 63
- FWCI
- 20.22
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.