The DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Grounding Action Descriptions in Videos
Open accessAbstract
Recent work has shown that the integration of visual information into text-based models can substantially improve model predictions, but so far only visual information extracted from static images has been used. In this paper, we consider the problem of grounding sentences describing actions in visual information extracted from videos. We present a general purpose corpus that aligns high quality videos with multiple natural language descriptions of the actions portrayed in the videos, together with an annotation of how similar the action descriptions are to each other. Experimental results demonstrate that a text-based model of similarity between actions improves substantially when combined with visual information from videos depicting the described actions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
From Frequency to Meaning: Vector Space Models of Semantics
Turney, P. D., & Pantel, P. (2010) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2934 · Cited by 2,866
Glenberg, A. M., & Kaschak, M. P. (2002) · Psychonomic Bulletin & Review · doi:10.3758/bf03196313 · Cited by 2,361
Labeling images with a computer game
von Ahn, L., & Dabbish, L. (2004) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/985692.985733 · Cited by 2,204
Action recognition by dense trajectories
Wang, H., Klaser, A., Schmid, C., & Liu, C.-L. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995407 · Cited by 2,154
The PASCAL Recognising Textual Entailment Challenge
Dagan, I., Glickman, O., & Magnini, B. (2006) · Lecture Notes in Computer Science · doi:10.1007/11736790_9 · Cited by 1,628
Collecting Highly Parallel Data for Paraphrase Evaluation
Chen, D. J., & Dolan, W. B. (2011) · Cited by 904
Vector-based Models of Semantic Composition
Mitchell, J., & Lapata, M. (2008) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 675
A database for fine grained activity detection of cooking activities
Rohrbach, M., Amin, S., Andriluka, M., & Schiele, B. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247801 · Cited by 521
Self-citationVondrick, C., Patterson, D., & Ramanan, D. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-012-0564-1 · Cited by 448
Gupta, A., Srinivasan, P., Shi, J., & Davis, L. S. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206492 · Cited by 231
Script Data for Attribute-Based Recognition of Composite Activities
Rohrbach, M., Regneri, M., Andriluka, M., Amin, S., Pinkal, M., & Schiele, B. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33718-5_11 · Cited by 185
Self-citationMovie/Script: Alignment and Parsing of Video and Text Transcription
Cour, T., Jordan, C., Miltsakaki, E., & Taskar, B. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-88693-8_12 · Cited by 124
Visual Information in Semantic Representation
Feng, Y., & Lapata, M. (2010) · Edinburgh Research Explorer (University of Edinburgh) · Cited by 99
Howell, S. R., Jankowicz, D., & Becker, S. (2005) · Journal of Memory and Language · doi:10.1016/j.jml.2005.03.002 · Cited by 77
Measuring Word Meaning in Context
Erk, K., McCarthy, D., & Gaylord, N. (2012) · Computational Linguistics · doi:10.1162/coli_a_00142 · Cited by 76
Investigations on word senses and word usages
Erk, K., McCarthy, D., & Gaylord, N. (2009) · doi:10.3115/1687878.1687882 · Cited by 76
Grounded Models of Semantic Representation
Silberer, C., & Lapata, M. (2012) · RECERCAT (Consorci de Serveis Universitaris de Catalunya) · Cited by 76
Combining feature norms and text data with topic models
Mark Steyvers, B. (2009) · Acta Psychologica · doi:10.1016/j.actpsy.2009.10.010 · Cited by 73
Automatic learning and generation of social behavior from collective human gameplay
Orkin, J., & Roy, D. K. (2009) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/lltq1667 · Cited by 62
Distributional semantics from text and images
Bruni, E., Tran, G. B., & Baroni, M. (2011) · Institutional Research Information System (Università degli Studi di Trento) · Cited by 54
Erk, K., & McCarthy, D. (2009) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/1699510.1699568 · Cited by 49
Improving Video Activity Recognition using Object Recognition and Text Mining
Motwani, T. S., & Mooney, R. J. (2012) · Frontiers in artificial intelligence and applications · doi:10.3233/978-1-61499-098-7-600 · Cited by 49
Dodge, J., Goyal, A., Han, X., Mensch, A., Mitchell, M., Stratos, K., Yamaguchi, K., Choi, Y., Daumé, H., Berg, A., et al. (2013) · Cited by 42
Using Closed Captions as Supervision for Video Activity Recognition
Gupta, S., & Mooney, R. J. (2010) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v24i1.7738 · Cited by 24
Learning the abstract motion semantics of verbs from captioned videos
Mathe, S., Fazly, A., Dickinson, S., & Stevenson, S. (2008) · 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2008.4563042 · Cited by 7
Cited by
Multimodal Machine Learning: A Survey and Taxonomy
Baltrusaitis, T., Ahuja, C., & Morency, L.-P. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2798607 · Cited by 5,018
MSR-VTT: A Large Video Description Dataset for Bridging Video and Language
Xu, J., Mei, T., Yao, T., & Rui, Y. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.571 · Cited by 1,759
The “Something Something” Video Database for Learning and Evaluating Visual Common Sense
Goyal, R., Kahou, S. E., Michalski, V., Materzynska, J., Westphal, S., Kim, H., Haenel, V., Fruend, I., Yianilos, P., Mueller-Freitag, M., et al. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.622 · Cited by 1,471
Dense-Captioning Events in Videos
Krishna, R., Hata, K., Ren, F., Fei-Fei, L., & Niebles, J. C. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.83 · Cited by 1,251
Visual Relationship Detection with Language Priors
Lu, C., Krishna, R., Bernstein, M., & Fei-Fei, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_51 · Cited by 1,069
Metadata
- Authors
- Regneri, M., Rohrbach, M., Wetzel, D., Thater, S., Schiele, B., & Pinkal, M.
- Venue
- Transactions of the Association for Computational Linguistics
- Publisher
- MIT Press - Journals
- Published
- 2013-12
- Volume
- 1
- Pages
- 25-36
- ISSN
- 2307-387X (ISSN-L)
- OpenAlex
- W2111078031
- Cited by
- 467
- References
- 25
- FWCI
- 19.70
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.