Finding Actors and Actions in Movies
Open accessThe DOI matches its Crossref record.
Abstract
We address the problem of learning a joint model of actors and actions in movies using weak supervision provided by scripts. Specifically, we extract actor/action pairs from the script and use them as constraints in a discriminative clustering framework. The corresponding optimization problem is formulated as a quadratic program under linear constraints. People in video are represented by automatically extracted and tracked faces together with corresponding motion features. First, we apply the proposed framework to the task of learning names of characters in the movie and demonstrate significant improvements over previous methods used for this task. Second, we explore the joint actor/action constraint and show its advantage for weakly supervised action learning. We validate our method in the challenging setting of localizing and recognizing characters and their actions in feature length movies Casablanca and American Beauty.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cited by
MovieQA: Understanding Stories in Movies through Question-Answering
Tapaswi, M., Zhu, Y., Stiefelhagen, R., Torralba, A., Urtasun, R., & Fidler, S. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.501 · Cited by 639
End-to-End Learning of Visual Representations From Uncurated Instructional Videos
Miech, A., Alayrac, J.-B., Smaira, L., Laptev, I., Sivic, J., & Zisserman, A. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00990 · Cited by 580
Self-citationA Review of Human Activity Recognition Methods
Vrigkas, M., Nikou, C., & Kakadiaris, I. A. (2015) · Frontiers in Robotics and AI · doi:10.3389/frobt.2015.00028 · Cited by 566
UntrimmedNets for Weakly Supervised Action Recognition and Detection
Wang, L., Xiong, Y., Lin, D., & Van Gool, L. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.678 · Cited by 546
A dataset for Movie Description
Rohrbach, A., Rohrbach, M., Tandon, N., & Schiele, B. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298940 · Cited by 435
W-TALC: Weakly-Supervised Temporal Activity Localization and Classification
Paul, S., Roy, S., & Roy-Chowdhury, A. K. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01225-0_35 · Cited by 340
Rohrbach, A., Torabi, A., Rohrbach, M., Tandon, N., Pal, C., Larochelle, H., Courville, A., & Schiele, B. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-016-0987-1 · Cited by 300
Large-Scale Weakly-Supervised Pre-Training for Video Action Recognition
Ghadiyaram, D., Tran, D., & Mahajan, D. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01232 · Cited by 281
Unsupervised Learning from Narrated Instruction Videos
Alayrac, J.-B., Bojanowski, P., Agrawal, N., Sivic, J., Laptev, I., & Lacoste-Julien, S. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.495 · Cited by 278
Self-citationConnectionist Temporal Modeling for Weakly Supervised Action Labeling
Huang, D.-A., Fei-Fei, L., & Niebles, J. C. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_9 · Cited by 226
SceneSkim: Searching and Browsing Movies Using Synchronized Captions, Scripts and Plot Summaries
Pavel, A., Goldman, D. B., Hartmann, B., & Agrawala, M. (2015) · Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology · doi:10.1145/2807442.2807502 · Cited by 67
“Prior work also aligns scripts to movies and TV shows in order to identify characters [14, 37] and actions [8, 23, 12] in the video.”
BackgroundAutoAD: Movie Description in Context
Han, T., Bain, M., Nagrani, A., Varol, G., Xie, W., & Zisserman, A. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52729.2023.01815 · Cited by 43
Speech2Action: Cross-Modal Supervision for Action Recognition
Nagrani, A., Sun, C., Ross, D., Sukthankar, R., Schmid, C., & Zisserman, A. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01033 · Cited by 12
Self-citation
Metadata
- Authors
- Bojanowski, P., Bach, F., Laptev, I., Ponce, J., Schmid, C., & Sivic, J.
- Venue
- 2013 IEEE International Conference on Computer Vision
- Publisher
- IEEE
- Published
- 2013-12
- Pages
- 2280-2287
- OpenAlex
- W2119031011
- Cited by
- 128
- References
- 0
- FWCI
- 8.53
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.