CoChat
Sign Up
Cited by 67Video Analysis and SummarizationInformation Retrieval and Search BehaviorMultimedia Communication and Technology

SceneSkim: Searching and Browsing Movies Using Synchronized Captions, Scripts and Plot Summaries

Open access
Pavel, Amy, Goldman, Dan B., Hartmann, Björn, Agrawala, Maneesh·2015·Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology
DOI 10.1145/2807442.2807502
Open this paper in CoChat — free accountOpen PDF

Abstract

Searching for scenes in movies is a time-consuming but crucial task for film studies scholars, film professionals, and new media artists. In pilot interviews we have found that such users search for a wide variety of clips---e.g., actions, props, dialogue phrases, character performances, locations---and they return to particular scenes they have seen in the past. Today, these users find relevant clips by watching the entire movie, scrubbing the video timeline, or navigating via DVD chapter menus. Increasingly, users can also index films through transcripts---however, dialogue often lacks visual context, character names, and high level event descriptions. We introduce SceneSkim, a tool for searching and browsing movies using synchronized captions, scripts and plot summaries. Our interface integrates information from such sources to allow expressive search at several levels of granularity: Captions provide access to accurate dialogue, scripts describe shot-by-shot actions and settings, and plot summaries contain high-level event descriptions. We propose new algorithms for finding word-level caption to script alignments, parsing text scripts, and aligning plot summaries to scripts. Film studies graduate students evaluating SceneSkim expressed enthusiasm about the usability of the proposed system for their research and teaching.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/2807442.2807502

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Existing tools leverage big data for screenwriting and feedback analysis.

    “Additionally, while existing tools leverage big data for screenwriting and feedback analysis [72, 77, 84]”

    From Understanding Screenwriters' Practices, Attitudes, and Future Expectations in Human-AI Co-Creation (2025) · cites this paper as [77]

    The Additional information section (page unavailable) describes a dataset of 816 movies, and the Conclusion (page unavailable) says SceneSkim uses aligned scripts, captions, and summaries to support film-studies analysis. The paper does not describe a tool for screenwriting or for analyzing feedback, so those attributed purposes are unsupported. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 37

  1. Barnes, C., Goldman, D. B., Shechtman, E., and Finkelstein, A. Video tapestries with continuous temporal zoom. ACM Transactions on Graphics (TOG) 29, 4 (2010), 89.
  2. Berthouzoz, F., Li, W., and Agrawala, M. Tools for placing cuts and transitions in interview video. ACM Transactions on Graphics (TOG) 31, 4 (2012), 67.
  3. Bird, S. NLTK: The natural language toolkit. In Proceedings of the COLING/ACL on Interactive presentation sessions, Association for Computational Linguistics (2006), 69-72.
  4. Bojanowski, P., Bach, F., Laptev, I., Ponce, J., Schmid, C., and Sivic, J. Finding actors and actions in movies. In Proc. IEEE International Conference on Computer Vision (2013).
  5. Brode, D., and Deyneka, L. Sex, Politics, and Religion in Star Wars: An Anthology. Scarecrow Press, 2012.
  6. Casares, J., Long, A. C., Myers, B. A., Bhatnagar, R., Stevens, S. M., Dabbish, L., Yocum, D., and Corbett, A. Simplifying video editing using metadata. In Proceedings of DIS, ACM (2002), 157-166.
  7. Cour, T., Jordan, C., Miltsakaki, E., and Taskar, B. Movie/script: Alignment and parsing of video and text transcription. In Computer Vision-ECCV 2008. Springer, 2008, 158-171.
  8. Everingham, M., Sivic, J., and Zisserman, A. Hello! My name is... Buffy -automatic naming of characters in TV video. In British Machine Vision Conference (2006).
  9. Fouse, A., Weibel, N., Hutchins, E., and Hollan, J. D. Chronoviz: A system for supporting navigation of time-coded data. In CHI'11 Extended Abstracts on Human Factors in Computing Systems, ACM (2011), 299-304.
  10. Goldman, D. B., Curless, B., Seitz, S. M., and Salesin, D. Schematic storyboarding for video visualization and editing. ACM Transactions on Graphics (Proc. SIGGRAPH) 25, 3 (2006).
  11. Grossman, T., Matejka, J., and Fitzmaurice, G. Chronicle: Capture, exploration, and playback of document workflow histories. In Proceedings of UIST, ACM (2010), 143-152.
  12. Haubold, A., and Kender, J. R. VAST MM: Multimedia browser for presentation video. In Proceedings of CIVR, ACM (2007), 41-48.
  13. Jackson, D., Nicholson, J., Stoeckigt, G., Wrobel, R., Thieme, A., and Olivier, P. Panopticon: A parallel video overview system. In Proceedings of UIST, ACM (2013), 123-130.
  14. Kim, J., Guo, P. J., Cai, C. J., Li, S.-W. D., Gajos, K. Z., and Miller, R. C. Data-driven interaction techniques for improving navigation of educational videos. In Proceedings of UIST, ACM (2014), 563-572.
  15. Kim, J., Nguyen, P. T., Weir, S., Guo, P. J., Miller, R. C., and Gajos, K. Z. Crowdsourcing step-by-step information extraction to enhance existing how-to videos. In Proceedings of CHI, ACM (2014), 4017-4026.
  16. Laptev, I., Marszalek, M., Schmid, C., and Rozenfeld, B. Learning realistic human actions from movies. In Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on, IEEE (2008), 1-8.
  17. Mackay, W. E., and Beaudouin-Lafon, M. Diva: Exploratory data analysis with multimedia streams. In Proceedings of the SIGCHI conference on human factors in computing systems, ACM Press/Addison-Wesley Publishing Co. (1998), 416-423.
  18. Matejka, J., Grossman, T., and Fitzmaurice, G. Ambient help. In Proceedings of CHI, ACM (2011), 2751-2760.
  19. Matejka, J., Grossman, T., and Fitzmaurice, G. Swift: Reducing the effects of latency in online video scrubbing. In Proceedings of CHI, ACM (2012), 637-646.
  20. Matejka, J., Grossman, T., and Fitzmaurice, G. Swifter: Improved online video scrubbing. In Proceedings of CHI, ACM (2013), 1159-1168.
  21. Matejka, J., Grossman, T., and Fitzmaurice, G. Video lens: Rapid playback and exploration of large video collections and associated metadata. In Proceedings of UIST, ACM (2014), 541-550.
  22. Mohamad Ali, N., Smeaton, A. F., and Lee, H. Designing an interface for a digital movie browsing system in the film studies domain. International Journal of Digital Content Technology and Its Applications 5, 9 (2011), 361-370.
  23. Monserrat, T.-J. K. P., Zhao, S., McGee, K., and Pandey, A. V. NoteVideo: Facilitating navigation of blackboard-style lecture videos. In Proceedings of CHI, ACM (2013), 1139-1148.
  24. Needleman, S. B., and Wunsch, C. D. A general method applicable to the search for similarities in the amino acid sequence of two proteins. Journal of molecular biology 48, 3 (1970), 443-453.
  25. Noah, D. B. B. O., and Smith, A. Learning latent personas of film characters. ACL (2013).
  26. Olsen, D. R., Partridge, B., and Lynn, S. Time warp sports for internet television. ACM Transactions on Computer-Human Interaction (TOCHI) 17, 4 (2010), 16.
  27. Pavel, A., Reed, C., Hartmann, B., and Agrawala, M. Video digests: A browsable, skimmable format for informational lecture videos. In Proceedings of UIST, ACM (2014), 573-582.
  28. Pongnumkul, S., Dontcheva, M., Li, W., Wang, J., Bourdev, L., Avidan, S., and Cohen, M. F. Pause-and-play: Automatically linking screencast video tutorials with applications. In Proceedings of UIST, ACM (2011), 135-144.
  29. Ramanathan, V., Joulin, A., Liang, P., and Fei-Fei, L. Linking people in videos with "their" names using coreference resolution. In Computer Vision-ECCV 2014. Springer, 2014, 95-110.
  30. Ronfard, R. Reading movies: An integrated DVD player for browsing movies and their scripts. In Proceedings of the 12th annual ACM international conference on Multimedia, ACM (2004), 740-741.
  31. Ronfard, R., and Thuong, T. T. A framework for aligning and indexing movies with their script. In Multimedia and Expo, 2003. ICME'03. Proceedings. 2003 International Conference on, vol. 1, IEEE (2003), I-21.
  32. Rubin, S., Berthouzoz, F., Mysore, G. J., Li, W., and Agrawala, M. Content-based tools for editing audio stories. In Proceedings of UIST, ACM (2013), 113-122.
  33. Sparck Jones, K. A statistical interpretation of term specificity and its application in retrieval. Journal of documentation 28, 1 (1972), 11-21.
  34. Tapaswi, M., Bauml, M., and Stiefelhagen, R. Aligning plot synopses ¨ to videos for story-based retrieval. International Journal of Multimedia Information Retrieval 4, 1 (2015), 3-16.
  35. Tapaswi, M., Bauml, M., and Stiefelhagen, R. Book2mo ¨vie: Aligning video scenes with book chapters. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2015), 1827-1835.
  36. Tiedemann, J. Parallel data, tools and interfaces in OPUS. In Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC'12), European Language Resources Association (ELRA) (2012).
  37. Yuan, J., and Liberman, M. Speaker identification on the SCOTUS corpus. Journal of the Acoustical Society of America 123, 5 (2008), 3878.

Cited by

Sorted by citations

Authors
Pavel, A., Goldman, D. B., Hartmann, B., & Agrawala, M.
Affiliations
  • University of California, Berkeley (US)
  • Adobe, Seattle, WA, USA
  • Stanford University (US)
ORCID
Venue
Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology
Publisher
ACM
Published
2015-11-05
Pages
181-190
DOI
10.1145/2807442.2807502
OpenAlex
W1834627372
Cited by
67
References
2
Citations by year
2026: 5 · 2025: 7 · 2024: 11 · 2023: 9 · 2022: 3 · 2021: 9 · 2020: 5 · 2019: 6 · 2018: 5 · 2017: 4 · 2016: 3
FWCI
1.64
Type
conference paper
Language
en
Open access
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations
ACM Symposium on User Interface Software and Technology (UIST) (publishedVersion)
Funders
  • Office of Naval Research (NDSEG)
  • National Science Foundation (IIS grant 1210836)
  • CITRIS Connected Communities Initiative
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Analysis and Summarization
  • Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
  • Social Sciences › Sociology and Political Science › Multimedia Communication and Technology
Keywords
video browsing, screenplays, plot summary, text parsing, film studies, user interface design

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review