The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
COIN: A Large-Scale Dataset for Comprehensive Instructional Video Analysis
Abstract
There are substantial instruction videos on the Internet, which enables us to acquire knowledge for completing various tasks. However, most existing datasets for instruction video analysis have the limitations in diversity and scale, which makes them far from many real-world applications where more diverse activities occur. Moreover, it still remains a great challenge to organize and harness such data. To address these problems, we introduce a large-scale dataset called “COIN" for COmprehensive INstruction video analysis. Organized with a hierarchical structure, the COIN dataset contains 11,827 videos of 180 tasks in 12 domains (e.g., vehicles, gadgets, etc.) related to our daily life. With a new developed toolbox, all the videos are annotated effectively with a series of step descriptions and the corresponding temporal boundaries. Furthermore, we propose a simple yet effective method to capture the dependencies among different steps, which can be easily plugged into conventional proposal-based action detection methods for localizing important steps in instruction videos. In order to provide a benchmark for instruction video analysis, we evaluate plenty of approaches on the COIN dataset under different evaluation criteria. We expect the introduction of the COIN dataset will promote the future in-depth research on instruction video analysis for the community.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
WordNet: An Electronic Lexical Database
Kilgarriff, A., & Fellbaum, C. (2000) · Language · doi:10.2307/417141 · Cited by 11,464
Learning Spatiotemporal Features with 3D Convolutional Networks
Tran, D., Bourdev, L., Fergus, R., Torresani, L., & Paluri, M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.510 · Cited by 9,593
Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset
Carreira, J., & Zisserman, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.502 · Cited by 9,570
WordNet: An Electronic Lexical Database
(1998) · The MIT Press eBooks · doi:10.7551/mitpress/7287.001.0001 · Cited by 4,857
Ucf101: A dataset of 101 human actions classes from videos in the wild
Soomro, K., Zamir, A. R., & Shah, M. (2012) · arXiv · doi:10.48550/arxiv.1212.0402 · Cited by 4,575
Temporal Segment Networks: Towards Good Practices for Deep Action Recognition
Wang, L., Xiong, Y., Wang, Z., Qiao, Y., Lin, D., Tang, X., & Van Gool, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_2 · Cited by 3,958
Action Recognition with Improved Trajectories
Wang, H., & Schmid, C. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.441 · Cited by 3,407
ActivityNet: A large-scale video benchmark for human activity understanding
Heilbron, F. C., Escorcia, V., Ghanem, B., & Niebles, J. C. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298698 · Cited by 2,578
MSR-VTT: A Large Video Description Dataset for Bridging Video and Language
Xu, J., Mei, T., Yao, T., & Rui, Y. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.571 · Cited by 1,759
A Duality Based Approach for Realtime TV-L 1 Optical Flow
Zach, C., Pock, T., & Bischof, H. (2007) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-74936-3_22 · Cited by 1,658 · cited 1×
“Here SSN−Flow adopted the optical flows calculated by [48], and SSN−Fusion combined the predicted scores of SSN−RGB and SSN−Flow.”
MethodLong-Term Recurrent Convolutional Networks for Visual Recognition and Description
Donahue, J., Hendricks, L. A., Rohrbach, M., Venugopalan, S., Guadarrama, S., Saenko, K., & Darrell, T. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2599174 · Cited by 1,547
Dense-Captioning Events in Videos
Krishna, R., Hata, K., Ren, F., Fei-Fei, L., & Niebles, J. C. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.83 · Cited by 1,251
AVA: A Video Dataset of Spatio-Temporally Localized Atomic Visual Actions
Gu, C., Sun, C., Ross, D. A., Vondrick, C., Pantofaru, C., Li, Y., Vijayanarasimhan, S., Toderici, G., Ricco, S., Sukthankar, R., et al. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00633 · Cited by 986
Temporal Action Detection with Structured Segment Networks
Zhao, Y., Xiong, Y., Wang, L., Wu, Z., Tang, X., & Lin, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.317 · Cited by 892
R-C3D: Region Convolutional 3D Network for Temporal Activity Detection
Xu, H., Das, A., & Saenko, K. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.617 · Cited by 783
Creating Summaries from User Videos
Gygli, M., Grabner, H., Riemenschneider, H., & Van Gool, L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10584-0_33 · Cited by 752
Video Summarization with Long Short-Term Memory
Zhang, K., Chao, W.-L., Sha, F., & Grauman, K. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46478-7_47 · Cited by 739
VSUMM: A mechanism designed to produce static video summaries and a novel evaluation method
de Avila, S. E. F., Lopes, A. P. B., da Luz, A., & de Albuquerque Araújo, A. (2011) · Pattern Recognition Letters · doi:10.1016/j.patrec.2010.08.004 · Cited by 679
TVSum: Summarizing web videos using titles
Song, Y., Vallmitjana, J., Stent, A., & Jaimes, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299154 · Cited by 623
End-to-End Dense Video Captioning with Masked Transformer
Zhou, L., Zhou, Y., Corso, J. J., Socher, R., & Xiong, C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00911 · Cited by 605
The Language of Actions: Recovering the Syntax and Semantics of Goal-Directed Human Activities
Kuehne, H., Arslan, A., & Serre, T. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.105 · Cited by 571
Towards Automatic Learning of Procedures From Web Instructional Videos
Zhou, L., Xu, C., & Corso, J. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12342 · Cited by 526
A database for fine grained activity detection of cooking activities
Rohrbach, M., Amin, S., Andriluka, M., & Schiele, B. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247801 · Cited by 521
Scaling Egocentric Vision: The "Equation missing" Dataset
Damen, D., Doughty, H., Farinella, G. M., Fidler, S., Furnari, A., Kazakos, E., Moltisanti, D., Munro, J., Perrett, T., Price, W., et al. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01225-0_44 · Cited by 504
Combining embedded accelerometers with computer vision for recognizing food preparation activities
Stein, S., & McKenna, S. J. (2013) · Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing · doi:10.1145/2493432.2493482 · Cited by 448
Das, P., Xu, C., Doell, R. F., & Corso, J. J. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.340 · Cited by 282
Unsupervised Learning from Narrated Instruction Videos
Alayrac, J.-B., Bojanowski, P., Agrawal, N., Sivic, J., Laptev, I., & Lacoste-Julien, S. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.495 · Cited by 278
Scaling Egocentric Vision: The EPIC-KITCHENS Dataset
Damen, D., Doughty, H., Farinella, G. M., Fidler, S., Furnari, A., Kazakos, E., Moltisanti, D., Munro, J., Perrett, T., Price, W., et al. (2018) · arXiv · doi:10.48550/arxiv.1804.02748 · Cited by 274
Towards Automatic Learning of Procedures from Web Instructional Videos
Zhou, L., Xu, C., & Corso, J. J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.09788 · Cited by 193
Dissimilarity-Based Sparse Subset Selection
Elhamifar, E., Sapiro, G., & Sastry, S. S. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2511748 · Cited by 183
PKU-MMD: A Large Scale Benchmark for Continuous Multi-Modal Human Action UnderstandingPreprint
Liu, C., Hu, Y., Li, Y., Song, S., & Liu, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.07475 · Cited by 169
Weakly-Supervised Action Segmentation with Iterative Soft Boundary Assignment
Xu, C., & Ding, L. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00681 · Cited by 150
NeuralNetwork-Viterbi: A Framework for Weakly Supervised Video Learning
Richard, A., Kuehne, H., Iqbal, A., & Gall, J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00771 · Cited by 133
Weakly-supervised action segmentation with iterative soft boundary assignment
Ding, L., & Xu, C. (2018) · arXiv · doi:10.48550/arxiv.1803.10699 · Cited by 124 · cited 4×
“[17] proposed a temporal convolutional feature pyramid network to predict frame-wise labels and use soft 1The language signal should not be treated as supervision since the steps are not directly given, but need to be further explored in an unsupervised manner.”
BackgroundMethodInfluentialAction Sets: Weakly Supervised Action Segmentation Without Ordering Constraints
Richard, A., Kuehne, H., & Gall, J. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00627 · Cited by 92
Unsupervised Semantic Parsing of Video Collections
Şener, O., Zamir, A., Savarese, S., & Saxena, A. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.509 · Cited by 89
Finding "It": Weakly-Supervised Reference-Aware Visual Grounding in Instructional Videos
Huang, D.-A., Buch, S., Dery, L. M., Garg, A., Fei-Fei, L., & Niebles, J. C. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00623 · Cited by 89
Weakly-Supervised Video Object Grounding from Text by Loss Weighting and Object InteractionPreprint
Zhou, L., Louis, N., & Corso, J. J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.02834 · Cited by 87
Fine-Grained Video Captioning for Sports Narrative
Yu, H., Cheng, S., Ni, B., Wang, M., Zhang, J., & Yang, X. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00629 · Cited by 71
Optimizing the number of steps in learning tasks for complex skills
Nadolski, R. J., Kirschner, P. A., & Van Merriënboer, J. J. G. (2005) · British Journal of Educational Psychology · doi:10.1348/000709904x22403 · Cited by 70
Diversity-Aware Multi-Video Summarization
Panda, R., Mithun, N. C., & Roy-Chowdhury, A. K. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2708902 · Cited by 69
Unsupervised Visual-Linguistic Reference Resolution in Instructional Videos
Huang, D.-A., Lim, J. J., Fei-Fei, L., & Niebles, J. C. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.116 · Cited by 51
Human Pose Forecasting via Deep Markov Models
Toyer, S., Cherian, A., Han, T., & Gould, S. J. (2017) · International Conference on Digital Image Computing: Techniques and Applications (DICTA) · doi:10.1109/dicta.2017.8227441 · Cited by 47
Cited by
End-to-End Learning of Visual Representations From Uncurated Instructional Videos
Miech, A., Alayrac, J.-B., Smaira, L., Laptev, I., Sivic, J., & Zisserman, A. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00990 · Cited by 580
VideoMamba: State Space Model for Efficient Video Understanding
Li, K., Li, X., Wang, Y., He, Y., Wang, Y., Wang, L., & Qiao, Y. (2025) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-73347-5_14 · Cited by 312
Uncertainty-Aware Score Distribution Learning for Action Quality Assessment
Tang, Y., Ni, Z., Zhou, J., Zhang, D., Lu, J., Wu, Y., & Zhou, J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00986 · Cited by 209
Self-citationVision-Language Pre-Training: Basics, Recent Advances, and Future Trends
Gan, Z., Li, L., Li, C., Wang, L., Liu, Z., & Gao, J. (2022) · Foundations and Trends® in Computer Graphics and Vision · doi:10.1561/0600000105 · Cited by 155
Self-supervised learning for videos: A survey
Schiappa, M. C., Rawat, Y. S., & Shah, M. (2023) · ACM Computing Surveys · doi:10.1145/3577925 · Cited by 155
Metadata
- Authors
- Tang, Y., Ding, D., Rao, Y., Zheng, Y., Zhang, D., Zhao, L., Lu, J., & Zhou, J.
- Venue
- 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2019-06
- Pages
- 1207-1216
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2964094654
- Cited by
- 253
- References
- 46
- FWCI
- 7.67
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.