The DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Learning and Using the Arrow of Time
Open accessAbstract
We seek to understand the arrow of time in videos - what makes videos look like they are playing forwards or backwards? Can we visualize the cues? Can the arrow of time be a supervisory signal useful for activity analysis? To this end, we build three large-scale video datasets and apply a learning-based approach to these tasks. To learn the arrow of time efficiently and reliably, we design a ConvNet suitable for extended temporal footprints and for class activation visualization, and study the effect of artificial cues, such as cinematographic conventions, on learning. Our trained model achieves state-of-the-art performance on large-scale real-world video datasets. Through cluster analysis and localization of important regions for the prediction, we examine learned visual cues that are consistent among many samples and show when and where they occur. Lastly, we use the trained ConvNet for two applications: self-supervision for action recognition, and video forensics - determining whether Hollywood film clips have been deliberately reversed in time, often used as special effects.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Self-citationGoing deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Learning Deep Features for Discriminative Localization
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.319 · Cited by 11,428
Two-Stream Convolutional Networks for Action Recognition in Videos
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1406.2199 · Cited by 6,756
Self-citationUcf101: A dataset of 101 human actions classes from videos in the wild
Soomro, K., Zamir, A. R., & Shah, M. (2012) · arXiv · doi:10.48550/arxiv.1212.0402 · Cited by 4,575
Temporal Segment Networks: Towards Good Practices for Deep Action Recognition
Wang, L., Xiong, Y., Wang, Z., Qiao, Y., Lin, D., Tang, X., & Van Gool, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_2 · Cited by 3,958
Striving for Simplicity: The All Convolutional Net
Springenberg, J. T., Dosovitskiy, A., Brox, T., & Riedmiller, M. (2014) · arXiv · doi:10.48550/arxiv.1412.6806 · Cited by 3,447
Convolutional Two-Stream Network Fusion for Video Action Recognition
Feichtenhofer, C., Pinz, A., & Zisserman, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.213 · Cited by 2,684
Self-citationUnsupervised Visual Representation Learning by Context Prediction
Doersch, C., Gupta, A., & Efros, A. A. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.167 · Cited by 2,629
Research works printed but not linked: 2
- dekel (basha), 2014, Photo sequencing, IJCV
- basha, 2012, Photo sequencing, ECCV
Cited by
Transformers in Vision: A Survey
Khan, S., Naseer, M., Hayat, M., Zamir, S. W., Khan, F. S., & Shah, M. (2022) · ACM Computing Surveys · doi:10.1145/3505244 · Cited by 3,595
Transformers in Vision: A Survey
Khan, S., Naseer, M., Hayat, M., Zamir, S. W., Khan, F. S., & Shah, M. (2021) · arXiv · doi:10.48550/arxiv.2101.01169 · Cited by 3,594
Self-Supervised Visual Feature Learning With Deep Neural Networks: A Survey
Jing, L., & Tian, Y. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.2992393 · Cited by 2,036
Self-Supervised Learning of Pretext-Invariant Representations
Misra, I., & van der Maaten, L. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00674 · Cited by 1,435
Audio-Visual Scene Analysis with Self-Supervised Multisensory Features
Owens, A., & Efros, A. A. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01231-1_39 · Cited by 704
A Survey on Self-Supervised Learning: Algorithms, Applications, and Future Trends
Gui, J., Chen, T., Zhang, J., Cao, Q., Sun, Z., Luo, H., & Tao, D. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3415112 · Cited by 552
Soviany, P., Ionescu, R. T., Rota, P., & Sebe, N. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01611-x · Cited by 431
Video Representation Learning by Dense Predictive Coding
Han, T., Xie, W., & Zisserman, A. (2019) · 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW) · doi:10.1109/iccvw.2019.00186 · Cited by 377
Self-citationSelf-Supervised Learning in Remote Sensing: A review
Wang, Y., Albrecht, C. M., Braham, N. A. A., Mou, L., & Zhu, X. X. (2022) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2022.3198244 · Cited by 364
Anomaly Detection in Video via Self-Supervised and Multi-Task Learning
Georgescu, M.-I., Barbalau, A., Ionescu, R. T., Shahbaz Khan, F., Popescu, M., & Shah, M. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01255 · Cited by 350
Multi-task Self-Supervised Learning for Human Activity Detection
Saeed, A., Ozcelebi, T., & Lukkien, J. (2019) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3328932 · Cited by 346
Self-Supervised Video Representation Learning with Space-Time Cubic Puzzles
Kim, D., Cho, D., & Kweon, I. S. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33018545 · Cited by 319
Qi, G.-J., & Luo, J. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3031898 · Cited by 315
A Review on Deep Learning Techniques for Video Prediction
Oprea, S., Martinez-Gonzalez, P., Garcia-Garcia, A., Castro-Vargas, J. A., Orts-Escolano, S., Garcia-Rodriguez, J., & Argyros, A. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3045007 · Cited by 301
Self-Supervised Learning by Cross-Modal Audio-Video Clustering
Alwassel, H., Mahajan, D., Korbar, B., Torresani, L., Ghanem, B., & Tran, D. (2019) · arXiv · doi:10.48550/arxiv.1911.12667 · Cited by 298
Pose-Guided Visible Part Matching for Occluded Person ReID
Gao, S., Wang, J., Lu, H., & Liu, Z. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01176 · Cited by 296
Self-supervised Co-training for Video Representation Learning
Han, T., Xie, W., & Zisserman, A. (2020) · arXiv · doi:10.48550/arxiv.2010.09709 · Cited by 292
Self-citationHard Negative Mixing for Contrastive Learning
Kalantidis, Y., Sarıyıldız, M. B., Pion, N., Weinzaepfel, P., & Larlus, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.01028 · Cited by 257
Self-Supervised Contrastive Representation Learning for Semi-Supervised Time-Series Classification
Eldele, E., Ragab, M., Chen, Z., Wu, M., Kwoh, C.-K., Li, X., & Guan, C. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3308189 · Cited by 224
Self-supervised learning for human activity recognition using 700,000 person-days of wearable data
Yuan, H., Chan, S., Creagh, A. P., Tong, C., Acquah, A., Clifton, D. A., & Doherty, A. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01062-3 · Cited by 166
“…(SSL) methods to leverage unlabelled data in a similar manner as language models (Mikolov et al., 2013; Devlin et al., 2018; Radford et al., 2018, 2019) and vision models (Doersch et al., 2015; Zhang et al., 2016; Noroozi and Favaro, 2016; Wei et al., 2018; He et al., 2020; Chen et al., 2020).”
Self-supervised learning for videos: A survey
Schiappa, M. C., Rawat, Y. S., & Shah, M. (2023) · ACM Computing Surveys · doi:10.1145/3577925 · Cited by 155
Contrastive Predictive Coding for Human Activity Recognition
Haresamudram, H., Essa, I., & Plötz, T. (2021) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3463506 · Cited by 150
“Identifying the odd or unrelated video sub-sequence [22], sequence verification of temporal order [53], and learning the arrow of time [78] have been used as pretext tasks.”
MethodSubject-Aware Contrastive Learning for Biosignals
Cheng, J. Y., Goh, H., Dogrusoz, K., Tuzel, O., & Azemi, E. (2020) · arXiv · doi:10.48550/arxiv.2007.04871 · Cited by 74
Vi2CLR: Video and Image for Visual Contrastive Learning of Representation
Diba, A., Sharma, V., Safdari, R., Lotfi, D., Sarfraz, M. S., Stiefelhagen, R., & Van Gool, L. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00153 · Cited by 53
Breaking away from labels: The promise of self-supervised machine learning in intelligent health
Spathis, D., Perez-Pozuelo, I., Marques-Fernandez, L., & Mascolo, C. (2022) · Patterns · doi:10.1016/j.patter.2021.100410 · Cited by 39
Unsupervised Video Representation Learning by Bidirectional Feature Prediction
Behrmann, N., Gall, J., & Noroozi, M. (2021) · 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv48630.2021.00171 · Cited by 28
LT-GAN: Self-Supervised GAN with Latent Transformation Detection
Patel, P., Kumari, N., Singh, M., & Krishnamurthy, B. (2021) · 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv48630.2021.00323 · Cited by 18
Self-Supervised Video GANs: Learning for Appearance Consistency and Motion Coherency
Hyun, S., Kim, J., & Heo, J.-P. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01068 · Cited by 16
Towards robust and label-efficient time series representation learning
Emadeldeen, A. I. A. E. (2023) · doi:10.32657/10356/170673 · Cited by 1
Metadata
- Authors
- Wei, D., Lim, J., Zisserman, A., & Freeman, W. T.
- Venue
- 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
- Publisher
- IEEE
- Published
- 2018-06
- Pages
- 8052-8060
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2799087757
- Cited by
- 315
- References
- 29
- FWCI
- 13.85
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.