The DOI matches its Crossref record; 74 of the 74 references it cites were found in a registry or an index.
Semantic Pooling for Complex Event Analysis in Untrimmed Videos
Open accessAbstract
Pooling plays an important role in generating a discriminative video representation. In this paper, we propose a new semantic pooling approach for challenging event analysis tasks (e.g., event detection, recognition, and recounting) in long untrimmed Internet videos, especially when only a few shots/segments are relevant to the event of interest while many other shots are irrelevant or even misleading. The commonly adopted pooling strategies aggregate the shots indifferently in one way or another, resulting in a great loss of information. Instead, in this work we first define a novel notion of semantic saliency that assesses the relevance of each shot with the event of interest. We then prioritize the shots according to their saliency scores since shots that are semantically more salient are expected to contribute more to the final event analysis. Next, we propose a new isotonic regularizer that is able to exploit the constructed semantic ordering information. The resulting nearly-isotonic support vector machine classifier exhibits higher discriminative power in event analysis tasks. Computationally, we develop an efficient implementation using the proximal gradient algorithm, and we prove new and closed-form proximal steps. We conduct extensive experiments on three real-world video datasets and achieve promising improvements.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533
Nonlinear total variation based noise removal algorithms
Rudin, L. I., Osher, S., & Fatemi, E. (1992) · Physica D: Nonlinear Phenomena · doi:10.1016/0167-2789(92)90242-f · Cited by 15,704
Learning the parts of objects by non-negative matrix factorization
Lee, D. D., & Seung, H. S. (1999) · Nature · doi:10.1038/44565 · Cited by 13,868
Learning Spatiotemporal Features with 3D Convolutional Networks
Tran, D., Bourdev, L., Fergus, R., Torresani, L., & Paluri, M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.510 · Cited by 9,593
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248
Two-Stream Convolutional Networks for Action Recognition in Videos
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1406.2199 · Cited by 6,756
Large-Scale Video Classification with Convolutional Neural Networks
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.223 · Cited by 6,354
Ucf101: A dataset of 101 human actions classes from videos in the wild
Soomro, K., Zamir, A. R., & Shah, M. (2012) · arXiv · doi:10.48550/arxiv.1212.0402 · Cited by 4,575
Shifts in Selective Visual Attention: Towards the Underlying Neural Circuitry
Koch, C., & Ullman, S. (1987) · Matters of Intelligence · doi:10.1007/978-94-009-3833-5_5 · Cited by 3,722
Action Recognition with Improved Trajectories
Wang, H., & Schmid, C. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.441 · Cited by 3,407
NLTK: the Natural Language Toolkit
Loper, E., & Bird, S. (2002) · Proceedings of the ACL-02 Workshop on Effective tools and methodologies for teaching natural language processing and computational linguistics - · doi:10.3115/1118108.1118117 · Cited by 3,218
Vlfeat: an open and portable library of computer vision algorithms
Vedaldi, A., & Fulkerson, B. (2010) · Proceedings of the 18th ACM international conference on Multimedia · doi:10.1145/1873951.1874249 · Cited by 2,664
Improving the Fisher Kernel for Large-Scale Image Classification
Perronnin, F., Sánchez, J., & Mensink, T. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15561-1_11 · Cited by 2,588
Training linear SVMs in linear time
Joachims, T. (2006) · Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1150402.1150429 · Cited by 1,920
On the algorithmic implementation of multiclass kernel-based vector machines
Crammer, K., & Singer, Y. (2002) · Cited by 1,887
Learning to detect unseen object classes by between-class attribute transfer
Lampert, C. H., Nickisch, H., & Harmeling, S. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206594 · Cited by 1,849
Proximal alternating linearized minimization for nonconvex and nonsmooth problems
Bolte, J., Sabach, S., & Teboulle, M. (2014) · Mathematical Programming · doi:10.1007/s10107-013-0701-9 · Cited by 1,688
YFCC100M: the new data in multimedia research
Thomee, B., Shamma, D. A., Friedland, G., Elizalde, B., Ni, K., Poland, D., Borth, D., & Li, L.-J. (2016) · Communications of the ACM · doi:10.1145/2812802 · Cited by 1,541
Image Classification with the Fisher Vector: Theory and Practice
Sánchez, J., Perronnin, F., Mensink, T., & Verbeek, J. (2013) · International Journal of Computer Vision · doi:10.1007/s11263-013-0636-x · Cited by 1,498
Large-scale Video Classification with Convolutional Neural Networks
Karpathy, A., Toderici, G. D., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014) · Cited by 1,266
On the algorithmic implementation of multiclass kernel-based vector machines
CrammerKoby, & SingerYoram (2002) · Journal of Machine Learning Research · doi:10.5555/944790.944813 · Cited by 874
Statistical Inference under Order Restrictions (The Theory and Application of Isotonic Regression)
Vincze, I., Barlow, R. E., Bartholomew, D. J., Bremner, J. M., & Brunk, H. D. (1973) · International Statistical Review / Revue Internationale de Statistique · doi:10.2307/1402630 · Cited by 842
NLTK: The Natural Language Toolkit
Bird, S. (2006) · Proceedings of the COLING/ACL on Interactive presentation sessions - · doi:10.3115/1225403.1225421 · Cited by 841
What, where and who? Classifying events by scene and object recognition
Li, L.-J., & Fei-Fei, L. (2007) · 2007 IEEE 11th International Conference on Computer Vision · doi:10.1109/iccv.2007.4408872 · Cited by 781
Discovering important people and objects for egocentric video summarization
Lee, Y. J., Ghosh, J., & Grauman, K. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247820 · Cited by 658
Weighted Sums of Random Kitchen Sinks: Replacing minimization with randomization in learning
Rahimi, A., & Recht, B. (2008) · Cited by 558
Segmenting Salient Objects from Images and Videos
Rahtu, E., Kannala, J., Salo, M., & Heikkilä, J. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15555-0_27 · Cited by 520
Modeling Spatial-Temporal Clues in a Hybrid Deep Learning Framework for Video Classification
Wu, Z., Wang, X., Jiang, Y.-G., Ye, H., & Xue, X. (2015) · Proceedings of the 23rd ACM international conference on Multimedia · doi:10.1145/2733373.2806222 · Cited by 479
A discriminative CNN video representation for event detection
Xu, Z., Yang, Y., & Hauptmann, A. G. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298789 · Cited by 435
Self-citationParallel support vector machines: The cascade svm
Graf, H. P., Cosatto, E., Bottou, L., Dourdanovic, I., & Vapnik, V. N. (2005) · Cited by 415
Visual Event Recognition in Videos by Learning from Web Data
Duan, L., Xu, D., Tsang, I. W.-H., & Luo, J. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.265 · Cited by 409
Action and Event Recognition with Fisher Vectors on a Compact Feature Set
Oneata, D., Verbeek, J., & Schmid, C. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.228 · Cited by 376
Learning latent temporal structure for complex event detection
Tang, K., Fei-Fei, L., & Koller, D. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247808 · Cited by 355
Jiang, Y.-G., Ye, G., Chang, S.-F., Ellis, D., & Loui, A. C. (2011) · Proceedings of the 1st ACM International Conference on Multimedia Retrieval · doi:10.1145/1991996.1992025 · Cited by 324
Action Recognition using Visual Attention
Sharma, S., Kiros, R., & Salakhutdinov, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.04119 · Cited by 298
Local Extremes, Runs, Strings and Multiresolution
Davies, P. L., & Kovac, A. (2001) · The Annals of Statistics · doi:10.1214/aos/996986501 · Cited by 265
Beyond Gaussian Pyramid: Multi-skip Feature Stacking for action recognition
Lan, Z., Lin, M., Li, X., Hauptmann, A. G., & Raj, B. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298616 · Cited by 264
Gupta, A., Srinivasan, P., Shi, J., & Davis, L. S. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206492 · Cited by 231
Exploiting Image-trained CNN Architectures for Unconstrained Video Classification
Zha, S., Luisier, F., Andrews, W., Srivastava, N., & Salakhutdinov, R. (2015) · Procedings of the British Machine Vision Conference 2015 · doi:10.5244/c.29.60 · Cited by 210
Multimodal feature fusion for robust event detection in web videos
Natarajan, P., Wu, S., Vitaladevuni, S., Zhuang, X., Tsakalidis, S., Park, U., Prasad, R., & Natarajan, P. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247814 · Cited by 178
A generalized proximal point algorithm for certain non-convex minimization problems
FUKUSHIMA, M., & MINE, H. (1981) · International Journal of Systems Science · doi:10.1080/00207728108963798 · Cited by 166
Image Labeling on a Network: Using Social-Network Metadata for Image Classification
McAuley, J., & Leskovec, J. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33765-9_59 · Cited by 162
Local velocity-adapted motion events for spatio-temporal recognition
Laptev, I., Barbara Caputo, B., Schüldt, C., & Lindeberg, T. (2007) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2006.11.023 · Cited by 155
Feature grouping and selection over an undirected graph
Yang, S., Yuan, L., Lai, Y., Shen, X., Wonka, P., & Ye, J. (2012) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2339530.2339675 · Cited by 132
Tamrakar, A., Ali, S., Yu, Q., Liu, J., Javed, O., Divakaran, A., Cheng, H., & Sawhney, H. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6248114 · Cited by 115
(2011) · doi:10.1198/tech.2010.10111 · Cited by 103
Recognizing Complex Events Using Large Margin Joint Low-Level Event Model
Izadinia, H., & Shah, M. A. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33765-9_31 · Cited by 94
Video event recognition using concept attributes
Liu, J., Yu, Q., Javed, O., Ali, S., Tamrakar, A., Divakaran, A., Cheng, H., & Sawhney, H. (2013) · Proceedings · doi:10.1109/wacv.2013.6475038 · Cited by 85
Combining the Right Features for Complex Event Recognition
Tang, K., Yao, B., Fei-Fei, L., & Koller, D. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.335 · Cited by 84
Knowledge Adaptation with PartiallyShared Features for Event DetectionUsing Few Exemplars
Ma, Z., Yang, Y., Sebe, N., & Hauptmann, A. G. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2306419 · Cited by 77
Self-citationVideo2vec Embeddings Recognize Events When Examples Are Scarce
Habibian, A., Mensink, T., & Snoek, C. G. M. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2627563 · Cited by 77
YFCC100M: The New Data in Multimedia Research
Thomee, B., Shamma, D. A., Friedland, G., Elizalde, B., Ni, K., Poland, D., Borth, D., & Li, L.-J. (2015) · arXiv · doi:10.48550/arxiv.1503.01817 · Cited by 69
Sample-Specific Late Fusion for Visual Category Recognition
Liu, D., Lai, K.-T., Ye, G., et al. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.109 · Cited by 68
Multimedia Event Detection Using A Classifier-Specific Intermediate Representation
Ma, Z., Yang, Y., Sebe, N., Zheng, K., & Hauptmann, A. G. (2013) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2013.2264928 · Cited by 67
Self-citationComplex Event Detection using Semantic Saliency and Nearly-Isotonic SVM
Chang, X., Yang, Y., Xing, E. P., & Yu, Y. (2015) · UTS ePRESS (University of Technology Sydney) · Cited by 66
Self-citationTowards textually describing complex video contents with audio-visual concept classifiers
Tan, C. C., Jiang, Y., & Ngo, C. (2011) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2072298.2072411 · Cited by 55
Mettes, P., van Gemert, J. C., Cappallo, S., Mensink, T., & Snoek, C. G. M. (2015) · International Conference on Multimedia Retrieval · doi:10.1145/2671188.2749404 · Cited by 53
Event Recognition in Photo Collections with a Stopwatch HMM
Bossard, L., Guillaumin, M., & Gool, L. V. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.151 · Cited by 53
Compositional Models for Video Event Detection: A Multiple Kernel Learning Latent Variable Approach
Vahdat, A., Cannons, K. J., Mori, G., Oh, S., & Kim, I. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.463 · Cited by 49
Scene Aligned Pooling for Complex Video Recognition
Cao, L., Mu, Y., Natsev, A., Chang, S., Hua, G., & Smith, J. R. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33709-3_49 · Cited by 48
Dynamic Pooling for Complex Event Recognition
Li, W., Yu, Q., Divakaran, A., & Vasconcelos, N. M. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.339 · Cited by 48
Searching Persuasively: Joint Event Detection and Evidence Recounting with Limited Supervision
Chang, X., Yu, Y., Yang, Y., & Hauptmann, A. G. (2015) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2733373.2806218 · Cited by 45
Self-citationDISCOVER: Discovering Important Segments for Classification of Video Events and Recounting
Sun, C., & Nevatia, R. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.329 · Cited by 42
Recognizing Complex Events in Videos by Learning Key Static-Dynamic Evidences
Lai, K.-T., Liu, D., Chen⋆, M.-S., & Chang, S. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10578-9_44 · Cited by 39
Enhancing Video Event Recognition Using Automatically Constructed Semantic-Visual Knowledge Base
Zhang, X., Yang, Y., Zhang, Y., Luan, H., Li, J., Zhang, H., & Chua, T. (2015) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2015.2449660 · Cited by 38
Self-citationTensor-Based Transductive Learning for Multimodality Video Semantic Concept Detection
Wu, F. Q., Liu, Y., & Zhuang, Y. (2009) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2009.2021724 · Cited by 32
Informedia@TRECVID 2014 MED and MER
Yu, S.-I., Jiang, L., Xu, Z., Lan, Z., Xu, S., Chang, X., & Martin, L. J. (2014) · Maryland Shared Open Access Repository (USMAI Consortium) · doi:10.13016/m2pcms-bkos · Cited by 23
Self-citationEvent Fisher Vectors: Robust Encoding Visual Diversity of Visual Streams
Nagel, M., Mensink, T., & Snoek, C. G. M. (2015) · British Machine Vision Conference (BMVC) · doi:10.5244/c.29.178 · Cited by 22
The AXES submissions at TrecVid 2013
Aly, R., Arandjelović, R., Chatfield, K., Douze, M., Fernando, B., Harchaoui, Z., McGuinness, K., O'Connor, N. E., Oneaţă, D., Parkhi, O., et al. (2013) · Lirias · Cited by 20
Sun, C., Burns, B., Nevatia, R., Snoek, C. G. M., Bolles, B., Myers, G., Wang, W., & Yeh, E. (2014) · International Conference on Multimedia Retrieval · doi:10.1145/2578726.2578757 · Cited by 19
Order Preserving Sparse Coding
Ni, B., Moulin, P., & Yan, S. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2362935 · Cited by 12
Textual Similarity with a Bag-of-Embedded-Words Model
Clinchant, S., & Perronnin, F. (2013) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/2499178.2499180 · Cited by 12
Cited by
A Discriminatively Learned CNN Embedding for Person Reidentification
Zheng, Z., Zheng, L., & Yang, Y. (2018) · ACM Transactions on Multimedia Computing, Communications, and Applications · doi:10.1145/3159171 · Cited by 874
Self-citationAn overview of Internet of Things (IoT): Architectural aspects, challenges, and protocols
Gupta, B. B., & Quamara, M. (2020) · Concurrency and Computation: Practice and Experience · doi:10.1002/cpe.4946 · Cited by 493
Security, privacy and trust of different layers in Internet-of-Things (IoTs) framework
Tewari, A., & Gupta, B. B. (2020) · Future Generation Computer Systems · doi:10.1016/j.future.2018.04.027 · Cited by 411
A detection algorithm for cherry fruits based on the improved YOLO-v4 model
Gai, R., Chen, N., & Yuan, H. (2023) · Neural Computing and Applications · doi:10.1007/s00521-021-06029-z · Cited by 388
An Adaptive Semisupervised Feature Analysis for Video Semantic Recognition
Luo, M., Chang, X., Nie, L., Yang, Y., Hauptmann, A. G., & Zheng, Q. (2018) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2017.2647904 · Cited by 329
Self-citation
Metadata
- Authors
- Chang, X., Yu, Y.-L., Yang, Y., & Xing, E. P.
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2017-08-01
- Volume
- 39
- Issue
- 8
- Pages
- 1617-1632
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W2520861906
- Cited by
- 324
- References
- 74
- FWCI
- 53.17
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.