Visual Semantic Information Pursuit: A Survey
Open accessThe DOI matches its Crossref record; 128 of the 136 references it cites were found in a registry or an index, and 2 are not checked yet.
Abstract
Visual semantic information comprises two important parts: the meaning of each visual semantic unit and the coherent visual semantic relation conveyed by these visual semantic units. Essentially, the former one is a visual perception task while the latter corresponds to visual context reasoning. Remarkable advances in visual perception have been achieved due to the success of deep learning. In contrast, visual semantic information pursuit, a visual scene semantic interpretation task combining visual perception and visual context reasoning, is still in its early stage. It is the core task of many different computer vision applications, such as object detection, visual semantic segmentation, visual relationship detection, or scene graph generation. Since it helps to enhance the accuracy and the consistency of the resulting interpretation, visual context reasoning is often incorporated with visual perception in current deep end-to-end visual semantic information pursuit methods. Surprisingly, a comprehensive review for this exciting area is still lacking. In this survey, we present a unified theoretical paradigm for all these methods, followed by an overview of the major developments and the future trends in each potential direction. The common benchmark datasets, the evaluation metrics and the comparisons of the corresponding methods are also introduced.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Deep learning has recently begun to dominate semantic visual segmentation [45,46]”
From Semantic Terrain Segmentation in the Navigation Vision of Planetary Rovers—A Systematic Literature Review (2022) · cites this paper as [45]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1506.01497 · Cited by 70,812
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584
(2016) · Cited by 31,590
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
Distributed Representations of Words and Phrases and their Compositionality
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1310.4546 · Cited by 24,533 · cited 1×
“edge bases like word2vec [58], [59] are often used as external prior knowledge since the embeddings can be used to measure the semantic similarity among different words.”
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699184 · Cited by 22,392
SSD: Single Shot MultiBox Detector
Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., & Berg, A. C. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_2 · Cited by 22,128 · cited 5×
“dataset, we select 6 current object detection methods including Fast R-CNN [53], Faster R-CNN [17], SSD500 [19], ION [110], SIN [23] and RFB Net300 [111], as shown in Table 2;”
MethodInfluentialYOLO9000: Better, Faster, Stronger
Redmon, J., & Farhadi, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.690 · Cited by 19,892
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204
Robust Real-Time Face Detection
Viola, P., & Jones, M. J. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000013087.49260.fb · Cited by 14,097
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
Lafferty, J., McCallum, A., & Pereira, F. C. N. (2001) · ScholarlyCommons (University of Pennsylvania) · Cited by 12,935
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2389824 · Cited by 11,971
Fully convolutional networks for semantic segmentation
Shelhamer, E., Long, J., & Darrell, T. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2572683 · Cited by 11,898
Object Detection with Discriminatively Trained Part-Based Models
Felzenszwalb, P. F., Girshick, R. B., McAllester, D., & Ramanan, D. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.167 · Cited by 9,853
Georges, A., Kotliar, G., Krauth, W., & Rozenberg, M. J. (1996) · Reviews of Modern Physics · doi:10.1103/revmodphys.68.13 · Cited by 6,901
Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations
Krishna, R., Zhu, Y., Groth, O., Johnson, J., Hata, K., Kravitz, J., Chen, S., Kalantidis, Y., Li, L.-J., Shamma, D. A., et al. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-016-0981-7 · Cited by 5,107
Deep visual-semantic alignments for generating image descriptions
Karpathy, A., & Fei-Fei, L. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298932 · Cited by 4,902
Domain Adaptation via Transfer Component Analysis
Pan, S. J., Tsang, I. W., Kwok, J. T., & Yang, Q. (2011) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2010.2091281 · Cited by 4,753
Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2014) · arXiv · doi:10.48550/arxiv.1412.7062 · Cited by 4,213 · cited 6×
“Specifically, 10 current visual semantic segmentation methods including CFM [112], DeepLab [66], DeepLab +”
MethodInfluentialNeural Networks and the Bias/Variance Dilemma
Geman, S., Bienenstock, E., & Doursat, R. (1992) · Neural Computation · doi:10.1162/neco.1992.4.1.1 · Cited by 3,578
Neural Message Passing for Quantum Chemistry
Gilmer, J., Schoenholz, S. S., Riley, P. F., Vinyals, O., & Dahl, G. E. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.01212 · Cited by 3,440
Graphical Models, Exponential Families, and Variational Inference
Wainwright, M. J., & Jordan, M. I. (2007) · now publishers, Inc. eBooks · doi:10.1561/9781601981851 · Cited by 2,959
Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
Krähenbühl, P., & Koltun, V. (2012) · arXiv · doi:10.48550/arxiv.1210.5644 · Cited by 2,908
Robust real-time object detection.
Viola, P. A. (2001) · IEEE/CVF International Conference on Computer Vision (ICCV) · Cited by 2,874
Learning Hierarchical Features for Scene Labeling
Farabet, C., Couprie, C., Najman, L., & LeCun, Y. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.231 · Cited by 2,620
Unsupervised Domain Adaptation by Backpropagation
Ganin, Y., & Lempitsky, V. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.7495 · Cited by 2,521
Conditional Random Fields as Recurrent Neural Networks
Zheng, S., Jayasumana, S., Romera-Paredes, B., Vineet, V., Su, Z., Du, D., Huang, C., & Torr, P. H. S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.179 · Cited by 2,386 · cited 11×
“, DeepLab + CRF [66], CRF-RNN [24], UoA-Context + CRF [115] andDAG-RNN+CRF [26], to solve the above issue.”
BackgroundMethodInfluentialMean-field theory for scale-free random networks
Barabási, A.-L., Albert, R., & Jeong, H. (1999) · Physica A: Statistical Mechanics and its Applications · doi:10.1016/s0378-4371(99)00291-5 · Cited by 2,255
The PASCAL visual object classes challenge 2006 (VOC2006) results
Everingham, M., Zisserman, A., Williams, C. K. I., & Gool, L. V. (2006) · Cited by 2,223
Domain Adaptation for Large-Scale Sentiment Classification: A Deep Learning Approach
Glorot, X., Bordes, A., & Bengio, Y. (2011) · HAL · Cited by 2,020
Receptive Field Block Net for Accurate and Fast Object Detection
Liu, S., Huang, D., & Wang, Y. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01252-6_24 · Cited by 1,885
Graphical Models, Exponential Families, and Variational Inference
Wainwright, M. J., & Jordan, M. I. (2008) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000001 · Cited by 1,820
Meta-learning with memory-augmented neural networks
Santoro, A., Bartunov, S., Botvinick, M., Wierstra, D., & Lillicrap, T. (2016) · International Conference on Machine Learning · Cited by 1,504
Quantum-chemical insights from deep tensor neural networks
Schütt, K. T., Arbabzadah, F., Chmiela, S., Müller, K. R., & Tkatchenko, A. (2017) · Nature Communications · doi:10.1038/ncomms13890 · Cited by 1,445
The Role of Context for Object Detection and Semantic Segmentation in the Wild
Mottaghi, R., Chen, X., Liu, X., Cho, N.-G., Lee, S.-W., Fidler, S., Urtasun, R., & Yuille, A. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.119 · Cited by 1,438 · cited 6×
“In this survey, three main benchmarks are chosen out from the abundant datasets for visual semantic segmentation methods: Pascal Context [104], Sift Flow [105] and COCO Stuff [106].”
MethodInfluentialLoopy Belief Propagation for Approximate Inference: An Empirical Study
Murphy, K. P., Weiss, Y., & Jordan, M. I. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1301.6725 · Cited by 1,397
Inside-Outside Net: Detecting Objects in Context with Skip Pooling and Recurrent Neural Networks
Bell, S., Zitnick, C. L., Bala, K., & Girshick, R. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.314 · Cited by 1,292
COCO-Stuff: Thing and Stuff Classes in Context
Caesar, H., Uijlings, J., & Ferrari, V. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00132 · Cited by 1,277
Scene Graph Generation by Iterative Message Passing
Xu, D., Zhu, Y., Choy, C. B., & Fei-Fei, L. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.330 · Cited by 1,271 · cited 14×
“As shown in Table 9, the first four methods (IMP [38], MSDN [39], NM-Freq [40] and Graph R-CNN [41]) use graph-based inference to propagate local contextual information in both directions between object and relationship nodes, while the last three methods (MotifNet [40], GPI [43] and LinkNet [44]) tend to incorporate…”
MethodInfluentialFirst- and Second-Order Methods for Learning: Between Steepest Descent and Newton's Method
Battiti, R. (1992) · Neural Computation · doi:10.1162/neco.1992.4.2.141 · Cited by 1,193
Measuring the Objectness of Image Windows
Alexe, B., Deselaers, T., & Ferrari, V. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.28 · Cited by 1,186
Visual Relationship Detection with Language Priors
Lu, C., Krishna, R., Bernstein, M., & Fei-Fei, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_51 · Cited by 1,069
Graves, A., Wayne, G., & Danihelka, I. (2014) · arXiv · doi:10.48550/arxiv.1410.5401 · Cited by 1,058 · cited 1×
“Unlike the above models, instead of using the internal memory cells like long-short term memory (LSTM) or gated recurrent unit (GRU), the external memory reasoning models apply the external memory cells, such as the neural turing machine (NTM) [81] or the memory-augmented neural network (MANN) [82], to store the gener…”
BackgroundNeural Motifs: Scene Graph Parsing with Global Context
Zellers, R., Yatskar, M., Thomson, S., & Choi, Y. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00611 · Cited by 1,052
An Embarrassingly Simple Approach to Zero-Shot Learning
Romera-Paredes, B., & Torr, P. H. S. (2017) · Advances in Computer Vision and Pattern Recognition · doi:10.1007/978-3-319-50077-5_2 · Cited by 1,003
Zero-Shot Learning Through Cross-Modal Transfer
Socher, R., Ganjoo, M., Sridhar, H., Bastani, O., Manning, C. D., & Ng, A. Y. (2013) · arXiv · doi:10.48550/arxiv.1301.3666 · Cited by 997
Interaction Networks for Learning about Objects, Relations and Physics
Battaglia, P. W., Pascanu, R., Lai, M., Rezende, D., & Kavukcuoglu, K. (2016) · arXiv · doi:10.48550/arxiv.1612.00222 · Cited by 969
Generalized Belief Propagation
Yedidia, J. S., Freeman, W. T., & Weiss, Y. (2000) · Cited by 918
Visual Domain Adaptation: A survey of recent advances
Patel, V. M., Gopalan, R., Li, R., & Chellappa, R. (2015) · IEEE Signal Processing Magazine · doi:10.1109/msp.2014.2347059 · Cited by 899
Efficient Piecewise Training of Deep Structured Models for Semantic Segmentation
Lin, G., Shen, C., van den Hengel, A., & Reid, I. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.348 · Cited by 862
Matching Networks for One Shot Learning
Vinyals, O., Blundell, C., Lillicrap, T., Kavukcuoglu, K., & Wierstra, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.04080 · Cited by 854
Graph R-CNN for Scene Graph Generation
Yang, J., Lu, J., Lee, S., Batra, D., & Parikh, D. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01246-5_41 · Cited by 851
Multiple Instance Boosting for Object Detection
Zhang, C., Platt, J. C., & Viola, P. A. (2005) · neural information processing systems · Cited by 769
Recurrent Convolutional Neural Networks for Scene Labeling
Pinheiro, P. O., & Collobert, R. (2014) · Infoscience (Ecole Polytechnique Fédérale de Lausanne) · Cited by 757
Advances in Variational Inference
Zhang, C., Butepage, J., Kjellstrom, H., & Mandt, S. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2889774 · Cited by 671
Advanced Deep-Learning Techniques for Salient and Category-Specific Object Detection: A Survey
Han, J., Zhang, D., Cheng, G., Liu, N., & Xu, D. (2018) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2749125 · Cited by 621
Visual Translation Embedding Network for Visual Relation Detection
Zhang, H., Kyaw, Z., Chang, S.-F., & Chua, T.-S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.331 · Cited by 586 · cited 7×
“Unlike the above methods, [33] proposes to model the predicate as a vector translation between the subject and the object, in which both subject and object are mapped into”
BackgroundMethodInfluentialSTC: A Simple to Complex Framework for Weakly-Supervised Semantic Segmentation
Wei, Y., Liang, X., Chen, Y., Shen, X., Cheng, M.-M., Feng, J., Zhao, Y., & Yan, S. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2636150 · Cited by 576
VQA: Visual Question Answering: www.visualqa.org
Agrawal, A., Lu, J., Antol, S., Mitchell, M., Zitnick, C. L., Parikh, D., & Batra, D. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-016-0966-6 · Cited by 570
An Introduction to Conditional Random Fields
Sutton, C., & McCallum, A. (2012) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000013 · Cited by 564
Harnessing Deep Neural Networks with Logic Rules
Hu, Z., Ma, X., Liu, Z., Hovy, E., & Xing, E. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1228 · Cited by 545
Detecting Visual Relationships with Deep Relational Networks
Dai, B., Zhang, Y., & Lin, D. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.352 · Cited by 502
Scene Graph Generation from Objects, Phrases and Region Captions
Li, Y., Ouyang, W., Zhou, B., Wang, K., & Wang, X. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.142 · Cited by 500
Convolutional feature masking for joint object and stuff segmentation
Dai, J., He, K., & Sun, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299025 · Cited by 422
Weakly Supervised Object Localization with Multi-Fold Multiple Instance Learning
Cinbis, R. G., Verbeek, J., & Schmid, C. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2535231 · Cited by 420
Image Segmentation by Probabilistic Bottom-Up Aggregation and Cue Integration
Alpert, S., Galun, M., Brandt, A., & Basri, R. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.130 · Cited by 399
A New Class of Upper Bounds on the Log Partition Function
Wainwright, M. J., Jaakkola, T. S., & Willsky, A. S. (2005) · IEEE Transactions on Information Theory · doi:10.1109/tit.2005.850091 · Cited by 381
Scene labeling with LSTM recurrent neural networks
Byeon, W., Breuel, T. M., Raue, F., & Liwicki, M. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298977 · Cited by 375
Gated Graph Sequence Neural Networks
Li, Y., Tarlow, D., Brockschmidt, M., & Zemel, R. S. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.05493 · Cited by 362
A Linear Programming Approach to Max-Sum Problem: A Review
Werner, T. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1036 · Cited by 342
Nonparametric scene parsing: Label transfer via dense scene alignment
Liu, C., Yuen, J., & Torralba, A. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206536 · Cited by 315
Visual Relationship Detection with Internal and External Linguistic Knowledge Distillation
Yu, R., Li, A., Morariu, V. I., & Davis, L. S. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.121 · Cited by 310
Markov Random Fields for Vision and Image Processing
(2011) · The MIT Press eBooks · doi:10.7551/mitpress/8579.001.0001 · Cited by 290
Factorizable Net: An Efficient Subgraph-Based Framework for Scene Graph Generation
Li, Y., Ouyang, W., Zhou, B., Shi, J., Zhang, C., & Wang, X. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01246-5_21 · Cited by 283
Structure Inference Net: Object Detection Using Scene-Level Context and Instance-Level Relationships
Liu, Y., Wang, R., Shan, S., & Chen, X. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00730 · Cited by 252
Deep Variation-Structured Reinforcement Learning for Visual Relationship and Attribute Detection
Liang, X., Lee, L., & Xing, E. P. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.469 · Cited by 243
What's going on? Discovering spatio-temporal dependencies in dynamic scenes
Kuettel, D., Breitenstein, M. D., Van Gool, L., & Ferrari, V. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539869 · Cited by 241
Iterative Visual Reasoning Beyond Convolutions
Chen, X., Li, L.-J., Fei-Fei, L., & Gupta, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00756 · Cited by 236
Gupta, A., Srinivasan, P., Shi, J., & Davis, L. S. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206492 · Cited by 231
ViP-CNN: Visual Phrase Guided Convolutional Neural Network
Li, Y., Ouyang, W., Wang, X., & Tang, X. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.766 · Cited by 229
Finding Things: Image Parsing with Regions and Per-Exemplar Detectors
Tighe, J., & Lazebnik, S. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.386 · Cited by 207
Weakly supervised object detection with convex clustering
Bilen, H., Pedersoli, M., & Tuytelaars, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298711 · Cited by 206
A Comparative Study of Modern Inference Techniques for Discrete Energy Minimization Problems
Kappes, J. H., Andres, B., Hamprecht, F. A., Schnorr, C., Nowozin, S., Batra, D., Kim, S., Kausler, B. X., Lellmann, J., Komodakis, N., et al. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.175 · Cited by 187
Large-Scale Visual Relationship Understanding
Zhang, J., Kalantidis, Y., Rohrbach, M., Paluri, M., Elgammal, A., & Elhoseiny, M. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33019185 · Cited by 182
Towards Context-Aware Interaction Recognition for Visual Relationship Detection
Zhuang, B., Liu, L., Shen, C., & Reid, I. D. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.71 · Cited by 177
Fu, K., Jin, J., Cui, R., Sha, F., & Zhang, C. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2642953 · Cited by 176
Deep Learning Markov Random Field for Semantic Segmentation
Liu, Z., Li, X., Luo, P., Loy, C. C., & Tang, X. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2737535 · Cited by 174
Weakly-Supervised Learning of Visual Relations
Peyre, J., Laptev, I., Schmid, C., & Sivic, J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.554 · Cited by 172
Zoom-Net: Mining Deep Feature Interactions for Visual Relationship Recognition
Yin, G., Sheng, L., Liu, B., Yu, N., Wang, X., Shao, J., & Loy, C. C. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01219-9_20 · Cited by 169
Spatial Memory for Context Reasoning in Object Detection
Chen, X., & Gupta, A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.440 · Cited by 167
Zhang, D., Han, J., Zhao, L., & Meng, D. (2019) · International Journal of Computer Vision · doi:10.1007/s11263-018-1112-4 · Cited by 164 · cited 1×
“To alleviate the annotation burden, weakly-supervised pursuit methods [86], [87], [88], [89], [90] have been proposed in recent years, which only require the vocabulary information”
MethodLinkNet: Relational Embedding for Scene GraphPreprint
Woo, S., Kim, D., Cho, D., & Kweon, I. S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.06410 · Cited by 160
Arnab, A., Zheng, S., Jayasumana, S., Romera‐Paredes, B., Larsson, M., Kirillov, A., Savchynskyy, B., Rother, C., Kahl, F., & Torr, P. H. S. (2018) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2762355 · Cited by 158
Context Driven Scene Parsing with Attention to Rare Classes
Yang, J., Price, B., Cohen, S., & Yang, M.-H. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.415 · Cited by 154
DAG-Recurrent Neural Networks for Scene Labeling
Shuai, B., Zuo, Z., Wang, B., & Wang, G. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.394 · Cited by 151
PPR-FCN: Weakly Supervised Visual Relation Detection via Parallel Pairwise R-FCN
Zhang, H., Kyaw, Z., Yu, J., & Chang, S. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.454 · Cited by 150
Weakly Supervised Deep Detection Networks
Bilen, H., & Vedaldi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.311 · Cited by 144
Crafting GBD-Net for Object Detection
Zeng, X., Ouyang, W., Yan, J., Li, H., Xiao, T., Wang, K., Liu, Y., Zhou, Y., Yang, B., Wang, Z., et al. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2745563 · Cited by 142 · cited 1×
“methods with visual context reasoning modules [20], [21], [22], [23] have been proposed recently, which are the focus”
MethodCOCO-Stuff: Thing and Stuff Classes in ContextPreprint
Caesar, H., Uijlings, J., & Ferrari, V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1612.03716 · Cited by 141
Scene Segmentation with DAG-Recurrent Neural Networks
Shuai, B., Zuo, Z., Wang, B., & Wang, G. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2712691 · Cited by 132
Relationship Proposal Networks
Zhang, J., Elhoseiny, M., Cohen, S. D., Chang, W. H., & Elgammal, A. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.555 · Cited by 130
Deep Learning for Image-to-Text Generation: A Technical Overview
He, X., & Deng, L. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2741510 · Cited by 121
MAP Estimation, Linear Programming and Belief Propagation with Convex Free EnergiesPreprint
Weiss, Y., Yanover, C., & Meltzer, T. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.5286 · Cited by 120
Ask Your Neurons: A Deep Learning Approach to Visual Question Answering
Malinowski, M., Rohrbach, M., & Fritz, M. (2017) · International Journal of Computer Vision · doi:10.1007/s11263-017-1038-2 · Cited by 105
Convergent message passing algorithms - a unifying viewPreprint
Meltzer, T., Globerson, A., & Weiss, Y. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1205.2625 · Cited by 101
Mapping Images to Scene Graphs with Permutation-Invariant Structured Prediction
Herzig, R., Raboh, M., Chechik, G., Berant, J., & Globerson, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.05451 · Cited by 91
Learning message-passing inference machines for structured prediction
Ross, S., Munoz, D., Hebert, M., & Bagnell, J. A. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995724 · Cited by 88
Zhang, Y., Zhang, Y., Swears, E., Larios, N., Wang, Z., & Ji, Q. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.33 · Cited by 76
Deep Neural Networks with Massive Learned Knowledge
Hu, Z., Yang, Z., Salakhutdinov, R., & Xing, E. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1173 · Cited by 76
Learning, detection and representation of multi-agent events in videos
Hakeem, A., & Shah, M. A. (2007) · Artificial Intelligence · doi:10.1016/j.artint.2007.04.002 · Cited by 68
GLoMo: Unsupervisedly Learned Relational Graphs as Transferable RepresentationsPreprint
Yang, Z., Zhao, J., Dhingra, B., He, K., Cohen, W. W., Salakhutdinov, R., & LeCun, Y. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.05662 · Cited by 59
Visual Question Answering: A Tutorial
Teney, D., Wu, Q., & van den Hengel, A. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2739826 · Cited by 48
Top-Down Learning for Structured Labeling with Convolutional Pseudoprior
Xie, S., Huang, X., & Tu, Z. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_19 · Cited by 38
Visual Relationship Detection Based on Guided Proposals and Semantic Knowledge Distillation
Plesse, F., Ginsca, A., Delezoide, B., & Preteux, F. J. (2018) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2018.8486503 · Cited by 31
Variational Algorithms for Marginal MAPPreprint
Liu, Q., & Ihler, A. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1302.6584 · Cited by 31
Robust Image Segmentation Using Resampling and Shape Constraints
Zöller, T., & Buhmann, J. M. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1150 · Cited by 24
Scene Parsing With Integration of Parametric and Non-Parametric Models
Shuai, B., Zuo, Z., Wang, G., & Wang, B. (2016) · IEEE Transactions on Image Processing · doi:10.1109/tip.2016.2533862 · Cited by 21
Large-Scale Visual Relationship UnderstandingPreprint
Zhang, J., Kalantidis, Y., Rohrbach, M., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.10660 · Cited by 14
Information Pursuit: A Bayesian Framework for Sequential Scene ParsingPreprint
Jahangiri, E., Yörük, E., Vidal, R., Younes, L., & Geman, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.02343 · Cited by 9
CRVI: Convex Relaxation for Variational Inference
Fazelnia, G., & Paisley, J. W. (2018) · International Conference on Machine Learning · Cited by 6
Research works printed but not linked: 2
- yang, 2018, GLoMo: Unsupervised learning of transferable relational graphs, Proc Int Conf Neural Inf Process, 8964
- sharma, 2014, Recursive context propagation network for semantic scene labeling, Proc Int Conf Neural Inf Process, 2447
Cited by
Artificial intelligence for the metaverse: A survey
Huynh-The, T., Pham, Q.-V., Pham, X.-Q., Nguyen, T. T., Han, Z., & Kim, D.-S. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.105581 · Cited by 710
Trends in Integration of Vision and Language Research: A Survey of Tasks, Datasets, and Methods
Mogadala, A., Kalimuthu, M., & Klakow, D. (2021) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.11688 · Cited by 47
Semantic-Guided Zero-Shot Learning for Low-Light Image/Video Enhancement
Zheng, S., & Gupta, G. (2022) · 2022 IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW) · doi:10.1109/wacvw54805.2022.00064 · Cited by 160
Federated Semantic Learning Driven by Information Bottleneck for Task-Oriented Communications
Wei, H., Ni, W., Xu, W., Wang, F., Niyato, D. T., & Zhang, P. (2023) · IEEE Communications Letters · doi:10.1109/lcomm.2023.3307096 · Cited by 49
Spatial–Temporal Knowledge-Embedded Transformer for Video Scene Graph Generation
Pu, T., Chen, T., Wu, H., Lu, Y., & Lin, L. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3345652 · Cited by 40
Metadata
- Authors
- Liu, D., Bober, M., & Kittler, J.
- Affiliations
- University of Surrey (GB)
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2021-04-01
- Volume
- 43
- Issue
- 4
- Pages
- 1404-1422
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- 1939-3539
- OpenAlex
- W2986758741
- Semantic Scholar
- c817ea222d18d3b611c6fa0dc6e92db70d8ef8e1
- Cited by
- 39
- References
- 136
- Citations by year
- 2026: 3 · 2025: 4 · 2024: 5 · 2023: 6 · 2022: 6 · 2021: 6
- FWCI
- 2.35
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Surrey Research Insight Open Access (The University of Surrey) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Defence Science and Technology Laboratory
- Engineering and Physical Sciences Research Council (EP/R018456/1)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image and Video Retrieval Techniques
- Keywords
- visual perception, visual relationship detection, scene graph generation, object detection, benchmark dataset, evaluation metrics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.