The DOI matches its Crossref record; 40 of the 40 references it cites were found in a registry or an index.
QPIC: Query-Based Pairwise Human-Object Interaction Detection with Image-Wide Contextual Information
Open accessAbstract
We propose a simple, intuitive yet powerful method for human-object interaction (HOI) detection. HOIs are so diverse in spatial distribution in an image that existing CNN-based methods face the following three major drawbacks; they cannot leverage image-wide features due to CNN’s locality, they rely on a manually defined location-of-interest for the feature aggregation, which sometimes does not cover contextually important regions, and they cannot help but mix up the features for multiple HOI instances if they are located closely. To overcome these drawbacks, we propose a transformer-based feature extractor, in which an attention mechanism and query-based detection play key roles. The attention mechanism is effective in aggregating contextually important information imagewide, while the queries, which we design in such a way that each query captures at most one human-object pair, can avoid mixing up the features from multiple instances. This transformer-based feature extractor produces so effective embeddings that the subsequent detection heads may be fairly simple and intuitive. The extensive analysis reveals that the proposed method successfully extracts contextually important features, and thus outperforms existing methods by large margins (5.37 mAP on HICO-DET, and 5.6 mAP on V-COCO). The source codes are available at https://github.com/hitachi-rd-cv/qpic.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
Focal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.324 · Cited by 30,581
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_13 · Cited by 18,285
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · arXiv · doi:10.48550/arxiv.2005.12872 · Cited by 18,253
The Hungarian method for the assignment problem
Kuhn, H. W. (1955) · Naval Research Logistics Quarterly · doi:10.1002/nav.3800020109 · Cited by 13,120
Decoupled Weight Decay Regularization
Loshchilov, I., & Hutter, F. (2017) · arXiv · doi:10.48550/arxiv.1711.05101 · Cited by 12,522
Generalized Intersection Over Union: A Metric and a Loss for Bounding Box Regression
Rezatofighi, H., Tsoi, N., Gwak, J., Sadeghian, A., Reid, I., & Savarese, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00075 · Cited by 6,612
Attention Augmented Convolutional Networks
Bello, I., Zoph, B., Le, Q., Vaswani, A., & Shlens, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00338 · Cited by 1,205
Parmar, N., Vaswani, A., Uszkoreit, J., Kaiser, Ł., Shazeer, N., Ku, A., & Tran, D. (2018) · arXiv · doi:10.48550/arxiv.1802.05751 · Cited by 1,085 · cited 1×
“A fixed positional encoding p ∈ R D c × H (cid:48) × W [1, 2, 22] is additionally input to the encoder to supplement the positional information, which the self-attention mechanism alone cannot inherently incorporate.”
MethodDetecting and Recognizing Human-Object Interactions
Gkioxari, G., Girshick, R., Dollar, P., & He, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00872 · Cited by 602
Learning Human-Object Interactions by Graph Parsing Neural Networks
Qi, S., Wang, W., Jia, B., Shen, J., & Zhu, S.-C. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01240-3_25 · Cited by 572
Learning to Detect Human-Object Interactions
Chao, Y.-W., Liu, Y., Liu, X., Zeng, H., & Deng, J. (2018) · 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2018.00048 · Cited by 520
Gupta, S., & Malik, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1505.04474 · Cited by 289
PPDM: Parallel Point Detection and Matching for Real-Time Human-Object Interaction Detection
Liao, Y., Liu, S., Wang, F., Chen, Y., Qian, C., & Feng, J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00056 · Cited by 289
Zhou, X., Wang, D., & Krähenbühl, P. (2019) · arXiv · doi:10.48550/arxiv.1904.07850 · Cited by 288
Learning Human-Object Interaction Detection Using Interaction Points
Wang, T., Yang, T., Danelljan, M., Khan, F. S., Zhang, X., & Sun, J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00417 · Cited by 252
VSGNet: Spatial Attention Network for Detecting Human Object Interactions Using Graph Convolutions
Ulutan, O., Iftekhar, A. S. M., & Manjunath, B. S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01363 · Cited by 241
Pose-Aware Multi-Level Feature Network for Human Object Interaction Detection
Wan, B., Zhou, D., Liu, Y., Li, R., & He, X. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00956 · Cited by 240
Transferable Interactiveness Knowledge for Human-Object Interaction Detection
Li, Y.-L., Zhou, S., Huang, X., Xu, L., Ma, Z., Fang, H.-S., Wang, Y., & Lu, C. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00370 · Cited by 232
iCAN: Instance-Centric Attention Network for Human-Object Interaction Detection
Gao, C., Zou, Y., & Huang, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.10437 · Cited by 204
DRG: Dual Relation Graph for Human-Object Interaction Detection
Gao, C., Xu, J., Zou, Y., & Huang, J.-B. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58610-2_41 · Cited by 198
Detailed 2D-3D Joint Representation for Human-Object Interaction
Li, Y., Liu, X., Han, L., Wang, S., Liu, J., Li, J., & Lu, C. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01018 · Cited by 181
Visual Compositional Learning for Human-Object Interaction Detection
Hou, Z., Peng, X., Qiao, Y., & Tao, D. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58555-6_35 · Cited by 176
UnionDet: Union-Level Detector Towards Real-Time Human-Object Interaction Detection
Kim, B., Choi, T., Kang, J., & Kim, H. J. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58555-6_30 · Cited by 170
Deep Contextual Attention for Human-Object Interaction Detection
Wang, T., Anwer, R. M., Khan, M. H., Khan, F. S., Pang, Y., Shao, L., & Laaksonen, J. T. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00579 · Cited by 161
Cascaded Human-Object Interaction Recognition
Zhou, T., Wang, W., Qi, S., Ling, H., & Shen, J. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00432 · Cited by 159
Relation Parsing Neural Network for Human-Object Interaction Detection
Zhou, P., & Chi, M. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00093 · Cited by 153
Gupta, T., Schwing, A. G., & Hoiem, D. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00977 · Cited by 145
Interact as You Intend: Intention-Driven Human-Object Interaction Detection
Xu, B., Li, J., Wong, Y., Zhao, Q., & Kankanhalli, M. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2943753 · Cited by 126
Detecting Human-Object Interactions with Action Co-occurrence Priors
Kim, D.-J., Sun, X. N., Choi, J., Lin, S. C.-F., & Kweon, I. S. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58589-1_43 · Cited by 121
ConsNet: Learning Consistency Graph for Zero-Shot Human-Object Interaction Detection
Liu, Y., Yuan, J., & Chen, C. W. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3413600 · Cited by 103
Contextual Heterogeneous Graph Network for Human-Object Interaction Detection
Wang, H., Zheng, W., & Yingbiao, L. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58520-4_15 · Cited by 97
Amplifying Key Cues for Human-Object-Interaction Detection
Liu, Y., Chen, Q., & Zisserman, A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58568-6_15 · Cited by 93
Polysemy Deciphering Network for Human-Object Interaction Detection
Zhong, X., Ding, C., Qu, X., & Tao, D. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58565-5_5 · Cited by 92
Polysemy Deciphering Network for Robust Human–Object Interaction Detection
Zhong, X., Ding, C., Qu, X., & Tao, D. (2021) · International Journal of Computer Vision · doi:10.1007/s11263-021-01458-8 · Cited by 68
A Graph-based Interactive Reasoning for Human-Object Interaction Detection
Yang, D., & Zou, Y. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/155 · Cited by 37
Action-Guided Attention Mining and Relation Reasoning Network for Human-Object Interaction Detection
Lin, X., Zou, Q., & Xu, X. (2020) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2020/154 · Cited by 23
Metadata
- Authors
- Tamura, M., Ohashi, H., & Yoshinaga, T.
- Venue
- 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2021-06
- Pages
- 10405-10414
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W3174164794
- Cited by
- 256
- References
- 40
- FWCI
- 15.16
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.