The DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.
Weakly Aligned Cross-Modal Learning for Multispectral Pedestrian Detection
Open accessAbstract
Multispectral pedestrian detection has shown great advantages under poor illumination conditions, since the thermal modality provides complementary information for the color image. However, real multispectral data suffers from the position shift problem, i.e. the color-thermal image pairs are not strictly aligned, making one object has different positions in different modalities. In deep learning based methods, this problem makes it difficult to fuse the feature maps from both modalities and puzzles the CNN training. In this paper, we propose a novel Aligned Region CNN (AR-CNN) to handle the weakly aligned multispectral data in an end-to-end way. Firstly, we design a Region Feature Alignment (RFA) module to capture the position shift and adaptively align the region features of the two modalities. Secondly, we present a new multimodal fusion method, which performs feature re-weighting to select more reliable features and suppress the useless ones. Besides, we propose a novel RoI jitter strategy to improve the robustness to unexpected shift patterns of different devices and system settings. Finally, since our method depends on a new kind of labelling: bounding boxes that match each modality, we manually relabel the KAIST dataset by locating bounding boxes in both modalities and building their relationships, providing a new KAIST-Paired Annotation. Extensive experimental validations on existing datasets are performed, demonstrating the effectiveness and robustness of the proposed method. Code and data are available at https://github.com/luzhang16/AR-CNN.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
Ren, S., He, K., Girshick, R., & Sun, J. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2577031 · Cited by 73,472
(2016) · Cited by 31,590
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105
Image registration methods: a survey
Zitová, B., & Flusser, J. (2003) · Image and Vision Computing · doi:10.1016/s0262-8856(03)00137-9 · Cited by 6,423
A survey of image registration techniques
Brown, L. G. (1992) · ACM Computing Surveys · doi:10.1145/146370.146374 · Cited by 4,057
A survey of medical image registration
Maintz, J. B. A., & Viergever, M. A. (1998) · Medical Image Analysis · doi:10.1016/s1361-8415(01)80026-8 · Cited by 3,327
Pedestrian Detection: An Evaluation of the State of the Art
Dollar, P., Wojek, C., Schiele, B., & Perona, P. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.155 · Cited by 3,190
Fast Feature Pyramids for Object Detection
Dollar, P., Appel, R., Belongie, S., & Perona, P. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2300479 · Cited by 1,955
Learning Rich Features from RGB-D Images for Object Detection and Segmentation
Gupta, S., Girshick, R., Arbeláez, P., & Malik, J. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10584-0_23 · Cited by 1,528
Multispectral pedestrian detection: Benchmark dataset and baseline
Hwang, S., Park, J., Kim, N., Choi, Y., & Kweon, I. S. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298706 · Cited by 1,249
Dollar, P., Tu, Z., Perona, P., & Belongie, S. (2009) · Procedings of the British Machine Vision Conference 2009 · doi:10.5244/c.23.91 · Cited by 1,237
DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusion
Wang, C., Xu, D., Zhu, Y., Martín-Martín, R., Lu, C., Fei-Fei, L., & Savarese, S. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00346 · Cited by 1,154
CityPersons: A Diverse Dataset for Pedestrian Detection
Zhang, S., Benenson, R., & Schiele, B. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.474 · Cited by 1,061
PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimation
Xu, D., Anguelov, D., & Jain, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00033 · Cited by 856
Is Faster R-CNN Doing Well for Pedestrian Detection?
Zhang, L., Lin, L., Liang, X., & He, K. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_28 · Cited by 852
Self-citationScale-aware Fast R-CNN for Pedestrian Detection
Li, J., Liang, X., Shen, S., Xu, T., Feng, J., & Yan, S. (2017) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2017.2759508 · Cited by 787
Pedestrian Detection with Unsupervised Multi-stage Feature Learning
Sermanet, P., Kavukcuoglu, K., Chintala, S., & Lecun, Y. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.465 · Cited by 742
Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D Images
Song, S., & Xiao, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.94 · Cited by 681
Repulsion Loss: Detecting Pedestrians in a Crowd
Wang, X., Xiao, T., Jiang, Y., Shao, S., Sun, J., & Shen, C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00811 · Cited by 592
Multispectral Deep Neural Networks for Pedestrian Detection
Liu, J., Zhang, S., Wang, S., & Metaxas, D. (2016) · Procedings of the British Machine Vision Conference 2016 · doi:10.5244/c.30.73 · Cited by 543
Illumination-aware faster R-CNN for robust multispectral pedestrian detection
Li, C., Song, D., Tong, R., & Tang, M. (2019) · Pattern Recognition · doi:10.1016/j.patcog.2018.08.005 · Cited by 533
Corneanu, C. A., Simon, M. O., Cohn, J. F., & Guerrero, S. E. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2515606 · Cited by 527
Occlusion-Aware R-CNN: Detecting Pedestrians in a Crowd
Zhang, S., Wen, L., Bian, X., Lei, Z., & Li, S. Z. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01219-9_39 · Cited by 488
Self-citationFiltered channel features for pedestrian detection
Zhang, S., Benenson, R., & Schiele, B. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298784 · Cited by 395
KAIST Multi-Spectral Day/Night Data Set for Autonomous and Assisted Driving
Choi, Y., Kim, N., Hwang, S., Park, K., Yoon, J. S., An, K., & Kweon, I. S. (2018) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2018.2791533 · Cited by 370
Guan, D., Cao, Y., Yang, J., Cao, Y., & Yang, M. Y. (2019) · Information Fusion · doi:10.1016/j.inffus.2018.11.017 · Cited by 342
Local Decorrelation for Improved Pedestrian Detection
Nam, W., Stradvision Inc, & Han, J. H. (2015) · Cited by 336
Pedestrian Detection at Day/Night Time with Visible and FIR Cameras: A Comparison
González, A., Fang, Z., Socarras, Y., Serrat, J., Vázquez, D., Xu, J., & López, A. (2016) · Sensors · doi:10.3390/s16060820 · Cited by 322
Cross-modality interactive attention network for multispectral pedestrian detection
Zhang, L., Liu, Z., Zhang, S., Yang, X., Qiao, H., Huang, K., & Hussain, A. (2019) · Information Fusion · doi:10.1016/j.inffus.2018.09.015 · Cited by 320
Self-citationWhat Can Help Pedestrian Detection?
Mao, J., Xiao, T., Jiang, Y., & Cao, Z. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.639 · Cited by 306
Convolutional Channel Features
Yang, B., Yan, J., Lei, Z., & Li, S. Z. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.18 · Cited by 301
Self-citationFully Convolutional Region Proposal Networks for Multispectral Person Detection
Konig, D., Adam, M., Jarvers, C., Layher, G., Neumann, H., & Teutsch, M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.36 · Cited by 287
Taking a deeper look at pedestrians
Hosang, J., Omran, M., Benenson, R., & Schiele, B. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299034 · Cited by 285
Illuminating Pedestrians via Simultaneous Detection and Segmentation
Brazil, G., Yin, X., & Liu, X. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.530 · Cited by 270
Multispectral Pedestrian Detection using Deep Fusion Convolutional Neural Networks.
Wagner, J., Fischer, V., Herman, M., & Behnke, S. (2016) · The European Symposium on Artificial Neural Networks · Cited by 244
Learning Cross-Modal Deep Representations for Robust Pedestrian Detection
Xu, D., Ouyang, W., Ricci, E., Wang, X., & Sebe, N. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.451 · Cited by 237
Physiology-Based Face Recognition in the Thermal Infrared Spectrum
Buddharaju, P., Pavlidis, I. T., Tsiamyrtzis, P., & Bazakos, M. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1007 · Cited by 236
Face recognition with visible and thermal infrared imagery
Socolinsky, D. A., Selinger, A., & Neuheisel, J. D. (2003) · Computer Vision and Image Understanding · doi:10.1016/s1077-3142(03)00075-4 · Cited by 220
Multiscale Fusion of Visible and Thermal IR Images for Illumination-Invariant Face Recognition
Kong, S. G., Heo, J., Boughorbel, F., Zheng, Y., Abidi, B. R., Koschan, A., Yi, M., & Abidi, M. A. (2007) · International Journal of Computer Vision · doi:10.1007/s11263-006-6655-0 · Cited by 218
Multispectral Pedestrian Detection via Simultaneous Detection and SegmentationPreprint
Li, C., Song, D., Tong, R., & Tang, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.04818 · Cited by 195
Thermal Image Enhancement using Convolutional Neural Network
Choi, Y., Kim, N., Hwang, S., & Kweon, I. S. (2016) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2016.7759059 · Cited by 166
Torabi, A., Massé, G., & Bilodeau, G.-A. (2012) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2011.10.006 · Cited by 161
Physiology-Based Face Recognition in the Thermal Infrared Spectrum
Pavlidis, I., & Buddharaju, P. (2007) · Medical Infrared Imaging · doi:10.1201/9781420008340.ch13 · Cited by 160
Unified multi-spectral pedestrian detection based on probabilistic fusion networks
Park, K., Kim, S., & Sohn, K. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2018.03.007 · Cited by 153
Remote Sensing Image Registration Techniques: A Survey
Dawn, S., Saxena, V., & Sharma, B. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-13681-8_13 · Cited by 133
InfAR dataset: Infrared action recognition at different times
Gao, C., Du, Y., Liu, J., Lv, J. G., Yang, L., Meng, D., & Hauptmann, A. G. (2016) · Neurocomputing · doi:10.1016/j.neucom.2016.05.094 · Cited by 111
Amodal Detection of 3D Objects: Inferring 3D Bounding Boxes from 2D Ones in RGB-Depth Images
Deng, Z., & Latecki, L. J. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.50 · Cited by 109
Choi, H., Kim, S., Park, K., & Sohn, K. (2016) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2016.7899703 · Cited by 101
Thermal-Visible Video Fusion for Moving Target Tracking and Pedestrian Classification
Leykin, A., Yang, R., & Hammoud, R. I. (2007) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2007.383444 · Cited by 68
CATS: A Color and Thermal Stereo Benchmark
Treible, W., Saponaro, P. J., Sorensen, S., Kolagunda, A., O’Neal, M., Phelan, B. R., Sherbondy, K. D., & Kambhamettu, C. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.22 · Cited by 65
Cao, Y., Guan, D., Wu, Y., Yang, J., Cao, Y., & Yang, M. Y. (2019) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2019.02.005 · Cited by 60
Multispectral Transfer Network: Unsupervised Depth Estimation for All-Day Vision
Kim, N., Choi, Y., Hwang, S., & Kweon, I. S. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.12297 · Cited by 55
Exploiting fusion architectures for multispectral pedestrian detection and segmentation
Guan, D., Cao, Y., Yang, J., Cao, Y., & Tisse, C.-L. (2018) · Applied Optics · doi:10.1364/ao.57.00d108 · Cited by 31
A Study on Visible to Infrared Action Recognition
Zhu, Y., & Guo, G. (2013) · IEEE Signal Processing Letters · doi:10.1109/lsp.2013.2272920 · Cited by 28
Cited by
ICAFusion: Iterative cross-attention guided feature fusion for multispectral object detection
Shen, J., Chen, Y., Liu, Y., Zuo, X., Fan, H., & Yang, W. (2024) · Pattern Recognition · doi:10.1016/j.patcog.2023.109913 · Cited by 544
Visible and Infrared Image Fusion Using Deep Learning
Zhang, X., & Demiris, Y. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3261282 · Cited by 392
LasHeR: A Large-Scale High-Diversity Benchmark for RGBT Tracking
Li, C., Xue, W., Jia, Y., Qu, Z., Luo, B., Tang, J., & Sun, D. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3130533 · Cited by 307
Improving Multispectral Pedestrian Detection by Addressing Modality Imbalance Problems
Zhou, K., Chen, L., & Cao, X. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58523-5_46 · Cited by 301
Multispectral Fusion for Object Detection with Cyclic Fuse-and-Refine Blocks
Zhang, H., Fromont, E., Lefevre, S., & Avignon, B. (2020) · 2020 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip40778.2020.9191080 · Cited by 297
Metadata
- Authors
- Zhang, L., Zhu, X., Chen, X., Yang, X., Lei, Z., & Liu, Z.
- Venue
- 2019 IEEE/CVF International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2019-10
- Pages
- 5126-5136
- OpenAlex
- W2987131085
- Cited by
- 271
- References
- 57
- FWCI
- 5.40
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.