The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
PST900: RGB-Thermal Calibration, Dataset and Segmentation Network
Open accessAbstract
In this work we propose long wave infrared (LWIR) imagery as a viable supporting modality for semantic segmentation using learning-based techniques. We first address the problem of RGB-thermal camera calibration by proposing a passive calibration target and procedure that is both portable and easy to use. Second, we present PST900, a dataset of 894 synchronized and calibrated RGB and Thermal image pairs with per pixel human annotations across four distinct classes from the DARPA Subterranean Challenge. Lastly, we propose a CNN architecture for fast semantic segmentation that combines both RGB and Thermal imagery in a way that leverages RGB imagery independently. We compare our method against the state-of-the-art and show that our method outperforms them in our dataset.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2644615 · Cited by 19,631
The pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599 · cited 1×
“segmentation from RGB imagery [3], [4], [5], [6], [7].”
MethodZhao, H., Shi, J., Qi, X., Wang, X., & Jia, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.660 · Cited by 16,438
The Cityscapes Dataset for Semantic Urban Scene Understanding
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.350 · Cited by 12,433 · cited 1×
“segmentation from RGB imagery [3], [4], [5], [6], [7].”
MethodScene Parsing through ADE20K Dataset
Zhou, B., Zhao, H., Puig, X., Fidler, S., Barriuso, A., & Torralba, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.544 · Cited by 3,326
Understanding Convolution for Semantic Segmentation
Wang, P., Chen, P., Yuan, Y., Liu, D., Huang, Z., Hou, X., & Cottrell, G. (2018) · 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2018.00163 · Cited by 1,957
ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation
Paszke, A., Chaurasia, A., Kim, S., & Culurciello, E. (2016) · arXiv · doi:10.48550/arxiv.1606.02147 · Cited by 1,826 · cited 1×
“[35].”
ERFNet: Efficient Residual Factorized ConvNet for Real-Time Semantic Segmentation
Romera, E., Alvarez, J. M., Bergasa, L. M., & Arroyo, R. (2018) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2017.2750080 · Cited by 1,484
MFNet: Towards real-time semantic segmentation for autonomous vehicles with multi-spectral scenes
Ha, Q., Watanabe, K., Karasawa, T., Ushiku, Y., & Harada, T. (2017) · 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8206396 · Cited by 693
Automatic camera and range sensor calibration using a single shot
Geiger, A., Moosmann, F., Car, O., & Schuster, B. (2012) · 2012 IEEE International Conference on Robotics and Automation · doi:10.1109/icra.2012.6224570 · Cited by 647
Rtfnet: Rgb-thermal fusion network for semantic segmentation of urban scenes
Sun, Y., Zuo, W., & Liu, M. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2904733 · Cited by 546 · cited 3×
“In this section, we compare our method against relevant methods MFNet [9], and RTFNet [10].”
MethodAugmented Reality Meets Computer Vision: Efficient Data Generation for Urban Driving Scenes
Abu Alhaija, H., Mustikovela, S. K., Mescheder, L., Geiger, A., & Rother, C. (2018) · International Journal of Computer Vision · doi:10.1007/s11263-018-1070-x · Cited by 428
Counting Apples and Oranges With Deep Learning: A Data-Driven Approach
Chen, S. W., Shivakumar, S. S., Dcunha, S., Das, J., Okon, E., Qu, C., Taylor, C. J., & Kumar, V. (2017) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2017.2651944 · Cited by 406
Self-citationFast-SCNN: Fast Semantic Segmentation Network
Poudel, R. P. K., Liwicki, S., & Cipolla, R. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.04502 · Cited by 358
Infrared Thermal Imaging: Fundamentals, Research and Applications
Vollmer, M., & Möllmann, K. (2017) · Wiley eBooks · doi:10.1002/9783527693306 · Cited by 317
Thermal Image Enhancement using Convolutional Neural Network
Choi, Y., Kim, N., Hwang, S., & Kweon, I. S. (2016) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2016.7759059 · Cited by 166
Torabi, A., Massé, G., & Bilodeau, G.-A. (2012) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2011.10.006 · Cited by 161
A Mask-Based Approach for the Geometric Calibration of Thermal-Infrared Cameras
Vidas, S., Lakemond, R., Denman, S., Fookes, C., Sridharan, S., & Wark, T. (2012) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2012.2182851 · Cited by 144
Thermal Stereo Odometry for UAVs
Mouats, T., Aouf, N., Chermak, L., & Richardson, M. A. (2015) · IEEE Sensors Journal · doi:10.1109/jsen.2015.2456337 · Cited by 104
3D Thermal Imaging: Fusion of Thermography and Depth Cameras
Rangel, J., & Soldan, S. (2014) · International Conference on Quantitative InfraRed Thermography (QIRT) · doi:10.21611/qirt.2014.035 · Cited by 90
Thermal–visible registration of human silhouettes: A similarity measure performance evaluation
Bilodeau, G.-A., Torabi, A., St-Charles, P.-L., & Riahi, D. (2014) · Infrared Physics & Technology · doi:10.1016/j.infrared.2014.02.005 · Cited by 74
CATS: A Color and Thermal Stereo Benchmark
Treible, W., Saponaro, P. J., Sorensen, S., Kolagunda, A., O’Neal, M., Phelan, B. R., Sherbondy, K. D., & Kambhamettu, C. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.22 · Cited by 65
MAVNet: An Effective Semantic Segmentation Micro-Network for MAV-Based Tasks
Nguyen, T., Wozencraft, J. M., Taylor, C. J., Kumar, V., Shivakumar, S. S., Miller, I. D., Keller, J. F., Lee, E. S., Zhou, A., Özaslan, T., et al. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2928734 · Cited by 46
Self-citationCombination of colour and thermal sensors for enhanced object detection
St-Laurent, L., Maldague, X., & Prévost, D. (2007) · International Conference on Information Fusion (FUSION) · doi:10.1109/icif.2007.4408003 · Cited by 43
THERMALNET: A DEEP CONVOLUTIONAL NETWORK FOR SYNTHETIC THERMAL IMAGE GENERATION
Kniaz, V. V., Сергеевич, Г. В., & Mizginov, V. A. (2017) · The international archives of the photogrammetry, remote sensing and spatial information sciences/International archives of the photogrammetry, remote sensing and spatial information sciences · doi:10.5194/isprs-archives-xlii-2-w4-41-2017 · Cited by 40
Thermal Infrared Pedestrian Image Segmentation Using Level Set Method
Qiao, Y., Wei, Z., & Zhao, Y. J. (2017) · Sensors · doi:10.3390/s17081811 · Cited by 30
Scene Semantic Reconstruction from Egocentric RGB-D-Thermal Videos
Luo, R., Şener, O., & Savarese, S. (2017) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2017.00073 · Cited by 15
Multimodal Sensor Fusion In Single Thermal image Super-ResolutionPreprint
Almasri, F., & Debeir, O. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.09276 · Cited by 13
Cited by
CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation With Transformers
Zhang, J., Liu, H., Yang, K., Hu, X., Liu, R., & Stiefelhagen, R. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3300537 · Cited by 731
GMNet: Graded-Feature Multilabel-Learning Network for RGB-Thermal Urban Scene Semantic Segmentation
Zhou, W., Liu, J., Lei, J., Yu, L., & Hwang, J.-N. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3109518 · Cited by 344
Liu, J., Liu, Z., Wu, G., Ma, L., Liu, R., Zhong, W., Luo, Z., & Fan, X. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00745 · Cited by 316
Deep multimodal fusion for semantic image segmentation: A survey
Zhang, Y., Sidibé, D., Morel, O., & Mériaudeau, F. (2021) · Image and Vision Computing · doi:10.1016/j.imavis.2020.104042 · Cited by 261
Zhang, Q., Zhao, S., Luo, Y., Zhang, D., Huang, N., & Han, J. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00266 · Cited by 225
Metadata
- Authors
- Shivakumar, S. S., Rodrigues, N., Zhou, A., Miller, I. D., Kumar, V., & Taylor, C. J.
- Venue
- 2020 IEEE International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2020-05
- Pages
- 9441-9447
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W2974989938
- Cited by
- 263
- References
- 32
- FWCI
- 8.92
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.