The DOI matches its Crossref record; 59 of the 59 references it cites were found in a registry or an index.
Thermal Object Detection in Difficult Weather Conditions Using YOLO
Open accessAbstract
Global terrorist threats and illegal migration have intensified concerns for the security of citizens, and every effort is made to exploit all available technological advances to prevent adverse events and protect people and their property. Due to the ability to use at night and in weather conditions where RGB cameras do not perform well, thermal cameras have become an important component of sophisticated video surveillance systems. In this paper, we investigate the task of automatic person detection in thermal images using convolutional neural network models originally intended for detection in RGB images. We compare the performance of the standard state-of-the-art object detectors such as Faster R-CNN, SSD, Cascade R-CNN, and YOLOv3, that were retrained on a dataset of thermal images extracted from videos that simulate illegal movements around the border and in protected areas. Videos are recorded at night in clear weather, rain, and in the fog, at different ranges, and with different movement types. YOLOv3 was significantly faster than other detectors while achieving performance comparable with the best, so it was used in further experiments. We experimented with different training dataset settings in order to determine the minimum number of images needed to achieve good detection results on test datasets. We achieved excellent detection results with respect to average accuracy for all test scenarios although a modest set of thermal images was used for training. We test our trained model on different well known and widely used thermal imaging datasets as well. In addition, we present the results of the recognition of humans and animals in thermal images, which is particularly important in the case of sneaking around objects and illegal border crossings. Also, we present our original thermal dataset used for experimentation that contains surveillance videos recorded at different weather and shooting conditions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138
(2016) · Cited by 31,590
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105
YOLO9000: Better, Faster, Stronger
Redmon, J., & Farhadi, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.690 · Cited by 19,892
The pascal visual object classes (VOC) challenge
Everingham, M., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-009-0275-4 · Cited by 19,599
Rapid object detection using a boosted cascade of simple features
Viola, P., & Jones, M. (2005) · Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001 · doi:10.1109/cvpr.2001.990517 · Cited by 18,300
A Multi-Modal Distributed Real-Time IoT System for Urban Traffic Control (Invited Paper)
Khanam, Z., Achari, V. P. S., Boukhennoufa, I., Jindal, A., & Singh, A. K. (2024) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/oasics.ng-res.2024.2 · Cited by 14,040
Cascade R-CNN: Delving Into High Quality Object Detection
Cai, Z., & Vasconcelos, N. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00644 · Cited by 7,370
FCOS: Fully Convolutional One-Stage Object Detection
Tian, Z., Shen, C., Chen, H., & He, T. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00972 · Cited by 7,013
Multispectral pedestrian detection: Benchmark dataset and baseline
Hwang, S., Park, J., Kim, N., Choi, Y., & Kweon, I. S. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298706 · Cited by 1,249
A Robust Real-Time Automatic License Plate Recognition Based on the YOLO Detector
Laroca, R., Severo, E., Zanlorensi, L. A., Oliveira, L. S., Goncalves, G. R., Schwartz, W. R., & Menotti, D. (2018) · 2018 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2018.8489629 · Cited by 602
Pedestrian Detection at Day/Night Time with Visible and FIR Cameras: A Comparison
González, A., Fang, Z., Socarras, Y., Serrat, J., Vázquez, D., Xu, J., & López, A. (2016) · Sensors · doi:10.3390/s16060820 · Cited by 322
A Two-Stage Template Approach to Person Detection in Thermal Imagery
Davis, J. W., & Keck, M. A. (2005) · 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1 · doi:10.1109/acvmot.2005.14 · Cited by 320
Deep convolutional neural networks for thermal infrared object tracking
Liu, Q., Lu, X., He, Z., Zhang, C., & Chen, W.-S. (2017) · Knowledge-Based Systems · doi:10.1016/j.knosys.2017.07.032 · Cited by 239
Bag of soft biometrics for person identification: New trends and challenges
Dantcheva, A., Velardo, C., D’Angelo, A., & Dugelay, J.-L. (2011) · Multimedia Tools and Applications · doi:10.1007/s11042-010-0635-7 · Cited by 221
Yolo Real Time Object Detection
Chaudhari, S., Malkan, N., Momin, A., & Bonde, M. (2020) · International Journal of Computer Trends and Technology · doi:10.14445/22312803/ijctt-v68i6p112 · Cited by 186
A Thermal Infrared Video Benchmark for Visual Analysis
Wu, Z., Fuller, N., Theriault, D., & Betke, M. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2014.39 · Cited by 179
People detection and tracking from aerial thermal views
Portmann, J., Lynen, S., Chli, M., & Siegwart, R. (2014) · 2014 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2014.6907094 · Cited by 174
Efficient Night Gait Recognition Based on Template Matching
Tan, D., Huang, K., Yu, S., & Tan, T. (2006) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2006.478 · Cited by 162
Flame detection using deep learning
Shen, D., Chen, X., Nguyen, M., & Yan, W. Q. (2018) · International Conference on Control, Automation and Robotics (ICCAR) · doi:10.1109/iccar.2018.8384711 · Cited by 157
Ramachandran, S., George, J., Skaria, S., & Varun, V. V. (2018) · SPIE Medical Imaging: Computer-Aided Diagnosis · doi:10.1117/12.2293699 · Cited by 141
Human Detection in Thermal Imaging Using YOLO
Ivašić-Kos, M., Krišto, M., & Pobar, M. (2019) · International Conference on Computer Technology Applications (ICCTA) · doi:10.1145/3323933.3324076 · Cited by 129
Self-citationThe Thermal Infrared Visual Object Tracking VOT-TIR2015 Challenge Results
Felsberg, M., Berg, A., Hager, G., Ahlberg, J., Kristan, M., Matas, J., Leonardis, A., Zajc, L. Č., Fernandez, G., Vojır, T., et al. (2015) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2015.86 · Cited by 122
A New Metric for Evaluating Semantic Segmentation: Leveraging Global and Contour Accuracy
Fernández-Moral, E., Martins, R., Wolf, D. F., & Rives, P. (2018) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2018.8500497 · Cited by 121
Pedestrian Detection in Thermal Images Using Saliency Maps
Ghose, D., Desai, S., Bhattacharya, S., Chakraborty, D., Fiterau, M., & Rahman, T. (2019) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00130 · Cited by 116
InfAR dataset: Infrared action recognition at different times
Gao, C., Du, Y., Liu, J., Lv, J. G., Yang, L., Meng, D., & Hauptmann, A. G. (2016) · Neurocomputing · doi:10.1016/j.neucom.2016.05.094 · Cited by 111
Zhang, X., Yang, W., Tang, X., & Liu, J. (2018) · Sensors · doi:10.3390/s18124308 · Cited by 92
Thermal–visible registration of human silhouettes: A similarity measure performance evaluation
Bilodeau, G.-A., Torabi, A., St-Charles, P.-L., & Riahi, D. (2014) · Infrared Physics & Technology · doi:10.1016/j.infrared.2014.02.005 · Cited by 74
Ball Detection Using Yolo and Mask R-CNN
Burić, M., Pobar, M., & Ivašić-Kos, M. (2018) · International Conference on Computational Science and Computational Intelligence (CSCI) · doi:10.1109/csci46756.2018.00068 · Cited by 72
Self-citationThe use of infrared thermography to detect the skin temperature response to physical activity
Tanda, G. (2015) · Journal of Physics Conference Series · doi:10.1088/1742-6596/655/1/012062 · Cited by 70
Multispectral pedestrian detection based on deep convolutional neural networks
Hou, Y.-L., Song, Y., Hao, X., Shen, Y., Qian, M., & Chen, H. (2018) · Infrared Physics & Technology · doi:10.1016/j.infrared.2018.08.029 · Cited by 70
Tracking human faces in infrared video
Eveland, C. K., Socolinsky, D. A., & Wolff, L. B. (2003) · Image and Vision Computing · doi:10.1016/s0262-8856(03)00056-8 · Cited by 67
CNN-based thermal infrared person detection by domain adaptation
Beyerer, J., Ruf, M., & Herrmann, C. (2018) · Autonomous Systems: Sensors, Processing, and Security for Ground, Air, Sea, and Space Vehicles and Infrastructure · doi:10.1117/12.2304400 · Cited by 65
Detection of Airplanes on the Ground Using YOLO Neural Network
Kharchenko, V. P., & Chyrka, I. (2018) · International Conference on Mathematical Methods in Electromagnetic Theory · doi:10.1109/mmet.2018.8460392 · Cited by 62
A novel infrared video surveillance system using deep learning based techniques
Zhang, H., Luo, C., Wang, Q., Kitchin, M. R., Parmley, A., Monge-Álvarez, J., & Casaseca‐de‐la‐Higuera, P. (2018) · Multimedia Tools and Applications · doi:10.1007/s11042-018-5883-y · Cited by 61
An overview of thermal face recognition methods
Krišto, M., & Ivašić-Kos, M. (2018) · International Convention on Information and Communication Technology, Electronics and Microelectronics · doi:10.23919/mipro.2018.8400200 · Cited by 58
Thermal image-based CNN's for ultra-low power people recognition
Gomez, A., Conti, F., & Benini, L. (2018) · ACM International Conference on Computing Frontiers (CF) · doi:10.1145/3203217.3204465 · Cited by 48
Multispectral pedestrian detection based on deep convolutional neural networks
Hou, Y.-L., Song, Y., Hao, X., Shen, Y., & Qian, M. (2017) · IEEE International Conference on Signal Processing, Communications and Computing (ICSPCC) · doi:10.1109/icspcc.2017.8242507 · Cited by 47
A survey on human detection surveillance systems for Raspberry Pi
Khalifa, A. F., Badr, E., & Elmahdy, H. N. (2019) · Image and Vision Computing · doi:10.1016/j.imavis.2019.02.010 · Cited by 47
Near infrared nighttime road pedestrians recognition based on convolutional neural network
Dai, X., Duan, Y.-X., Hu, J., Liu, S., Hu, C., He, Y., Chen, D., Luo, C., & Meng, J.-Q. (2018) · Infrared Physics & Technology · doi:10.1016/j.infrared.2018.11.028 · Cited by 46
Adapting YOLO Network for Ball and Player Detection
Burić, M., Pobar, M., & Ivašić-Kos, M. (2019) · International Conference on Pattern Recognition Applications and Methods · doi:10.5220/0007582008450851 · Cited by 42
Self-citationThermal Imaging Dataset for Person Detection
Krišto, M., & Ivašić-Kos, M. (2019) · International Convention on Information and Communication Technology, Electronics and Microelectronics · doi:10.23919/mipro.2019.8757208 · Cited by 41
Recognizing pedestrian’s unsafe behaviors in far-infrared imagery at night
Lee, E., Ko, B. C., & Nam, J.-Y. (2016) · Infrared Physics & Technology · doi:10.1016/j.infrared.2016.03.006 · Cited by 37
Yu, T., Mo, B., Liu, F., Qi, H., & Liu, Y. (2019) · Infrared Physics & Technology · doi:10.1016/j.infrared.2019.02.012 · Cited by 34
Deep residual infrared action recognition by integrating local and global spatio-temporal cues
Imran, J., & Raman, B. (2019) · Infrared Physics & Technology · doi:10.1016/j.infrared.2019.103014 · Cited by 34
Domain Adaptation for Privacy-Preserving Pedestrian Detection in Thermal Imagery
Kieu, M., Bagdanov, A. D., Bertini, M., & Bimbo, A. D. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-30645-8_19 · Cited by 34
Ivašić-Kos, M., Pobar, M., & Ribarič, S. (2015) · Pattern Recognition · doi:10.1016/j.patcog.2015.10.017 · Cited by 31
Self-citationBuilding a labeled dataset for recognition of handball actions using mask R-CNN and STIPS
Ivašić-Kos, M., & Pobar, M. (2018) · European Workshop on Visual Information Processing · doi:10.1109/euvip.2018.8611642 · Cited by 24
Self-citationClassifying objects in LWIR imagery via CNNs
Rodger, I., Connor, B., & Robertson, N. M. (2016) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.2241858 · Cited by 21
Person Detection in Thermal Videos Using YOLO
Ivašić-Kos, M., Krišto, M., & Pobar, M. (2019) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-29513-4_18 · Cited by 21
Self-citationHUMAN DETECTION BASED ON A SEQUENCE OF THERMAL IMAGES USING DEEP LEARNING
Wang, X., & Hosseinyalamdary, S. (2019) · The international archives of the photogrammetry, remote sensing and spatial information sciences/International archives of the photogrammetry, remote sensing and spatial information sciences · doi:10.5194/isprs-archives-xlii-2-w13-127-2019 · Cited by 18
An overview of action recognition in videos
Burić, M., Pobar, M., & Ivašić-Kos, M. (2017) · International Convention on Information and Communication Technology, Electronics and Microelectronics · doi:10.23919/mipro.2017.7973588 · Cited by 13
Self-citationReal-Time Implementation of Human Detection in Thermal Imagery Based on CNN
Shahid, N., Yu, G.-H., Trinh, T. D., Sin, D.-S., & Kim, J. Y. (2019) · The Journal of Korean Institute of Information Technology · doi:10.14801/jkiit.2019.17.1.107 · Cited by 10
Cited by
Vision-based navigation and guidance for agricultural autonomous vehicles and robots: A review
Bai, Y., Zhang, B., Xu, N., Zhou, J., Shi, J., & Diao, Z. (2023) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2022.107584 · Cited by 353
YOLO-FIRI: Improved YOLOv5 for Infrared Image Object Detection
Li, S., Li, Y., Li, Y., Li, M., & Xu, X. (2021) · IEEE Access · doi:10.1109/access.2021.3120870 · Cited by 300
Realtime Vehicle Tracking Method Based on YOLOv5 + DeepSORT
Lin, L., He, H., Xu, Z., & Wu, D. (2023) · Computational Intelligence and Neuroscience · doi:10.1155/2023/7974201 · Cited by 18
Edge AI System Using a Thermal Camera for Industrial Anomaly Detection
Oliveira, V. M., & Moreira, A. H. J. (2022) · Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-031-06371-8_12 · Cited by 10
Metadata
- Authors
- Kristo, M., Ivasic-Kos, M., & Pobar, M.
- Venue
- IEEE Access
- Published
- 2020
- Volume
- 8
- Pages
- 125459-125476
- ISSN
- 2169-3536 (ISSN-L)
- OpenAlex
- W3038835693
- Cited by
- 309
- References
- 59
- FWCI
- 14.99
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.