Improving single-stage object detectors for nighttime pedestrian detection
Open accessAbstract
Improving the reliability of nighttime pedestrian detection is a crucial challenge towards the design of robust autonomous systems. Not surprisingly, most pedestrian fatalities occur in low-illumination settings, thus emphasizing the need for new algorithmic advances. This work presents a novel pedestrian detection approach that makes a number of crucial modifications to the state-of-the-art YOLOV5-PANet architecture, in order to improve the reliability of features extracted from nighttime images. More specifically, the proposed architecture systematically incorporates powerful shuffle attention mechanisms and a transformer module to improve the feature learning pipeline. Instead of advocating the use of other sensing modalities that are better suited for nighttime detection, our approach relies only on conventional RGB cameras and is hence broadly applicable. Our empirical studies with nighttime pedestrian detection benchmarks show that with only minimal increase in model complexity, our approach provides significant improvements in detection efficacy over existing solutions. Finally, we explore the impact of post-hoc network pruning on the speed-accuracy trade-off of our approach and demonstrate that it is well suited for reduced memory/compute requirements.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803 · see citing works
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684 · see citing works
(2016) · Cited by 31,590 · see citing works
Girshick, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.169 · Cited by 28,105 · see citing works
SSD: Single Shot MultiBox Detector
Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., & Berg, A. C. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_2 · Cited by 22,128 · see citing works
YOLO9000: Better, Faster, Stronger
Redmon, J., & Farhadi, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.690 · Cited by 19,892 · see citing works
Advances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,881 · see citing works
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2389824 · Cited by 11,971 · see citing works
YOLOv4: Optimal Speed and Accuracy of Object Detection
Bochkovskiy, A., Wang, C.-Y., & Liao, H.-Y. M. (2020) · arXiv · doi:10.48550/arxiv.2004.10934 · Cited by 10,307 · see citing works
Path Aggregation Network for Instance Segmentation
Liu, S., Qi, L., Qin, H., Shi, J., & Jia, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00913 · Cited by 10,011 · see citing works
CSPNet: A New Backbone that can Enhance Learning Capability of CNN
Wang, C.-Y., Mark Liao, H.-Y., Wu, Y.-H., Chen, P.-Y., Hsieh, J.-W., & Yeh, I.-H. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00203 · Cited by 5,265 · see citing works
Deep Learning for Generic Object Detection: A Survey
Liu, L., Ouyang, W., Wang, X., Fieguth, P., Chen, J., Liu, X., & Pietikäinen, M. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-019-01247-4 · Cited by 2,965 · see citing works
Pruning Filters for Efficient ConvNets
Li, H., Kadav, A., Durdanovic, I., Samet, H., & Graf, H. P. (2016) · arXiv · doi:10.48550/arxiv.1608.08710 · Cited by 2,928 · see citing works
Channel Pruning for Accelerating Very Deep Neural Networks
He, Y., Zhang, X., & Sun, J. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.155 · Cited by 2,647 · see citing works
BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning
Yu, F., Chen, H., Wang, X., Xian, W., Chen, Y., Liu, F., Madhavan, V., & Darrell, T. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00271 · Cited by 2,641 · see citing works
Monocular Pedestrian Detection: Survey and Experiments
Enzweiler, M., & Gavrila, D. M. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.260 · Cited by 1,213 · see citing works
SA-Net: Shuffle Attention for Deep Convolutional Neural Networks
Zhang, Q., & Yang, Y.-B. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414568 · Cited by 1,148 · see citing works
HRank: Filter Pruning Using High-Rank Feature Map
Lin, M., Ji, R., Wang, Y., Zhang, Y., Zhang, B., Tian, Y., & Shao, L. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00160 · Cited by 918 · see citing works
Computer Vision and Pattern Recognition 2020
Akata, Z., Geiger, A., & Sattler, T. (2021) · International Journal of Computer Vision · doi:10.1007/s11263-021-01522-3 · Cited by 646 · see citing works
NetAdapt: Platform-Aware Neural Network Adaptation for Mobile Applications
Yang, T.-J., Howard, A., Chen, B., Zhang, X., Go, A., Sandler, M., Sze, V., & Adam, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01249-6_18 · Cited by 572 · see citing works
Pedestrian Detection at Day/Night Time with Visible and FIR Cameras: A Comparison
González, A., Fang, Z., Socarras, Y., Serrat, J., Vázquez, D., Xu, J., & López, A. M. (2016) · Sensors · doi:10.3390/s16060820 · Cited by 325 · see citing works
On the future of transportation in an era of automated and autonomous vehicles
Hancock, P. A., Nourbakhsh, I., & Stewart, J. (2019) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1805770115 · Cited by 312 · see citing works
Guided Attentive Feature Fusion for Multispectral Pedestrian Detection
Zhang, H., Fromont, É., Lefèvre, S., & Avignon, B. (2021) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv48630.2021.00012 · Cited by 307 · see citing works
Gate Decorator: Global Filter Pruning Method for Accelerating Deep Convolutional Neural NetworksPreprint
You, Z., Yan, K., Ye, J., Ma, M., & Wang, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.08174 · Cited by 200 · see citing works
EagleEye: Fast Sub-net Evaluation for Efficient Neural Network Pruning
Li, B., Wu, B., Su, J., & Wang, G. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58536-5_38 · Cited by 182 · see citing works
Efficient Pedestrian Detection at Nighttime Using a Thermal Camera
Baek, J., Hong, S., Kim, J., et al. (2017) · Sensors · doi:10.3390/s17081850 · Cited by 75 · see citing works
Kim, J. H., Batchuluun, G., & Park, K. R. (2018) · Expert Systems with Applications · doi:10.1016/j.eswa.2018.07.020 · Cited by 71 · see citing works
Multispectral Deep Neural Networks for Pedestrian DetectionPreprint
Liu, J., Zhang, S., Wang, S., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1611.02644 · Cited by 53 · see citing works
Pedestrian Detection Using Multispectral Images and a Deep Neural Network
Nataprawira, J., Gu, Y., Гончаренко, И. А., et al. (2021) · Sensors · doi:10.3390/s21072536 · Cited by 46 · see citing works
A multispectral feature fusion network for robust pedestrian detection
Song, X., Gao, S., & Chen, C. (2020) · Alexandria Engineering Journal · doi:10.1016/j.aej.2020.05.035 · Cited by 45 · see citing works
Near-Infrared-Based Nighttime Pedestrian Detection Using Grouped Part Models
Lee, Y.-S., Chan, Y.-M., Fu, L., et al. (2015) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2014.2385707 · Cited by 36 · see citing works
Deep Learning based Pedestrian Detection at all Light Conditions
Chebrolu, K. N. R., & Kumar, P. N. (2019) · doi:10.1109/iccsp.2019.8698101 · Cited by 33 · see citing works
Low-light pedestrian detection from RGB images using multi-modal knowledge distillation
Kruthiventi, S. S. S., Sahay, P., & Biswal, R. R. (2017) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2017.8297075 · Cited by 30 · see citing works
Ground Plane Context Aggregation Network for Day-and-Night on Vehicular Pedestrian Detection
Xu, Z., Vong, C., Wong, C.-C., et al. (2020) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.2991848 · Cited by 22 · see citing works
A New Region Proposal Network for Far-Infrared Pedestrian Detection
Cao, Z., Yang, H., Zhao, J., et al. (2019) · IEEE Access · doi:10.1109/access.2019.2932749 · Cited by 13 · see citing works
Nighttime Pedestrian Detection Based on Feature Attention and Transformation
Li, G., Zhang, S., & Yang, J. J. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9412889 · Cited by 10 · see citing works
Real-time on-board pedestrian detection using generic single-stage algorithms and on-road databases
Ortiz Castelló, V., del Tejo Catalá, O., Salvador Igual, I., & Perez-Cortes, J.-C. (2020) · International Journal of Advanced Robotic Systems · doi:10.1177/1729881420929175 · Cited by 7 · see citing works
Improving the visibility of nighttime images for pedestrian recognition using in‐vehicle camera
Ogura, R., Nagasaki, T., & Matsubara, H. (2020) · Electronics and Communications in Japan · doi:10.1002/ecj.12268 · Cited by 4 · see citing works
Flagged first, then by citations
Research works printed but not linked: 5
- M. Braun, S. Krebs, F. Flohr and D. M. Gavrila, The eurocity persons dataset: A novel benchmark for object detection, arXiv preprint arXiv:1805.07193 (2018). · arXiv
- A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly et al., An image is worth 16x16 words: Transformers for image recognition at scale, in International Conference on Learning Representations (2020)
- R. Girshick, Fast r-cnn, in Proceedings of the IEEE international conference on com- puter vision (2015) pp. 1440-1448.
- L. Neumann, M. Karg, S. Zhang, C. Scharfenberger, E. Piegert, S. Mistr, O. Proko- fyeva, R. Thiel, A. Vedaldi, A. Zisserman et al., Nightowls: A pedestrians at night dataset, in Asian Conference on Computer Vision (2018) pp. 691-705.
- A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser and I. Polosukhin, Attention is all you need, in Advances in neural information processing systems (2017) pp. 5998-6008.
Cited by
Sorted by citations
Ali, M. L., & Zhang, Z. (2024) · Computers · doi:10.3390/computers13120336 · Cited by 383 · see citing works
- Authors
- Devi, S., Thopalli, K., Malarvezhi, P., & Thiagarajan, J. J.
- Affiliations
- SRM Institute of Science and Technology (IN)
- Arizona State University (US)
- Lawrence Livermore National Laboratory (US)
- Corresponding author
- Thiagarajan, Jayaraman J.
- Venue
- International Journal of Pattern Recognition and Artificial Intelligence
- Publisher
- World Scientific Pub Co Pte Ltd
- Published
- 2022-07
- Volume
- 36
- Issue
- 09
- ISSN
- 0218-0014 (ISSN-L)
- 1793-6381
- OpenAlex
- W4290043875
- Cited by
- 15
- References
- 38
- Citations by year
- 2026: 1 · 2024: 5 · 2023: 9
- FWCI
- 1.15
- Type
- journal article
- Language
- en
- Open access
- green
- www.osti.gov
- Locations
- International Journal of Pattern Recognition and Artificial Intelligence (publishedVersion)
- OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) (submittedVersion)
- Funders
- U.S. Department of Energy Lawrence Livermore National Laboratory (DE-AC52-07NA27344, LLNL-JRNL-832628)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- one-stage object detection, shuffle attention, transformer modules, low-light images, autonomous driving safety, network pruning, speed-accuracy trade-off
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
