Raindrops on Windshield: Dataset and Lightweight Gradient-Based Detection Algorithm
Open accessAbstract
Autonomous vehicles use cameras as one of the primary sources of information about the environment. Adverse weather conditions such as raindrops, snow, mud, and others, can lead to various image artifacts. These artifacts significantly degrade the quality and reliability of the obtained visual data and can lead to accidents if they are not detected in time. This paper presents ongoing work on a new dataset for training and assessing vision algorithms' performance for different tasks of image artifacts detection on either camera lens or car windshield. At the moment, we present a publicly available set of images containing 8190 images, of which 3390 contain raindrops. Images are annotated with the binary mask representing areas with raindrops. We demonstrate the applicability of the dataset in the problems of raindrops presence detection and raindrop region segmentation. To augment the data, we also propose a data augmentation algorithm that allows the generation of synthetic raindrops on images. Apart from the dataset, we present a novel gradient-based algorithm for raindrop presence detection in a video sequence. The experimental evaluation proves that the algorithm reliably detects raindrops. Moreover, compared with the state-of-the-art cross-correlation-based algorithm, the proposed algorithm showed a higher quality of raindrop presence detection and image processing speed, making it applicable for the self-check procedure of real autonomous systems. The dataset is available at github.com/EvoCargo/RaindropsOnWindshield.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Image restoration using inpainting or generative methods carries risks of hallucinated visual cues that may mislead the perception system.
“Other works explored image restoration using inpainting or generative methods [3,4], though such approaches carry risks of hallucinated visual cues that may mislead the perception system.”
From AI-Based Image Quality and Lens Defect Analysis in Autonomous Driving: A Framework with U-Net-Based Soiling Detection (2025) · cites this paper as [3]
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404 · see citing works
Vision meets robotics: The KITTI dataset
Geiger, A., Lenz, P., Stiller, C., & Urtasun, R. (2013) · The International Journal of Robotics Research · doi:10.1177/0278364913491297 · Cited by 9,913 · see citing works
1 year, 1000 km: The Oxford RobotCar dataset
Maddern, W., Pascoe, G., Linegar, C., & Newman, P. (2017) · The International Journal of Robotics Research · doi:10.1177/0278364916679498 · Cited by 1,632 · see citing works
Attentive Generative Adversarial Network for Raindrop Removal from A Single Image
Qian, R., Tan, R. T., Yang, W., Su, J., & Liu, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00263 · Cited by 821 · see citing works
Adherent Raindrop Modeling, Detection and Removal in Video
You, S., Tan, R. T., Kawakami, R., Mukaigawa, Y., & Ikeuchi, K. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2491937 · Cited by 169 · see citing works
Video-based raindrop detection for improved image registration
Roser, M., & Geiger, A. (2009) · 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops · doi:10.1109/iccvw.2009.5457650 · Cited by 126 · see citing works
Multi-class weather classification on single images
Zhang, Z., & Ma, H. (2015) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2015.7351637 · Cited by 68 · see citing works
Camera as weather sensor: Estimating weather information from single images
Chu, W.-T., Zheng, X.-Y., & Ding, D.-S. (2017) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2017.04.002 · Cited by 68 · see citing works
Removing image artifacts due to dirty camera lenses and thin occluders
Gu, J., Ramamoorthi, R., Belhumeur, P. N., & Nayar, S. K. (2009) · ACM Transactions on Graphics · doi:10.1145/1618452.1618490 · Cited by 65 · see citing works
Detection of unfocused raindrops on a windscreen using low level image processing
Nashashibi, F., de Charrette, R., & Lia, A. (2010) · 2010 11th International Conference on Control Automation Robotics & Vision · doi:10.1109/icarcv.2010.5707398 · Cited by 38 · see citing works
Real-time raindrop detection based on cellular neural networks for ADAS
Machot, F. A., Ali, M., Mosa, A. H., Schwarzlmüller, C., Gutmann, M., & Kyamakya, K. (2016) · Journal of Real-Time Image Processing · doi:10.1007/s11554-016-0569-z · Cited by 34 · see citing works
Towards rain detection through use of in-vehicle multipurpose cameras
Cord, A., & Aubert, D. (2011) · 2011 IEEE Intelligent Vehicles Symposium (IV) · doi:10.1109/ivs.2011.5940484 · Cited by 25 · see citing works
Detection of camera artifacts from camera images
Einecke, N., Gandhi, H. K., & Deigmoeller, J. (2014) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2014.6957756 · Cited by 17 · see citing works
Hirohashi, Y., Narioka, K., Suganuma, M., Liu, X., Tamatsu, Y., & Okatani, T. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00115 · Cited by 8 · see citing works
Visibility Loss Detection for Video Camera Using Deep Convolutional Neural Networks
Ivanov, A. I., & Yudin, D. A. (2018) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-01818-4_43 · Cited by 5 · see citing works
Gray spot detection in surveillance video using convolutional neural network
Hu, L., Chen, L., & Cheng, J. S. (2018) · IEEE Conference on Industrial Electronics and Applications (ICIEA) · doi:10.1109/iciea.2018.8398187 · Cited by 4 · see citing works
X-NET For Single Image Raindrop Removal
Lin, J., & Dai, L. (2020) · Proceedings - International Conference on Image Processing · doi:10.1109/icip40778.2020.9191073 · Cited by 4 · see citing works
Lens adhering contaminant detection using spatio-temporal blur
Akkala, V., Parikh, P., Mahesh, B. L., Deshmukh, A. S., & Medasani, S. S. (2016) · International Conference on Signal Processing and Communications (SPCOM) · doi:10.1109/spcom.2016.7746664 · Cited by 3 · see citing works
Convolutional Neural Network for Raindrop Detection
Bae, I.-G., Bang, S., & Kim, W. (2019) · International Conference on Ubiquitous Robots (UR) · doi:10.1109/urai.2019.8768613 · Cited by 3 · see citing works
Adherent Mist and Raindrop Removal from a Single Image Using Attentive Convolutional NetworkPreprint
He, D., Shang, X., & Luo, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.01466 · Cited by 3 · see citing works
Flagged first, then by citations
1 of the 21 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
NTIRE 2025 Challenge on Day and Night Raindrop Removal for Dual-Focused Images: Methods and Results
Li, X., Jin, Y., Jin, X., Wu, Z., Li, B., Wang, Y., Yang, W., Li, Y., Chen, Z., Wen, B., et al. (2025) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw67362.2025.00111 · Cited by 31 · see citing works
Feature-Aligned Video Raindrop Removal With Temporal Constraints
Yan, W., Xu, L., Yang, W., & Tan, R. T. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3170726 · Cited by 20 · see citing works
Panoptic Water Surface Visual Perception for USVs Using Monocular Camera Sensor
Xu, H., Zhang, X., He, J., Geng, Z., Yu, Y. F., & Cheng, Y. (2024) · IEEE Sensors Journal · doi:10.1109/jsen.2024.3413088 · Cited by 12 · see citing works
Raindrop Clarity: A Dual-Focused Dataset for Day and Night Raindrop Removal
Jin, Y., Li, X., Wang, J., Zhang, Y., & Zhang, M. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72658-3_1 · Cited by 12 · see citing works
Deep Learning Strategies for Analysis of Weather-Degraded Optical Sea Ice Images
Panchi, N., & Kim, E. (2024) · IEEE Sensors Journal · doi:10.1109/jsen.2024.3376518 · Cited by 10 · see citing works
Automotive DNN-Based Object Detection in the Presence of Lens Obstruction and Video Compression
Baris, G., Li, B., Chan, P. H., Avizzano, C. A., & Donzella, V. (2025) · IEEE Access · doi:10.1109/access.2025.3544773 · Cited by 7 · see citing works
Lee, C., Schätzle, S., Lang, S. A., & Oksanen, T. (2024) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2024.109622 · Cited by 6 · see citing works
Eberhardt, T. D., Brühl, T., Schwager, R., Sohn, T. S., & Stork, W. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00507 · Cited by 2 · see citing works
DSBEV: Docking Space Segmentation for Autonomous Surface Vehicle in Bird's Eye View
Pang, C., Zhang, X., Chen, X., Xu, H., Yu, Y. F., & Cheng, Y. (2025) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2025.3567389 · Cited by 2 · see citing works
Chernyshova, Y. S., Sheshkus, A. V., Bulatov, K. B., & Arlazarov, V. V. (2023) · Pattern Recognition and Image Analysis · doi:10.1134/s1054661823040107 · Cited by 2 · see citing works
Road Target Detection in Different Weather Conditions Based on Deep Learning
Yang, Z., Zhao, Y., Li, B., & Guo, L. (2024) · IEEJ Transactions on Electrical and Electronic Engineering · doi:10.1002/tee.24153 · Cited by 2 · see citing works
Wu, P., Kong, M., Zhang, H., Fathollahi‐Fard, A. M., & Yaseen, Z. M. (2024) · Annals of Operations Research · doi:10.1007/s10479-024-06410-w · Cited by 3 · see citing works
Discussion on Redundant Processing Algorithm of Association Rules Based on Hypergraph in Data Mining
Zhu, J. (2022) · Journal of Robotics · doi:10.1155/2022/1007464 · Cited by 2 · see citing works
RetractedEberhardt, T. D., Brühl, T., Schwager, R., Sohn, T. S., & Stork, W. (2024) · IEEE International Conference on Vehicular Electronics and Safety (ICVES) · doi:10.1109/icves61986.2024.10927915 · Cited by 1 · see citing works
Kuznetsov, V. I., & Yudin, D. A. (2022) · Optical Memory and Neural Networks · doi:10.3103/s1060992x22030043 · Cited by 1 · see citing works
O’g’li, A. (2025) · American Journal of Mechanics and Applications · doi:10.11648/j.ajma.20251204.14
Anderson, A., Olafarson, E. J., Andersen, P., & Noori, N. S. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-21441-7_14
Computer Vision-based Rain-sensing Windshield Wiper System
Hong, W.-K., Wi, J.-W., Jeong, D.-W., Kim, M.-W., Im, S., & Son, K. J. (2023) · The Journal of Korean Institute of Information Technology · doi:10.14801/jkiit.2023.21.3.117
Analysis of Faster R-CNN network prediction in the presence of lens occlusion and video compressionPreprint
Baris, G., Li, B., Chan, P. H., Avizzano, C. A., & Donzella, V. (2023) · doi:10.36227/techrxiv.23047412.v1
Analysis of Faster R-CNN network prediction in the presence of lens occlusion and video compressionPreprint
Baris, G., Li, B., Chan, P. H., Avizzano, C. A., & Donzella, V. (2023) · doi:10.36227/techrxiv.23047412
Beyond the Smudge: Simulating Opaque and Transparent Automotive Camera Lens Soiling
Eberhardt, T. D., Brühl, T., Schwager, R., Sohn, T. S., & Stork, W. (2025) · IEEE Symposia on Computational Intelligence for Energy, Transport and Environmental Sustainability (CIETES) · doi:10.1109/cietes63869.2025.10995030
A Real-Time Drone Image Processing System using Deep Learning and Big Data Technology
Anh-Tuan Nguyen, D., & Do, T.-H. (2024) · RIVF International Conference on Computing and Communication Technologies (RIVF) · doi:10.1109/rivf64335.2024.11009059
Lee, C., Schätzle, S., Lang, S. A., & Oksanen, T. (2025) · Smart Agricultural Technology · doi:10.1016/j.atech.2025.101102
A Real-Time Deraining Network with Rain Feature Perception for Autonomous Driving
Yin, S. W., & Liu, H. (2025) · International Conference on Machine Learning and Intelligent Systems Engineering (MLISE) · doi:10.1109/mlise66443.2025.11100248
Lee, C., Schätzle, S., Lang, S. A., Maier, M., & Oksanen, T. (2025) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra55743.2025.11128195
Go, Y.-H., & Lee, S.-H. (2025) · Mathematics · doi:10.3390/math13193176
Cheng, X., Verma, P., Slama, D., & Lachenmaier, J. F. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11227586
Rainy-nuScenes: A Data Partition for Benchmarking Contaminated Vehicle Cameras through Rain
Eberhardt, T. D., Klingler, M., Brühl, T., Schwager, R., Sohn, T. S., Schneider, S.-A., & Stork, W. (2025) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc58881.2025.11342952
Unified Removal of Raindrops and Reflections: A New Benchmark and A Novel Pipeline
Liu, X., He, Z., Chen, Y. F., Zhu, C., Chen, Z., Luo, X., & Lu, Z.-M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37026-6_36
RD-ROSL: A Real-World Dataset and Restoration Method for Obstructed Smartphone Lenses
Sakib, N., Hossain, D. T., & Kheya, T. H. (2025) · International Conference on Computer and Information Technology (ICCIT) · doi:10.1109/iccit68739.2025.11491453
- Authors
- Soboleva, V., & Shipitko, O.
- Affiliations
- Moscow Institute of Physics and Technology (RU)
- Institute for Information Transmission Problems (RU)
- Venue
- 2021 IEEE Symposium Series on Computational Intelligence (SSCI)
- Publisher
- IEEE
- Published
- 2021-12-05
- Pages
- 1-7
- OpenAlex
- W3158989573
- Cited by
- 30
- References
- 21
- Citations by year
- 2026: 1 · 2025: 13 · 2024: 8 · 2023: 4 · 2022: 4
- FWCI
- 3.96
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Symposium Series on Computational Intelligence (SSCI) (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Enhancement Techniques
- Keywords
- raindrop detection, autonomous vehicles, adverse weather, windshields, synthetic data augmentation, cross-correlation comparison
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
