Exploring Sensor Impact and Architectural Robustness in Adverse Weather on BEV Perception

“Uricar et al. introduced the Desoiling Dataset [43] with paired clean and soiled fisheye videos, and applied CycleGAN-based [44] restoration to improve segmentation using ResNet50 [45] and FCN8 [46].”

“The model utilises Vision Mamba (ViM) [60] for image encoding and a deep Q-network (DQN) [61] to learn scene-consistent fusion weights, achieving high detection accuracy under fog and snow on the nuScenes and Radiate datasets [62].”

“To quantify the effect of their perceptual quality on reproducibility, we measured the mean Structural Similarity Index (SSIM) [75] between clean and occluded images (5 k random samples per camera).”

“Uricar et al. introduced the Desoiling Dataset [43] with paired clean and soiled fisheye videos, and applied CycleGAN-based [44] restoration to improve segmentation using ResNet50 [45] and FCN8 [46].”

“…et al. investigated the adversarial robustness of real-time semantic segmentation models by proposing [37] a pipeline for generating targeted and untargeted adversarial patch attacks, evaluated on ICNet [38], BiSeNet [39], and DDR-Net [40] across CARLA [41], Cityscapes [42], and real-world data.”

“We simulate occlusion scenarios based on environmental conditions using the nuScenes dataset [23], and evaluate their impact on vehicle and map segmentation tasks in BEV space.”

“Nawaz et al. proposed a hybrid Red, Green, Blue (RGB), thermal, and LiDAR fusion system [5] using CNNs [54], YOLOv8 [55], and PointPillars [56], achieving robust detection in fog, rain, and nighttime on the FLIR [57], TarDAL [58], and CUHK [59] datasets.”

“…et al. investigated the adversarial robustness of real-time semantic segmentation models by proposing [37] a pipeline for generating targeted and untargeted adversarial patch attacks, evaluated on ICNet [38], BiSeNet [39], and DDR-Net [40] across CARLA [41], Cityscapes [42], and real-world data.”

“Brophy et al. [52] analysed rain’s impact on object detection using BDD100k [53].”

“Nawaz et al. proposed a hybrid Red, Green, Blue (RGB), thermal, and LiDAR fusion system [5] using CNNs [54], YOLOv8 [55], and PointPillars [56], achieving robust detection in fog, rain, and nighttime on the FLIR [57], TarDAL [58], and CUHK [59] datasets.”

“…et al. investigated the adversarial robustness of real-time semantic segmentation models by proposing [37] a pipeline for generating targeted and untargeted adversarial patch attacks, evaluated on ICNet [38], BiSeNet [39], and DDR-Net [40] across CARLA [41], Cityscapes [42], and real-world data.”

“Nawaz et al. proposed a hybrid Red, Green, Blue (RGB), thermal, and LiDAR fusion system [5] using CNNs [54], YOLOv8 [55], and PointPillars [56], achieving robust detection in fog, rain, and nighttime on the FLIR [57], TarDAL [58], and CUHK [59] datasets.”

“We used the official pre-trained checkpoints released by the authors of SimpleBEV [24], BEVFusion [25], and BEVCar [26].”

“For camera-only BEV segmentation, we compare the fusion architectures with existing baselines such as OFT [84], LSS [27], CVT [29], and MMTRaP [30], reporting vehicle IoU scores ranging from 30.1% to 37.9% under clean sensor conditions.”

“After lifting, the image and RADAR aligned features are fused through a cross-attention mechanism inspired by TransFu-sion [83], where RADAR features serve as queries and camera features as keys and values.”

“…et al. investigated the adversarial robustness of real-time semantic segmentation models by proposing [37] a pipeline for generating targeted and untargeted adversarial patch attacks, evaluated on ICNet [38], BiSeNet [39], and DDR-Net [40] across CARLA [41], Cityscapes [42], and real-world data.”

“Teresa et al. developed a deep learning-based weather classification framework [31] using transfer learning with MobileNetV2 [32], DenseNet121 [33], and VGG-16 [34] to classify six weather types on the DAWN2020 [35] and MCWRD2018 [36] datasets.”

“Teresa et al. developed a deep learning-based weather classification framework [31] using transfer learning with MobileNetV2 [32], DenseNet121 [33], and VGG-16 [34] to classify six weather types on the DAWN2020 [35] and MCWRD2018 [36] datasets.”

“Automated driving technology has made significant advances, enabling vehicles to operate with increasing independence and less oversight in diverse and complex environments [1], [2].”

“Existing studies include both camera-only and multi-sensor models, explored in both image space and BEV representations; however, most do not address the effects of sensor occlusion [28], [29].”

“Uricar et al. proposed DirtyGAN [51], which introduces synthetic soiling generation to reduce annotation effort and boost detection accuracy, supported by the WoodScape dataset [47].”

“The model utilises Vision Mamba (ViM) [60] for image encoding and a deep Q-network (DQN) [61] to learn scene-consistent fusion weights, achieving high detection accuracy under fog and snow on the nuScenes and Radiate datasets [62].”

“Third, we introduce angular occlusion by removing LiDAR points within a specified angular sector of the point cloud, simulating degradation from obstructions that block certain directions in the field of view [78].”

“By fusing these sensor inputs, automated vehicles aim to achieve reliable perception for object detection [8], tracking [9], segmentation [10], lane marking [11], depth estimation [12], and motion planning by leveraging the relative strengths of each sensing modality [13].”

“We used the official pre-trained checkpoints released by the authors of SimpleBEV [24], BEVFusion [25], and BEVCar [26].”

“For the camera and RADAR fusion, we compare the fusion architectures against clean-sensor baselines such as MM-TraP [30] and CRN [85], which achieve vehicle IoU scores up to 58.8% and mIoU scores ranging from 53.8% to 70.5%.”

“Teresa et al. developed a deep learning-based weather classification framework [31] using transfer learning with MobileNetV2 [32], DenseNet121 [33], and VGG-16 [34] to classify six weather types on the DAWN2020 [35] and MCWRD2018 [36] datasets.”

“By fusing these sensor inputs, automated vehicles aim to achieve reliable perception for object detection [8], tracking [9], segmentation [10], lane marking [11], depth estimation [12], and motion planning by leveraging the relative strengths of each sensing modality [13].”

“Automated driving technology has made significant advances, enabling vehicles to operate with increasing independence and less oversight in diverse and complex environments [1], [2].”

“For instance, Uricar et al. [18] and Yahiaoui et al. [19] explored opaque and transparent soiling detection using image-level annotations, while Nesti et al. [20] investigated adversarial patch attacks for semantic segmentation.”

“Physics-based approaches such as LISA [79], Teufel et al. [80], and Yan et al. [77] model light scattering and attenuation under adverse weather (rain, fog, snow) to improve training robustness for detection tasks.”

“…weather or low-light conditions, and cannot provide accurate depth information [5]; RADAR is robust in poor visibility but lacks fine spatial detail [6]; and LiDAR delivers accurate 3D spatial information but can be affected by weather conditions, reflec-tive surfaces, or extreme temperatures [7].”

“Teresa et al. developed a deep learning-based weather classification framework [31] using transfer learning with MobileNetV2 [32], DenseNet121 [33], and VGG-16 [34] to classify six weather types on the DAWN2020 [35] and MCWRD2018 [36] datasets.”

“Third, we add Gaussian noise to the RADAR data to reflect increased measurement uncertainty, as commonly observed in cluttered urban environments or under radio-frequency interference [76].”

“By fusing these sensor inputs, automated vehicles aim to achieve reliable perception for object detection [8], tracking [9], segmentation [10], lane marking [11], depth estimation [12], and motion planning by leveraging the relative strengths of each sensing modality [13].”

“These systems rely on multiple complementary sensors: camera, Radio Detection And Ranging (RADAR), and Light Detection and Ranging (LiDAR), to perceive their surroundings [3], [4].”

“This vulnerability poses a critical risk to safety, as occlusions may lead to perception failures and poor decision-making [17].”

“Existing studies include both camera-only and multi-sensor models, explored in both image space and BEV representations; however, most do not address the effects of sensor occlusion [28], [29].”

“Rossolini et al. investigated the adversarial robustness of real-time semantic segmentation models by proposing [37] a pipeline for generating targeted and untargeted adversarial patch attacks, evaluated on ICNet [38], BiSeNet [39], and DDR-Net [40] across CARLA [41], Cityscapes [42], and…”

“Uricar et al. proposed DirtyGAN [51], which introduces synthetic soiling generation to reduce annotation effort and boost detection accuracy, supported by the WoodScape dataset [47].”

“Other studies focus on contamination of the LiDAR cover, where dust, dirt, or water droplets on the lens can distort or block returns, with approaches developed for fault detection and signal modelling [81], [82].”

“…weather or low-light conditions, and cannot provide accurate depth information [5]; RADAR is robust in poor visibility but lacks fine spatial detail [6]; and LiDAR delivers accurate 3D spatial information but can be affected by weather conditions, reflec-tive surfaces, or extreme temperatures [7].”

“Physics-based approaches such as LISA [79], Teufel et al. [80], and Yan et al. [77] model light scattering and attenuation under adverse weather (rain, fog, snow) to improve training robustness for detection tasks.”

“First, to mimic adverse weather such as fog and rain, we include random point dropout, a commonly used corruption where scattering and absorption reduce point density and effective sensing range [77].”

“Uricar et al. introduced the Desoiling Dataset [43] with paired clean and soiled fisheye videos, and applied CycleGAN-based [44] restoration to improve segmentation using ResNet50 [45] and FCN8 [46].”

“We used the official pre-trained checkpoints released by the authors of SimpleBEV [24], BEVFusion [25], and BEVCar [26].”

“Other studies focus on contamination of the LiDAR cover, where dust, dirt, or water droplets on the lens can distort or block returns, with approaches developed for fault detection and signal modelling [81], [82].”

“On the mono-sensor side, Zhang et [72], integrating camera and LiDAR features for HD map construction.”

“Building on the SoilingNet approach, Das et al. extended this work through TiledSoilingNet [50], which performs tile-level soiling detection and supports embedded multi-task deployment.”

“Nawaz et al. proposed a hybrid Red, Green, Blue (RGB), thermal, and LiDAR fusion system [5] using CNNs [54], YOLOv8 [55], and PointPillars [56], achieving robust detection in fog, rain, and nighttime on the FLIR [57], TarDAL [58], and CUHK [59] datasets.”

“By fusing these sensor inputs, automated vehicles aim to achieve reliable perception for object detection [8], tracking [9], segmentation [10], lane marking [11], depth estimation [12], and motion planning by leveraging the relative strengths of each sensing modality [13].”

“Yahiaoui et al. [19] addressed glare detection under extreme brightness with thresholding and CNN-based methods [49].”

“Nawaz et al. proposed a hybrid Red, Green, Blue (RGB), thermal, and LiDAR fusion system [5] using CNNs [54], YOLOv8 [55], and PointPillars [56], achieving robust detection in fog, rain, and nighttime on the FLIR [57], TarDAL [58], and CUHK [59] datasets.”

“Only a few works, RoboBEV and MSA-Fusion [21], [22] have investigated occlusion effects in the BEV domain.”

“Yahiaoui et al. [19] addressed glare detection under extreme brightness with thresholding and CNN-based methods [49].”

“In addition, Kumar et al. investigated [64] the effects of camera occlusion on BEV vehicle segmentation using multi-sensor fusion with SimpleBEV architecture.”

“The three severity levels (0.1, 0.2, 0.3) were generated by tuning the opacity parameter in the camera obstruction repository [74].”

“These systems rely on multiple complementary sensors: camera, Radio Detection And Ranging (RADAR), and Light Detection and Ranging (LiDAR), to perceive their surroundings [3], [4].”

“As shown in Table 9, baseline methods such as MMTraP [30] and BEVMOSNet [10], evaluated under full sensor availability, achieve strong results with vehicle IoU scores of 63.2% and 61.8%, respectively, and an mIoU score of 64.6% from MMTraP.”

“By fusing these sensor inputs, automated vehicles aim to achieve reliable perception for object detection [8], tracking [9], segmentation [10], lane marking [11], depth estimation [12], and motion planning by leveraging the relative strengths of each sensing modality [13].”

“Brophy et al. [52] analysed rain’s impact on object detection using BDD100k [53].”

“By fusing these sensor inputs, automated vehicles aim to achieve reliable perception for object detection [8], tracking [9], segmentation [10], lane marking [11], depth estimation [12], and motion planning by leveraging the relative strengths of each sensing modality [13].”

“Teresa et al. developed a deep learning-based weather classification framework [31] using transfer learning with MobileNetV2 [32], DenseNet121 [33], and VGG-16 [34] to classify six weather types on the DAWN2020 [35] and MCWRD2018 [36] datasets.”

“Nawaz et al. proposed a hybrid Red, Green, Blue (RGB), thermal, and LiDAR fusion system [5] using CNNs [54], YOLOv8 [55], and PointPillars [56], achieving robust detection in fog, rain, and nighttime on the FLIR [57], TarDAL [58], and CUHK [59] datasets.”

“The tasks include: 1) vehicle segmentation [21], [19]; 2) map segmentation [19]; and 3) 3-”

“Table 2 summarizes the degradation observed in BEV perception tasks across the three severity levels, as reported by the data team based on the Dirt dataset [31].”