The DOI matches its Crossref record; 53 of the 53 references it cites were found in a registry or an index.
Polarization-driven semantic segmentation via efficient attention-bridged fusion
Open accessAbstract
Semantic segmentation (SS) is promising for outdoor scene perception in safety-critical applications like autonomous vehicles, assisted navigation and so on. However, traditional SS is primarily based on RGB images, which limits the reliability of SS in complex outdoor scenes, where RGB images lack necessary information dimensions to fully perceive unconstrained environments. As a preliminary investigation, we examine SS in an unexpected obstacle detection scenario, which demonstrates the necessity of multimodal fusion. Thereby, in this work, we present EAFNet, an Efficient Attention-bridged Fusion Network, to exploit complementary information coming from different optical sensors. Specifically, we incorporate polarization sensing to obtain supplementary information, considering its optical characteristics for robust representation of diverse materials. By using a single-shot polarization sensor, we build the first RGB-P dataset which consists of 394 annotated pixel-aligned RGB-polarization images. A comprehensive variety of experiments shows the effectiveness of EAFNet to fuse polarization and RGB information, as well as its flexibility to be adapted to other sensor combination scenarios.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“ZJU [79] is a real-world dataset and the only among the one listed supporting the light polarization modality.”
From Multimodal semantic segmentation in autonomous driving: A review of current approaches and future perspectives (2022) · cites this paper as [79]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699184 · Cited by 22,392
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2644615 · Cited by 19,631
Zhao, H., Shi, J., Qi, X., Wang, X., & Jia, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.660 · Cited by 16,438
The Cityscapes Dataset for Semantic Urban Scene Understanding
Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., & Schiele, B. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.350 · Cited by 12,433
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2389824 · Cited by 11,971
ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks
Wang, Q., Wu, B., Zhu, P., Li, P., Zuo, W., & Hu, Q. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01155 · Cited by 9,641
Dual Attention Network for Scene Segmentation
Fu, J., Liu, J., Tian, H., Li, Y., Bao, Y., Fang, Z., & Lu, H. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00326 · Cited by 7,351
LabelMe: A Database and Web-Based Tool for Image Annotation
Russell, B. C., Torralba, A., Murphy, K. P., & Freeman, W. T. (2008) · International Journal of Computer Vision · doi:10.1007/s11263-007-0090-8 · Cited by 4,210
BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning
Yu, F., Chen, H., Wang, X., Xian, W., Chen, Y., Liu, F., Madhavan, V., & Darrell, T. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00271 · Cited by 2,641
Accurate and Efficient Stereo Processing by Semi-Global Matching and Mutual Information
Hirschmüller, H. (2005) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2005.56 · Cited by 1,731
Context Encoding for Semantic Segmentation
Zhang, H., Dana, K., Shi, J., Zhang, Z., Wang, X., Tyagi, A., & Agrawal, A. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00747 · Cited by 1,528
ERFNet: Efficient Residual Factorized ConvNet for Real-Time Semantic Segmentation
Romera, E., Álvarez, J. M., Bergasa, L. M., & Arroyo, R. (2017) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2017.2750080 · Cited by 1,514
Feng, D., Haase-Schutz, C., Rosenbaum, L., Hertlein, H., Glaser, C., Timm, F., Wiesbeck, W., & Dietmayer, K. (2021) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2020.2972974 · Cited by 1,425
Li, B., Shen, C., Dai, Y., van den Hengel, A., & He, M. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298715 · Cited by 621
OCNet: Object Context Network for Scene ParsingPreprint
Yuan, Y., Huang, L., Guo, J., Zhang, C., Chen, X., & Wang, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.00916 · Cited by 520
ACNET: Attention Based Network to Exploit Complementary Features for RGBD Semantic Segmentation
Hu, X., Yang, K., Fei, L., & Wang, K. (2019) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2019.8803025 · Cited by 500
Self-citationOršić, M., Krešo, I., Bevandić, P., & Šegvić, S. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01289 · Cited by 440
Deep multimodal fusion for semantic image segmentation: A survey
Zhang, Y., Sidibé, D., Morel, O., & Mériaudeau, F. (2021) · Image and Vision Computing · doi:10.1016/j.imavis.2020.104042 · Cited by 261
Segmenting Transparent Objects in the Wild
Xie, E., Wang, W., Wang, W., Ding, M., Shen, C., & Luo, P. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58601-0_41 · Cited by 229
Deep Multispectral Semantic Scene Understanding of Forested Environments Using Multimodal Fusion
Valada, A., Oliveira, G. L., Brox, T., & Burgard, W. (2017) · Springer proceedings in advanced robotics · doi:10.1007/978-3-319-50115-4_41 · Cited by 211
Choi, S., Kim, J. T., & Choo, J. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00939 · Cited by 201
Lost and Found: detecting small road hazards for self-driving vehicles
Pinggera, P., Ramos, S., Gehrig, S. K., Franke, U., Rother, C., & Mester, R. (2016) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2016.7759186 · Cited by 200
Sun, L., Yang, K., Hu, X., Hu, W., & Wang, K. (2020) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2020.3007457 · Cited by 178
Self-citationSegmenting Objects in Day and Night: Edge-Conditioned CNN for Thermal Image Semantic Segmentation
Li, C., Xia, W., Yan, Y., Luo, B., & Tang, J. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3009373 · Cited by 169
EV-SegNet: Semantic Segmentation for Event-Based Cameras
Alonso, Í., & Murillo, A. C. (2019) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00205 · Cited by 168
Bridging the Day and Night Domain Gap for Semantic Segmentation
Romera, E., Bergasa, L. M., Yang, K., Álvarez, J. M., & Barea, R. (2019) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2019.8813888 · Cited by 116
Self-citationSun, L., Wang, K., Yang, K., & Xiang, K. (2019) · Artificial Intelligence and Machine Learning in Defense Applications · doi:10.1117/12.2532477 · Cited by 96
Self-citationHeatNet: Bridging the Day-Night Domain Gap in Semantic Segmentation with Thermal Images
Vertens, J., Zürn, J., & Burgard, W. (2020) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros45743.2020.9341192 · Cited by 91
Target enhanced 3D reconstruction based on polarization-coded structured light
Huang, X. M., Bai, J. Y., Wang, K., Liu, Q., Luo, Y., Yang, K., & Zhang, X. (2017) · Optics Express · doi:10.1364/oe.25.001173 · Cited by 73
Self-citationISSAFE: Improving Semantic Segmentation in Accidents by Fusing Event-based Data
Zhang, J., Yang, K., & Stiefelhagen, R. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636109 · Cited by 69
Self-citationBlin, R., Aïnouz, S., Canu, S., & Mériaudeau, F. (2019) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2019.8916853 · Cited by 60
Yang, K., Hu, X., Chen, H., Xiang, K., Wang, K., & Stiefelhagen, R. (2020) · IEEE Intelligent Vehicles Symposium · doi:10.1109/iv47402.2020.9304706 · Cited by 55
Self-citationRANUS: RGB and NIR Urban Scene Dataset for Deep Scene Parsing
Choe, G., Kim, S., Im, S., Lee, J., Narasimhan, S. G., & Kweon, I. S. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2801390 · Cited by 45
Detecting Traversable Area and Water Hazards for the Visually Impaired with a pRGB-D Sensor
Yang, K., Wang, K., Cheng, R., Hu, W., Huang, X. M., & Bai, J. Y. (2017) · Sensors · doi:10.3390/s17081890 · Cited by 43
Self-citationSingle Image Water Hazard Detection Using FCN with Reflection Attention Units
Han, X., Nguyen, C. V., You, S., & Lu, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01231-1_7 · Cited by 40
Depth from stereo polarization in specular scenes for urban robotics
Berger, K., Voorhies, R. C., & Matthies, L. H. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989227 · Cited by 38
PALVO: visual odometry based on panoramic annular lens
Chen, H., Wang, K., Hu, W., Yang, K., Cheng, R., Huang, X. M., & Bai, J. Y. (2019) · Optics Express · doi:10.1364/oe.27.024481 · Cited by 27
Self-citationMultimodality semantic segmentation based on polarization and color images
Wang, F., Aïnouz, S., Lian, C., & Bensrhair, A. (2017) · Neurocomputing · doi:10.1016/j.neucom.2016.10.090 · Cited by 25
Robustifying semantic cognition of traversability across wearable RGB-depth cameras
Yang, K., Bergasa, L. M., Romera, E., & Wang, K. (2019) · Applied Optics · doi:10.1364/ao.58.003141 · Cited by 25
Self-citationA multimodal vision sensor for autonomous driving
Sun, D., Huang, X. M., & Yang, K. (2019) · Counterterrorism, Crime Fighting, Forensics, and Surveillance Technologies · doi:10.1117/12.2535552 · Cited by 23
Self-citationPredicting Polarization Beyond Semantics for Wearable Robotics
Yang, K., Bergasa, L. M., Romera, E., Huang, X. M., & Wang, K. (2018) · IEEE-RAS International Conference on Humanoid Robots (Humanoids) · doi:10.1109/humanoids.2018.8625005 · Cited by 21
Self-citationXiang, K., Wang, K., & Yang, K. (2019) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2019.8917096 · Cited by 19
Self-citationA comparative study of high-recall real-time semantic segmentation based on swift factorized network
Xiang, K., Wang, K., & Yang, K. (2019) · Artificial Intelligence and Machine Learning in Defense Applications · doi:10.1117/12.2532697 · Cited by 10
Self-citationSegmenting Transparent Objects in the WildPreprint
Xie, E., Wang, W., Wang, W., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.13948 · Cited by 10
Zhou, K., Wang, K., & Yang, K. (2020) · Journal of Physics Conference Series · doi:10.1088/1742-6596/1518/1/012051 · Cited by 7
Self-citationYang, K., Bergasa, L. M., Romera, E., Wang, J., Wang, K., & López-Guillén, E. (2018) · IEEE International Conference on Robotics and Biomimetics (ROBIO) · doi:10.1109/robio.2018.8664804 · Cited by 5
Self-citationWang, Y., Liu, Q., Zu, H., Liu, X., Xie, R., & Wang, F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.08740 · Cited by 4
A Multimodal Vision Sensor for Autonomous DrivingPreprint
Sun, D., Huang, X. M., & Yang, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.05649 · Cited by 3
Self-citationISSAFE: Improving Semantic Segmentation in Accidents by Fusing Event-based DataPreprint
Zhang, J., Yang, K., & Stiefelhagen, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.08974 · Cited by 3
Self-citation
Cited by
Sorted by citations
CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation With Transformers
Zhang, J., Liu, H., Yang, K., Hu, X., Liu, R., & Stiefelhagen, R. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3300537 · Cited by 731
Self-citationDelivering Arbitrary-Modal Semantic Segmentation
Zhang, J., Liu, R., Shi, H., Yang, K., Reiß, S., Peng, K., Fu, H., Wang, K., & Stiefelhagen, R. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.00116 · Cited by 215
Self-citationGlass Segmentation using Intensity and Spectral Polarization Cues
Mei, H., Dong, B., Dong, W., Yang, J., Baek, S., Heide, F., Peers, P., Wei, X., & Yang, X. S. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01229 · Cited by 116
RGB-D and Thermal Sensor Fusion: A Systematic Literature Review
Brenner, M., Reyes, N. H., Sušnjak, T., & Barczak, A. L. C. (2023) · IEEE Access · doi:10.1109/access.2023.3301119 · Cited by 119
Zhang, J., Yang, K., Constantinescu, A., Peng, K., Müller, K., & Stiefelhagen, R. (2022) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2022.3161141 · Cited by 98
Self-citation
Metadata
- Authors
- Xiang, K., Yang, K., & Wang, K.
- Affiliations
- Zhejiang University (CN)
- Karlsruhe Institute of Technology (DE)
- Venue
- Optics Express
- Publisher
- Optica Publishing Group
- Published
- 2021-02-01
- Volume
- 29
- Issue
- 4
- Pages
- 4802
- ISSN
- 1094-4087 (ISSN-L)
- OpenAlex
- W3110089052
- Cited by
- 111
- References
- 50
- Citations by year
- 2026: 20 · 2025: 20 · 2024: 26 · 2023: 18 · 2022: 20 · 2021: 7
- FWCI
- 6.39
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Optics Express (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Figshare (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- ZJU-Sunny Photonics Innovation Center (2020-03)
- Bundesministerium für Arbeit und Soziales (01KM151112)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Physical Sciences › Engineering › Automotive Engineering › Autonomous Vehicle Technology and Safety
- Keywords
- semantic segmentation, polarization sensing, multimodal fusion, obstacle detection, autonomous vehicles, attention mechanism
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
