Hybrid semantic segmentation for tunnel lining cracks based on Swin Transformer and convolutional neural network
Open accessThe DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
Abstract
In the field of tunnel lining crack identification, the semantic segmentation algorithms based on convolution neural network (CNN) are extensively used. Owing to the inherent locality of CNN, these algorithms cannot make full use of context semantic information, resulting in difficulty in capturing the global features of crack. Transformer‐based networks can capture global semantic information, but this method also has the deficiencies of strong data dependence and easy loss of local features. In this paper, a hybrid semantic segmentation algorithm for tunnel lining crack, named SCDeepLab, is proposed by fusing Swin Transformer and CNN in the encoding and decoding framework of DeepLabv3+ to address the above issues. In SCDeepLab, a joint backbone network is introduced with CNN‐based Inverse Residual Block and Swin Transformer Block. The former is used to extract the local detailed information of the crack to generate the shallow feature layer, whereas the latter is used to extract the global semantic information to obtain the deep feature layer. In addition, Efficient Channel Attention enhanced Feature Fusion Module is proposed to fuse the shallow and deep features to combine the advantages of the two types of features. Furthermore, the strategy of transfer learning is adopted to solve the data dependency of Swin Transformer. The results show that the mean intersection over union ( mIoU ) and mean pixel accuracy ( mPA ) of SCDeepLab on the data sets constructed in this paper are 77.41% and 84.42%, respectively, which have higher segmentation accuracy than previous CNN‐based and transformer‐based semantic segmentation algorithms.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Swin Transformer: Hierarchical Vision Transformer using Shifted Windows
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00986 · Cited by 35,534
Feature Pyramid Networks for Object Detection
Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.106 · Cited by 30,939
Focal Loss for Dense Object Detection
Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollar, P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.324 · Cited by 30,581
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513
Xception: Deep Learning with Depthwise Separable Convolutions
Chollet, F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.195 · Cited by 20,380 · cited 3×
“The former is composed of the backbone Xception (Chollet, 2017) and atrous spatial pyramid pooling (ASPP) to extract the target feature.”
BackgroundMethodEncoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · arXiv · doi:10.48550/arxiv.1802.02611 · Cited by 20,063
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_49 · Cited by 20,054
V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation
Milletari, F., Navab, N., & Ahmadi, S.-A. (2016) · 2016 Fourth International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2016.79 · Cited by 11,931 · cited 1×
“To solve this problem, Dice loss (Milletari et al., 2016) is selected as the loss function of SCDeepLab and the five other comparison networks.”
MethodPath Aggregation Network for Instance Segmentation
Liu, S., Qi, L., Qin, H., Shi, J., & Jia, J. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00913 · Cited by 10,011
EfficientDet: Scalable and Efficient Object Detection
Tan, M., Pang, R., & Le, Q. V. (2019) · arXiv · Cited by 9,841 · cited 2×
“The results (Table 6) show that the accuracy of the model with EFFM is higher than that with FPN and Bi-FPN, which indicates that the feature fusion module designed in this paper has a better fusion effect.”
MethodHoward, A., Sandler, M., Chen, B., Wang, W., Chen, L.-C., Tan, M., Chu, G., Vasudevan, V., Zhu, Y., Pang, R., et al. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00140 · Cited by 9,708
ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networks
Wang, Q., Wu, B., Zhu, P., Li, P., Zuo, W., & Hu, Q. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.01155 · Cited by 9,641 · cited 8×
“To further verify the effectiveness of the EFFM, three different feature fusion modules were selected for further comparison, including FPN (Lin et al., 2017a), Bi-FPN (Tan et al., 2020), and FFM (referring to the remaining modules of EFFM after removing ECA in this paper).”
BackgroundInfluentialRethinking Atrous Convolution for Semantic Image Segmentation
Chen, L.-C., Papandreou, G., Schroff, F., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1706.05587 · Cited by 7,517
Swin-Unet: Unet-like pure transformer for medical image segmentation
Cao, H., Wang, Y., Chen, J., Jiang, D., Zhang, X., Tian, Q., & Wang, M. (2023) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-25066-8_9 · Cited by 5,640
Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2014) · arXiv · doi:10.48550/arxiv.1412.7062 · Cited by 4,213 · cited 1×
“In addition, for other DeepLab series networks (Chen et al., 2014, 2017) with dilated convolution, bilinear interpolation is directly used to upsample the feature layer to restore the original resolution, which cannot restore the information lost in the downsampling process.”
MethodLabelMe: A Database and Web-Based Tool for Image Annotation
Russell, B. C., Torralba, A., Murphy, K. P., & Freeman, W. T. (2008) · International Journal of Computer Vision · doi:10.1007/s11263-007-0090-8 · Cited by 4,210
UnitBox: An Advanced Object Detection Network
Yu, J., Jiang, Y., Wang, Z., Cao, Z., & Huang, T. (2016) · Proceedings of the 24th ACM international conference on Multimedia · doi:10.1145/2964284.2967274 · Cited by 1,731
Autonomous concrete crack detection using deep fully convolutional neural network
Dung, C. V., & Anh, L. D. (2019) · Automation in Construction · doi:10.1016/j.autcon.2018.11.028 · Cited by 1,170
ConViT: improving vision transformers with soft convolutional inductive biases
d’Ascoli, S., Touvron, H., Leavitt, M. L., Morcos, A. S., Biroli, G., & Sagun, L. (2022) · Journal of Statistical Mechanics: Theory and Experiment · doi:10.1088/1742-5468/ac9830 · Cited by 782 · cited 1×
“…such networks lack the inductive biases inherent to CNNs, including translation invariance and local sensitivity, which lead to the short-comings of strong data dependence and easy loss of local features in the tunnel lining crack segmentation (d’Ascoli et al., 2021; Liu, Sangineto, et al., 2021).”
Background
Cited by
A survey of the vision transformers and their CNN-transformer based variants
Khan, A., Rauf, Z., Sohail, A., Khan, A. R., Asif, H., Asif, A., & Farooq, U. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10595-0 · Cited by 451
Metadata
- Authors
- Zhou, Z., Zhang, J., & Gong, C.
- Venue
- Computer-Aided Civil and Infrastructure Engineering
- Publisher
- Elsevier BV
- Published
- 2023-04-04
- Volume
- 38
- Issue
- 17
- Pages
- 2491-2510
- ISSN
- 1093-9687 (ISSN-L)
- 1467-8667
- OpenAlex
- W4362551393
- Cited by
- 175
- References
- 22
- FWCI
- 20.58
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.