CoChat

A Generic Automated Surface Defect Detection Based on a Bilinear Model

Open access
Zhou, Fei, Liu, Guihua, Xu, Feng, Deng, Hao · 2019 · Applied Sciences

The DOI matches its Crossref record; 32 of the 33 references it cites were found in a registry or an index.

Abstract

Aiming at the problems of complex texture, variable interference factors and large sample acquisition in surface defect detection, a generic method of automated surface defect detection based on a bilinear model was proposed. To realize the automatic classification and localization of surface defects, a new Double-Visual Geometry Group16 (D-VGG16) is firstly designed as feature functions of the bilinear model. The global and local features fully extracted from the bilinear model by D-VGG16 are output to the soft-max function to realize the automatic classification of surface defects. Then the heat map of the original image is obtained by applying Gradient-weighted Class Activation Mapping (Grad-CAM) to the output features of D-VGG16. Finally, the defects in the original input image can be located automatically after processing the heat map with a threshold segmentation method. The training process of the proposed method is characterized by a small sample, end-to-end, and is weakly-supervised. Furthermore, experiments are performed on two public and two industrial datasets, which have different defective features in texture, shape and color. The results show that the proposed method can simultaneously realize the classification and localization of defects with different defective features. The average precision of the proposed method is above 99% on the four datasets, and is higher than the known latest algorithms.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3390/app9153159

Cites

32 of 32 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Network overall structure. Figure 1. Network overall structure. Figure 1. Network overall structure.
Figure 1 Network overall structure. Figure 1. Network overall structure. Figure 1. Network overall structure.CC-BY · From the PDF
Image 2
CC-BY · From the PDF
Figure 3 Data stream of the bilinear model. Figure 3. Data stream of the bilinear model.
Figure 3 Data stream of the bilinear model. Figure 3. Data stream of the bilinear model.CC-BY · From the PDF
Figure 4 Examples of the DAGM_2007 defect dataset. Each column represents a type of defect, and the defect areas are labeled by the red bounding boxes. ( a ) classes1; ( b ) classes2; ( c ) classes3; ( d ) classes4; ( e ) classses5; ( f ) classes6. Figure 4. Examples of the DAGM_2007 defect dataset. Each column represents a type of defect, and the defect areas are labeled by the red bounding boxes. ( a ) classes1; ( b ) classes2; ( c ) classes3; ( d ) classes4; ( e ) classses5; ( f ) classes6.
Figure 4 Examples of the DAGM_2007 defect dataset. Each column represents a type of defect, and the defect areas are labeled by the red bounding boxes. ( a ) classes1; ( b ) classes2; ( c ) classes3; ( d ) classes4; ( e ) classses5; ( f ) classes6. Figure 4. Examples of the DAGM_2007 defect dataset. Each column represents a type of defect, and the defect areas are labeled by the red bounding boxes. ( a ) classes1; ( b ) classes2; ( c ) classes3; ( d ) classes4; ( e ) classses5; ( f ) classes6.CC-BY · From the PDF
Figure 5 Examples of the NEU defect dataset. Each column represents a type of defect, and the defect areas are labeled by the red bounding boxes. ( a ) crazing; ( b ) inclusion; ( c ) patches; ( d ) pitted-surface; ( e ) rolled-in-scale; ( f ) scratches. Figure 5. Examples of the NEU defect dataset. Each column represents a type of defect, and the defect areas are labeled by the red bounding boxes. ( a ) crazing; ( b ) inclusion; ( c ) patches; ( d ) pitted-surface; ( e ) rolled-in-scale; ( f ) scratches.
Figure 5 Examples of the NEU defect dataset. Each column represents a type of defect, and the defect areas are labeled by the red bounding boxes. ( a ) crazing; ( b ) inclusion; ( c ) patches; ( d ) pitted-surface; ( e ) rolled-in-scale; ( f ) scratches. Figure 5. Examples of the NEU defect dataset. Each column represents a type of defect, and the defect areas are labeled by the red bounding boxes. ( a ) crazing; ( b ) inclusion; ( c ) patches; ( d ) pitted-surface; ( e ) rolled-in-scale; ( f ) scratches.CC-BY · From the PDF
Image 6
CC-BY · From the PDF
( a ) ( b ) Figure 7. Examples of the fluorescent magnetic powder surface defects dataset. The defect areas are Figure 7. Examples of the fluorescent magnetic powder surface defects dataset. The defect areas are labeled by the red bounding boxes. ( a ) bad; ( b ) good. Figure 7. Examples of the fluorescent magnetic powder surface defects dataset. The defect areas are labeled by the red bounding boxes. ( a ) bad; ( b ) good.
( a ) ( b ) Figure 7. Examples of the fluorescent magnetic powder surface defects dataset. The defect areas are Figure 7. Examples of the fluorescent magnetic powder surface defects dataset. The defect areas are labeled by the red bounding boxes. ( a ) bad; ( b ) good. Figure 7. Examples of the fluorescent magnetic powder surface defects dataset. The defect areas are labeled by the red bounding boxes. ( a ) bad; ( b ) good.CC-BY · From the PDF
Figure 8 Examples of localization on DAGM_2007 defect dataset. From top to bottom are the original image, the combination of the original image and the heat map, and the location results of the defects. The Ground-Truth of the defect is marked with the red bounding boxes, while the localization results of the proposed method is marked with the blue bounding boxes. ( a ) classes1; ( b ) classes2; ( c ) classes3; ( d ) classes4; ( e ) classses5; ( f ) classes6. Figure 8. Examples of localization on DAGM_2007 defect dataset. From top to bottom are the original image, the combination of the original image and the heat map, and the location results of the defects. The Ground-Truth of the defect is marked with the red bounding boxes, while the localization results of the proposed method is marked with the blue bounding boxes. ( a ) classes1; ( b ) classes2; ( c ) classes3; ( d ) classes4; ( e ) classses5; ( f ) classes6.
Figure 8 Examples of localization on DAGM_2007 defect dataset. From top to bottom are the original image, the combination of the original image and the heat map, and the location results of the defects. The Ground-Truth of the defect is marked with the red bounding boxes, while the localization results of the proposed method is marked with the blue bounding boxes. ( a ) classes1; ( b ) classes2; ( c ) classes3; ( d ) classes4; ( e ) classses5; ( f ) classes6. Figure 8. Examples of localization on DAGM_2007 defect dataset. From top to bottom are the original image, the combination of the original image and the heat map, and the location results of the defects. The Ground-Truth of the defect is marked with the red bounding boxes, while the localization results of the proposed method is marked with the blue bounding boxes. ( a ) classes1; ( b ) classes2; ( c ) classes3; ( d ) classes4; ( e ) classses5; ( f ) classes6.CC-BY · From the PDF
Image 9
CC-BY · From the PDF
Figure 10 Examples of localization on the diode glass bulb surface defect dataset. From top to bottom are the original image, the combination of the original image and the heat map, and the location results of these defects. The Ground-Truth of the defect is marked with the red bounding boxes, while the localization results of the proposed method is marked with the blue bounding boxes. ( a ) break; ( b ) shell wall damage; ( c ) stain. Figure 10. Examples of localization on the diode glass bulb surface defect dataset. From top to bottom are the original image, the combination of the original image and the heat map, and the location results of these defects. The Ground-Truth of the defect is marked with the red bounding boxes, while the localization results of the proposed method is marked with the blue bounding boxes. ( a ) break; ( b ) shell wall damage; ( c ) stain.
Figure 10 Examples of localization on the diode glass bulb surface defect dataset. From top to bottom are the original image, the combination of the original image and the heat map, and the location results of these defects. The Ground-Truth of the defect is marked with the red bounding boxes, while the localization results of the proposed method is marked with the blue bounding boxes. ( a ) break; ( b ) shell wall damage; ( c ) stain. Figure 10. Examples of localization on the diode glass bulb surface defect dataset. From top to bottom are the original image, the combination of the original image and the heat map, and the location results of these defects. The Ground-Truth of the defect is marked with the red bounding boxes, while the localization results of the proposed method is marked with the blue bounding boxes. ( a ) break; ( b ) shell wall damage; ( c ) stain.CC-BY · From the PDF
Figure 11 Localization results of the proposed method on the fluorescent magnetic powder surface defect dataset. From top to bottom are the original image, the combination of the original image and the heat map, and the location results of these defects. The Ground-Truth of the defect is marked with the red bounding boxes, and the localization result of the proposed method is marked with the blue bounding boxes. Figure 11. Localization results of the proposed method on the fluorescent magnetic powder surface defect dataset. From top to bottom are the original image, the combination of the original image and the heat map, and the location results of these defects. The Ground-Truth of the defect is marked with the red bounding boxes, and the localization result of the proposed method is marked with the blue bounding boxes.
Figure 11 Localization results of the proposed method on the fluorescent magnetic powder surface defect dataset. From top to bottom are the original image, the combination of the original image and the heat map, and the location results of these defects. The Ground-Truth of the defect is marked with the red bounding boxes, and the localization result of the proposed method is marked with the blue bounding boxes. Figure 11. Localization results of the proposed method on the fluorescent magnetic powder surface defect dataset. From top to bottom are the original image, the combination of the original image and the heat map, and the location results of these defects. The Ground-Truth of the defect is marked with the red bounding boxes, and the localization result of the proposed method is marked with the blue bounding boxes.CC-BY · From the PDF
Figure 12 Comparison of 1 F curve obtained from four methods. ( a ) Diode glass bulb surface defect dataset; ( b ) Magnetic powder surface defect dataset. Figure 12. Comparison of F 1 curve obtained from four methods. ( a ) Diode glass bulb surface defect dataset; ( b ) Magnetic powder surface defect dataset.
Figure 12 Comparison of 1 F curve obtained from four methods. ( a ) Diode glass bulb surface defect dataset; ( b ) Magnetic powder surface defect dataset. Figure 12. Comparison of F 1 curve obtained from four methods. ( a ) Diode glass bulb surface defect dataset; ( b ) Magnetic powder surface defect dataset.CC-BY · From the PDF

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 5

  1. Santoyo, 2019, Reconocimiento de patrones y evaluación del daño generado en aceros de baja aleación a partir del procesamiento digital de imágenes e inteligencia artificial, DYNA Ing. Ind., 94, 357
  2. Lin, M., Chen, Q., and Yan, S. (2013, January 2–4). Network in network. Proceedings of the International Conference on Learning Representations (ICLR), Scottsdale, AZ, USA.
  3. Yosinski, J., Clune, J., Bengio, Y., and Lipson, H. How transferable are features in deep neural networks? In Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada, 8–13 December 2014; pp.
  4. Ostu, 1975, A threshold selection method from gray-histogram, IEEE Trans. Syst. Man Cybern., 9, 62
  5. Song, 2014, Automatic recognition of surface defects on hot-rolled steel strip using scattering convolution network, J. Comput. Inf. Syst., 10, 3049

Cited by

See all 47

Metadata

Authors
Zhou, F., Liu, G., Xu, F., & Deng, H.
Affiliations
Southwest University of Science and Technology (CN)
Corresponding author
Liu, Guihua
Venue
Applied Sciences
Publisher
MDPI AG
Published
2019-08-03
Volume
9
Issue
15
Pages
3159
ISSN
2076-3417 (ISSN-L)
OpenAlex
W2965258271
Cited by
47
References
33
Citations by year
2026: 2 · 2025: 4 · 2024: 7 · 2023: 7 · 2022: 8 · 2021: 10 · 2020: 8 · 2019: 1
FWCI
4.98
Type
journal article
Language
en
Open access
Locations
  • Applied Sciences (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
Funders
National Natural Science Foundation of China (11602292, 61701421, 61601381)
Topics
  • Physical Sciences › Engineering › Industrial and Manufacturing Engineering › Industrial Vision Systems and Defect Detection
  • Physical Sciences › Engineering › Mechanical Engineering › Non-Destructive Testing Techniques
  • Physical Sciences › Engineering › Civil and Structural Engineering › Infrastructure Maintenance and Monitoring
Keywords
surface defect detection, bilinear model, Grad-CAM, heatmaps, defect localization, defect classification, weakly supervised learning, end-to-end learning, thresholding, industrial inspection

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.