Defective samples simulation through neural style transfer for automatic surface defect segment
Open accessAbstract
Owing to the lack of defect samples in industrial product quality inspection, trained segmentation model tends to overfit when applied online. To address this problem, we propose a defect sample simulation algorithm based on neural style transfer. The simulation algorithm requires only a small number of defect samples for training, and can efficiently generate simulation samples for next-step segmentation task. In our work, we introduce a masked histogram matching module to maintain color consistency of the generated area and the true defect. To preserve the texture consistency with the surrounding pixels, we take the fast style transfer algorithm to blend the generated area into the background. At the same time, we also use the histogram loss to further improve the quality of the generated image. Besides, we propose a novel structure of segment net to make it more suitable for defect segmentation task. We train the segment net with the real defect samples and the generated simulation samples separately on the button datasets. The results show that the F1 score of the model trained with only the generated simulation samples reaches 0.80, which is better than the real sample result.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834 · see citing works
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699184 · Cited by 22,392 · cited 1× · see citing works
“Chen et al.18 proposed Encoder-Decoder structure segmentation net Deeplab V3+, which is mainly composed of Deeplab v316 as the encoder, while the decoder uses a simple but effective module.”
MethodEncoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_49 · Cited by 20,054 · cited 1× · see citing works
“Chen et al. 18 proposed Encoder-Decoder structure segmentation net Deeplab V3+, which is mainly composed of Deeplab v3 16 as the encoder, while the decoder uses a simple but effective module.”
MethodSegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2644615 · Cited by 19,631 · see citing works
Nonlinear total variation based noise removal algorithms
Rudin, L., Osher, S., & Fatemi, E. (1992) · Physica D Nonlinear Phenomena · doi:10.1016/0167-2789(92)90242-f · Cited by 17,214 · cited 1× · see citing works
“Using tv loss can increase the smoothness of the transitions between specified transferred region and background 8,21 , making the composite region less abrupt.”
BackgroundZhao, H., Shi, J., Qi, X., Wang, X., & Jia, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.660 · Cited by 16,438 · see citing works
Perceptual Losses for Real-Time Style Transfer and Super-Resolution
Johnson, J., Alahi, A., & Fei-Fei, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_43 · Cited by 11,283 · cited 2× · see citing works
“The experiment uses the F1 score as an evaluation index, and its definition is as shown in (8).”
MethodRethinking Atrous Convolution for Semantic Image Segmentation
Chen, L.-C., Papandreou, G., Schroff, F., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1706.05587 · Cited by 7,517 · see citing works
Image Style Transfer Using Convolutional Neural Networks
Gatys, L. A., Ecker, A. S., & Bethge, M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.265 · Cited by 6,025 · cited 1× · see citing works
“We use masked cross-entropy as the loss function, and the optimization goal of Buttonlab is shown in (7).”
MethodInstance Normalization: The Missing Ingredient for Fast StylizationPreprint
Ulyanov, D., Vedaldi, A., & Lempitsky, V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.08022 · Cited by 4,325 · see citing works
Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2014) · arXiv · doi:10.48550/arxiv.1412.7062 · Cited by 4,213 · see citing works
Rother, C., Kolmogorov, V., & Blake, A. (2004) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/1186562.1015720 · Cited by 935 · see citing works
Wavelet based methods on patterned fabric defect detection
Ngan, H. Y. T., Pang, G. K. H., Yung, S. P., & Ng, M. K. (2004) · Pattern Recognition · doi:10.1016/j.patcog.2004.07.009 · Cited by 264 · cited 1× · see citing works
“( ) ( ) ( ) conte 2 nt 2 , 1 ˆ ˆ , j j j j j j y y y y C H W φ φ φ = − (1)”
Stable and Controllable Neural Texture Synthesis and Style Transfer Using Histogram LossesPreprint
Risser, E., Wilmot, P., & Barnes, C. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1701.08893 · Cited by 197 · cited 1× · see citing works
“The first step of the simulation is to match the specified area with the reference area by histogram matching 19 .”
Anomaly detection with convolutional neural networks for industrial surface inspection
Staar, B., Lütjen, M., & Freitag, M. (2019) · Procedia CIRP · doi:10.1016/j.procir.2019.02.123 · Cited by 160 · cited 2× · see citing works
“We train 10 epochs, and the weights in (6) are c 1e5 w = , s 2.”
MethodHe, D., Xu, K., & Zhou, P. (2019) · Computers & Industrial Engineering · doi:10.1016/j.cie.2018.12.043 · Cited by 140 · cited 1× · see citing works
“( ) ( ) 2 hist 1 ˆ ˆ j j j j j F y R y C H W φ − = (4)”
Automatic fabric defect detection with a wide-and-compact network
Li, Y., Zhang, D., & Lee, D.-J. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.10.070 · Cited by 115 · cited 1× · see citing works
“( ) ( ) 2 2 tv , , 1 , 1, , ˆ ˆ ˆ ˆ ( ) ˆ x y x y x y x y x y y y y y y − − = − + − (5)”
Luan, F., Paris, S., Shechtman, E., et al. (2018) · Computer graphics forum (Print) · doi:10.1111/cgf.13478 · Cited by 78 · cited 2× · see citing works
“Apart from these global stylization algorithms, Luan et al. 9 designed a local style transfer algorithm in 2018, focusing on the simulation of the local area in the painting.”
BackgroundMethodMorphology-based defect detection in machined surfaces with circular tool-mark patterns
Tsai, D., & Molina, D. E. R. (2018) · Measurement · doi:10.1016/j.measurement.2018.10.079 · Cited by 51 · cited 1× · see citing works
“( ) ( ) ( ) 2 , style ˆ ˆ , j j j F y y G y G y φ φ φ = − (2)”
Large-Scale Sparse Learning From Noisy Tags for Semantic Segmentation
Li, A., Lu, Z., Wang, L., Han, P., & Wen, J.-R. (2016) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2016.2631528 · Cited by 11 · cited 1× · see citing works
“At present, a large number of related works use conventional image processing techniques 1–3 or neural network 4–6 for defect segmentation.”
BackgroundSelf-supervised Monocular Depth Estimation with Multiple Resolution Images
Ryu, G., & Kim, W. (2024) · Journal of Broadcast Engineering · doi:10.5909/jbe.2024.29.4.399
Flagged first, then by citations
Cited by
Sorted by citations
Deep Industrial Image Anomaly Detection: A Survey
Liu, J., Xie, G., Wang, J., Li, S., Wang, C., Zheng, F., & Jin, Y. (2024) · Machine Intelligence Research · doi:10.1007/s11633-023-1459-z · Cited by 463 · see citing works
“[157] propose a model named DST to simulate defect samples.”
BackgroundHuman-centric artificial intelligence architecture for industry 5.0 applications
Rožanec, J. M., Novalija, I., Zajec, P., Kenda, K., Ghinani, H. T., Suh, S., Bian, S., Veliou, E., Papamartzivanos, D., Giannetsos, T., et al. (2022) · International Journal of Production Research · doi:10.1080/00207543.2022.2138611 · Cited by 372 · see citing works
“Neural Style Transfer (Wei et al. (2020)) has been successfully used to generate synthetic samples by fusing defect snippets with images of non-defective manufactured pieces.”
Computer vision defect detection on unseen backgrounds for manufacturing inspection
Mezher, A. M., & Marble, A. E. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122749 · Cited by 49 · see citing works
Wu, K., Tan, J., & Liu, C. (2022) · IEEE Sensors Journal · doi:10.1109/jsen.2022.3161331 · Cited by 29 · see citing works
MST: Multiscale Flow-Based Student–Teacher Network for Unsupervised Anomaly Detection
Yang, Y., Yang, Y., Zhou, S., Gao, Y., Zhu, Y., Wan, X., Hu, W., & Jiang, X. (2024) · Electronics · doi:10.3390/electronics13163224 · Cited by 4 · see citing works
Self-citationSemi-supervised deep learning for robust anomaly detection in smart manufacturing systems
Le, T. (2024) · theses.fr (ABES) · doi:10.70675/19d23228zf030z4df7za3abz596d36577818
Deployment-Oriented Memory-Based Visual Inspection of Injection-Molded Parts
Raza, M. M., Ali, B. B., Qureshi, M. S., Rizvi, O., Anees, M., Khalid, A., Wasay, M. A., Hassan, W., & Ahsan, U. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-26217-2_16
- Authors
- Wei, T., Cao, D., Jiang, X., Zheng, C., & Liu, L.
- Affiliations
- Huazhong University of Science and Technology (CN)
- Venue
- 2019 International Conference on Optical Instruments and Technology: Optoelectronic Measurement Technology and Systems
- Publisher
- SPIE
- Published
- 2020-03-12
- Pages
- 6
- OpenAlex
- W3010725630
- Cited by
- 10
- References
- 24
- Citations by year
- 2026: 1 · 2024: 4 · 2023: 2 · 2022: 2 · 2020: 1
- FWCI
- 1.78
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- International Conference on Optical Instruments and Technology: Optoelectronic Measurement Technology and Systems (OIT) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Engineering › Industrial and Manufacturing Engineering › Industrial Vision Systems and Defect Detection
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Physical Sciences › Engineering › Civil and Structural Engineering › Infrastructure Maintenance and Monitoring
- Keywords
- surface defect segmentation, neural style transfer, color consistency, texture consistency
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
