CoChat
Sign Up
Cited by 27Infrastructure Maintenance and MonitoringAdvanced Neural Network ApplicationsIndustrial Vision Systems and Defect Detection

The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.

Detecting surface defects of heritage buildings based on deep learning

Open access
Fu, Xiaoli, Angkawisittpan, Niwat·2024·Journal of Intelligent Systems
DOI 10.1515/jisys-2023-0048
Open this paper in CoChat — free accountOpen PDF

Abstract

The present study examined the usage of deep convolutional neural networks (DCNNs) for the classification, segmentation, and detection of the images of surface defects in heritage buildings. A survey was conducted on the building surface defects in Gulang Island (a UNESCO World Cultural Heritage Site), which were subsequently classified into six categories according to relevant standards. A Swin Transformer- and YOLOv5-based model was built for the automated detection of surface defects. Experimental results suggested that the proposed model was 99.2% accurate at classifying plant penetration and achieved a mean intersection-over-union (mIoU) of over 92% in relation to moss, cracking, alkalization, staining, and deterioration, outperforming CNN-based semantic segmentation networks such as FCN, PSPNet, and DeepLabv3plus. The Swin Transformer-based approach for the segmentation of building surface defect images achieved the highest accuracy regardless of the evaluation metric (with an mIoU of 90.96% and an mAcc of 95.78%), when contrasted to mainstream DCNNs such as SegFormer, PSPNet, and DANet.

Overview

Integrity signals

Registry

Crossref · 10.1515/jisys-2023-0048

Cites

5 of 5 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Flagged first, then by citations

Show all 5 references

Research works printed but not linked: 54

  1. Mimura T, Mita A. Automatic estimation of natural frequencies and damping ratios of building structures. Procedia Eng. 2017;88:163–9. · DOI
  2. Perez H, Tah JHM, Mosavi A. Deep learning for detecting building defects using convolutional neural networks. Sens (Basel). 2019;19(16):3556. · DOI
  3. Martin G, Selvakumaran S, Marinoni A, Sadeghi Z, Middleton C. Structural health monitoring on urban areas by using multi temporal insar and deep learning. Proceedings of IEEE International Geoscience and Remote Sensing Symposium IGARSS; 2021. p. 176–9. · DOI
  4. Dang HV, Tatipamula M, Nguyen HX. Cloud-based digital twinning for structural health monitoring using deep learning. IEEE Trans Ind Inform. 2022;18(6):3820–30. · DOI
  5. Davoudi R, Miller GR, Kutz JN. Structural load estimation using machine vision and surface crack patterns for shear-critical RC beams and slabs. J Comput Civ Eng. 2018;32(4):04018024. 10.1061/(ASCE)CP.1943-5487.0000766. · DOI
  6. Hoang ND. Image processing-based recognition of wall defects using machine learning approaches and steerable filters. Comput Intell Neurosci. 2018;2018:7913952. 10.1155/2018/7913952. · DOI
  7. Jo J, Jadidi Z, Stantic BA. Drone-based building inspection system using software-agents. In Studies in computational intelligence. Vol. 737. New York, NY, USA: Springer; 2017. p. 115–21. · DOI
  8. Valero E, Forster A, Bosché F, Hyslop E, Wilson L, Turmel A. Automated defect detection and classification in ashlar masonry walls using machine learning. Autom Constr. 2019;106:102846. 10.1016/j.autcon.2019.102846. · DOI
  9. Valero E, Forster A, Bosché F, Renier C, Hyslop E, Wilson L. High level-of-detail BIM and machine learning for automated masonry wall defect surveying. In Proceedings of the International Symposium on Automation and Robotics in Construction. Berlin, Germany: 2018. p. 20–5. · DOI
  10. Chu W-T, Tsai M-H. Visual pattern discovery for architecture image classification and product image search. In Proceedings of the 2nd ACM International Conference on Multimedia Retrieval. Hong Kong, China: 2012. p. 5–8. · DOI
  11. Goel A, Juneja M, Jawahar CV. Are buildings only instances?: Exploration in architectural style categories. In Proceedings of the Eighth Indian Conference on Computer Vision, Graphics and Image Processing. Mumbai, India: 2012. p. 1–8. · DOI
  12. Mathias M, Martinovic A, Weissenberg J, Haegler S, Van Gool L. Automatic architectural style recognition. In Proceedings of the 4th ISPRS International Workshop 3D-ARCH 2011. XXXVIII-5/W16, Trento, Italy; 2011. p. 171–6. · DOI
  13. Shalunts G, Haxhimusa Y, Sablatni R. Architectural style classification of building facade windows. In Advances in visual computing. 6939, Las Vegas, NV, USA: Springer; 2011. p. 280–9. · DOI
  14. Zhang L, Song M, Liu X, Sun L, Chen C, Bu J. Recognizing architecture styles by hierarchical sparse coding of blocklets. Inf Sci. 2014;254:141–54. · DOI
  15. Xu Z, Tao D, Zhang Y, Wu J, Tsoi AC. Architectural style classification using multinomial latent logistic regression. In: Fleet D, Pajdla T, Schiele B, Tuytelaars T, editors. Computer vision – ECCV 2014. ECCV 2014. Lecture notes in computer science. Vol. 8689. Cham: Springer; 2014. 10.1007/978-3-319-10590-1_39. · DOI
  16. Llamas J, Lerones PM, Medina R, Zalama E, Gómez-García-Bermejo J. Classification of architectural heritage images using deep learning techniques. Appl Sci. 2017;7:992. 10.3390/app7100992. · DOI
  17. Grinzato E, Bison PG, Marinetti S. Monitoring of ancient buildings by the thermal method. J Cultural Herit. 2002;3(1):21–9. · DOI
  18. Avdelidis N, Moropoulou A. Applications of infrared thermography for the investigation of historic structures. J Cultural Herit. 2004;5(1):119–27. · DOI
  19. Mansuri LE, Patel D. Artificial intelligence-based automatic visual inspection system for built heritage. Smart Sustain Built Environ. 2021;11(2):622–46. · DOI
  20. Trillo C, Aburamadan R, Chikomborero B, Makore N, Udeaja C, Moustaka A, et al. Towards smart planning conservation of heritage cities: digital technologies and heritage conservation planning. In International Conference on Human-Computer Interaction, Lecture Notes in Computer Science. 12794, Cham: Springer; 2021. 10.1007/978-3-030-77411-0_10. · DOI
  21. Lerones PM, Vélez DO, Rojo FG, Gómez-García-Bermejo J, Casanova EZ. Moisture detection in heritage buildings by 3D laser scanning. Stud Conserv. 2016;61:46–54. · DOI
  22. Tavukçuoğlu A. Non-destructive testing for building diagnostics and monitoring: Experience achieved with case studies. MATEC Web Conf. 2018;149:01015. 10.1051/matecconf/201814901015. · DOI
  23. Błaszczak-Bąk W, Suchocki C, Janicka J, Dumalski A, Duchnowski R. Defect detection of historic structures in dark places based on the point cloud analysis by modified Optd method. Int Arch Photogram Remote Sens Spat Inf Sci. 2019;XLII-3/W8:71–7. · DOI
  24. Wong CW. Applications of non-destructive tests for diagnosis of heritage buildings: Case studies from Singapore and Malaysia. Built Herit. 2019;3:14–25. · DOI
  25. Radnić J, Matešan. D, Abaza A. Restoration and strengthening of historical buildings: The example of Minceta fortress in Dubrovnik. Adv Civ Eng. 2020;2020:8854397. 10.1155/2020/8854397. · DOI
  26. Błaszczak-Bąk W, Suchocki C, Janicka J, Dumalski A, Sobieraj-Żłobińska A. Automatic threat detection for historic buildings in dark places based on the modified OptD method. ISPRS Int J Geo-Inf. 2020;9(2):123. 10.3390/ijgi9020123. · DOI
  27. Wu Y, Qin Y, Qian Y, Guo F, Wang Z, Jia L. Hybrid deep learning architecture for rail surface segmentation and surface defect detection. Comput Civ Infrastruct Eng. 2022;37(2):227–44. · DOI
  28. Wood RL, Mohammadi ME. Feature-based point cloud-based assessment of heritage structures for nondestructive and noncontact surface damage detection. Heritage. 2021;4:775–93. · DOI
  29. Al-Sakkaf A, Ghodke S, An C, Bagchi A. Defect detection in heritage buildings using ground penetrating radar – A review. In proceedings of SHMII-11: 11th International Conference on Structural Health Monitoring of Intelligent Infrastructure. Montreal, QC, Canada; 2022. p. 1–4.
  30. Masrour T, El Hassani I, Bouchama MS. Deep convolutional neural networks with transfer learning for old buildings pathologies automatic detection. In Proceedings of International Conference on Advanced Intelligent Systems for Sustainable Development. Springer; 2019. p. 204–16. · DOI
  31. Sharma T, Agrawal P, Verma NK. Detection of dust deposition using convolutional neural network for heritage images. In Proceedings of Computational Intelligence: Theories, Applications and Future Directions-Volume II. Springer; 2019. p. 347–59. · DOI
  32. Zou Z, Zhao X, Zhao P, Qi F, Wang N. CNN-based statistics and location estimation of missing components in routine inspection of historic buildings. J Cultural Herit. 2019;38:221–30. · DOI
  33. Zou Z, Zhao P, Zhao X. Automatic segmentation, inpainting, and classification of defective patterns on ancient architecture using multiple deep learning algorithms. Struct Contr Health Monit. 2021;28(7):e2742. 10.1002/stc.2742. · DOI
  34. Wang N, Zhao X, Zhao P, Zhang Y, Zou Z, Ou J. Automatic damage detection of historic masonry buildings based on mobile deep learning. Autom Constr. 2019;103:53–66. · DOI
  35. Ye S, Sun L. Method for detecting surface defects of ceramic tableware based on deep learning. J Phys: Conf Ser. 2020;1650:032045. 10.1088/1742-6596/1650/3/032045. · DOI
  36. Stephen O, Maduh UJ, Sain MA. Machine learning method for detection of surface defects on ceramic tiles using convolutional neural networks. Electronics. 2022;11(1):55. 10.3390/electronics11010055. · DOI
  37. Teng S, Liu Z, Li X. Improved YOLOv3-based bridge surface defect detection by combining high- and low-resolution feature images. Buildings. 2022;12(8):1225. 10.3390/buildings12081225. · DOI
  38. Shao W, Kakizaki K, Araki S, Mukai T. Automated two-stage approach for damage detection of surface defects in historical buildings. In proceedings of 2022 IEEE 46th Annual Computers, Software, and Applications Conference (COMPSAC). Los Alamitos, CA, USA; 2022. p. 816–1821. · DOI
  39. Bolourian N. Point cloud-based deep learning and UAV path planning for surface defect detection of concrete bridges. PhD Thesis. Canada: Concordia University; 2022.
  40. Meklati S, Boussora K, Abdi MEH, Sid-Ahmed B. Surface damage identification for heritage site protection: A mobile crowd-sensing solution based on deep learning. J Comput Cultural Herit. 2023;16(2):25, 1–24. · DOI
  41. Chen Y, Zhu Z, Lin Z, Zhou Y. Building surface crack detection using deep learning technology. Buildings. 2023;13:1814. 10.3390/buildings13071814. · DOI
  42. Bruno S, Galantucci RA, Musicco A. Decay detection in historic buildings through image-based deep learning. VITRUVIO - Int J Archit Technol Sustain. 2023;8:6–17. · DOI
  43. Yang W. A survey of surface defect detection based on deep learning. In Proceedings of the 2022 7th International Conference on Modern Management and Education Technology (MMET 2022). Shanghai, China: 2022. 10.2991/978-2-494069-51-0_51. · DOI
  44. Kwon N, Song K, Ahn Y, Park M, Jang Y. Maintenance cost prediction for aging residential buildings based on case-based reasoning and genetic algorithm. J Build Eng. 2020;28:101006. 10.1016/j.jobe.2019.101006. · DOI
  45. Prieto A, Verichev K, Silva A, de Brito J. On the impacts of climate change on the functional deterioration of heritage buildings in South Chile. Build Environ. 2020;183:107138. 10.1016/j.buildenv.2020.107138. · DOI
  46. Seeley IH. Building maintenance. London, UK: Macmillan International Higher Education; 1987. · DOI
  47. Richardson B. Defects and deterioration in buildings: A practical guide to the science and technology of material failure. London, UK: Routledge; 2002. · DOI
  48. Panboonyuen T, Jitkajornwanich K, Lawawirojwong S, Srestasathiern P, Vateekul P. Transformer-based decoder designs for semantic segmentation on remotely sensed images. Remote Sens. 2021;13(24):5100. 10.3390/rs13245100. · DOI
  49. Wang K, Teng Z, Zou T. Metal defect detection based on Yolov5. J Phys: Conf Ser. 2022;2218(1):012050. 10.1088/1742-6596/2218/1/012050. · DOI
  50. Zhou J, Jiang P, Zou A, Chen X, Hu W. Ship target detection algorithm based on improved YOLOv5. J Mar Sci Eng. 2021;9(8):908. 10.3390/jmse9080908. · DOI
  51. Kasper-Eulaers M, Hahn N, Berger S, Sebulonsen T, Myrland Ø, Kummervold P. Short communication: Detecting heavy goods vehicles in rest areas in winter conditions using YOLOv5. Algorithms. 2021;14:114. 10.3390/a14040114. · DOI
  52. Shelhamer E, Long J, Darrell T. Fully convolutional networks for semantic segmentation. IEEE Trans Pattern Anal Mach Intell. 2017;39(4):640–51. · DOI · Paper page
  53. Han K, Wang Y, Chen H, Chen X, Guo J, Liu Z, et al. A survey on vision transformer. IEEE Trans Pattern Anal Mach Intell. 2023;45(1):87–110. · DOI
  54. Wright L, Demeure N. Ranger21: A synergistic deep learning optimizer. ArXiv, 2021. 10.48550/arXiv.2106.13731. · DOI

Cited by

Sorted by citations

See all 27

Metadata

Authors
Fu, X., & Angkawisittpan, N.
Affiliations
  • Xiamen Institute of Technology (CN)
  • Mahasarakham University (TH)
Corresponding author
Angkawisittpan, Niwat
ORCID
Venue
Journal of Intelligent Systems
Publisher
Walter de Gruyter GmbH
Published
2024-01-01
Volume
33
Issue
1
ISSN
  • 0334-1860 (ISSN-L)
  • 2191-026X
DOI
10.1515/jisys-2023-0048
OpenAlex
W4392249028
Cited by
27
References
5
Citations by year
2026: 8 · 2025: 12 · 2024: 7
FWCI
4.52
Type
journal article
Language
en
Open access
Licence
Locations
  • Journal of Intelligent Systems (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • Greater South Information System
Topics
  • Physical Sciences › Engineering › Civil and Structural Engineering › Infrastructure Maintenance and Monitoring
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Engineering › Industrial and Manufacturing Engineering › Industrial Vision Systems and Defect Detection
Keywords
defect detection, Swin Transformer, YOLOv5, semantic segmentation, bryophytes, building deterioration, cultural heritage preservation, automated image classification

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review