Semantic segmentation based on Deep learning for the detection of Cyanobacterial Harmful Algal Blooms (CyanoHABs) using synthetic images
Open accessAbstract
Cyanobacterial Harmful Algal Blooms (CyanoHABs) in lakes and reservoirs have increased substantially in recent decades due to different environmental factors. Its early detection is a crucial issue to minimize health effects, particularly in potential drinking and recreational water bodies. The use of Autonomous Surface Vehicles (ASVs) equipped with machine vision systems (cameras) onboard, represents a useful alternative at this time. In this regard, we propose an image Semantic Segmentation approach based on Deep Learning with Convolutional Neural Networks (CNNs) for the early detection of CyanoHABs considering an ASV perspective. The use of these models is justified by the fact that with their convolutional architecture, it is possible to capture both, spectral and textural information considering the context of a pixel and its neighbors. To train these models it is necessary to have data, but the acquisition of real images is a difficult task, due to the capricious appearance of the algae on water surfaces sporadically and intermittently over time and after long periods of time, requiring even years and the permanent installation of the image capture system. This justifies the generation of synthetic data so that sufficiently trained models are required to detect CyanoHABs patches when they emerge on the water surface. The data generation for training and the use of the semantic segmentation models to capture contextual information determine the need for the proposal, as well as its novelty and contribution. Three datasets of images containing CyanoHABs patches are generated: (a) the first contains real patches of CyanoHABs as foreground and images of lakes and reservoirs as background, but with a limited number of examples; (b) the second, contains synthetic patches of CyanoHABs generated with state-of-the-art Style-based Generative Adversarial Network Adaptive Discriminator Augmentation (StyleGAN2-ADA) and Neural Style Transfer as foreground and images of lakes and reservoirs as background, and (c) the third set, is the combination of the previous two. Four model architectures for semantic segmentation (UNet++, FPN, PSPNet, and DeepLabV3+), with two encoders as backbone (ResNet50 and EfficientNet-b6), are evaluated from each dataset on real test images and different distributions. The results show the feasibility of the approach and that the UNet++ model with EfficientNet-b6, trained on the third dataset, achieves good generalization and performance for the real test images.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
With various copyrighted software or sensor applications, the data might be closely connected to tools, and information might consequently be omitted at the point of software interaction.
“Additionally, with various copyrighted software or sensor applications [34], the data might be closely connected to tools, and information might consequently be omitted at the point of software interaction [35].”
From Reinvigorating algal cultivation for biomass production with digital twin technology - a smart sustainable infrastructure (2024) · cites this paper as [35]
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · see citing works
Feature Pyramid Networks for Object Detection
Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.106 · Cited by 30,939 · see citing works
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699184 · Cited by 22,392 · see citing works
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741 · see citing works
Pattern Recognition with Fuzzy Objective Function Algorithms
Bezdek, J. C. (1981) · Springer eBooks · doi:10.1007/978-1-4757-0450-1 · Cited by 16,868 · see citing works
Zhao, H., Shi, J., Qi, X., Wang, X., & Jia, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.660 · Cited by 16,438 · see citing works
A Style-Based Generator Architecture for Generative Adversarial Networks
Karras, T., Laine, S., & Aila, T. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00453 · Cited by 11,714 · see citing works
Rethinking Atrous Convolution for Semantic Image Segmentation
Chen, L.-C., Papandreou, G., Schroff, F., & Adam, H. (2017) · arXiv · doi:10.48550/arxiv.1706.05587 · Cited by 7,517 · see citing works
(2007) · doi:10.5555/1283383.1283494 · Cited by 7,364 · see citing works
Analyzing and Improving the Image Quality of StyleGAN
Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., & Aila, T. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00813 · Cited by 6,332 · see citing works
Huisman, J., Codd, G. A., Paerl, H. W., Ibelings, B. W., Verspagen, J. M. H., & Visser, P. (2018) · Nature Reviews Microbiology · doi:10.1038/s41579-018-0040-1 · Cited by 2,935 · see citing works
Albumentations: fast and flexible image augmentations
Buslaev, A., Iglovikov, V. I., Khvedchenya, E., Parinov, A., Druzhinin, M., & Kalinin, A. A. (2020) · Information · doi:10.3390/info11020125 · Cited by 2,714 · see citing works
A Neural Algorithm of Artistic Style
Gatys, L., Ecker, A., & Bethge, M. (2016) · Journal of Vision · doi:10.1167/16.12.326 · Cited by 2,612 · see citing works
The Effectiveness of Data Augmentation in Image Classification using Deep Learning
Perez, L., & Wang, J. (2017) · arXiv · doi:10.48550/arxiv.1712.04621 · Cited by 2,499 · see citing works
A survey on deep learning techniques for image and video semantic segmentation
García-García, A., Orts‐Escolano, S., Oprea, S., Villena-Martínez, V., Martínez-González, P., & García‐Rodríguez, J. (2018) · Applied Soft Computing · doi:10.1016/j.asoc.2018.05.018 · Cited by 2,489 · see citing works
Frid-Adar, M., Diamant, I., Klang, E., Amitai, M. M., Goldberger, J., & Greenspan, H. K. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.09.013 · Cited by 1,952 · see citing works
Semantic Understanding of Scenes Through the ADE20K Dataset
Zhou, B., Zhao, H., Puig, X., Xiao, T., Fidler, S., Barriuso, A., & Torralba, A. (2019) · International Journal of Computer Vision · doi:10.1007/s11263-018-1140-0 · Cited by 1,831 · see citing works
Wider or Deeper: Revisiting the ResNet Model for Visual Recognition
Wu, Z., Shen, C., & van den Hengel, A. (2019) · Pattern Recognition · doi:10.1016/j.patcog.2019.01.006 · Cited by 1,743 · see citing works
Data augmentation for improving deep learning in image classification problem
Mikołajczyk, A., & Grochowski, M. (2018) · International Interdisciplinary PhD Workshop (IIPhDW) · doi:10.1109/iiphdw.2018.8388338 · Cited by 1,737 · see citing works
Panoptic Feature Pyramid Networks
Kirillov, A., Girshick, R., He, K., & Dollár, P. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00656 · Cited by 1,460 · see citing works
A wavelet-based image fusion tutorial
Pájares, G., & García, J. M. D. L. C. (2004) · Pattern Recognition · doi:10.1016/j.patcog.2004.03.010 · Cited by 1,368 · see citing works
Training Generative Adversarial Networks with Limited Data
Karras, T., Aittala, M., Hellsten, J., Laine, S., Lehtinen, J., & Aila, T. (2020) · arXiv · doi:10.48550/arxiv.2006.06676 · Cited by 1,170 · see citing works
A unifying view on dataset shift in classification
Moreno-Torres, J. G., Raeder, T., Alaiz-Rodríguez, R., Chawla, N. V., & Herrera, F. (2012) · Pattern Recognition · doi:10.1016/j.patcog.2011.06.019 · Cited by 1,029 · see citing works
A review of medical image data augmentation techniques for deep learning applications
Chlap, P., Min, H., Vandenberg, N., Dowling, J., Holloway, L. C., & Haworth, A. (2021) · Journal of Medical Imaging and Radiation Oncology · doi:10.1111/1754-9485.13261 · Cited by 1,008 · see citing works
A novel ocean color index to detect floating algae in the global oceans
Hu, C. (2009) · Remote Sensing of Environment · doi:10.1016/j.rse.2009.05.012 · Cited by 979 · see citing works
A survey of loss functions for semantic segmentation
Jadon, S. (2020) · IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB) · doi:10.1109/cibcb48159.2020.9277638 · Cited by 951 · see citing works
Deep semantic segmentation of natural and medical images: a review
Taghanaki, S. A., Abhishek, K., Cohen, J., Cohen‐Adad, J., & Hamarneh, G. (2020) · Artificial Intelligence Review · doi:10.1007/s10462-020-09854-1 · Cited by 888 · see citing works
Patki, N., Wedge, R., & Veeramachaneni, K. K. (2016) · IEEE International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa.2016.49 · Cited by 716 · see citing works
Sandfort, V., Yan, K., Pickhardt, P. J., & Summers, R. M. (2019) · Scientific Reports · doi:10.1038/s41598-019-52737-x · Cited by 684 · see citing works
Monitoring cyanobacterial blooms by satellite remote sensing
Kutser, T., Metsamaa, L., Strömbeck, N., & Vahtmäe, E. (2006) · Estuarine Coastal and Shelf Science · doi:10.1016/j.ecss.2005.11.024 · Cited by 296 · see citing works
Yang, J. R., Lv, H., Isabwe, A., Liu, L., Yu, X., Chen, H., & Yang, J. (2017) · Water Research · doi:10.1016/j.watres.2017.04.062 · Cited by 233 · see citing works
Preserving Color in Neural Artistic Style TransferPreprint
Gatys, L. A., Bethge, M., Hertzmann, A., & Shechtman, E. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.05897 · Cited by 113 · see citing works
Brain tumor image generation using an aggregation of GAN models with style transfer
Mukherkjee, D., Saha, P., Kaplun, D., Sinitca, A., & Sarkar, R. (2022) · Scientific Reports · doi:10.1038/s41598-022-12646-y · Cited by 106 · see citing works
A deep learning method for cyanobacterial harmful algae blooms prediction in Taihu Lake, China
Cao, H., Han, L., & Li, L. (2022) · Harmful Algae · doi:10.1016/j.hal.2022.102189 · Cited by 96 · see citing works
Satellite detection of harmful algal bloom occurrences in Korean waters
Ahn, Y.-H., Shanmugam, P., Ryu, J., & Jeong, J.-C. (2005) · Harmful Algae · doi:10.1016/j.hal.2005.07.007 · Cited by 94 · see citing works
Autonomous Inland Water Monitoring: Design and Application of a Surface Vessel
Hitz, G., Pomerleau, F., Garneau, M., Pradalier, C., Posch, T., Pernthaler, J., & Siegwart, R. (2012) · IEEE Robotics & Automation Magazine · doi:10.1109/mra.2011.2181771 · Cited by 80 · see citing works
Feist, S. M., & Lance, R. F. (2021) · Harmful Algae · doi:10.1016/j.hal.2021.102124 · Cited by 48 · see citing works
Cannizzaro, J. P., Barnes, B. B., Hu, C., et al. (2019) · Remote Sensing of Environment · doi:10.1016/j.rse.2019.111227 · Cited by 39 · see citing works
Zyuzin, V., & Chumarnaya, T. V. (2019) · Ural Symposium on Biomedical Engineering, Radioelectronics and Information Technology (USBEREIT) · doi:10.1109/usbereit.2019.8736616 · Cited by 39 · see citing works
Touzout, W., Benmoussa, Y., Benazzouz, D., et al. (2021) · Ocean Engineering · doi:10.1016/j.oceaneng.2020.108560 · Cited by 36 · see citing works
A risk assessment method for remote sensing of cyanobacterial blooms in inland waters
Chen, N., Wang, S., Zhang, X., et al. (2020) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2020.140012 · Cited by 34 · see citing works
Sharan, T. S., Tripathi, S., Sharma, S., & Sharma, N. (2021) · IETE Technical Review · doi:10.1080/02564602.2021.1955760 · Cited by 30 · see citing works
Tan, F., Xiao, P., Yang, J. R., et al. (2021) · Harmful Algae · doi:10.1016/j.hal.2021.102125 · Cited by 25 · see citing works
Improving the Neural Algorithm of Artistic StylePreprint
Novak, R., & Nikulin, Y. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.04603 · Cited by 14 · see citing works
Romero‐Vivas, E., Borstel, F. D. V., Pérez-Estrada, C. J., et al. (2015) · Environmental Science Processes & Impacts · doi:10.1039/c5em00097a · Cited by 4 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Sheik, A. G., Kumar, A., Ansari, F. A., Raj, V., Peleato, N. M., Patan, A. K., Kumari, S., & Bux, F. (2024) · Algal Research · doi:10.1016/j.algal.2024.103779 · Cited by 37 · see citing works
- Authors
- Barrientos-Espillco, F., Gascó, E., López-González, C. I., Gómez-Silva, M. J., & Pajares, G.
- Affiliations
- Universidad Complutense de Madrid (ES)
- Corresponding author
- Barrientos-Espillco, Fredy
- ORCID
- Venue
- Applied Soft Computing
- Publisher
- Elsevier BV
- Published
- 2023-07
- Volume
- 141
- Pages
- 110315
- ISSN
- 1568-4946 (ISSN-L)
- OpenAlex
- W4366256777
- Cited by
- 26
- References
- 47
- Citations by year
- 2026: 2 · 2025: 8 · 2024: 14 · 2023: 2
- FWCI
- 2.87
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Applied Soft Computing (publishedVersion, cc-by)
- Funders
- Comunidad de Madrid (Y2020/TCS-6420)
- Ministerio de Ciencia, Innovación y Universidades (RTI2018-098962-BC21)
- Topics
- Physical Sciences › Engineering › Media Technology › Remote-Sensing Image Classification
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Environmental Science › Environmental Chemistry › Aquatic Ecosystems and Phytoplankton Dynamics
- Keywords
- cyanobacterial harmful algal blooms, semantic segmentation, unmanned surface vehicles, synthetic image generation, StyleGAN2-ADA, neural style transfer, convolutional neural networks, U-Net
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
