WeedMap: a large-scale semantic weed mapping framework using aerial multispectral imaging and deep neural network for precision farming
Open accessAbstract
The ability to automatically monitor agricultural fields is an important capability in precision farming, enabling steps towards more sustainable agriculture. Precise, high-resolution monitoring is a key prerequisite for targeted intervention and the selective application of agro-chemicals. The main goal of this paper is developing a novel crop/weed segmentation and mapping framework that processes multispectral images obtained from an unmanned aerial vehicle (UAV) using a deep neural network (DNN). Most studies on crop/weed semantic segmentation only consider single images for processing and classification. Images taken by UAVs often cover only a few hundred square meters with either color only or color and near-infrared (NIR) channels. Although a map can be generated by processing single segmented images incrementally, this requires additional complex information fusion techniques which struggle to handle high fidelity maps due to their computational costs and problems in ensuring global consistency. Moreover, computing a single large and accurate vegetation map (e.g., crop/weed) using a DNN is non-trivial due to difficulties arising from: (1) limited ground sample distances (GSDs) in high-altitude datasets, (2) sacrificed resolution resulting from downsampling high-fidelity images, and (3) multispectral image alignment. To address these issues, we adopt a stand sliding window approach that operates on only small portions of multispectral orthomosaic maps (tiles), which are channel-wise aligned and calibrated radiometrically across the entire map. We define the tile size to be the same as that of the DNN input to avoid resolution loss. Compared to our baseline model (i.e., SegNet with 3 channel RGB (red, green, and blue) inputs) yielding an area under the curve (AUC) of [background=0.607, crop=0.681, weed=0.576], our proposed model with 9 input channels achieves [0.839, 0.863, 0.782]. Additionally, we provide an extensive analysis of 20 trained models, both qualitatively and quantitatively, in order to evaluate the effects of varying input channels and tunable network hyperparameters. Furthermore, we release a large sugar beet/weed aerial dataset with expertly guided annotations for further research in the fields of remote sensing, precision agriculture, and agricultural robotics.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Better cropping systems, fertilization, irrigation practices, and advanced digital agriculture technologies can reduce N fertilizer input and N2O emissions.
“Better cropping systems, fertilization, and irrigation practices, and the use of advanced digital agriculture technologies, such as multi-sensor drone technology to allow farmers to manage crops, soil, fertilization, and irrigation more effectively and precisely, can reduce N fertilizer input and N2O emissions.117,118”
From Technologies and perspectives for achieving carbon neutrality (2021) · cites this paper as [118]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404 · see citing works
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834 · see citing works
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684 · see citing works
Rethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138 · see citing works
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699184 · Cited by 22,392 · see citing works
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2016) · arXiv · doi:10.48550/arxiv.1606.00915 · Cited by 22,392 · see citing works
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2644615 · Cited by 19,631 · see citing works
Deep learning in agriculture: A survey
Kamilaris, A., & Prenafeta-Boldú, F. X. (2018) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2018.02.016 · Cited by 5,579 · see citing works
Accurate, Dense, and Robust Multiview Stereopsis
Furukawa, Y., & Ponce, J. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.161 · Cited by 3,031 · see citing works
Snavely, N., Seitz, S. M., & Szeliski, R. (2006) · ACM Transactions on Graphics · doi:10.1145/1141911.1141964 · Cited by 2,975 · see citing works
Eigen, D., & Fergus, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.304 · Cited by 2,862 · see citing works
H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation From CT Volumes
Li, X., Chen, H., Qi, X., Dou, Q., Fu, C., & Heng, P. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2845918 · Cited by 2,530 · see citing works
The Effectiveness of Data Augmentation in Image Classification using Deep Learning
Perez, L., & Wang, J. (2017) · arXiv · doi:10.48550/arxiv.1712.04621 · Cited by 2,499 · see citing works
A survey on deep learning techniques for image and video semantic segmentation
García-García, A., Orts‐Escolano, S., Oprea, S., Villena-Martínez, V., Martínez-González, P., & García‐Rodríguez, J. (2018) · Applied Soft Computing · doi:10.1016/j.asoc.2018.05.018 · Cited by 2,488 · see citing works
The application of small unmanned aerial systems for precision agriculture: a review
Zhang, C., & Kovacs, J. M. (2012) · Precision Agriculture · doi:10.1007/s11119-012-9274-5 · Cited by 2,037 · see citing works
ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation
Paszke, A., Chaurasia, A., Kim, S., & Culurciello, E. (2016) · arXiv · doi:10.48550/arxiv.1606.02147 · Cited by 1,826 · see citing works
ERFNet: Efficient Residual Factorized ConvNet for Real-Time Semantic Segmentation
Romera, E., Álvarez, J. M., Bergasa, L. M., & Arroyo, R. (2017) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2017.2750080 · Cited by 1,491 · see citing works
BoxSup: Exploiting Bounding Boxes to Supervise Convolutional Networks for Semantic Segmentation
Dai, J., He, K., & Sun, J. (2015) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.191 · Cited by 1,118 · see citing works
A Review on Deep Learning Techniques Applied to Semantic SegmentationPreprint
García-García, A., Orts‐Escolano, S., Oprea, S., Villena-Martínez, V., & García‐Rodríguez, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.06857 · Cited by 1,049 · see citing works
Understanding Data Augmentation for Classification: When to Warp?
Wong, S. C., Gatt, A., Stamatescu, V., & McDonnell, M. D. (2016) · 2016 International Conference on Digital Image Computing: Techniques and Applications (DICTA) · doi:10.1109/dicta.2016.7797091 · Cited by 1,007 · see citing works
Smart farming is key to developing sustainable agriculture
Walter, A., Finger, R., Huber, R., & Buchmann, N. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1707462114 · Cited by 798 · see citing works
Self-citationAlgorithms for semantic segmentation of multispectral remote sensing imagery using deep learning
Kemker, R., Salvaggio, C., & Kanan, C. (2018) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2018.04.014 · Cited by 558 · see citing works
Area under the Precision-Recall Curve: Point Estimates and Confidence Intervals
Boyd, K., Eng, K., & Page, D. (2013) · ECML/PKDD · doi:10.1007/978-3-642-40994-3_29 · Cited by 549 · see citing works
What is a good evaluation measure for semantic segmentation?
Csurka, G., Larlus, D., & Perronnin, F. (2013) · British Machine Vision Conference (BMVC) · doi:10.5244/c.27.32 · Cited by 536 · see citing works
Milioto, A., Lottes, P., & Stachniss, C. (2018) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2018.8460962 · Cited by 509 · see citing works
Self-citationUAV-based crop and weed classification for smart farming
Lottes, P., Khanna, R., Pfeifer, J., et al. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989347 · Cited by 392 · see citing works
Self-citationDeep machine learning provides state-of-the-art performance in image-based plant phenotyping
Pound, M. P., Atkinson, J. A., Townsend, A. J., Wilson, M. H., Griffiths, M., Jackson, A. S., Bulat, A., Tzimiropoulos, G., Wells, D. M., Murchie, E. H., et al. (2017) · GigaScience · doi:10.1093/gigascience/gix083 · Cited by 390 · see citing works
weedNet: Dense Semantic Weed Classification Using Multispectral Images and MAV for Smart Farming
Sa, I., Chen, Z., Popović, M., Khanna, R., Liebisch, F., Nieto, J. I., & Siegwart, R. (2017) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2017.2774979 · Cited by 378 · see citing works
Self-citationWeed detection for site‐specific weed management: mapping and real‐time approaches
Granados, F. L. (2010) · Weed Research · doi:10.1111/j.1365-3180.2010.00829.x · Cited by 371 · see citing works
A Review of Deep Learning Methods and Applications for Unmanned Aerial Vehicles
Carrio, A., Sampedro, C., Rodríguez-Ramos, A., & Campoy, P. (2017) · Journal of Sensors · doi:10.1155/2017/3296874 · Cited by 356 · see citing works
Lottes, P., Behley, J., Milioto, A., & Stachniss, C. (2018) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2018.2846289 · Cited by 287 · see citing works
Self-citationde Castro, A. I., Torres‐Sánchez, J., Peña, J. M., et al. (2018) · Remote Sensing · doi:10.3390/rs10020285 · Cited by 282 · see citing works
Support Vector Machines for crop/weeds identification in maize fields
Guerrero, J. M., Pájares, G., Montalvo, M., Romeo, J., & Guijarro, M. (2012) · Expert Systems with Applications · doi:10.1016/j.eswa.2012.03.040 · Cited by 266 · see citing works
Pérez‐Ortiz, M., Peña, J. M., Gutiérrez, P. A., Torres‐Sánchez, J., Hervás‐Martínez, C., & Granados, F. L. (2015) · Applied Soft Computing · doi:10.1016/j.asoc.2015.08.027 · Cited by 211 · see citing works
Selecting patterns and features for between- and within- crop-row weed mapping using UAV-imagery
Pérez‐Ortiz, M., Peña, J. M., Gutiérrez, P. A., Torres‐Sánchez, J., Hervás‐Martínez, C., & Granados, F. L. (2015) · Expert Systems with Applications · doi:10.1016/j.eswa.2015.10.043 · Cited by 188 · see citing works
Mixtures of Lightweight Deep Convolutional Neural Networks: Applied to Agricultural Robotics
McCool, C., Pérez, T., & Upcroft, B. (2017) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2017.2667039 · Cited by 178 · see citing works
Automatic model based dataset generation for fast and accurate crop and weeds detection
Cicco, M. D., Potena, C., Grisetti, G., & Pretto, A. (2017) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2017.8206408 · Cited by 177 · see citing works
Fast and Accurate Crop and Weed Identification with Summarized Train Sets for Precision Agriculture
Potena, C., Nardi, D., & Pretto, A. (2017) · Advances in intelligent systems and computing · doi:10.1007/978-3-319-48036-7_9 · Cited by 173 · see citing works
Fusion of pixel and object-based features for weed mapping using unmanned aerial vehicle imagery
Gao, J., Liao, W., Nuyttens, D. R. E., et al. (2018) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2017.12.012 · Cited by 135 · see citing works
H-DenseUNet: Hybrid Densely Connected UNet for Liver and Tumor Segmentation from CT VolumesPreprint
Li, X., Chen, H., Qi, X., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1709.07330 · Cited by 105 · see citing works
Semantic Segmentation of Mixed Crops using Deep Convolutional Neural Network
Mortensen, A. K., Dyrmann, M., Karstoft, H., Jørgensen, R. N., & Gislum, R. (2016) · Cited by 88 · see citing works
UAVs and Machine Learning Revolutionising Invasive Grass and Vegetation Surveys in Remote Arid Lands
Sandino, J., Gonzalez, F., Mengersen, K. L., et al. (2018) · Sensors · doi:10.3390/s18020605 · Cited by 60 · see citing works
Oettershagen, P., Stastny, T., Hinzmann, T., et al. (2017) · Journal of Field Robotics · doi:10.1002/rob.21765 · Cited by 53 · see citing works
Hinzmann, T., Schönberger, J. L., Pollefeys, M., et al. (2017) · Springer proceedings in advanced robotics · doi:10.1007/978-3-319-67361-5_25 · Cited by 48 · see citing works
Aerial and Ground Based Sensing of Tolerance to Beet Cyst Nematode in Sugar Beet
Joalland, S., Screpanti, C., Varella, H., Reuther, M., Schwind, M., Lång, C., Walter, A., & Liebisch, F. (2018) · Remote Sensing · doi:10.3390/rs10050787 · Cited by 48 · see citing works
Self-citationGarcía-Ruíz, F., Wulfsohn, D., & Rasmussen, J. (2015) · Biosystems Engineering · doi:10.1016/j.biosystemseng.2015.07.012 · Cited by 35 · see citing works
Environmental Reviews and Case Studies: Bringing Unmanned Aerial Systems Closer to the Environment
Detweiler, C., Ore, J., Anthony, D., Elbaum, S. G., Burgin, A. J., & Lorenz, A. J. (2015) · Environmental Practice · doi:10.1017/s1466046615000174 · Cited by 23 · see citing works
On field radiometric calibration for multispectral cameras
Khanna, R., Sa, I., Nieto, J. I., et al. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989768 · Cited by 13 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Technologies and perspectives for achieving carbon neutrality
Wang, F., Harindintwali, J. D., Yuan, Z., Wang, M., Wang, F., Li, S., Yin, Z., Huang, L., Fu, Y., Li, L., et al. (2021) · The Innovation · doi:10.1016/j.xinn.2021.100180 · Cited by 1,330 · see citing works
David, E., Madec, S., Sadeghi-Tehran, P., Aasen, H., Zheng, B., Liu, S., Kirchgessner, N., Ishikawa, G., Nagasawa, K., Badhon, M. A., et al. (2020) · Plant Phenomics · doi:10.34133/2020/3521852 · Cited by 277 · see citing works
- Authors
- Sa, I., Popović, M., Khanna, R., Chen, Z., Lottes, P., Liebisch, F., Nieto, J., Stachniss, C., Walter, A., & Siegwart, R.
- Affiliations
- ETH Zurich (CH)
- University of Bonn (DE)
- Corresponding author
- Sa, Inkyu
- ORCID
- Venue
- Remote Sensing
- Publisher
- MDPI AG
- Published
- 2018-09-07
- Volume
- 10
- Issue
- 9
- Pages
- 1423
- ISSN
- 2072-4292 (ISSN-L)
- OpenAlex
- W2886554959
- Cited by
- 305
- References
- 49
- Citations by year
- 2026: 37 · 2025: 42 · 2024: 43 · 2023: 49 · 2022: 33 · 2021: 43 · 2020: 37 · 2019: 20 · 2018: 1
- FWCI
- 22.35
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Remote Sensing (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Repository for Publications and Research Data (ETH Zurich) (submittedVersion, cc-by)
- ETH Zürich Research Collection (cc-by)
- Funders
- Horizon 2020 (644227)
- The Swiss State Secretariat for Education, Research and Innovation (15.0029)
- Topics
- Life Sciences › Agricultural and Biological Sciences › Plant Science › Smart Agriculture and AI
- Physical Sciences › Environmental Science › Ecology › Remote Sensing in Agriculture
- Physical Sciences › Engineering › Media Technology › Remote-Sensing Image Classification
- Keywords
- weed mapping, crop segmentation, semantic segmentation, multispectral imagery, unmanned aerial vehicles, precision agriculture, sugar beet, sliding window, SegNet, near-infrared channel, radiometric calibration, agricultural robotics, aerial image dataset
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
