The DOI matches its Crossref record; 63 of the 63 references it cites were found in a registry or an index.
Recognition and Localization Methods for Vision-Based Fruit Picking Robots: A Review
Open accessAbstract
The utilization of machine vision and its associated algorithms improves the efficiency, functionality, intelligence, and remote interactivity of harvesting robots in complex agricultural environments. Machine vision and its associated emerging technology promise huge potential in advanced agricultural applications. However, machine vision and its precise positioning still have many technical difficulties, making it difficult for most harvesting robots to achieve true commercial applications. This article reports the application and research progress of harvesting robots and vision technology in fruit picking. The potential applications of vision and quantitative methods of localization, target recognition, 3D reconstruction, and fault tolerance of complex agricultural environment are focused, and fault-tolerant technology designed for utilization with machine vision and robotic systems are also explored. The two main methods used in fruit recognition and localization are reviewed, including digital image processing technology and deep learning-based algorithms. The future challenges brought about by recognition and localization success rates are identified: target recognition in the presence of illumination changes and occlusion environments; target tracking in dynamic interference-laden environments, 3D target reconstruction, and fault tolerance of the vision system for agricultural robots. In the end, several open research problems specific to recognition and localization applications for fruit harvesting robots are mentioned, and the latest development and future development trends of machine vision are described.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Micro/nanorobots for biomedicine: Delivery, surgery, sensing, and detoxification
Li, J., Esteban-Fernández de Ávila, B., Gao, W., Zhang, L., & Wang, J. (2017) · Science Robotics · doi:10.1126/scirobotics.aam6431 · Cited by 1,565
Self-citationThe grand challenges of Science Robotics
Yang, G.-Z., Bellingham, J., Dupont, P. E., Fischer, P., Floridi, L., Full, R., Jacobstein, N., Kumar, V., McNutt, M., Merrifield, R., et al. (2018) · Science Robotics · doi:10.1126/scirobotics.aar7650 · Cited by 1,177
DeepFruits: A Fruit Detection System Using Deep Neural Networks
Sa, I., Ge, Z., Dayoub, F., Upcroft, B., Perez, T., & McCool, C. (2016) · Sensors · doi:10.3390/s16081222 · Cited by 1,104
Harvesting Robots for High‐value Crops: State‐of‐the‐art Review and Challenges Ahead
Bac, C. W., Henten, E. J. V., Hemming, J., & Edan, Y. (2014) · Journal of Field Robotics · doi:10.1002/rob.21525 · Cited by 686
Sensors and systems for fruit detection and localization: A review
Gongal, A., Amatya, S., Karkee, M., Zhang, Q., & Lewis, K. (2015) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2015.05.021 · Cited by 572
Deep Count: Fruit Counting Based on Deep Simulated Learning
Rahnemoonfar, M., & Sheppard, C. (2017) · Sensors · doi:10.3390/s17040905 · Cited by 520
Image Segmentation for Fruit Detection and Yield Estimation in Apple Orchards
Bargoti, S., & Underwood, J. P. (2017) · Journal of Field Robotics · doi:10.1002/rob.21699 · Cited by 495
Sii-Mobility: An IoT/IoE Architecture to Enhance Smart City Mobility and Transportation Services
Badii, C., Bellini, P., Difino, A., & Nesi, P. (2018) · Sensors · doi:10.3390/s19010001 · Cited by 472
A review of key techniques of vision-based control for harvesting robot
Zhao, Y., Gong, L., Huang, Y., & Liu, C. (2016) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2016.06.022 · Cited by 407
Counting Apples and Oranges With Deep Learning: A Data-Driven Approach
Chen, S. W., Shivakumar, S. S., Dcunha, S., Das, J., Okon, E., Qu, C., Taylor, C. J., & Kumar, V. (2017) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2017.2651944 · Cited by 406
Design, integration, and field evaluation of a robotic apple harvester
Silwal, A., Davidson, J. R., Karkee, M., Mo, C., Zhang, Q., & Lewis, K. (2017) · Journal of Field Robotics · doi:10.1002/rob.21715 · Cited by 406
Design and control of an apple harvesting robot
De-An, Z., Jidong, L., Wei, J., Ying, Z., & Yu, C. (2011) · Biosystems Engineering · doi:10.1016/j.biosystemseng.2011.07.005 · Cited by 382
Robotic kiwifruit harvesting using machine vision, convolutional neural networks, and robotic arms
Williams, H. A. M., Jones, M. H., Nejati, M., Seabright, M. J., Bell, J., Penhall, N. D., Barnett, J. J., Duke, M. D., Scarfe, A. J., Ahn, H. S., et al. (2019) · Biosystems Engineering · doi:10.1016/j.biosystemseng.2019.03.007 · Cited by 346
Li, Y., & Tong, S. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2598580 · Cited by 314
On Plant Detection of Intact Tomato Fruits Using Image Analysis and Machine Learning Methods
Yamamoto, K., Guo, W., Yoshioka, Y., & Ninomiya, S. (2014) · Sensors · doi:10.3390/s140712191 · Cited by 248
Tanigaki, K., Fujiura, T., Akase, A., & Imagawa, J. (2008) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2008.01.018 · Cited by 235
Computer vision for fruit harvesting robots state of the art and challenges ahead
Kapach, K., Barnea, E., Mairon, R., Edan, Y., & Shahar, O. B. (2012) · International Journal of Computational Vision and Robotics · doi:10.1504/ijcvr.2012.046419 · Cited by 225
Strawberry Yield Prediction Based on a Deep Neural Network Using High-Resolution Aerial Orthoimages
Chen, Y., Lee, W. S., Gan, H., Peres, N., Fraisse, C., Zhang, Y., & He, Y. (2019) · Remote Sensing · doi:10.3390/rs11131584 · Cited by 224
A SURVEY OF COMPUTER VISION METHODS FOR LOCATING FRUIT ON TREES
Jiménez, A. R., Ceres, R., & Pons, J. L. (2000) · Transactions of the ASAE · doi:10.13031/2013.3096 · Cited by 213
A field‐tested robotic harvesting system for iceberg lettuce
Birrell, S., Hughes, J., Cai, J. Y., & Iida, F. (2020) · Journal of Field Robotics · doi:10.1002/rob.21888 · Cited by 192
On-Tree Mango Fruit Size Estimation Using RGB-D Images
Wang, Z., Walsh, K., & Verma, B. (2017) · Sensors · doi:10.3390/s17122738 · Cited by 182
Automatic fruit recognition and counting from multiple images
Song, Y., Glasbey, C. A., Horgan, G. W., et al. (2014) · Biosystems Engineering · doi:10.1016/j.biosystemseng.2013.12.008 · Cited by 176
Kurtulmus, F., Lee, W. S., & Vardar, A. (2011) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2011.07.001 · Cited by 172
Single-Shot Convolution Neural Networks for Real-Time Fruit Detection Within the Tree
Bresilla, K., Perulli, G. D., Boini, A., Morandi, B., Corelli Grappadelli, L., & Manfrini, L. (2019) · Frontiers in Plant Science · doi:10.3389/fpls.2019.00611 · Cited by 168
Guava Detection and Pose Estimation Using a Low-Cost RGB-D Sensor in the Field
Lin, G., Tang, Y., Zou, X., Xiong, J., & Li, J. (2019) · Sensors · doi:10.3390/s19020428 · Cited by 160
Self-citationAutomatic apple recognition based on the fusion of color and 3D feature for robotic fruit picking
Tao, Y., & Zhou, J. (2017) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2017.09.019 · Cited by 159
Apple crop-load estimation with over-the-row machine vision system
Gongal, A., Silwal, A., Amatya, S., Karkee, M., Zhang, Q., & Lewis, K. (2015) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2015.10.022 · Cited by 157
Fruit Detection and Segmentation for Apple Harvesting Using Visual Sensor in Orchards
Kang, H., & Chen, C. (2019) · Sensors · doi:10.3390/s19204599 · Cited by 143
Luo, L., Tang, Y., Lu, Q., Chen, X., Zhang, P., & Zou, X. (2018) · Computers in Industry · doi:10.1016/j.compind.2018.03.017 · Cited by 138
Self-citationOptimal manipulator design for a cucumber harvesting robot
Henten, E. J. V., Slot, D. A. V. T., Hol, C. W. J., & van Willigenburg, L. G. (2008) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2008.11.004 · Cited by 132
A novel image processing algorithm to separate linearly clustered kiwifruits
Fu, L., Tola, E., Al-Mallahi, A. A., Li, R., & Cui, Y. (2019) · Biosystems Engineering · doi:10.1016/j.biosystemseng.2019.04.024 · Cited by 124
Improvements to and large‐scale evaluation of a robotic kiwifruit harvester
Williams, H. A., Ting, C., Nejati, M., Jones, M. H., Penhall, N., Lim, J., Seabright, M., Bell, J., Ahn, H. S., Scarfe, A. J., et al. (2019) · Journal of Field Robotics · doi:10.1002/rob.21890 · Cited by 124
Rakun, J., Stajnko, D., & Zazula, D. (2011) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2011.01.007 · Cited by 118
Mango Fruit Load Estimation Using a Video Based MangoYOLO—Kalman Filter—Hungarian Algorithm Method
Wang, Z., Walsh, K. B., & Koirala, A. (2019) · Sensors · doi:10.3390/s19122742 · Cited by 118
An integrated system for perception-driven autonomy with modular robots
Daudelin, J., Jing, G., Tosun, T., Yim, M. H., Kress‐Gazit, H., & Campbell, M. (2018) · Science Robotics · doi:10.1126/scirobotics.aat4983 · Cited by 115
On-tree fruit recognition using texture properties and color data
Zhao, J., Tow, J., & Katupitiya, J. (2005) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2005.1545592 · Cited by 114
Zhuang, J., Luo, S. M., Hou, C., Tang, Y., He, Y., & Xue, X. Y. (2018) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2018.07.004 · Cited by 113
Self-citationClassification of plant structures from uncalibrated image sequences
Dey, D., Mummert, L., & Sukthankar, R. (2012) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2012.6163017 · Cited by 104
Colour-agnostic shape-based 3D fruit detection for crop harvesting robots
Barnea, E., Mairon, R., & Ben‐Shahar, O. (2016) · Biosystems Engineering · doi:10.1016/j.biosystemseng.2016.01.013 · Cited by 103
Stem localization of sweet-pepper plants using the support wire as a visual cue
Bac, C. W., Hemming, J., & Henten, E. J. V. (2014) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2014.04.011 · Cited by 97
Orchard fruit segmentation using multi-spectral feature learning
Hung, C., Nieto, J. I., Taylor, Z., Underwood, J., & Sukkarieh, S. (2013) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros.2013.6697125 · Cited by 97
Xiong, J., Lin, R., Liu, Z., He, Z., Tang, L., Yang, Z., & Zou, X. (2017) · Biosystems Engineering · doi:10.1016/j.biosystemseng.2017.11.005 · Cited by 94
Self-citationEnd-effector with a bite mode for harvesting citrus fruit in random stalk orientation environment
Wang, Y., Yang, Y. H., Yang, C., Zhao, H., Chen, G., Zhang, Z., Fu, S., Zhang, M., & Xu, H. (2019) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2019.01.015 · Cited by 89
Visual positioning technology of picking robots for dynamic litchi clusters with disturbance
Xiong, J., He, Z., Lin, R., Liu, Z., Bu, R., Yang, Z., Peng, H., & Zou, X. (2018) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2018.06.007 · Cited by 86
Self-citationHonkavaara, E., Kaivosoja, J., Mäkynen, J., Pellikka, I., Pesonen, L. A., Saari, H. K., Salo, H., Hakala, T., Marklelin, L., & Rosnell, T. (2012) · ISPRS annals of the photogrammetry, remote sensing and spatial information sciences · doi:10.5194/isprsannals-i-7-353-2012 · Cited by 82
Breeding new varieties for controlled environments
Folta, K. M. (2018) · Plant Biology · doi:10.1111/plb.12914 · Cited by 82
Pattern recognition of fruit shape based on the concept of chaos and neural networks
Morimoto, T., Takeuchi, T., Miyata, H., & Hashimoto, Y. (2000) · Computers and Electronics in Agriculture · doi:10.1016/s0168-1699(00)00070-3 · Cited by 77
Kirk, R., Cielniak, G., & Mangan, M. (2020) · Sensors · doi:10.3390/s20010275 · Cited by 75
Arad, B., Kurtser, P., Barnea, E., Harel, B., Edan, Y., & Ben‐Shahar, O. (2019) · Sensors · doi:10.3390/s19061390 · Cited by 74
Zhuang, J., Hou, C., Tang, Y., He, Y., Guo, Q., Zhong, Z., & Luo, S. (2019) · Biosystems Engineering · doi:10.1016/j.biosystemseng.2019.08.016 · Cited by 72
Self-citationMajeed, Y., Zhang, J., Zhang, X., Fu, L., Karkee, M., Zhang, Q., & Whiting, M. D. (2018) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2018.08.064 · Cited by 68
Multispectral mapping in agriculture: Terrain mosaic using an autonomous quadcopter UAV
Navia, J., Mondragon, I. F., Patino, D. A., & Colorado, J. D. (2016) · International Conference on Unmanned Aircraft Systems (ICUAS) · doi:10.1109/icuas.2016.7502606 · Cited by 61
Fruit recognition based on convolution neural network
Hou, L., Wu, Q., Sun, Q., et al. (2016) · International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD) · doi:10.1109/fskd.2016.7603144 · Cited by 52
The use of agricultural robots in orchard management
Zhang, Q., Karkee, M., Tabb, A., et al. (2019) · Burleigh Dodds series in agricultural science · doi:10.19103/as.2019.0056.14 · Cited by 49
Angle estimation between plant parts for grasp optimisation in harvest robots
Barth, R., Hemming, J., & Henten, E. J. V. (2019) · Biosystems Engineering · doi:10.1016/j.biosystemseng.2019.04.006 · Cited by 40
Masuzawa, H., Miura, J., & Oishi, S. (2017) · IEEE/SICE International Symposium on System Integration (SII) · doi:10.1109/sii.2017.8279277 · Cited by 39
Xu, W., Chen, H., Su, Q., Ji, C., Xu, W., Memon, M. S., & Zhou, J. C. (2019) · Biosystems Engineering · doi:10.1016/j.biosystemseng.2019.06.016 · Cited by 38
GREENHOUSE MECHANIZATION: STATE OF THE ART AND FUTURE PERSPECTIVE
Henten, E. J. V. (2006) · Acta Horticulturae · doi:10.17660/actahortic.2006.710.3 · Cited by 34
A Visual System of Citrus Picking Robot Using Convolutional Neural Networks
Yan-ping, L., Yang, C., Huang, L., et al. (2018) · International Conference on Systems and Informatics (ICSAI) · doi:10.1109/icsai.2018.8599325 · Cited by 30
A Vision Servo System for Automated Harvest of Sweet Pepper in Korean Greenhouse Environment
Lee, B., Kam, D., Min, B., Hwa, J.-H., & Oh, S. (2019) · Applied Sciences · doi:10.3390/app9122395 · Cited by 29
Stereo vision with texture learning for fault-tolerant automatic baling
Blas, M. R., & Blanke, M. (2010) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2010.10.012 · Cited by 22
Matsuzaki, S., Masuzawa, H., Miura, J., & Oishi, S. (2018) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc.2018.00070 · Cited by 22
Design and implementation of an automatic peach-harvesting robot system
Yu, Y., Sun, Z., Zhao, X., et al. (2018) · International Conference on Advanced Computational Intelligence (ICACI) · doi:10.1109/icaci.2018.8377546 · Cited by 9
Cited by
Multi-class fruit-on-plant detection for apple in SNAP system using Faster R-CNN
Gao, F., Fu, L., Zhang, X., Majeed, Y., Li, R., Karkee, M., & Zhang, Q. (2020) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2020.105634 · Cited by 333
Fu, L., Gao, F., Wu, J., Li, R., Karkee, M., & Zhang, Q. (2020) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2020.105687 · Cited by 331
Intelligent robots for fruit harvesting: recent developments and future challenges
Zhou, H., Wang, X., Au, W., Kang, H., & Chen, C. (2022) · Precision Agriculture · doi:10.1007/s11119-022-09913-3 · Cited by 316
Rehman, A., Saba, T., Kashif, M., Fati, S. M., Bahaj, S. A., & Chaudhry, H. (2022) · Agronomy · doi:10.3390/agronomy12010127 · Cited by 306
Plant Disease Recognition Model Based on Improved YOLOv5
Chen, Z., Wu, R., Lin, Y., Li, C., Chen, S., Yuan, Z., Chen, S., & Zou, X. (2022) · Agronomy · doi:10.3390/agronomy12020365 · Cited by 272
Self-citation
Metadata
- Authors
- Tang, Y., Chen, M., Wang, C., Luo, L., Li, J., Lian, G., & Zou, X.
- Venue
- Frontiers in Plant Science
- Publisher
- Frontiers Media SA
- Published
- 2020-05-19
- Volume
- 11
- Pages
- 510-510
- ISSN
- 1664-462X (ISSN-L)
- OpenAlex
- W3026254524
- Cited by
- 575
- References
- 63
- FWCI
- 43.20
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.