The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Robust Fruit Counting: Combining Deep Learning, Tracking, and Structure from Motion
Open accessAbstract
We present a novel fruit counting pipeline that combines deep segmentation, frame to frame tracking, and 3D localization to accurately count visible fruits across a sequence of images. Our pipeline works on image streams from a monocular camera, both in natural light, as well as with controlled illumination at night. We first train a Fully Convolutional Network (FCN) and segment video frame images into fruit and non-fruit pixels. We then track fruits across frames using the Hungarian Algorithm where the objective cost is determined from a Kalman Filter corrected Kanade-Lucas-Tomasi (KLT) Tracker. In order to correct the estimated count from tracking process, we combine tracking results with a Structure from Motion (SfM) algorithm to calculate relative 3D locations and size estimates to reject outliers and double counted fruit tracks. We evaluate our algorithm by comparing with ground-truth human-annotated visual counts. Our results demonstrate that our pipeline is able to accurately and reliably count fruits across image sequences, and the correction step can significantly improve the counting accuracy and robustness. Although discussed in the context of fruit counting, our work can extend to detection, tracking, and counting of a variety of other stationary features of interest such as leaf-spots, wilt, and blossom.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“For example, crop counting and yield estimation is an important topic of precision agriculture [202], [141].”
From Empowering things with intelligence: a survey of the progress, challenges, and opportunities in artificial intelligence of things (2021) · cites this paper as [141]
In the citing paper
“To address this issue, a detection-tracking-counting based method is proposed [141], which can reject outliers and double-counted fruits.”
From Empowering things with intelligence: a survey of the progress, challenges, and opportunities in artificial intelligence of things (2021) · cites this paper as [141]
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834
A New Approach to Linear Filtering and Prediction Problems
Kalman, R. E. (1960) · Journal of Basic Engineering · doi:10.1115/1.3662552 · Cited by 31,751
(1981) · Cited by 11,963
Structure-from-Motion Revisited
Schonberger, J. L., & Frahm, J.-M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.445 · Cited by 6,433
Algorithms for the Assignment and Transportation Problems
Munkres, J. R. (1957) · Journal of the Society for Industrial and Applied Mathematics · doi:10.1137/0105003 · Cited by 4,001
An algorithm for tracking multiple targets
Reid, D. (1979) · IEEE Transactions on Automatic Control · doi:10.1109/tac.1979.1102177 · Cited by 3,059
DeepFruits: A Fruit Detection System Using Deep Neural Networks
Sa, I., Ge, Z., Dayoub, F., Upcroft, B., Pérez, T., & McCool, C. (2016) · Sensors · doi:10.3390/s16081222 · Cited by 1,155
Deep fruit detection in orchards
Bargoti, S., & Underwood, J. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989417 · Cited by 607
Deep Count: Fruit Counting Based on Deep Simulated Learning
Rahnemoonfar, M., & Sheppard, C. (2017) · Sensors · doi:10.3390/s17040905 · Cited by 540
Image Segmentation for Fruit Detection and Yield Estimation in Apple Orchards
Bargoti, S., & Underwood, J. (2017) · Journal of Field Robotics · doi:10.1002/rob.21699 · Cited by 518
Deep Learning Based Oil Palm Tree Detection and Counting for High-Resolution Remote Sensing Images
Li, W., Fu, H., Yu, L., & Cracknell, A. P. (2016) · Remote Sensing · doi:10.3390/rs9010022 · Cited by 428
Counting Apples and Oranges With Deep Learning: A Data-Driven Approach
Chen, S. W., Shivakumar, S. S., Dcunha, S., Das, J., Okon, E., Qu, C., Taylor, C. J., & Kumar, V. (2017) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2017.2651944 · Cited by 406
Self-citationCoupled Detection and Trajectory Estimation for Multi-Object Tracking
Leibe, B., Konrad Schindler, B., & Gool, L. V. (2007) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2007.4408936 · Cited by 281
Automated Crop Yield Estimation for Apple Orchards
Wang, Q., Nuske, S. T., Bergerman, M., & Singh, S. (2013) · Springer tracts in advanced robotics · doi:10.1007/978-3-319-00065-7_50 · Cited by 233
Data synthesis methods for semantic segmentation in agriculture: A Capsicum annuum dataset
Barth, R., IJsselmuiden, J. M. M., Hemming, J., & Henten, E. J. V. (2017) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2017.12.001 · Cited by 191
Devices, systems, and methods for automated monitoring enabling precision agriculture
Das, J., Cross, G., Qu, C., Makineni, A., Tokekar, P., Mulgaonkar, Y., & Kumar, V. (2015) · IEEE International Conference on Automation Science and Engineering (CASE) · doi:10.1109/coase.2015.7294123 · Cited by 146
Self-citationTexture-based fruit detection via images using the smooth patterns on the fruit
Pothen, Z. S., & Nuske, S. T. (2016) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2016.7487722 · Cited by 41
Surveying apple orchards with a monocular vision system
Roy, P., & Isler, V. (2016) · 2022 IEEE 18th International Conference on Automation Science and Engineering (CASE) · doi:10.1109/coase.2016.7743500 · Cited by 25
Research works printed but not linked: 1
- 10.1109/COASE.2016.7743500 · DOI
Cited by
Sorted by citations
Zhang, J., & Tao, D. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3039359 · Cited by 774
Deep learning – Method overview and review of use for fruit detection and yield estimation
Koirala, A., Walsh, K. B., Wang, Z., & McCarthy, C. L. (2019) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2019.04.017 · Cited by 619
Machine learning in agriculture domain: A state-of-art survey
Meshram, V. A., Patil, K. R., Meshram, V. V., Hanchate, D. B., & Ramkteke, S. D. (2021) · Artificial Intelligence in the Life Sciences · doi:10.1016/j.ailsci.2021.100010 · Cited by 421
Atitallah, S. B., Driss, M., Boulila, W., & Ghézala, H. B. (2020) · Computer Science Review · doi:10.1016/j.cosrev.2020.100303 · Cited by 375
Mohyuddin, G., Khan, M. A., Haseeb, A., Mahpara, S., Waseem, M., & Saleh, A. M. (2024) · IEEE Access · doi:10.1109/access.2024.3390581 · Cited by 283
Metadata
- Authors
- Liu, X., Chen, S. W., Aditya, S., Sivakumar, N., Dcunha, S., Qu, C., Taylor, C. J., Das, J., & Kumar, V.
- Affiliations
- University of Pennsylvania (US)
- Venue
- 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
- Publisher
- IEEE
- Published
- 2018-10
- Pages
- 1045-1052
- ISSN
- 2153-0858 (ISSN-L)
- OpenAlex
- W2963353662
- Cited by
- 135
- References
- 21
- Citations by year
- 2026: 7 · 2025: 13 · 2024: 18 · 2023: 23 · 2022: 18 · 2021: 22 · 2020: 21 · 2019: 13
- FWCI
- 40.72
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- U.S. Department of Agriculture (2015-67021-23857)
- University of Pennsylvania
- Washington State University
- National Institute of Food and Agriculture (67021)
- Topics
- Life Sciences › Agricultural and Biological Sciences › Plant Science › Smart Agriculture and AI
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Video Surveillance and Tracking Methods
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- fruit counting, fruit segmentation, object tracking, structure from motion, 3D localization, monocular camera, nighttime imaging, fully convolutional network, Kalman filter, Kanade-Lucas-Tomasi tracker, Hungarian algorithm, outlier removal, double-counting correction, plant phenotyping
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
