CoChat
Sign Up
Cited by 78Robot Manipulation and LearningAdvanced Manufacturing and Logistics OptimizationRobotics and Automated Systems

What are the important technologies for bin picking? Technology analysis of robots in competitions based on a set of performance metrics

Open access
Fujita, M., Domae, Y., Noda, A., Garcia Ricardez, G. A., Nagatani, T., Zeng, A., Song, S., Rodriguez, A. +3 more·2019·Advanced Robotics
DOI 10.1080/01691864.2019.1698463
Open this paper in CoChat — free accountOpen PDF

Abstract

Bin picking is still a challenge in robotics, as patent in recent robot competitions. These competitions are an excellent platform for technology comparisons since some participants may use state-of-the-art technologies, while others may use conventional ones. Nevertheless, even though points are awarded or subtracted based on the performance in the frame of the competition rules, the final score does not directly reflect the suitability of the technology. Therefore, it is difficult to understand which technologies and their combination are optimal for various real-world problems. In this paper, we propose a set of performance metrics selected in terms of actual field use as a solution to clarify the important technologies in bin picking. Moreover, we use the selected metrics to compare our four original robot systems, which achieved the best performance in the Stow task of the Amazon Robotics Challenge 2017. Based on this comparison, we discuss which technologies are ideal for practical use in bin picking robots in the fields of factory and warehouse automation.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1080/01691864.2019.1698463

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • A set of performance metrics for bin picking compared four robotic systems, and the proposed system took first place in the Amazon Robotics Challenge 2017.

    “The authors proposed a set of performance metrics and compared four robotic systems for bin picking, and took first place in the Amazon Robotics Challenge 2017.”

    From Advanced Applications of Industrial Robotics: New Trends and Possibilities (2021) · cites this paper as [36]

    The Introduction and Conclusions say the paper proposes performance metrics and compares four systems from the 2017 Amazon Robotics Challenge. Section 5.2 identifies MIT-Princeton as the winning team, but the paper does not describe a single proposed system that took first place; it proposes metrics for analyzing the systems. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • The most promising solutions for bin picking are RGB-D sensors and CNN-based algorithms for object recognition, and a combination of suction-based and typical two-finger grippers for grasping different objects.

    “The survey results show that the most promising solutions for such a task are RGB-D sensors and CNN-based algorithms for object recognition, and a combination of suction-based and typical two-finger grippers for grasping different objects”

    From Advanced Applications of Industrial Robotics: New Trends and Possibilities (2021) · cites this paper as [36]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 33

  1. Marvel JA, Saidi K, Eastman R, et al. Technology readi- ness levels for randomized bin picking. Proceedings of the Workshop on Performance Metrics for Intelligent Sys- tems. PerMIS '12. College Park, MD. New York (NY): ACM; 2012. p. 109-113.
  2. Yokokohji Y, Kawai Y, Shibata M, et al. Assembly chal- lenge: a robot competition of the Industrial Robotics category, World Robot Summit -summary of the pre- competition in 2018. Adv Robot. 2019;33(17):876-899.
  3. Jolliffe IT. Principal Component Analysis. 2nd ed. New York (NY): Springer; 2002.
  4. Hernandez C, Bharatheesha M, Ko W, et al. Team Delft's robot winner of the Amazon Picking Challenge 2016. In: Robot World Cup. Springer; 2016. p. 613-624. Available from: https://link.springer.com/chapter/10.1007/978-3- 319-68792-6_51#citeas · DOI
  5. Morrison D, Tow AW, McTaggart M. Cartman: The low-cost Cartesian manipulator that won the Amazon Robotics Challenge. 2018 IEEE International Conference on Robotics and Automation (ICRA); Brisbane, Australia. IEEE; 2018. p. 7757-7764.
  6. Garcia Ricardez GA, El Hafi L, von Drigalski F. Stand- ing on giant's shoulders: newcomer's experience from the Amazon Robotics Challenge 2017. In: Advances on Robotics Item Picking -applications in ware- house & e-commerce. Springer; 2020. Available from: https://www.springer.com/gp/book/9783030356781.
  7. Zeng A, Song S, Yu KT, et al. Robotic pick-and-place of novel objects in clutter with multi-affordance grasping and cross-domain image matching. 2018 IEEE Interna- tional Conference on Robotics and Automation (ICRA); Brisbane, Australia. IEEE; 2018. p. 1-8.
  8. Causo A, Chong ZH, Luxman R, et al. A robust robot design for item picking. 2018 IEEE International Con- ference on Robotics and Automation (ICRA); Brisbane, Australia. IEEE; 2018. p. 7421-7426.
  9. Fujita M, Domae Y, Kawanishi R, et al. Bin-picking robot using a multi-gripper switching strategy based on object sparseness. 2019 IEEE International Conference on Automation Science and Engineering; Vancouver, BC, Canada. IEEE; 2019. p. 1540-1547.
  10. John A, Eric B, Rodenberg N, et al. A positive pres- sure universal gripper based on the jamming of granular material. IEEE Trans Robot. 2012;28(2):341-350.
  11. Domae Y, Okuda H, Taguchi Y, et al. Fast graspability evaluation on single depth maps for bin picking with general grippers. 2014 IEEE International Conference on Robotics and Automation (ICRA); Hong Kong. IEEE; 2014. p. 1997-2004.
  12. Jiang Y, Moseson S, Saxena A. Efficient grasping from RGBD images: Learning using a new rectangle rep- resentation. 2011 IEEE International Conference on Robotics and Automation; Shanghai, China. IEEE; 2011. p. 3304-3311.
  13. Lenz I, Lee H, Saxena A. Deep learning for detecting robotic grasps. Int J Robot Res. 2015;34(4-5):705-724.
  14. Pinto L, Gupta A. Supersizing self-supervision: learning to grasp from 50 k tries and 700 robot hours. 2016 IEEE International Conference on Robotics and Automation (ICRA). IEEE; 2016. p. 3406-3413.
  15. Levine S, Pastor P, Krizhevsky A, et al. Learning hand- eye coordination for robotic grasping with deep learn- ing and large-scale data collection. Int J Robot Res. 2018;37(4-5):421-436.
  16. Rodriguez A, Mason MT, Ferry S. From caging to grasp- ing. Int J Robot Res. 2012;31(7):886-900.
  17. Mahler J, Pokorny FT, Hou B, et al. Dex-Net 1.0: a cloud-based network of 3D objects for robust grasp plan- ning using a multi-armed bandit model with correlated rewards. 2016 IEEE International Conference on Robotics and Automation (ICRA). IEEE; 2016. p. 1957-1964.
  18. Mahler J, Liang J, Niyaz S, et al. Dex-Net 2.0: deep learn- ing to plan robust grasps with synthetic point clouds and analytic grasp metrics. arXiv preprint arXiv:170309312. 2017.
  19. Mahler J, Matl M, Liu X, et al. Dex-Net 3.0: comput- ing robust vacuum suction grasp targets in point clouds using a new analytic model and deep learning. 2018 IEEE International Conference on Robotics and Automation (ICRA); Brisbane, Australia. IEEE; 2018. p. 1-8.
  20. Matsumura R, Harada K, Domae Y, et al. Learning based industrial bin-picking trained with approximate physics simulator. International Conference on Intelligent Autonomous Systems; Baden-Baden, Germany. Springer; 2018. p. 786-798.
  21. Jonschkowski R, Eppner C, Höfer S. Probabilistic multi- class segmentation for the Amazon Picking Challenge. 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS); Daejeon, Korea. IEEE; 2016. p. 1-7.
  22. Ren S, He K, Girshick R, et al. R-CNN: towards real- time object detection with region proposal networks. Advances in Neural Information Processing Systems. 2015. p. 91-99.
  23. Liu W, Anguelov D, Erhan D, et al. SSD: single shot multi- box detector. In: Leibe B, Matas J, Sebe N, et al., editors. Computer Vision -ECCV 2016; Amsterdam, The Nether- lands. Cham: Springer International Publishing; 2016. p. 21-37.
  24. Besl PJ, McKay ND. A method for registration of 3-D shapes. IEEE Trans Pattern Anal Mach Intell. 1992 Feb;14(2):239-256.
  25. Liu MY, Tuzel O, Veeraraghavan A, et al. Fast directional chamfer matching. 2010 IEEE Computer Society Con- ference on Computer Vision and Pattern Recognition;
  26. Drost B, Ulrich M, Navab N, et al. Model globally, match locally: efficient and robust 3D object recognition. 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE; 2010. p. 998-1005.
  27. Choi C, Taguchi Y, Tuzel O, et al. Voting-based pose estimation for robotic assembly using a 3D sensor. 2012 IEEE International Conference on Robotics and Automa- tion; Saint Paul, Minnesota, USA. IEEE; 2012. p. 1724- 1731.
  28. Torii T, Hashimoto M. Model-less estimation method for robot grasping parameters using 3D shape primitive approximation. 2018 IEEE 14th International Confer- ence on Automation Science and Engineering (CASE); Munich, Germany. IEEE; 2018. p. 580-585.
  29. Zeng A, Yu KT, Song S, et al. Multi-view self-supervised deep learning for 6D pose estimation in the Amazon Pick- ing Challenge. 2017 IEEE International Conference on Robotics and Automation (ICRA); Singapore. IEEE; 2017. p. 1386-1383.
  30. Bian J, Lin W, Matsushita Y. Gms: grid-based motion statistics for fast, ultra-robust feature correspondence. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR); Honolulu, Hawaii, USA. 2017 Jul. p. 2828-2837.
  31. Garcia Ricardez GA, von Drigalski F, El Hafi L, et al. Lessons from the Airbus Shopfloor Challenge 2016 and the Amazon Robotics Challenge 2017. 18th SICE System Integration Division Annual Conf. Sendai, Japan. 2017 Dec. p. 572-575.
  32. Garcia Ricardez GA, von Drigalski F, El Hafi L, et al. Warehouse picking automation system with learning- and feature-based object recognition and grasping point estimation. 18th SICE System Integration Division Annual Conf. Sendai, Japan. 2017 Dec. p. 2249-2253.
  33. Zeng A, Song S, Lee J. Tossingbot: learning to throw arbitrary objects with residual physics. arXiv preprint arXiv:190311239. 2019.

Cited by

Sorted by citations

Authors
Fujita, M., Domae, Y., Noda, A., Garcia Ricardez, G. A., Nagatani, T., Zeng, A., Song, S., Rodriguez, A., Causo, A., Chen, I. M., & Ogasawara, T.
Affiliations
  • Mitsubishi Electric (Japan) (JP)
  • Artificial Intelligence Research Center (JP)
  • Osaka Institute of Technology (JP)
  • Division of Information Science, Nara Institute of Science and Technology, Nara, Japan
  • Princeton University (US)
  • Mechanical Engineering Department, Massachusetts Institute of Technology, Boston, MA, USA
  • Nanyang Technological University (SG)
  • Nara Institute of Science and Technology (JP)
Corresponding author
Domae, Y.
ORCID
Venue
Advanced Robotics
Publisher
Informa UK Limited
Published
2019-12-07
Pages
1-15
ISSN
  • 0169-1864 (ISSN-L)
  • 1568-5535
DOI
10.1080/01691864.2019.1698463
OpenAlex
W2993476843
Cited by
78
References
2
Citations by year
2026: 2 · 2025: 10 · 2024: 10 · 2023: 15 · 2022: 19 · 2021: 18 · 2020: 4
FWCI
5.49
Type
journal article
Language
en
Open access
Licence
Locations
  • Advanced Robotics (publishedVersion, cc-by-nc-nd)
  • DR-NTU (Nanyang Technological University) (submittedVersion, cc-by-nc-nd)
  • DSpace@MIT (Massachusetts Institute of Technology) (submittedVersion, cc-by-nc-nd)
Topics
  • Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
  • Physical Sciences › Engineering › Industrial and Manufacturing Engineering › Advanced Manufacturing and Logistics Optimization
  • Physical Sciences › Engineering › Control and Systems Engineering › Robotics and Automated Systems
Keywords
bin picking, performance indicators, technology comparison, robot competition, warehouse automation, factory automation, robotic grasping

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review