CoChat
Sign Up
Cited by 119Advanced Neural Network ApplicationsRobot Manipulation and LearningDomain Adaptation and Few-Shot Learning

A novel weakly-supervised approach for RGB-D-based nuclear waste object detection

Open access
Sun, Li, Zhao, Cheng, Yan, Zhi, Liu, Pengcheng, Duckett, Tom, Stolkin, Rustam·2019·IEEE Sensors Journal
DOI 10.1109/jsen.2018.2888815
Open this paper in CoChat — free accountOpen PDF

Abstract

This paper addresses the problem of RGBD-based detection and categorization of waste objects for nuclear decommissioning. To enable autonomous robotic manipulation for nuclear decommissioning, nuclear waste objects must be detected and categorized. However, as a novel industrial application, large amounts of annotated waste object data are currently unavailable. To overcome this problem, we propose a weakly supervised learning approach which is able to learn a deep convolutional neural network from unlabeled RGBD videos while requiring very few annotations. The proposed method also has the potential to be applied to other household or industrial applications. We evaluate our approach on the Washington RGB-D object recognition benchmark, achieving the state-of-the-art performance among semi-supervised methods. More importantly, we introduce a novel dataset, i.e., Birmingham nuclear waste simulants dataset, and evaluate our proposed approach on this novel industrial object recognition challenge. We further propose a complete real-time pipeline for RGBD-based detection and categorization of nuclear waste simulants. Our weakly supervised approach has demonstrated to be highly effective in solving a novel RGB-D object detection and recognition application with limited human annotations.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/jsen.2018.2888815

Cites

6 of 6 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • A novel multimodal convolutional neural network for RGB-D object detection achieved 49.1% mean average precision, processing images in real-time at 35.3 FPS on one single Nvidia GTX1080 GPU.

    “Achieved 49.1% mean average precision, processing images in real-time at 35.3 FPS on one single Nvidia GTX1080 GPU.”

    From Advanced Applications of Industrial Robotics: New Trends and Possibilities (2021) · cites this paper as [55]

    The running-time results (C. Industrial Object Recognition, p. 6) report 2–3 Hz on an NVIDIA TITAN X GPU, or up to 5 Hz with a lighter network and point-cloud down-sampling—not 35.3 FPS on a GTX1080. Table IV (p. 7) reports 80.85% instance-wise and 75.52% pixel-wise average precision, rather than the claimed 49.1% mean average precision. (LLM, conf=0.88)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 33

  1. [1][1] J. R. Uijlings, K. E. Van De Sande, T. Gevers, and A. W. Smeulders, 'Selective search for object recognition,' IJCV , 2013. -
  2. [2][2] P. Arbel´ aez, J. Pont-Tuset, J. T. Barron, F. Marques, and J. Malik, 'Multiscale combinatorial grouping,' in CVPR , 2014. -
  3. [3][3] R. Girshick, J. Donahue, T. Darrell, and J. Malik, 'Rich feature hierarchies for accurate object detection and semantic segmentation,' in CVPR , 2014. -
  4. [4][4] R. Girshick, 'Fast r-cnn,' in ICCV , 2015. -
  5. [6][6] W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C.-Y. Fu, and A. C. Berg, 'Ssd: Single shot multibox detector,' in ECCV , 2016. -
  6. [7][7] S. Gupta, R. Girshick, P. Arbel´ aez, and J. Malik, 'Learning rich features from rgb-d images for object detection and segmentation,' in ECCV , 2014. -
  7. [8][8] S. Song and J. Xiao, 'Sliding shapes for 3d object detection in depth images,' in ECCV , 2014. -
  8. [9][9] S. Song and J. Xiao, 'Deep sliding shapes for amodal 3d object detection in rgb-d images,' in CVPR , 2016. -
  9. [12][12] M. Schwarz, H. Schulz, and S. Behnke, 'Rgb-d object recognition and pose estimation based on pre-trained convolutional neural network features,' in ICRA , 2015. -
  10. [13][13] A. Eitel, J. T. Springenberg, L. Spinello, M. Riedmiller, and W. Burgard, 'Multimodal deep learning for robust rgb-d object recognition,' in IROS , 2015. -
  11. [15][15] Y. Cheng, R. Cai, C. Zhang, Z. Li, X. Zhao, K. Huang, and Y. Rui, 'Query adaptive similarity measure for rgb-d object recognition,' in ICCV , 2015. -
  12. [16][16] Y. Cheng, X. Zhao, R. Cai, Z. Li, K. Huang, and Y. Rui, 'Semisupervised multimodal deep learning for rgb-d object recognition,' in IJCAI , 2016. -
  13. [17][17] J. Hoffman, S. Gupta, J. Leong, S. Guadarrama, and T. Darrell, 'Crossmodal adaptation for rgb-d detection,' in ICRA , 2016. -
  14. [18][18] C. Wang and K. Siddiqi, 'Differential geometry boosts convolutional neural networks for object detection,' in CVPR Workshops , 2016. -
  15. [19][19] G. Papandreou, L.-C. Chen, K. P. Murphy, and A. L. Yuille, 'Weaklyand semi-supervised learning of a deep convolutional network for semantic image segmentation,' in ICCV , 2015. -
  16. [20][20] D. Barnes, W. Maddern, and I. Posner, 'Find your own way: Weaklysupervised segmentation of path proposals for urban autonomy,' arXiv preprint arXiv:1610.01238 , 2016. - · arXiv
  17. [21][21] A. Zeng, K.-T. Yu, S. Song, D. Suo, E. Walker Jr, A. Rodriguez, and J. Xiao, 'Multi-view self-supervised deep learning for 6d pose estimation in the amazon picking challenge,' arXiv preprint arXiv:1609.09475 , 2016. - · arXiv
  18. [23][23] Y. Jia, E. Shelh., J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell, 'Caffe: Convolutional architecture for fast feature embedding,' arXiv preprint arXiv:1408.5093 , 2014. - · arXiv
  19. [26][26] O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, M. Bernstein et al. , 'Imagenet large scale visual recognition challenge,' IJCV , 2015. -
  20. [27][27] Z. Wu, S. Song, A. K., F. Yu, L. Z., X. T., and J. Xiao, '3d shapenets: A deep representation for volumetric shapes,' in CVPR , 2015. -
  21. [28][28] S. Katz, A. Tal, and R. Basri, 'Direct visibility of point sets,' ACM Transactions on Graphics (TOG) , 2007. -
  22. [29][29] L. Sun, S. Rogers, G. Aragon-Camarasa, and J. P. Siebert, 'Recognising the clothing categories from free-configuration using gaussianprocess-based interactive perception,' in ICRA , 2016. -
  23. [30][30] K. Lai, L. Bo, X. Ren, and D. Fox, 'A large-scale hierarchical multiview rgb-d object dataset,' in ICRA , 2011. -
  24. [31][31] A. Sinha, J. Bai, and K. Ramani, 'Deep learning 3d shape surfaces using geometry images,' in ECCV , 2016. -
  25. [32][32] S. Ravanbakhsh, J. Schneider, and B. Poczos, 'Deep learning with sets and point clouds,' arXiv preprint arXiv:1611.04500 , 2016. - · arXiv
  26. [33][33] A. Brock, T. Lim, J. Ritchie, and N. Weston, 'Generative and discriminative voxel modeling with convolutional neural networks,' arXiv preprint arXiv:1608.04236 , 2016. - · arXiv
  27. [34][34] V. Hegde and R. Zadeh, 'Fusionnet: 3d object classification using multiple data representations,' CoRR , 2016. -
  28. [35][35] E. Johns, S. Leutenegger, and A. J. Davison, 'Pairwise decomposition of image sequences for active multi-view recognition,' in CVPR , 2016. -
  29. [36][36] H. Su, S. Maji, E. Kalogerakis, and E. Learned-Miller, 'Multi-view convolutional neural networks for 3d shape recognition,' in ICCV , 2015. -
  30. [37][37] K. Lai, L. Bo, X. Ren, and D. Fox, 'A large-scale hierarchical multiview rgb-d object dataset,' in ICRA , 2011. -
  31. [38][38] R. Socher, B. Huval, B. P. Bath, C. D. Manning, and A. Y. Ng, 'Convolutional-recursive deep learning for 3d object classification.' in NIPS , 2012. -
  32. [40][40] Y. Cheng, R. Cai, C. Zhang, Z. Li, X. Zhao, K. Huang, and Y. Rui, 'Query adaptive similarity measure for rgb-d object recognition,' in ICCV , 2015. -
  33. [41][41] H. F. Zaki, F. Shafait, and A. Mian, 'Convolutional hypercube pyramid for accurate rgb-d object category and instance recognition,' in ICRA , 2016. -

Cited by

Sorted by citations

Authors
Sun, L., Zhao, C., Yan, Z., Liu, P., Duckett, T., & Stolkin, R.
Affiliations
  • University of Oxford (GB)
  • University of Birmingham (GB)
  • Université de technologie de belfort-montbéliard (FR)
  • University of Lincoln (GB)
ORCID
Venue
IEEE Sensors Journal
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Published
2019-05-01
Volume
19
Issue
9
Pages
3487-3500
ISSN
  • 1530-437X (ISSN-L)
  • 1558-1748
  • 2379-9153
DOI
10.1109/jsen.2018.2888815
OpenAlex
W2905739540
Cited by
119
References
6
Citations by year
2026: 3 · 2025: 4 · 2024: 5 · 2023: 17 · 2022: 27 · 2021: 33 · 2020: 13 · 2019: 10 · 2018: 4 · 2017: 3
FWCI
7.27
Type
journal article
Language
en
Open access
Licence
Locations
  • IEEE Sensors Journal (publishedVersion)
  • Lincoln Repository (University of Lincoln) (acceptedVersion)
  • White Rose Research Online (University of Leeds, The University of Sheffield, University of York) (acceptedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • Figshare (submittedVersion)
  • Figshare (submittedVersion, cc-by)
  • Cardiff Metropolitan Research Repository (Cardiff Metropolitan University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • EU Projects (H2020 RoMaNS 645582, CHIST-ERA (Perception-Guided Robotic Grasping))
  • Engineering and Physical Sciences Research Council (EP/M026477/1 (UK-Korea Civil Nuclear Collaboration), EP/P017487/1 (Remote Sensing in Extreme Environments), EP/R02572X/1 (National Centre for Nuclear Robotics))
  • DISTINCTIVE
  • Research Councils UK
  • Royal Society
  • EU (H2020 ILIAD 732737)
  • National Natural Science Foundation of China (61803396)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Engineering › Control and Systems Engineering › Robot Manipulation and Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
Keywords
RGB-D object recognition, weakly supervised learning, nuclear decommissioning, semi-supervised learning, limited annotations

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review