NodeSLAM: Neural object descriptors for multi-view shape reconstruction
Open accessAbstract
The choice of scene representation is crucial in both the shape inference algorithms it requires and the smart applications it enables. We present efficient and optimisable multi-class learned object descriptors together with a novel probabilistic and differential rendering engine, for principled full object shape inference from one or more RGB-D images. Our framework allows for accurate and robust 3D object reconstruction which enables multiple applications including robot grasping and placing, augmented reality, and the first object-level SLAM system capable of optimising object poses and shapes jointly with camera trajectory.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Prior methods for object pose and segmentation typically operate on static inputs and do not allow the agent to interact with the scene.
“However, these methods typically operate on static inputs and do not allow the agent to interact with the scene.”
From Object-Centric Scene Representations Using Active Inference (2024) · cites this paper as Sucar et al., 2020
Section 6 describes NodeSLAM as building an object map incrementally from a stream of images, rather than operating only on static inputs. Section 7.4, “Robot Manipulation Application” (page unavailable), describes a robot using its reconstructions to pack and sort objects, contrary to the claim that the method does not allow interaction with the scene. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · see citing works
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137 · see citing works
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067 · see citing works
3D ShapeNets: A deep representation for volumetric shapes
Wu, Z., Song, S., Khosla, A., Yu, F., Zhang, L., Tang, X., & Xiao, J. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298801 · Cited by 5,413 · see citing works
Parallel Tracking and Mapping for Small AR Workspaces
Klein, G., & Murray, D. (2007) · 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality · doi:10.1109/ismar.2007.4538852 · Cited by 4,346 · see citing works
KinectFusion: Real-time dense surface mapping and tracking
Newcombe, R. A., Fitzgibbon, A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A. J., Kohi, P., Shotton, J., & Hodges, S. (2011) · 2011 10th IEEE International Symposium on Mixed and Augmented Reality · doi:10.1109/ismar.2011.6092378 · Cited by 4,008 · see citing works
Self-citationMonoSLAM: Real-time single camera SLAM
Davison, A. J., Reid, I. D., Molton, N. D., & Stasse, O. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1049 · Cited by 3,959 · see citing works
Self-citationDeepSDF: Learning Continuous Signed Distance Functions for Shape Representation
Park, J. J., Florence, P., Straub, J., Newcombe, R., & Lovegrove, S. J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00025 · Cited by 3,557 · see citing works
Engel, J. J., Koltun, V., & Cremers, D. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2658577 · Cited by 2,995 · see citing works
Occupancy Networks: Learning 3D Reconstruction in Function Space
Mescheder, L., Oechsle, M., Niemeyer, M., Nowozin, S., & Geiger, A. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00459 · Cited by 2,733 · see citing works
KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera
Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., Newcombe, R. A., Kohli, P., Shotton, J., Hodges, S., Freeman, D., Davison, A., et al. (2011) · ACM Symposium on User Interface Software and Technology · doi:10.1145/2047196.2047270 · Cited by 2,601 · see citing works
Self-citationShapeNet: An Information-Rich 3D Model Repository
Chang, A. X., Funkhouser, T., Guibas, L., Hanrahan, P., Huang, Q., Li, Z., Savarese, S., Savva, M., Song, S., Su, H., et al. (2015) · arXiv · doi:10.48550/arxiv.1512.03012 · Cited by 2,330 · see citing works
3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction
Choy, C., Xu, D., Gwak, J., Chen, K., & Savarese, S. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46484-8_38 · Cited by 1,741 · see citing works
ShapeNet: An Information-Rich 3D Model Repository
Chang, A. X., Funkhouser, T. A., Guibas, L. J., Hanrahan, P., Qi-xing Huang, A., Li, Z., Savarese, S., Savva, M., Song, S., Su, H., et al. (2023) · doi:10.5281/zenodo.10089705 · Cited by 1,624 · see citing works
Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modeling
Wu, J., Zhang, C., Xue, T., Freeman, W. T., & Tenenbaum, J. B. (2016) · arXiv · doi:10.48550/arxiv.1610.07584 · Cited by 1,393 · see citing works
Open3D: A Modern Library for 3D Data ProcessingPreprint
Zhou, Q.-Y., Park, J., & Koltun, V. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.09847 · Cited by 1,126 · see citing works
SLAM++: Simultaneous Localisation and Mapping at the Level of Objects
Salas-Moreno, R. F., Newcombe, R. A., Strasdat, H., Kelly, P. H. J., & Davison, A. J. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.178 · Cited by 945 · see citing works
Self-citationElasticFusion: Dense SLAM Without A Pose Graph
Whelan, T., Leutenegger, S., Salas Moreno, R., Glocker, B., & Davison, A. (2015) · Robotics: Science and Systems XI · doi:10.15607/rss.2015.xi.001 · Cited by 835 · see citing works
Self-citationDifferentiable Volumetric Rendering: Learning Implicit 3D Representations Without 3D Supervision
Niemeyer, M., Mescheder, L., Oechsle, M., & Geiger, A. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00356 · Cited by 832 · see citing works
Kajiya, J. T., & Herzen, B. P. V. (1984) · ACM SIGGRAPH Computer Graphics · doi:10.1145/964965.808594 · Cited by 709 · see citing works
SemanticFusion: Dense 3D semantic mapping with convolutional neural networks
McCormac, J., Handa, A., Davison, A. J., & Leutenegger, S. (2017) · Proceedings - IEEE International Conference on Robotics and Automation/Proceedings · doi:10.1109/icra.2017.7989538 · Cited by 684 · see citing works
Self-citationMulti-View Supervision for Single-View Reconstruction via Differentiable Ray Consistency
Tulsiani, S., Zhou, T., Efros, A. A., & Malik, J. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.30 · Cited by 528 · see citing works
Gkioxari, G., Johnson, J., & Malik, J. (2019) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00988 · Cited by 468 · see citing works
Pix3D: Dataset and Methods for Single-Image 3D Shape Modeling
Sun, X., Wu, J., Zhang, X., Zhang, Z., Zhang, C., Xue, T., Tenenbaum, J. B., & Freeman, W. T. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00314 · Cited by 456 · see citing works
3D-RCNN: Instance-Level 3D Object Reconstruction via Render-and-Compare
Kundu, A., Li, Y., & Rehg, J. M. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00375 · Cited by 331 · see citing works
Fusion++: Volumetric Object-Level SLAM
McCormac, J., Clark, R., Bloesch, M., Davison, A. J., & Leutenegger, S. (2018) · International Conference on 3D Vision (3DV) · doi:10.1109/3dv.2018.00015 · Cited by 317 · see citing works
Self-citationDIST: Rendering Deep Implicit Signed Distance Function With Differentiable Sphere Tracing
Liu, S., Zhang, Y., Peng, S., Shi, B., Pollefeys, M., & Cui, Z. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00209 · Cited by 256 · see citing works
Meaningful maps – Object-oriented semantic mapping
Sünderhauf, N., Pham, T. T., Latif, Y., Milford, M. J., & Reid, I. D. (2017) · QUT ePrints (Queensland University of Technology) · Cited by 219 · see citing works
Variational Autoencoders for Deforming 3D Mesh Models
Tan, Q., Gao, L., Lai, Y., et al. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00612 · Cited by 214 · see citing works
SDFDiff: Differentiable Rendering of Signed Distance Fields for 3D Shape Optimization
Jiang, Y., Ji, D., Han, Z., & Zwicker, M. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00133 · Cited by 211 · see citing works
Superquadrics Revisited: Learning 3D Shape Parsing Beyond Cuboids
Paschalidou, D., Ulusoy, A. O., & Geiger, A. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01059 · Cited by 187 · see citing works
Dense Reconstruction Using 3D Object Shape Priors
Dame, A., Prisacariu, V. A., Ren, C. Y., & Reid, I. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.170 · Cited by 135 · see citing works
DA-RNN: Semantic Mapping with Data Associated Recurrent Neural Networks
Yu, X., & Fox, D. (2017) · Robotics: Science and Systems · doi:10.15607/rss.2017.xiii.013 · Cited by 119 · see citing works
FroDO: From Detections to 3D Objects
Rünz, M., Li, K., Tang, M., Ma, L., Kong, C., Schmidt, T., Reid, I. D., Agapito, L., Straub, J., Lovegrove, S. J., et al. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.01473 · Cited by 69 · see citing works
SAMP: Shape and Motion Priors for 4D Vehicle Reconstruction
Engelmann, F., Stuckler, J., & Leibe, B. (2017) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2017.51 · Cited by 42 · see citing works
Object-Centric Photometric Bundle Adjustment with Deep Shape Prior
Zhu, R., Wang, C., Lin, C.-H., Wang, Z., & Lucey, S. (2018) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2018.00103 · Cited by 15 · see citing works
Deep-SLAM++: Object-level RGBD SLAM based on class-specific deep shape priorsPreprint
Hu, L., Xu, W., Huang, K., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.09691 · Cited by 12 · see citing works
Single-view Object Shape Reconstruction Using Deep Shape Prior and SilhouettePreprint
Li, K., Garg, R., Cai, M., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.11921 · Cited by 11 · see citing works
Wang, R., Yang, N., Stueckler, J., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.10097 · Cited by 9 · see citing works
DA-RNN: Semantic Mapping with Data Associated Recurrent Neural NetworksPreprint
Yu, X., & Fox, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.03098 · Cited by 6 · see citing works
Flagged first, then by citations
Research works printed but not linked: 6
- J. Engel, V. Koltun, and D. Cremers. Direct sparse odom- etry. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 2017.
- K. He, G. Gkioxari, P. Dollár, and R. Girshick. Mask r-cnn. In Proceedings of the International Conference on Computer Vision (ICCV), 2017.
- K. Simonyan and A. Zisserman. Very Deep Convolutional Networks for Large-Scale Image Recognition. In Proceed- ings of the International Conference on Learning Represen- tations (ICLR), 2015.
- K. Sohn, H. Lee, and X. Yan. Learning structured output representation using deep conditional generative models. In Neural Information Processing Systems (NIPS), 2015.
- N. Wang, Y. Zhang, Z. Li, Y. Fu, W. Liu, and Y.-G. Jiang. Pixel2Mesh: Generating 3D Mesh Models from Single RBG Images. In Proceedings of the European Conference on Computer Vision (ECCV), pages 52-67, 2018.
- J. Wu, Y. Wang, T. Xue, X. Sun, B. Freeman, and J. Tenen- baum. Marrnet: 3d shape reconstruction via 2.5 d sketches. In Neural Information Processing Systems (NIPS), 2017.
Cited by
Sorted by citations
Object-Centric Scene Representations Using Active Inference
Van de Maele, T., Verbelen, T., Mazzaglia, P., Ferraro, S., & Dhoedt, B. (2024) · Neural Computation · doi:10.1162/neco_a_01637 · Cited by 5 · see citing works
“Such representations can then be used to infer objects shape and pose in 3D from one or more RGB-D images (Sucar et al., 2020).”
Background
- Authors
- Sucar, E., Wada, K., & Davison, A.
- Affiliations
- Imperial College London (GB)
- Venue
- 2020 International Conference on 3D Vision (3DV)
- Publisher
- IEEE
- Published
- 2020-11
- Pages
- 949-958
- OpenAlex
- W3092584126
- Cited by
- 108
- References
- 40
- Citations by year
- 2026: 2 · 2025: 16 · 2024: 35 · 2023: 20 · 2022: 19 · 2021: 16
- FWCI
- 12.23
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- International Conference on 3D Vision (3DV) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Engineering and Physical Sciences Research Council (EP/S036636/1)
- Topics
- Physical Sciences › Engineering › Computational Mechanics › 3D Shape Modeling and Analysis
- Physical Sciences › Engineering › Aerospace Engineering › Robotics and Sensor-Based Localization
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Vision and Imaging
- Keywords
- 3D object reconstruction, RGB-D, differentiable rendering, object pose estimation, robotic grasping, augmented reality, scene representation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
