CoChat
Sign Up
PreprintCited by 65Space Satellite Systems and ControlAdvanced Neural Network ApplicationsSpacecraft Dynamics and Control

Towards Robust Learning-Based Pose Estimation of Noncooperative Spacecraft

Open access
Park, Tae Ha, Sharma, Sumant, D'Amico, Simone·2019·arXiv
DOI 10.48550/arxiv.1909.00392arXiv 1909.00392
Open this paper in CoChat — free accountOpen PDF

Abstract

This work presents a novel Convolutional Neural Network (CNN) architecture and a training procedure to enable robust and accurate pose estimation of a noncooperative spacecraft. First, a new CNN architecture is introduced that has scored a fourth place in the recent Pose Estimation Challenge hosted by Stanford's Space Rendezvous Laboratory (SLAB) and the Advanced Concepts Team (ACT) of the European Space Agency (ESA). The proposed architecture first detects the object by regressing a 2D bounding box, then a separate network regresses the 2D locations of the known surface keypoints from an image of the target cropped around the detected Region-of-Interest (RoI). In a single-image pose estimation problem, the extracted 2D keypoints can be used in conjunction with corresponding 3D model coordinates to compute relative pose via the Perspective-n-Point (PnP) problem. These keypoint locations have known correspondences to those in the 3D model, since the CNN is trained to predict the corners in a pre-defined order, allowing for bypassing the computationally expensive feature matching processes. This work also introduces and explores the texture randomization to train a CNN for spaceborne applications. Specifically, Neural Style Transfer (NST) is applied to randomize the texture of the spacecraft in synthetically rendered images. It is shown that using the texture-randomized images of spacecraft for training improves the network's performance on spaceborne images without exposure to them during training. It is also shown that when using the texture-randomized spacecraft images during training, regressing 3D bounding box corners leads to better performance on spaceborne images than regressing surface keypoints, as NST inevitably distorts the spacecraft's geometric features to which the surface keypoints have closer relation.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1909.00392

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The TANGO satellite model has a coarse dimension of 80 × 75 × 32 cm and leads to an operation range of approximately 10 m.

    “The TANGO satellite [145] model used in the multiple datasets [19,20,51,51,85,132] has a coarse dimension of 80 × 75 × 32 cm will lead to the operation range of ∼10 m.”

    From A survey on deep learning-based monocular spacecraft pose estimation: Current state, limitations and prospects (2023) · cites this paper as [51]

    The Single Image Pose Estimation section (p. 4) says the method uses a wireframe model of Tango, supporting the model identification. Our copy does not give dimensions of 80 × 75 × 32 cm or say that the model's size leads to an operating range of approximately 10 m. The Introduction (p. 3) instead describes performance at separations from 3 to over 30 m. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • The algorithms [44,54,99] and the SLAB Baseline [51] provide a good trade-off in terms of performance and number of parameters, with position and orientation errors of less than 0.30 m and 4 degrees, respectively, while using fewer than 20 million parameters.

    “The results show that the algorithms[44,54,99] and the SLAB Baseline[51] provide a good trade-off in terms of the performance and the number of parameters, as shown by the red regions of each plot.”

    From A survey on deep learning-based monocular spacecraft pose estimation: Current state, limitations and prospects (2023) · cites this paper as [51]

    In Experiment 1 (pp. 11, 15), the paper reports 11.17 million parameters for its proposed architecture, a mean rotation error of 3.097 degrees, and mean translation-error components of 0.010, 0.011, and 0.210 m on the synthetic validation set. These figures support a performance–parameter trade-off for that architecture, but the paper does not establish the stated trade-off for the other cited algorithms or a SLAB Baseline; nor does it report a single position-error figure below 0.30 m for that group. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • The baseline performance study uses three pose estimation CNNs: KRN, SPN, and HigherHRNet.

    “First, the baseline performance study uses three different pose estimation CNNs: the Keypoint Regression Network (KRN) [4] and Spacecraft Pose Network (SPN) [3], which are the two baseline models for SPEED, and HigherHRNet [41], the state-of-the art model for the bottom-up human joint detection task.”

    From SPEED+: Next-Generation Dataset for Spacecraft Pose Estimation across Domain Gap (2022) · cites this paper as [4]

  • KRN directly regresses the locations of 11 pre-designated keypoints of the Tango spacecraft's corners and antennae.

    “On the other hand, KRN directly regress the locations of 11 pre-designated keypoints of the Tango spacecraft's corners and antennae.”

    From SPEED+: Next-Generation Dataset for Spacecraft Pose Estimation across Domain Gap (2022) · cites this paper as [4]

  • The algorithms [44,54,99] and the SLAB Baseline [51] provide a good trade-off in terms of performance and number of parameters.

    “The results show that the algorithms [44,54,99] and the SLAB Baseline [51] provide a good trade-off in terms of the performance and the number of parameters.”

    From A survey on deep learning-based monocular spacecraft pose estimation: Current state, limitations and prospects (2023) · cites this paper as [51]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

No citation lists for this paper

No citation lists for this paper yet — checked again daily.

References 43 · Cited by 65

Research works printed but not linked: 33

  1. [1][1] J. L. Forshaw, G. S. Aglietti, N. Navarathinam, H. Kadhem, T. Salmon, A. Pisseloup, E. Joffre, T. Chabot, I. Retat, R. Axthelm, and e. al., 'RemoveDEBRIS: An in-orbit active debris removal demonstration mission,' Acta Astronautica , Vol. 127, 2016, p. 448463, 10.1016/j.actaastro.2016.06.018. - · DOI · Paper page
  2. [2][2] B. Sullivan, D. Barnhart, L. Hill, P. Oppenheimer, B. L. Benedict, G. V. Ommering, L. Chappell, J. Ratti, and P. Will, 'DARPA Phoenix Payload Orbital Delivery System (PODs): FedEx to GEO,' AIAA SPACE 2013 Conference and Exposition , 2013, 10.2514/6.2013-5484. - · DOI
  3. [5][5] S. Sharma, J. Ventura, and S. DAmico, 'Robust Model-Based Monocular Pose Initialization for Noncooperative Spacecraft Rendezvous,' Journal of Spacecraft and Rockets , 2018, p. 116, 10.2514/1.a34124. - · DOI · Paper page
  4. [7][7] S. Sharma and S. D'Amico, 'Comparative assessment of techniques for initial pose estimation using monocular vision,' Acta Astronautica , Vol. 123, 2016, p. 435445, 10.1016/j.actaastro.2015.12.032. - · DOI · Paper page
  5. [8][8] S. Tulsiani and J. Malik, 'Viewpoints and keypoints,' 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2015, 10.1109/cvpr.2015.7298758. - · DOI
  6. [9][9] H. Su, C. R. Qi, Y. Li, and L. J. Guibas, 'Render for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Views,' 2015 IEEE International Conference on Computer Vision (ICCV) , 2015, 10.1109/iccv.2015.308. - · DOI
  7. [10][10] P. Poirson, P. Ammirato, C. Fu, W. Liu, J. Kosecka, and A. C. Berg, 'Fast Single Shot Detection and Pose Estimation,' CoRR , Vol. abs/1609.05590, 2016. - · arXiv
  8. [11][11] W. Kehl, F. Manhardt, F. Tombari, S. Ilic, and N. Navab, 'SSD-6D: Making RGB-Based 3D Detection and 6D Pose Estimation Great Again,' 2017 IEEE International Conference on Computer Vision (ICCV) , 2017, 10.1109/iccv.2017.169. - · DOI · Paper page
  9. [12][12] M. Sundermeyer, Z.-C. Marton, M. Durner, M. Brucker, and R. Triebel, 'Implicit 3D Orientation Learning for 6D Object Detection from RGB Images,' The European Conference on Computer Vision (ECCV) , September 2018. -
  10. [13][13] A. Kendall, M. Grimes, and R. Cipolla, 'PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization,' 2015 IEEE International Conference on Computer Vision (ICCV) , 2015, 10.1109/iccv.2015.336. - · DOI · Paper page
  11. [15][15] Y. Xiang, T. Schmidt, V. Narayanan, and D. Fox, 'PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes,' Robotics: Science and Systems XIV , 2018, 10.15607/rss.2018.xiv.019. - · DOI · Paper page
  12. [16][16] Y. Li, G. Wang, X. Ji, Y. Xiang, and D. Fox, 'DeepIM: Deep Iterative Matching for 6D Pose Estimation,' Computer Vision ECCV 2018 Lecture Notes in Computer Science , 2018, p. 695711. -
  13. [17][17] M. Rad and V. Lepetit, 'BB8: A Scalable, Accurate, Robust to Partial Occlusion Method for Predicting the 3D Poses of Challenging Objects without Using Depth,' 2017 IEEE International Conference on Computer Vision (ICCV) , 2017, 10.1109/iccv.2017.413. - · DOI
  14. [18][18] B. Tekin, S. N. Sinha, and P. Fua, 'Real-Time Seamless Single Shot 6D Object Pose Prediction,' CVPR , 2018. -
  15. [19][19] J. Tremblay, T. To, B. Sundaralingam, Y. Xiang, D. Fox, and S. Birchfield, 'Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objects,' CoRR abs/1809.10790 , 2018. - · arXiv
  16. [20][20] Z. Zhao, G. Peng, H. Wang, H. Fang, C. Li, and C. Lu, 'Estimating 6D Pose From Localizing Designated Surface Keypoints,' ArXiv , Vol. abs/1812.01387, 2018. - · arXiv
  17. [21][21] S. Peng, Y. Liu, Q. Huang, X. Zhou, and H. Bao, 'PVNet: Pixel-wise Voting Network for 6DoF Pose Estimation,' The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Oral , 2019. -
  18. [22][22] J. Redmon and A. Farhadi, 'YOLO9000: Better, Faster, Stronger,' 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , 2017, 10.1109/cvpr.2017.690. - · DOI · Paper page
  19. [23][23] V. Lepetit, F. Moreno-Noguer, and P. Fua, 'EPnP: An Accurate O(n) Solution to the PnP Problem,' International Journal of Computer Vision , Vol. 81, No. 2, 2008, p. 155166, 10.1007/s11263-008-01526. - · DOI
  20. [24][24] J. Redmon and A. Farhadi, 'YOLOv3: An Incremental Improvement,' CoRR , Vol. abs/1804.02767, 2018. - · arXiv
  21. [27][27] M. A. Fischler and R. C. Bolles, 'Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography,' Readings in Computer Vision , 1987, p. 726740, 10.1016/b978-0-08-051581-6.50070-2. - · DOI
  22. [29][29] E. Brachmann, A. Krull, F. Michel, S. Gumhold, J. Shotton, and C. Rother, 'Learning 6D Object Pose Estimation Using 3D Object Coordinates,' Computer Vision ECCV 2014 Lecture Notes in Computer Science , 2014, p. 536551. -
  23. [30][30] S. Sharma, C. Beierle, and S. D'Amico, 'Pose estimation for non-cooperative spacecraft rendezvous using convolutional neural networks,' 2018 IEEE Aerospace Conference , March 2018, pp. 1-12, 10.1109/AERO.2018.8396425. - · DOI · Paper page
  24. [31][31] J.-F. Shi, S. Ulrich, and S. Ruel, CubeSat Simulation and Detection using Monocular Camera Images and Convolutional Neural Networks , 10.2514/6.2018-1604. - · DOI
  25. [32][32] S. Sharma and S. D'Amico, 'Pose Estimation for Non-Cooperative Rendezvous Using Neural Networks,' 2019 AAS/AIAA Astrodynamics Specialist Conference, Ka'anapali, Maui, HI , January 13-17 2019. -
  26. [33][33] S. D'Amico, P. Bodin, M. Delpech, and R. Noteborn, 'PRISMA,' Distributed Space Missions for Earth System Monitoring Space Technology Library (M. D'Errico, ed.), Vol. 31, ch. 21, pp. 599-637, 2013, 10.1007/978-1-4614-4541-8 21. - · DOI
  27. [34][34] M. A. Fischler and R. C. Bolles, 'Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography,' Communications of the ACM , Vol. 24, No. 6, 1981, p. 381395, 10.1145/358669.358692. - · DOI · Paper page
  28. [39][39] M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L. Chen, 'MobileNetV2: Inverted Residuals and Linear Bottlenecks,' 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition , June 2018, pp. 4510-4520, 10.1109/CVPR.2018.00474. - · DOI · Paper page
  29. [40][40] A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, 'MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications,' 2017. cite arxiv:1704.04861. - · arXiv
  30. [44][44] T.-Y. Lin, M. Maire, S. J. Belongie, L. D. Bourdev, R. B. Girshick, J. Hays, P. Perona, D. Ramanan, P. Doll´ ar, and C. L. Zitnick, 'Microsoft COCO: Common Objects in Context,' ECCV , 2014. -
  31. [45][45] K. Simonyan and A. Zisserman, 'Very Deep Convolutional Networks for Large-Scale Image Recognition,' CoRR , 2014. -
  32. [46][46] X. Huang and S. Belongie, 'Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization,' ICCV , 2017. -
  33. [48][48] I. Loshchilov and F. Hutter, 'Decoupled Weight Decay Regularization,' International Conference on Learning Representations , 2019. -

Authors
Park, T. H., Sharma, S., & D'Amico, S.
Affiliations
Stanford University (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2019
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.1909.00392
arXiv
1909.00392
OpenAlex
W2971436417
Cited by
65
References
43
Citations by year
2025: 9 · 2024: 12 · 2023: 21 · 2022: 6 · 2021: 11 · 2020: 5 · 2019: 1
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
King Abdulaziz City for Science and Technology
Topics
  • Physical Sciences › Engineering › Aerospace Engineering › Space Satellite Systems and Control
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Engineering › Aerospace Engineering › Spacecraft Dynamics and Control
Keywords
spacecraft pose estimation, convolutional neural networks, perspective-n-point, texture randomization, neural style transfer, relative pose estimation, object detection

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review