Cited by 207Space Satellite Systems and ControlSpacecraft Dynamics and ControlAdvanced Neural Network Applications

Pose estimation for non-cooperative spacecraft rendezvous using convolutional neural networks

Open access
Sharma, Sumant, Beierle, Connor, D'Amico, Simone·2018·2018 IEEE Aerospace Conference

Abstract

On-board estimation of the pose of an uncooperative target spacecraft is an essential task for future on-orbit servicing and close-proximity formation flying missions. However, two issues hinder reliable on-board monocular vision based pose estimation: robustness to illumination conditions due to a lack of reliable visual features and scarcity of image datasets required for training and benchmarking. To address these two issues, this work details the design and validation of a monocular vision based pose determination architecture for spaceborne applications. The primary contribution to the state-of-the-art of this work is the introduction of a novel pose determination method based on Convolutional Neural Networks (CNN) to provide an initial guess of the pose in real-time on-board. The method involves discretizing the pose space and training the CNN with images corresponding to the resulting pose labels. Since reliable training of the CNN requires massive image datasets and computational resources, the parameters of the CNN must be determined prior to the mission with synthetic imagery. Moreover, reliable training of the CNN requires datasets that appropriately account for noise, color, and illumination characteristics expected in orbit. Therefore, the secondary contribution of this work is the introduction of an image synthesis pipeline, which is tailored to generate high fidelity images of any spacecraft 3D model. In contrast to prior techniques demonstrated for close-range pose determination of spacecraft, the proposed architecture relies on neither hand-engineered image features nor a-priori relative state information. Hence, the proposed technique is scalable to spacecraft of different structural and physical properties as well as robust to the dynamic illumination conditions of space. Through metrics measuring classification and pose accuracy, it is shown that the presented architecture has desirable robustness and scalable properties. Therefore, the proposed technique can be used to augment the current state-of-the-art monocular vision-based pose estimation techniques used in spaceborne applications.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/aero.2018.8396425

Cites

19 of 19 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Pose errors of the CNN in [17] considerably increase when the network is tested with real images.

    “This can be clearly seen in Table 6, in which the pose errors considerably increase when the network is tested with real images.”

    From Review of the robustness and applicability of monocular pose estimation systems for relative navigation with an uncooperative spacecraft (2019) · cites this paper as [17]

    The Experiments section says the CNN was evaluated on imagery imitating actual PRISMA space imagery, not on real images. The Conclusions section explicitly says testing the networks on actual space imagery remains to be done, so the paper cannot report an increase in pose errors on real-image tests. No page number is available in our copy. (LLM, conf=0.89)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured
  • The scheme proposed in [56] returns better pose estimates than the AlexNet scheme while decreasing the size of the training set, and has comparable accuracy in 2D bounding box detection.

    “The scheme proposed in [56] proved to return better pose estimates than the AlexNet scheme while at the same decreasing the size of the training set, as well as a comparable accuracy in the 2D bounding box detection compared to the architecture in [56].”

    From Review of the robustness and applicability of monocular pose estimation systems for relative navigation with an uncooperative spacecraft (2019) · cites this paper as [56]

    The Convolutional Neural Network section says the proposed networks adopt AlexNet’s architecture, rather than outperforming a separate AlexNet scheme. The Conclusions state that training-set size correlates with accuracy and call for larger datasets, contrary to the claimed improvement from a smaller training set. Our copy does not report a comparison of 2D bounding-box detection accuracy. (LLM, conf=0.86)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured
  • The TANGO satellite model has a coarse dimension of 80 × 75 × 32 cm and leads to an operation range of approximately 10 m.

    “The TANGO satellite [145] model used in the multiple datasets [19,20,51,51,85,132] has a coarse dimension of 80 × 75 × 32 cm will lead to the operation range of ∼10 m.”

    From A survey on deep learning-based monocular spacecraft pose estimation: Current state, limitations and prospects (2023) · cites this paper as [85]

    The Synthetic Dataset Creation section (page unavailable) says the paper generated images of the Tango spacecraft, and the dataset description lists camera-to-target distances of 8–13 m, including 10 m. Our copy does not give dimensions of 80 × 75 × 32 cm or say that the satellite’s size leads to an approximately 10 m operating range. (LLM, conf=0.85)

    Experimental — accuracy not yet measured
  • Sharma et al. later proposed doing position estimation based on bounding box detection and orientation estimation based on soft classification.

    “but later they [29] proposed doing position estimation based on bounding box detection and orientation estimation based on soft classification.”

    From Deep Learning for Spacecraft Pose Estimation from Photorealistic Rendering (2020) · cites this paper as [29]

    The Introduction (page unavailable) says the architecture requires a sliding-window approach to locate the spacecraft in the image and uses bearing information to reconstruct its three-dimensional position; it does not describe position estimation based on bounding-box detection. The Type 2 section (page unavailable) describes softmax probabilities for pose labels, supporting classification-based pose estimation, but not a separate soft-classification method specifically for orientation. (LLM, conf=0.86)

    Experimental — accuracy not yet measured
  • In [16], initial conditions for relative state were assumed from separate results of the pose initialization subsystem, without modeling the interface between initial pose estimation and the filter, and no SIL/HIL tests were conducted.

    “Moreover, the initial conditions for the relative state in [16] were assumed from the separate results of the pose initialization subsystem, without modeling the interface between the initial pose estimation and the filter itself, and no SIL/HIL tests were conducted.”

    From Review of the robustness and applicability of monocular pose estimation systems for relative navigation with an uncooperative spacecraft (2019) · cites this paper as [16]

    The Conclusions (page unavailable) describe the CNN as a potential initializer for other pose-determination algorithms, not as an initializer connected to a filter. The Type 1 and Type 2 experimental sections (pages unavailable) report image-dataset tests, consistent with the claim that no SIL/HIL tests were conducted. Our copy does not support the more specific assertion that relative-state initial conditions were assumed from separate pose-initialization results. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 16

  1. [8]8. J. Ventura, "Autonomous Proximity Operations for Noncooperative Space Target," Ph.D. dissertation, Technical University of Munich, 2016.
  2. [9]9. A. Cropp and P. Palmer, "Pose Estimation and Relative Orbit Determination of a Nearby Target Microsatellite using Passive Imagery," 5th Cranfield Conference on Dynamics and Control of Systems and Structures in Space 2002, pp. 389-395, 2002.
  3. [10]10. M. Benn, "Vision Based Navigation Sensors for Space- craft Rendezvous and Docking," Ph.D. dissertation, Danish Technical University, 2010.
  4. [11]11. K. Kanani, A. Petit, E. Marchand, T. Chabot, and B. Gerber, "Vision Based Navigation for Debris Removal Missions," 63rd International Astronautical Congress, pp. 1-8, 2012. [Online]. Available: http: //hal.inria.fr/hal-00750610
  5. [12]12. S. Sharma and S. D'Amico, "Comparative Assessment of Techniques for Initial Pose Estimation using Monoc- ular Vision," Acta Astronautica, pp. 1-17, 2015.
  6. [14]14. S. D'Amico, M. Benn, and J. Jorgensen, "Pose Estima- tion of an Uncooperative Spacecraft," Proceedings of 5th International Conference on Spacecraft Formation Flying Missions and Technologies, no. 1, pp. 1-17, 2013.
  7. [15]15. S. Sharma and J. Ventura, "Robust Model-Based Monocular Pose Estimation for Noncooperative Space- craft Rendezvous," Journal of Spacecraft and Rockets, pp. 1-35, 2017.
  8. [19]19. A. R. Romano, "3D Pose Estimation Using Convo- lutional Neural Networks," Ph.D. dissertation, Escola Tècnica Superior d'Enginyeria Industrial de Barcelona, 2015.
  9. [20]20. H. Su, C. R. Qi, Y. Li, and L. J. Guibas, "Render for CNN: Viewpoint estimation in images using CNNs trained with rendered 3D model views," Proceedings of the IEEE International Conference on Computer Vision, vol. 11-18-Dece, pp. 2686-2694, 2016.
  10. [21]21. M. W. Wright and F. Fallside, "Object Pose Estimation by Neural Network," in International Conference on Image Processing and its Applications. Maastricht, Netherlands: IET, 1992, pp. 602-603.
  11. [27]27. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Ima- geNet Classification with Deep Convolutional Neural Networks," Advances In Neural Information Processing Systems, pp. 1-9, 2012.
  12. [28]28. B. Zhou, A. Lapedriza, J. Xiao, A. Torralba, and A. Oliva, "Learning Deep Features for Scene Recognition using Places Database," Advances in Neural Information Processing Systems 27, pp. 487-495, 2014. [Online]. Available: http://papers.nips.cc/paper/ 5349-learning-deep-features-for-scene-recognition-using-places-database. pdf
  13. [29]29. M. Bowick, A. Cacciuto, D. R. Nelson, and A. Travesset, "Crystalline Order on a Sphere and the Generalized Thomson Problem," Physical Review Letters, vol. 89, no. 24, p. 249902, 2002. [Online]. Available: https://link.aps.org/doi/10.1103/ PhysRevLett.89.249902
  14. [30]30. P. Bodin, R. Noteborn, R. Larsson, T. Karlsson, S. D'Amico, J.-S. Ardaens, M. Delpech, and J. C. Berges, "The prisma formation flying demonstrator: Overview and conclusions from the nominal mission," Advances in the Astronautical Sciences, vol. 144, pp. 441-460, 2012.
  15. [32]32. S. D'Amico, P. Bodin, M. Delpech, and R. Noteborn, "PRISMA," in Distributed Space Missions for Earth System Monitoring, M. D'Errico, Ed., 2013, pp. 599- 637.
  16. [36]36. G. E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R. R. Salakhutdinov, "Improving neural networks by preventing co-adaptation of feature detectors," pp. 1-18, 2012. [Online]. Available: http://arxiv.org/abs/ 1207.0580

Cited by

Sorted by citations

Authors
Sharma, S., Beierle, C., & D'Amico, S.
Affiliations
Stanford University (US)
Venue
2018 IEEE Aerospace Conference
Publisher
IEEE
Published
2018-03
Pages
1-12
ISSN
1095-323X (ISSN-L)
OpenAlex
W2810225457
Cited by
207
References
19
Citations by year
2026: 21 · 2025: 27 · 2024: 39 · 2023: 30 · 2022: 34 · 2021: 25 · 2020: 24 · 2019: 7
FWCI
24.30
Type
conference paper
Language
en
Open access
Locations
  • IEEE Aerospace Conference (AERO) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • Deutsches Zentrum für Luft- und Raumfahrt
  • King Abdulaziz City for Science and Technology
  • Danmarks Tekniske Universitet
Topics
  • Physical Sciences › Engineering › Aerospace Engineering › Space Satellite Systems and Control
  • Physical Sciences › Engineering › Aerospace Engineering › Spacecraft Dynamics and Control
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Keywords
spacecraft pose estimation, monocular vision, convolutional neural networks, on-orbit servicing, synthetic image generation, illumination robustness

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review