CoChat

Fast and Accurate Spacecraft Pose Estimation From Single Shot Space Imagery Using Box Reliability and Keypoints Existence Judgments

Open access
Huo, Yurong, Li, Zhi, Zhang, Feng · 2020 · IEEE Access

The DOI matches its Crossref record; 67 of the 67 references it cites were found in a registry or an index.

Abstract

Real-time 6DOF (6 Degree of Freedom) pose estimation of an uncooperative spacecraft is an important part of proximity operations, e.g., space debris removal, spacecraft rendezvous and docking, on-orbit servicing, etc. In this article, a novel efficient deep learning based approach is proposed to estimate the 6DOF pose of uncooperative spacecraft using monocular-vision measurement. Firstly, we introduce a new lightweight YOLO-liked CNN to detect spacecraft and predict 2D locations of the projected keypoints of a prior reconstructed 3D model in real-time. Then, we design two novel models for predicting the bounding box (bbox) reliability scores and the probability of keypoints existence. The two models not only significantly reduce the false positive, but also speed up convergence. Finally, the 6DOF pose is estimated and refined using Perspective-n-Point and geometric optimizer. Results demonstrate that the proposed approach achieves 73.2% average precision and 77.6% average recall for spacecraft detection on the SPEED dataset after only 200 training epochs. For the pose estimation task, the mean rotational error is 0.6812°, and the mean translation error is 0.0320m. The proposed approach achieves competitive pose estimation performance and extreme lightweight (~0.89 million learnable weights in total) on the SPEED dataset while being efficient for real-time applications.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/access.2020.3041415

Cites

67 of 67 references verified0 citation errors · 0 not checkable

Statistics

Figures

FIGURE 1 The architecture of keypoints regression network in our approach.
FIGURE 1 The architecture of keypoints regression network in our approach.CC-BY · From the PDF
FIGURE 2 Six satellite image examples of SPEED dataset with significant variation in view, lighting condition, background, and object size.
FIGURE 2 Six satellite image examples of SPEED dataset with significant variation in view, lighting condition, background, and object size.CC-BY · From the PDF
FIGURE 3 3D reconstruction wireframe with 11 keypoints and 4 examples of ground truth 2D bounding box enclosed 11 2D projected keypoints.
FIGURE 3 3D reconstruction wireframe with 11 keypoints and 4 examples of ground truth 2D bounding box enclosed 11 2D projected keypoints.CC-BY · From the PDF
Image 4
CC-BY · From the PDF
FIGURE 5 The outputs of the tiny-YOLOv3 and the spacecraft detection network of our approach.
FIGURE 5 The outputs of the tiny-YOLOv3 and the spacecraft detection network of our approach.CC-BY · From the PDF
FIGURE 6 The ground truth heatmap and 3 predicted heatmap examples of one keypoint.
FIGURE 6 The ground truth heatmap and 3 predicted heatmap examples of one keypoint.CC-BY · From the PDF
Image 7
CC-BY · From the PDF
FIGURE 8 The distribution of the relative attitude in our test images.
FIGURE 8 The distribution of the relative attitude in our test images.CC-BY · From the PDF
Image 9
CC-BY · From the PDF
Image 10
CC-BY · From the PDF
FIGURE 11 The comparison results of heatmap loss and total loss of KE and NoKE models. ( a ) The heatmap regression loss; ( b ) The total loss.
FIGURE 11 The comparison results of heatmap loss and total loss of KE and NoKE models. ( a ) The heatmap regression loss; ( b ) The total loss.CC-BY · From the PDF
FIGURE 12 The ξ T (left), ξ R (middle) and error dispersion (right) on the test set.
FIGURE 12 The ξ T (left), ξ R (middle) and error dispersion (right) on the test set.CC-BY · From the PDF
FIGURE 13 The mean ξ T and mean ξ R computed against to the mean relative distance. The shaded region shows the 25 and 75 percentile values.
FIGURE 13 The mean ξ T and mean ξ R computed against to the mean relative distance. The shaded region shows the 25 and 75 percentile values.CC-BY · From the PDF
FIGURE 14 The ξ T and ξ R for sorted test images.
FIGURE 14 The ξ T and ξ R for sorted test images.CC-BY · From the PDF
FIGURE 15 Visualization of bounding boxes detection and keypoints regression for SPEED images. The bounding boxes are presented in different colors randomly. (GT denotes ground truth; Pre denotes prediction).
FIGURE 15 Visualization of bounding boxes detection and keypoints regression for SPEED images. The bounding boxes are presented in different colors randomly. (GT denotes ground truth; Pre denotes prediction).CC-BY · From the PDF
FIGURE 16 Visualization of SPEED images with the predicted poses shown as green wireframes and axes.
FIGURE 16 Visualization of SPEED images with the predicted poses shown as green wireframes and axes.CC-BY · From the PDF
Image 17
CC-BY · From the PDF
FIGURE 18 Failed cases in spacecraft detection: ( a ) Poor light condition and small object; ( b ) Earth background and small object. (Green bounding box denotes ground truth; yellow bounding box denotes wrong box.)
FIGURE 18 Failed cases in spacecraft detection: ( a ) Poor light condition and small object; ( b ) Earth background and small object. (Green bounding box denotes ground truth; yellow bounding box denotes wrong box.)CC-BY · From the PDF
Image 19
CC-BY · From the PDF
Image 20
CC-BY · From the PDF
Image 21
CC-BY · From the PDF

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 4

  1. sobel, 1973, A $3\times3$ isotropic gradient operator for image processing, Pattern Classification and Scene Analysis, 271
  2. lourakis, 2005, A brief description of the Levenberg-Marquardt algorithm implemented by Levmar, Inst Comput Sci Found Res Technol, 4, 1
  3. 2019, ESA
  4. joseph, 2018, YOLO Series

Cited by

See all 47

Metadata

Authors
Huo, Y., Li, Z., & Zhang, F.
Affiliations
  • Space Engineering University (CN)
  • Chinese People's Liberation Army (CN)
Venue
IEEE Access
Published
2020
Volume
8
Pages
216283-216297
ISSN
2169-3536 (ISSN-L)
OpenAlex
W3108463056
Cited by
44
References
66
Citations by year
2026: 5 · 2025: 13 · 2024: 11 · 2023: 7 · 2022: 4 · 2021: 4
FWCI
2.25
Type
journal article
Language
en
Open access
Locations
  • IEEE Access (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
National Natural Science Foundation of China (61906213)
Topics
  • Physical Sciences › Engineering › Aerospace Engineering › Space Satellite Systems and Control
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Engineering › Aerospace Engineering › Spacecraft Dynamics and Control
Keywords
spacecraft pose estimation, monocular vision, 6DoF pose estimation, lightweight neural networks, real-time pose estimation, perspective-n-point, geometry optimization, proximity operations

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.