CoChat

Utilization of FPGA for Onboard Inference of Landmark Localization in CNN-Based Spacecraft Pose Estimation

Open access
Cosmas, Kiruki, Kenichi, Asami · 2020 · Aerospace

The DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.

Abstract

In the recent past, research on the utilization of deep learning algorithms for space applications has been widespread. One of the areas where such algorithms are gaining attention is in spacecraft pose estimation, which is a fundamental requirement in many spacecraft rendezvous and navigation operations. Nevertheless, the application of such algorithms in space operations faces unique challenges compared to application in terrestrial operations. In the latter, they are facilitated by powerful computers, servers, and shared resources, such as cloud services. However, these resources are limited in space environment and spacecrafts. Hence, to take advantage of these algorithms, an on-board inferencing that is power- and cost-effective is required. This paper investigates the use of a hybrid Field Programmable Gate Array (FPGA) and Systems-on-Chip (SoC) device for efficient onboard inferencing of the Convolutional Neural Network (CNN) part of such pose estimation methods. In this study, Xilinx’s Zynq UltraScale+ MPSoC device is used and proposed as an effective onboard-inferencing solution. The performance of the onboard and computer inferencing is compared, and the effectiveness of the hybrid FPGA-CPU architecture is verified. The FPGA-based inference has comparable accuracy to the PC-based inference with an average RMS error difference of less than 0.55. Two CNN models that are based on encoder-decoder architecture have been investigated in this study and three approaches demonstrated for landmarks localization.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3390/aerospace7110159

Cites

44 of 44 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 Typical Convolutional Neural Network (CNN)-based Pose Estimation Flow.
Figure 1 Typical Convolutional Neural Network (CNN)-based Pose Estimation Flow.CC-BY · From the PDF
Figure 2 Typical FPGA-based Inference Architecture.
Figure 2 Typical FPGA-based Inference Architecture.CC-BY · From the PDF
Figure 3 Onboard FPGA-based Implementation for CNN-based Pose Estimation.
Figure 3 Onboard FPGA-based Implementation for CNN-based Pose Estimation.CC-BY · From the PDF
Figure 4 ResNet50 Architecture for Direct Regression.
Figure 4 ResNet50 Architecture for Direct Regression.CC-BY · From the PDF
Figure 5 Direct Regression Performance after 40 Epochs Training.
Figure 5 Direct Regression Performance after 40 Epochs Training.CC-BY · From the PDF
Figure 6 U-Net Architecture for Heatmap-based Approach.
Figure 6 U-Net Architecture for Heatmap-based Approach.CC-BY · From the PDF
Figure 7 U-Net Model Keypoint Detection Performance on Full Images.
Figure 7 U-Net Model Keypoint Detection Performance on Full Images.CC-BY · From the PDF
Figure 8 Detection, Cropping and Keypoint Localization Flow.
Figure 8 Detection, Cropping and Keypoint Localization Flow.CC-BY · From the PDF
Figure 9 Spacecraft Detection and Bounding with YOLOv3.
Figure 9 Spacecraft Detection and Bounding with YOLOv3.CC-BY · From the PDF
Figure 10 U-Net Model Keypoint Detection Performance on Cropped Images.
Figure 10 U-Net Model Keypoint Detection Performance on Cropped Images.CC-BY · From the PDF
Figure 11 ResNet34-U-Net Architecture.
Figure 11 ResNet34-U-Net Architecture.CC-BY · From the PDF
Figure 12 Performance of ResNet34-U-Net on Keypoint Localization.
Figure 12 Performance of ResNet34-U-Net on Keypoint Localization.CC-BY · From the PDF
Figure 13 FPGA Inference Flow.
Figure 13 FPGA Inference Flow.CC-BY · From the PDF
Figure 14 Hardware Block Design in Vivado.
Figure 14 Hardware Block Design in Vivado.CC-BY · From the PDF
Figure 15 Hardware and Software Implementation Flow for MPSoC Inference.
Figure 15 Hardware and Software Implementation Flow for MPSoC Inference.CC-BY · From the PDF
Figure 16 Deployment Model (from quantization stage) and Compiled Network Summary by DNNC.
Figure 16 Deployment Model (from quantization stage) and Compiled Network Summary by DNNC.CC-BY · From the PDF
Image 17
CC-BY · From the PDF
Figure 18 Power Consumption of Ultra96v2 Evaluation Board Outside and During Inference.
Figure 18 Power Consumption of Ultra96v2 Evaluation Board Outside and During Inference.CC-BY · From the PDF

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 44 references

Research works printed but not linked: 4

  1. Diebel, 2006, Representing attitude: Euler angles, unit quaternions, and rotation vectors, Matrix, 58, 1
  2. Hirano, D., Kato, H., and Saito, T. (2018, January 4–6). Deep Learning based Pose Estimation in Space. Proceedings of the International Symposium on Artificial Intelligence, Robotics and Automation in Space (i-SAIRAS), Madrid, Spain.
  3. Sharma, S. (2019). Pose Estimation of Uncooperative Spacecraft Using Monocular Vision and Deep Learning. [Ph.D. Thesis, Stanford University].
  4. Sonawani, S., Alimo, R., Detry, R., Jeong, D., Hess, A., and Amor, H.B. (2020). Assistive Relative Pose Estimation for On-orbit Assembly using Convolutional Neural Networks. arXiv.

Cited by

See all 55

Metadata

Authors
Cosmas, K., & Kenichi, A.
Affiliations
  • Kyushu Institute of Technology (JP)
  • Embedded Systems Laboratory, Electrical and Space Engineering, Kyushu Institute of Technology, 1-1 Sensui, Tobata, Kitakyushu, Fukuoka 804-8550, Japan
Corresponding author
Cosmas, Kiruki
Venue
Aerospace
Publisher
MDPI AG
Published
2020-11-05
Volume
7
Issue
11
Pages
159
ISSN
2226-4310 (ISSN-L)
OpenAlex
W3097559709
Cited by
51
References
40
Citations by year
2026: 7 · 2025: 11 · 2024: 10 · 2023: 10 · 2022: 3 · 2021: 10
FWCI
2.24
Type
journal article
Language
en
Open access
Locations
  • Aerospace (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Hardware and Architecture › Embedded Systems Design Techniques
  • Physical Sciences › Engineering › Electrical and Electronic Engineering › Embedded Systems and FPGA Applications
Keywords
FPGA, spacecraft pose estimation, landmark detection, convolutional neural networks, Zynq UltraScale+ MPSoC, encoder-decoder architecture, spacecraft rendezvous, spacecraft navigation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Utilization of FPGA for Onboard Inference of Landmark Localization in CNN-Based Spacecraft Pose Estimation. https://app.cochat.ai/papers/10.3390/aerospace7110159
The paper
Cosmas, K., & Kenichi, A. (2020). Utilization of FPGA for Onboard Inference of Landmark Localization in CNN-Based Spacecraft Pose Estimation. Aerospace. https://doi.org/10.3390/aerospace7110159

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.