CoChat

Deep Learning for Spacecraft Pose Estimation from Photorealistic Rendering

Proenca, Pedro F., Gao, Yang · 2020 · 2020 IEEE International Conference on Robotics and Automation (ICRA)

The DOI matches its Crossref record; 48 of the 61 references it cites were found in a registry or an index; 13 of the 19 sources its claims cite were checked for support.

Abstract

On-orbit proximity operations in space rendezvous, docking and debris removal require precise and robust 6D pose estimation under a wide range of lighting conditions and against highly textured background, i.e., the Earth. This paper investigates leveraging deep learning and photorealistic rendering for monocular pose estimation of known uncooperative spacecraft. We first present a simulator built on Unreal Engine 4, named URSO, to generate labeled images of spacecraft orbiting the Earth, which can be used to train and evaluate neural networks. Secondly, we propose a deep learning framework for pose estimation based on orientation soft classification, which allows modelling orientation ambiguity as a mixture model. This framework was evaluated both on URSO datasets and the European Space Agency pose estimation challenge. In this competition, our best model achieved 3rdplace on the synthetic test set and 2ndplace on the real test set. Moreover, our results show the impact of several architectural and training aspects, and we demonstrate qualitatively how models learned on URSO datasets can perform on real images from space.

Overview

CoChat Summary

TL;DR

The paper presents URSO, a Unreal Engine 4 simulator for photorealistic labeled spacecraft images, and a deep-learning monocular 6D pose estimation framework using orientation soft classification. It reports that soft classification outperforms direct orientation regression and that models trained on URSO with sim-to-real augmentation placed 3rd on the ESA challenge synthetic test set and 2nd on the real test set.

Key findings

  • Orientation soft classification outperformed direct regression on the soyuz_easy dataset: test angular error was 8.0° for classification versus 13.5° for Regress 1, 13.4° for Regress 2, and 20.0° for Regress 3 (Table II).
  • Random camera orientation perturbations improved performance on soyuz_easy: without augmentation, location error was 1.06 m and orientation error 19.5°; with rotation augmentation, 0.56 m and 8.0° (Table V).
  • In the ESA pose estimation challenge, the best model achieved 3rd place on the synthetic test set and 2nd place on the real test set; the best model scored 0.1630 real error and 0.0604 synthetic error, while the triple ensemble scored 0.1555 real and 0.0571 synthetic (Table VII).
  • Multimodal orientation estimation via an EM Gaussian mixture can model orientation ambiguity; on a symmetrical marker dataset, scoring the best of two hypotheses frequently found the right solution (Fig. 5).

Methods

  • URSO, built on Unreal Engine 4, generated labeled RGB and depth images of Soyuz and Dragon spacecraft orbiting Earth; datasets contained 5000 images each, with soyuz_easy at [10–20] m, soyuz_hard at [10–40] m, and a dragon dataset, using 10% held out for testing and 10% for validation. Experiments also used the ESA & Stanford SPEED benchmark: 12000 synthetic training images and 5 real training images, with 2998 synthetic and 300 real test images at 1920 × 1200 px in grayscale.
  • The network used a ResNet backbone with a translation regression branch minimizing relative error and an orientation branch using soft classification over a 3D histogram of Euler-angle bins; an optional EM framework fit a Gaussian mixture for multimodal orientation. Default training used ResNet50, bottleneck width 32, 16 bins per Euler angle, camera rotation perturbations up to 10°, images resized to half size, SGD with momentum 0.9, weight decay 0.0001, batch size 4, and Mask R-CNN COCO backbone weights on one NVIDIA GTX 2080 Ti.
  • The paper reports ablations on orientation soft classification parameters, regression versus classification, architecture depth, image resolution, bottleneck width, and augmentation, using mean absolute location error, mean angular error, and the ESA Error metric.

Limitations

  • Stated: the architecture is not scalable in image and orientation resolution, dense connections are costly, results were obtained with a dedicated network per dataset, and sharing a backbone may be beneficial. The authors also note accuracy comes with around 500M parameters and remains far from the top 2 teams, gross errors appear after 20 m, and they only used half the original image size.
  • Stated: the network does not handle multiple objects per se; future work includes RNN tracking with video sequences and extending URSO to SLAM for targets with unknown geometry.
  • Evident from the text: evaluation is limited to Soyuz, Dragon, and the PRISMA mockup datasets, with only 5 real training images and 300 real test images in SPEED; sim-to-real generalization is demonstrated qualitatively.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1109/icra40945.2020.9197244

Cites

48 of 48 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

8 Valid · 5 Not valid · 2 Not checked · 40 Own results and interpretations

28 more claims

Create a free account to see all 55 claims and to verify the 1 reference still open.

See all claims — free account
  • It may state two separate claims as one. 11
  • Probably not a substantial claim. 13
  • The quote only partly states this claim. 1
  • Borderline: one of the tests for a substantial claim is unclear. 1

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 48 references

Research works printed but not linked: 1

  1. karis, 2013, Real shading in unreal engine 4, In Proc Physically Based Shading Theory Practice

Cited by

See all 229

Metadata

Authors
Proenca, P. F., & Gao, Y.
Affiliations
University of Surrey (GB)
Venue
2020 IEEE International Conference on Robotics and Automation (ICRA)
Publisher
IEEE
Published
2020-05
Pages
6007-6013
ISSN
1050-4729 (ISSN-L)
OpenAlex
W3089381104
Cited by
229
References
61
Citations by year
2026: 25 · 2025: 55 · 2024: 41 · 2023: 43 · 2022: 29 · 2021: 27 · 2020: 9
FWCI
39.76
Type
conference paper
Language
en
Open access
closed
Locations
  • Proceedings - IEEE International Conference on Robotics and Automation/Proceedings (publishedVersion)
  • Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
Topics
  • Physical Sciences › Engineering › Aerospace Engineering › Space Satellite Systems and Control
  • Physical Sciences › Engineering › Aerospace Engineering › Spacecraft Dynamics and Control
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Keywords
spacecraft pose estimation, 6D pose estimation, monocular pose estimation, photorealistic rendering, Unreal Engine 4, synthetic image generation, mixture models, spacecraft rendezvous, space debris removal

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Deep Learning for Spacecraft Pose Estimation from Photorealistic Rendering. https://app.cochat.ai/papers/10.1109/icra40945.2020.9197244
The paper
Proenca, P. F., & Gao, Y. (2020). Deep Learning for Spacecraft Pose Estimation from Photorealistic Rendering. 2020 IEEE International Conference on Robotics and Automation (ICRA). https://doi.org/10.1109/icra40945.2020.9197244

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.