Deep Learning for Spacecraft Pose Estimation from Photorealistic Rendering
The DOI matches its Crossref record; 48 of the 61 references it cites were found in a registry or an index; 13 of the 19 sources its claims cite were checked for support.
Abstract
On-orbit proximity operations in space rendezvous, docking and debris removal require precise and robust 6D pose estimation under a wide range of lighting conditions and against highly textured background, i.e., the Earth. This paper investigates leveraging deep learning and photorealistic rendering for monocular pose estimation of known uncooperative spacecraft. We first present a simulator built on Unreal Engine 4, named URSO, to generate labeled images of spacecraft orbiting the Earth, which can be used to train and evaluate neural networks. Secondly, we propose a deep learning framework for pose estimation based on orientation soft classification, which allows modelling orientation ambiguity as a mixture model. This framework was evaluated both on URSO datasets and the European Space Agency pose estimation challenge. In this competition, our best model achieved 3rdplace on the synthetic test set and 2ndplace on the real test set. Moreover, our results show the impact of several architectural and training aspects, and we demonstrate qualitatively how models learned on URSO datasets can perform on real images from space.
Overview
CoChat Summary
TL;DR
The paper presents URSO, a Unreal Engine 4 simulator for photorealistic labeled spacecraft images, and a deep-learning monocular 6D pose estimation framework using orientation soft classification. It reports that soft classification outperforms direct orientation regression and that models trained on URSO with sim-to-real augmentation placed 3rd on the ESA challenge synthetic test set and 2nd on the real test set.
Key findings
- Orientation soft classification outperformed direct regression on the soyuz_easy dataset: test angular error was 8.0° for classification versus 13.5° for Regress 1, 13.4° for Regress 2, and 20.0° for Regress 3 (Table II).
- Random camera orientation perturbations improved performance on soyuz_easy: without augmentation, location error was 1.06 m and orientation error 19.5°; with rotation augmentation, 0.56 m and 8.0° (Table V).
- In the ESA pose estimation challenge, the best model achieved 3rd place on the synthetic test set and 2nd place on the real test set; the best model scored 0.1630 real error and 0.0604 synthetic error, while the triple ensemble scored 0.1555 real and 0.0571 synthetic (Table VII).
- Multimodal orientation estimation via an EM Gaussian mixture can model orientation ambiguity; on a symmetrical marker dataset, scoring the best of two hypotheses frequently found the right solution (Fig. 5).
Methods
- URSO, built on Unreal Engine 4, generated labeled RGB and depth images of Soyuz and Dragon spacecraft orbiting Earth; datasets contained 5000 images each, with soyuz_easy at [10–20] m, soyuz_hard at [10–40] m, and a dragon dataset, using 10% held out for testing and 10% for validation. Experiments also used the ESA & Stanford SPEED benchmark: 12000 synthetic training images and 5 real training images, with 2998 synthetic and 300 real test images at 1920 × 1200 px in grayscale.
- The network used a ResNet backbone with a translation regression branch minimizing relative error and an orientation branch using soft classification over a 3D histogram of Euler-angle bins; an optional EM framework fit a Gaussian mixture for multimodal orientation. Default training used ResNet50, bottleneck width 32, 16 bins per Euler angle, camera rotation perturbations up to 10°, images resized to half size, SGD with momentum 0.9, weight decay 0.0001, batch size 4, and Mask R-CNN COCO backbone weights on one NVIDIA GTX 2080 Ti.
- The paper reports ablations on orientation soft classification parameters, regression versus classification, architecture depth, image resolution, bottleneck width, and augmentation, using mean absolute location error, mean angular error, and the ESA Error metric.
Limitations
- Stated: the architecture is not scalable in image and orientation resolution, dense connections are costly, results were obtained with a dedicated network per dataset, and sharing a backbone may be beneficial. The authors also note accuracy comes with around 500M parameters and remains far from the top 2 teams, gross errors appear after 20 m, and they only used half the original image size.
- Stated: the network does not handle multiple objects per se; future work includes RNN tracking with video sequences and extending URSO to SLAM for targets with unknown geometry.
- Evident from the text: evaluation is limited to Soyuz, Dragon, and the PRISMA mockup datasets, with only 5 real training images and 300 real test images in SPEED; sim-to-real generalization is demonstrated qualitatively.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
8 Valid · 5 Not valid · 2 Not checked · 40 Own results and interpretations
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“While several works [13, 16, 24] have used L 2 or L 1 loss to regress orientation. This does not represent correctly the actual angular distance for any orientation representation.”
What the sources say
- Not checked — no full text of the sourcecites [13] [13] A. Kendall, M. Grimes, and R. Cipolla, 'Posenet: A convolutional network for real-time 6-dof camera relocalization,' in IEEE International Conference on Computer Vision (ICCV) , pp. 2938-2946, 2015. - source
- Could not verifycites [16] [16] T.-T. Do, M. Cai, T. Pham, and I. Reid, 'Deep-6dpose: recovering 6d object pose from a single rgb image,' arXiv preprint arXiv:1802.10367 , 2018. - sourceExperimental — accuracy not yet measured
Section D, “3D Rotation Regression” (page unavailable), says a squared-distance loss can penalize orientations that are equivalent for symmetric objects. It does not assess whether L1 or L2 loss correctly represents angular distance across orientation representations; searches for those terms found no passage making that claim. (LLM, conf=0.72)
- Could not verifycites [24] [24] S. Wang, R. Clark, H. Wen, and N. Trigoni, 'Deepvo: Towards end-to-end visual odometry with deep recurrent convolutional neural networks,' in International Conference on Robotics and Automation (ICRA) , pp. 2043-2050, IEEE, 2017. - sourceExperimental — accuracy not yet measured
Our copy of 'Going Further with Point Pair Features' is a geometric Hough-voting paper: it discusses PPF discretization (Sec. 4.3, p. 7-8), smart sampling (Sec. 4.2, p. 7) and pose generation/postprocessing (Sec. 4.4, pp. 8-9), and its Related Work (p. 2) only mentions forest-based regression of object coordinates in passing. No passage addresses L2 or L1 loss functions for orientation regression, nor any claim about angular distance being misrepresented by such losses (searched: 'L2 loss to regress orientation', 'angular distance between rotations quaternion Euler angle', 'learning based regression of pose with a loss function'). The absence is not contrary evidence, so the attribution cannot be checked against this source. (LLM, conf=0.70)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“On-orbit proximity operations in space rendezvous, docking and debris removal require precise and robust 6D pose estimation under a wide range of lighting conditions and against highly textured background, i.e., the Earth.”
- Also stated in: § I. INTRODUCTION · p. 1 · plain
“These rely on precise and robust estimation of the relative pose and trajectory of object targets in close-proximity under harsh lighting conditions and against highly textured background (i.e. Earth).”
What the sources say
- Partly supportedcites [1] [1] S. Nanjangud, P. Blacker, S. Bandyopadhyay, and Y. Gao, 'Robotics and ai-enabled on-orbit operations with future generation of small satellites,' Proceedings of the IEEE , 2018. - sourceExperimental — accuracy not yet measured
The source's 'A. Sensing and Perception' section supports most of the claim: it states that sensing and perception is the first essential component for proximity operations by a chaser, that it is necessary to estimate the pose of the target (with 3-D model reconstruction for unfamiliar targets like debris), and it describes the illumination difficulty of space — images with a wide dynamic range from harsh illumination, and LiDAR being preferred for robustness to ambient lighting variations. However, no passage in our copy addresses the 'highly textured background, i.e., the Earth' element of the claim; searches for background clutter/Earth texture as a perception challenge returned nothing on that topic, so that qualifier is not attributable to this source. Our copy also does not use the '6D' formulation explicitly, though it does refer to determining 'position and orientations'. (LLM, conf=0.72)
- Not matchedcites [2] [2] B. Taylor, G. Aglietti, S. Fellowes, S. Ainley, T. Salmon, I. Retat, C. Burgess, A. Hall, T. Chabot, K. Kanani, et al. , 'Remove debris mission, from concept to orbit,' in 32nd Annual AIAA/USU Conference on Small Satellites , 2018. -
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § I. INTRODUCTION · p. 1 · plain
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“The latter contains both synthetic and real images with 1920 × 1200 px, generated in [29], of a mockup model of one satellite used in a flight mission, named PRISMA [44].”
What the sources say
- Supportedcites [29] [29] S. Sharma and S. D'Amico, 'Pose estimation for non-cooperative rendezvous using neural networks,' in AAS/AIAA Astrodynamics Specialist Conference , 2019. - source
The Synthetic Dataset Creation section identifies Tango as a spacecraft used in the PRISMA mission and says the paper generated synthetic images of it. The citation to this source follows the PRISMA identification; the citing sentence attributes image generation to a different reference. (LLM, conf=0.90)
- Not checked — no full text of the sourcecites [44] [44] S. D'Amico, J.-S. Ardaens, and R. Larsson, 'Spaceborne autonomous formation-flying experiment on the prisma mission,' Journal of Guidance, Control, and Dynamics , vol. 35, no. 3, pp. 834-850, 2012. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Quaternions, for example, are non-injective. While one can map quaternions to lie only on one hemisphere as in [38], L 2 distances to quaternions near the equator will still not express the geodesic distance.”
What the sources say
- Supportedcites [38] [38] A. Kendall and R. Cipolla, 'Geometric loss functions for camera pose regression with deep learning,' in IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pp. 5974-5983, 2017. - source
Section 2.1 (page not provided) defines quaternion geodesic distance using the absolute value of the quaternion inner product. That formula supports the claim: quaternions near the hemisphere boundary can have a small geodesic distance even when their L2 distance is large. Section 2.2 uses this geodesic distance as the quaternion loss rather than standard mean-squared error. (LLM, conf=0.86)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we present a probabilistic orientation estimation via soft classification that performs significantly better than direct orientation regression”
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
28 more claims
Create a free account to see all 55 claims and to verify the 1 reference still open.
- It may state two separate claims as one. 11
- Probably not a substantial claim. 13
- The quote only partly states this claim. 1
- Borderline: one of the tests for a substantial claim is unclear. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The post-processing procedure for lightbox images loosely follows that of the synthetic imagery of SPEED detailed in [3, 39].”
From SPEED+: Next-Generation Dataset for Spacecraft Pose Estimation across Domain Gap (2022) · cites this paper as [39]
Experimental — accuracy not yet measured
Citations
Cites
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137
Least-Squares Fitting of Two 3-D Point Sets
Arun, K. S., Huang, T. S., & Blostein, S. D. (1987) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.1987.4767965 · Cited by 3,952 · cited 1×
“To avoid having to fine-tune loss weights, we have also experimented instead in Section VI regressing three virtual 3D keypoints and then estimate pose using a closed-form solution [37].”
MethodCARLA: An open urban driving simulator
Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., & Koltun, V. (2017) · arXiv · doi:10.48550/arxiv.1711.03938 · Cited by 3,718 · cited 2×
“As for URSO, we collected one dataset for the dragon spacecraft and two datasets for the soyuz model with different operating ranges: soyuz_easy with [10-20] m and soyuz_hard with [10-40]”
BackgroundMethodPoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization
Kendall, A., Grimes, M., & Cipolla, R. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.336 · Cited by 2,489 · cited 2×
“As for URSO, we collected one dataset for the dragon spacecraft and two datasets for the soyuz model with different operating ranges: soyuz_easy with [10-20] m and soyuz_hard with [10-40]”
MethodAirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehicles
Shah, S., Dey, D., Lovett, C., & Kapoor, A. (2018) · Springer Proceedings in Advanced Robotics · doi:10.1007/978-3-319-67361-5_40 · Cited by 2,254 · cited 1×
“Nevertheless, state-of-the-art game engines (e.g. UE4), widely used in autonomous driving [32] and robotics [33, 34] offer far more resources to develop complex and photorealistic environments, but these have been criticized in [31] for being designed for human vision and lacking the photometric…”
BackgroundPoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes
Xiang, Y., Schmidt, T., Narayanan, V., & Fox, D. (2018) · Robotics: Science and Systems XIV · doi:10.15607/rss.2018.xiv.019 · Cited by 2,139 · cited 3×
“For lower space complexity, one could use instead ROIs as in [14, 16, 37], but this complicates our end-to-end pose estimation.”
MethodSSD-6D: Making RGB-Based 3D Detection and 6D Pose Estimation Great Again
Kehl, W., Manhardt, F., Tombari, F., Ilic, S., & Navab, N. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.169 · Cited by 1,047 · cited 1×
“As for URSO, we collected one dataset for the dragon spacecraft and two datasets for the soyuz model with different operating ranges: soyuz_easy with [10-20] m and soyuz_hard with [10-40]”
MethodDeepvo: towards end-to-end visual odometry with deep recurrent convolutional neural networks
Wang, S., Clark, R., Wen, H., & Trigoni, N. (2017) · 2017 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2017.7989236 · Cited by 970 · cited 4×
“DL has also been successfully applied to visual odometry [24, 25].”
BackgroundMethodInfluentialAn Analysis of Deep Neural Network Models for Practical Applications
Canziani, A., Paszke, A., & Culurciello, E. (2016) · arXiv · doi:10.48550/arxiv.1605.07678 · Cited by 969
Real-Time Seamless Single Shot 6D Object Pose Prediction
Tekin, B., Sinha, S. N., & Fua, P. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00038 · Cited by 900 · cited 1×
“As for URSO, we collected one dataset for the dragon spacecraft and two datasets for the soyuz model with different operating ranges: soyuz_easy with [10-20] m and soyuz_hard with [10-40]”
Method
Research works printed but not linked: 1
- karis, 2013, Real shading in unreal engine 4, In Proc Physically Based Shading Theory Practice
Cited by
Spacecraft pose estimation dataset (speed)
Sharma, S., Park, T. H., & D'Amico, S. (2026) · Stanford Digital Repository · doi:10.25740/dz692fn7184 · Cited by 304
Satellite Pose Estimation Challenge: Dataset, Competition Design, and Results
Kisantal, M., Sharma, S., Park, T. H., Izzo, D., Martens, M., & D'Amico, S. (2020) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2020.2989063 · Cited by 304
“pedro_fairspace [44] submitted the best submission that did not rely on PnP solvers, finishing on the third place.”
SPEED+: Next-Generation Dataset for Spacecraft Pose Estimation across Domain Gap
Park, T. H., Martens, M., Lecuyer, G., Izzo, D., & D'Amico, S. (2022) · 2022 IEEE Aerospace Conference (AERO) · doi:10.1109/aero53065.2022.9843439 · Cited by 170
“Moreover, these images mostly lack accurate pose labels, which has forced the previous evaluations to be strictly qualitative [8] or based on hand-labeled pose annotations [10].”
Robust multi-task learning and online refinement for spacecraft pose estimation across domain gap
Park, T. H., & D’Amico, S. (2024) · Advances in Space Research · doi:10.1016/j.asr.2023.03.036 · Cited by 125
“Afterwards, many top-performing entries of SPEC2019 have shown diverse CNN architectures and pose estimation strategies, such as prob-ablistic orientation estimation via soft classification (Proenc¸a & Gao, 2020) or estimation of a set of designated keypoints on the spacecraft surface via direction…”
BackgroundPauly, L., Rharbaoui, W., Shneider, C., Rathinam, A., Gaudillière, V., & Aouada, D. (2023) · Acta Astronautica · doi:10.1016/j.actaastro.2023.08.001 · Cited by 121
Metadata
- Authors
- Proenca, P. F., & Gao, Y.
- Affiliations
- University of Surrey (GB)
- ORCID
- Gao, Yang
- Venue
- 2020 IEEE International Conference on Robotics and Automation (ICRA)
- Publisher
- IEEE
- Published
- 2020-05
- Pages
- 6007-6013
- ISSN
- 1050-4729 (ISSN-L)
- OpenAlex
- W3089381104
- Cited by
- 229
- References
- 61
- Citations by year
- 2026: 25 · 2025: 55 · 2024: 41 · 2023: 43 · 2022: 29 · 2021: 27 · 2020: 9
- FWCI
- 39.76
- Type
- conference paper
- Language
- en
- Open access
- closed
- Licence
- Locations
- Proceedings - IEEE International Conference on Robotics and Automation/Proceedings (publishedVersion)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- Topics
- Physical Sciences › Engineering › Aerospace Engineering › Space Satellite Systems and Control
- Physical Sciences › Engineering › Aerospace Engineering › Spacecraft Dynamics and Control
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- spacecraft pose estimation, 6D pose estimation, monocular pose estimation, photorealistic rendering, Unreal Engine 4, synthetic image generation, mixture models, spacecraft rendezvous, space debris removal
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Deep Learning for Spacecraft Pose Estimation from Photorealistic Rendering. https://app.cochat.ai/papers/10.1109/icra40945.2020.9197244
- The paper
- Proenca, P. F., & Gao, Y. (2020). Deep Learning for Spacecraft Pose Estimation from Photorealistic Rendering. 2020 IEEE International Conference on Robotics and Automation (ICRA). https://doi.org/10.1109/icra40945.2020.9197244
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.