Learning with 3D rotations, a hitchhiker's guide to SO(3)
Open accessAbstract
Many settings in machine learning require the selection of a rotation representation. However, choosing a suitable representation from the many available options is challenging. This paper acts as a survey and guide through rotation representations. We walk through their properties that harm or benefit deep learning with gradient-based optimization. By consolidating insights from rotation-based learning, we provide a comprehensive overview of learning functions with rotation representations. We provide guidance on selecting representations based on whether rotations are in the model's input or output and whether the data primarily comprises small angles.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Euler angles and quaternions contain discontinuities that hinder rotation estimation with deep learning.
“While Euler angles and quaternions are common choices, both contain discontinuities that hinder rotation estimation with deep learning [32,33].”
From Rapid patient-specific neural networks for intraoperative X-ray to volume registration (2025) · cites this paper as [33]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Still being checked.
Research works printed but not linked: 22
- Alvarez-Tunon, O., Brodskiy, Y., and Kayacan, E. Loss it right: Euclidean and riemannian metrics in learning- based visual odometry. In ISR Europe; 56th International Symposium on Robotics, pp. 107-111, 2023.
- Brégier, R. Deep regression on manifolds: a 3d rotation case study. In 2021 International Conference on 3D Vision (3DV), pp. 166-174. IEEE, 2021.
- Brunton, S. L., Budišić, M., Kaiser, E., and Kutz, J. N. Modern koopman theory for dynamical systems. SIAM Review, 64(2):229-340, 2022.
- Chen, J., Yin, Y., Birdal, T., Chen, B., Guibas, L. J., and Wang, H. Projective manifold gradient layer for deep rotation regression. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6646-6655, 2022.
- Dinh, L., Pascanu, R., Bengio, S., and Bengio, Y. Sharp min- ima can generalize for deep nets. In International Con- ference on Machine Learning, pp. 1019-1028. PMLR, 2017.
- Dufter, P., Schmitt, M., and Schütze, H. Position infor- mation in transformers: an overview. Computational Linguistics, 48(3):733-763, 09 2022.
- Euler, L. Du mouvement de rotation des corps solides autour d'un axe variable. Mémoires de l'académie des sciences de Berlin, pp. 154-193, 1765.
- Gilmer, J., Ghorbani, B., Garg, A., Kudugunta, S., Neyshabur, B., Cardoze, D., Dahl, G. E., Nado, Z., and Firat, O. A loss curvature perspective on training instabili- ties of deep learning models. In International Conference on Learning Representations, 2022.
- Grassia, F. S. Practical parameterization of rotations using the exponential map. Journal of graphics tools, 3(3): 29-48, 1998.
- Hartley, R., Trumpf, J., Dai, Y., and Li, H. Rotation av- eraging. International journal of computer vision, 103: 267-305, 2013.
- Huynh, D. Q. Metrics for 3d rotations: Comparison and analysis. Journal of Mathematical Imaging and Vision, 35:155-164, 2009.
- Levinson, J., Esteves, C., Chen, K., Snavely, N., Kanazawa, A., Rostamizadeh, A., and Makadia, A. An analysis of svd for deep rotation estimation. Advances in Neural Information Processing Systems, 33:22554-22565, 2020.
- Llanas, B., Lantarón, S., and Sáinz, F. J. Constructive approximation of discontinuous functions by neural net- works. Neural Processing Letters, 27:209-226, 2008.
- Lyle, C., Zheng, Z., Nikishin, E., Pires, B. A., Pascanu, R., and Dabney, W. Understanding plasticity in neural net- works. In International Conference on Machine Learning, pp. 23190-23211. PMLR, 2023.
- Peretroukhin, V., Giamou, M., Rosen, D. M., Greene, W. N., Roy, N., and Kelly, J. A smooth representation of SO(3) for deep rotation learning with uncertainty. In Proceed- ings of Robotics: Science and Systems, Jul. 12-16 2020.
- Saxena, A., Driemeyer, J., and Ng, A. Y. Learning 3d object orientation from images. In 2009 IEEE International conference on robotics and automation, pp. 794-800. IEEE, 2009.
- Stuelpnagel, J. On the parametrization of the three- dimensional rotation group. SIAM review, 6(4):422-430, 1964.
- Terzakis, G., Lourakis, M., and Ait-Boudaoud, D. Modified rodrigues parameters: an efficient representation of orien- tation in 3d vision and graphics. Journal of Mathematical Imaging and Vision, 60:422-442, 2018.
- Todorov, E., Erez, T., and Tassa, Y. Mujoco: A physics engine for model-based control. In IEEE/RSJ Interna- tional Conference on Intelligent Robots and Systems, pp. 5026-5033, 2012.
- Xu, Z.-B. and Cao, F.-L. Simultaneous lp-approximation order for neural networks. Neural Networks, 18(7):914- 923, 2005.
- Zhang, H., Gao, X., Unterman, J., and Arodz, T. Approxi- mation capabilities of neural odes and invertible residual networks. In International Conference on Machine Learn- ing, pp. 11086-11095. PMLR, 2020.
- Zhou, Y., Barnes, C., Lu, J., Yang, J., and Li, H. On the con- tinuity of rotation representations in neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5745-5753, 2019.
Cited by
Sorted by citations
Demonstration of the Dyna Reinforcement Learning Framework for Reactive Close Proximity Operations
Majumdar, R., Sternberg, D. C., Albee, K., & Jia-Richards, O. (2025) · AIAA SciTech Forum · doi:10.2514/6.2025-1002 · Cited by 5 · see citing works
Liconti, D., Toshimitsu, Y., & Katzschmann, R. K. (2024) · IEEE-RAS International Conference on Humanoid Robots (Humanoids) · doi:10.1109/humanoids58906.2024.10769883 · Cited by 4 · see citing works
Multi-Modal Gesture Recognition from Video and Surgical Tool Pose Information via Motion Invariants
Atoum, J. K., Johnston, G. L. H., Simaan, N., & Wu, J. Y. (2025) · International Symposium on Medical Robotics · doi:10.1109/ismr67322.2025.11025959 · Cited by 3 · see citing works
Rapid patient-specific neural networks for intraoperative X-ray to volume registration
Gopalakrishnan, V., Chlorogiannis, D.-D., Abumoussa, A., Larson, A. M., Haouchine, N., Orbach, D. B., Frisken, S., Dey, N., & Golland, P. (2025) · arXiv · doi:10.48550/arxiv.2503.16309 · Cited by 2 · see citing works
- Authors
- Geist, A. R., Frey, J., Zhobro, M., Levina, A., & Martius, G.
- Affiliations
- Max Planck Institute for Intelligent Systems (DE)
- ETH Zurich (CH)
- University of Tübingen (DE)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2024
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2404.11735
- OpenAlex
- W4394991821
- Cited by
- 3
- References
- 0
- Citations by year
- 2025: 2 · 2024: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Funders
- Bundesministerium für Bildung und Forschung (01IS18039B, FKZ: 01IS18039B)
- HORIZON EUROPE European Research Council (101045454)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
- Physical Sciences › Engineering › Computational Mechanics › 3D Shape Modeling and Analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- rotation representation, SO(3), 3D rotation, gradient-based optimization, rotation group, representation selection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
