Student or educator? Sign up with your school email and get 6 months of Pro free.
Cited by 169

The DOI matches its Crossref record; 11 of the 11 references it cites were found in a registry or an index.

Synthesis of Compositional Animations from Textual Descriptions

Ghosh, Anindita, Cheema, Noshaba, Oguz, Cennet, Theobalt, Christian, Slusallek, Philipp·2021·2021 IEEE/CVF International Conference on Computer Vision (ICCV)

Abstract

"How can we animate 3D-characters from a movie script or move robots by simply telling them what we would like them to do?" "How unstructured and complex can we make a sentence and still generate plausible movements from it?" These are questions that need to be answered in the long-run, as the field is still in its infancy. Inspired by these problems, we present a new technique for generating compositional actions, which handles complex input sentences. Our output is a 3D pose sequence depicting the actions in the input sentence. We propose a hierarchical two-stream sequential model to explore a finer joint-level mapping between natural language sentences and 3D pose sequences corresponding to the given motion. We learn two manifold representations of the motion, one each for the upper body and the lower body movements. Our model can generate plausible pose sequences for short sentences describing single actions as well as long complex sentences describing multiple sequential and compositional actions. We evaluate our proposed model on the publicly available KIT Motion-Language Dataset containing 3D pose data with human-annotated sentences. Experimental results show that our model advances the state-of-the-art on text-based motion synthesis in objective evaluations by a margin of 50%. Qualitative evaluations based on a user study indicate that our synthesized motions are perceived to be the closest to the ground-truth motion captures for both short and compositional sentences.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/iccv48922.2021.00143

Cites

11 of 11 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 11 references

Research works printed but not linked: 63

  1. 10.3389/fnbot.2016.00005 · DOI
  2. 10.1109/LRA.2018.2852838 · DOI
  3. xu, 2020, Hierarchical stylebased networks for motion synthesis
  4. 10.1609/aaai.v33i01.33018553 · DOI
  5. 10.1109/WACV45572.2020.9093627 · DOI
  6. 10.1145/3072959.3073663 · DOI · Paper page
  7. hermann, 2017, Grounded language learning in a simulated 3d world
  8. henter, 2020, Moglow: Probabilistic and controllable motion synthesis using normalising flows, ACM Transactions on Graphics (TOG), 39, 1, 10.1145/3414685.3417836 · DOI · Paper page
  9. 10.1109/ICRA.2018.8460699 · DOI · Paper page
  10. 10.1111/cgf.14028 · DOI
  11. 10.1145/2820903.2820918 · DOI · Paper page
  12. 10.1145/2897824.2925975 · DOI · Paper page
  13. 10.1016/j.robot.2014.12.008 · doi:10.1016/j.robot.2014.12.008
  14. tang, 2018, Long-term human motion prediction by modeling motion context and enhancing motion dynamic
  15. 10.1145/3125739.3132594 · DOI
  16. tenney, 2019, What do you learn from context? probing for sentence structure in contextualized word representations
  17. hasan, 2019, Forecasting people trajectories and head poses by jointly reasoning on tracklets and vislets, IEEE Transactions on Pattern Analysis and Machine Intelligence
  18. 10.1109/HUMANOIDS.2014.7041470 · doi:10.1109/humanoids.2014.7041470
  19. hanser, 2009, Scenemaker: Intelligent multimodal visualisation of natural language scripts, Irish Conference on Artificial Intelligence and Cognitive Science, 144
  20. villegas, 2017, Learning to generate longterm future via hierarchical prediction, International Conference on Machine Learning, 3560
  21. 10.1109/WACV.2018.00134 · DOI
  22. 10.1109/ICCV.2017.361 · DOI · Paper page
  23. 10.1109/TVCG.2019.2938520 · DOI
  24. 10.1109/VR.2019.8798178 · DOI
  25. 10.1145/3340555.3356090 · doi:10.1145/3340555.3356090
  26. 10.1109/ICRA.2018.8460608 · DOI
  27. 10.1109/ICCV.2015.494 · DOI · Paper page
  28. 10.1109/CVPR.2016.265 · DOI · Paper page
  29. gatys, 2015, Texture Synthesis Using Convolutional Neural Networks
  30. 10.1109/3DV.2017.00059 · DOI · Paper page
  31. habibie, 2017, A recurrent variational autoencoder for human motion synthesis, 28th British Machine Vision Conference
  32. 10.1109/CVPR42600.2020.01443 · DOI · Paper page
  33. pavllo, 2018, Quaternet A quaternion-based recurrent model for human motion
  34. 10.1177/0278364915587923 · DOI
  35. 10.1109/ICRA.2012.6225331 · DOI
  36. 10.1109/IROS.2007.4399344 · doi:10.1109/iros.2007.4399344
  37. socher, 2013, Zeroshot learning through cross-modal transfer
  38. shimada, 2020, Physcap: Physically plausible monocular 3d motion capture in real time, ACM Transactions on Graphics (TOG), 39, 1, 10.1145/3414685.3417877 · DOI · Paper page
  39. 10.1016/j.robot.2018.07.006 · DOI
  40. 10.1089/big.2016.0028 · DOI · Paper page
  41. cheema, 2018, Dilated temporal fully-convolutional network for semantic segmentation of motion capture data
  42. 10.1109/WACV.2019.00156 · DOI
  43. 10.1037/xap0000189 · DOI
  44. cho, 2014, On the properties of neural machine translation Encoder-decoder approaches
  45. 10.18653/v1/W19-4828 · DOI · Paper page
  46. 10.1016/j.ins.2020.08.123 · DOI
  47. 10.1109/CVPR42600.2020.00655 · DOI · Paper page
  48. du, 2019, Stylistic locomotion modeling with conditional variational autoencoder, In Eurographics (Short papers), 9
  49. du, 2015, Hierarchical recurrent neural network for skeleton based action recognition, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1110
  50. 10.1016/j.cag.2020.09.009 · DOI
  51. 10.1109/3DV.2019.00084 · DOI · Paper page
  52. 10.1007/978-3-030-58523-5_15 · DOI
  53. 10.1145/3272127.3275038 · DOI
  54. 10.1109/ICCV.2019.00724 · DOI · Paper page
  55. 10.1109/CVPRW.2018.00191 · DOI · Paper page
  56. mikolov, 2013, Distributed Representations of Words and Phrases and Their Compositionality
  57. 10.1109/CVPR.2017.388 · DOI
  58. 10.1109/WACV45572.2020.9093350 · DOI
  59. 10.1109/TRO.2016.2572685 · DOI
  60. mei, 2016, Listen, attend, and walk: Neural mapping of navigational instructions to action sequences, Proceedings of the AAAI Conference on Artificial Intelligence, 30, 10.1609/aaai.v30i1.10364 · DOI · Paper page
  61. 10.1109/CVPR.2017.497 · DOI · Paper page
  62. lin, 2018, Generating animated videos of human activities from natural language descriptions, Learning, 2018, 2
  63. li, 2017, Auto-conditioned recurrent networks for extended complex human motion synthesis

Cited by

See all 169

Metadata

Authors
Ghosh, A., Cheema, N., Oguz, C., Theobalt, C., & Slusallek, P.
Venue
2021 IEEE/CVF International Conference on Computer Vision (ICCV)
Publisher
IEEE
Published
2021-10
Pages
1376-1386
OpenAlex
W3144253442
Cited by
169
References
11
FWCI
40.71
Type
conference paper
Language
en
Open access
closed

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Synthesis of Compositional Animations from Textual Descriptions. https://app.cochat.ai/papers/10.1109/iccv48922.2021.00143
The paper
Ghosh, A., Cheema, N., Oguz, C., Theobalt, C., & Slusallek, P. (2021). Synthesis of Compositional Animations from Textual Descriptions. 2021 IEEE/CVF International Conference on Computer Vision (ICCV). https://doi.org/10.1109/iccv48922.2021.00143

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.