Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.

Evaluation of Speech-to-Gesture Generation Using Bi-Directional LSTM Network

Hasegawa, Dai, Kaneko, Naoshi, Shirakawa, Shinichi, Sakuta, Hiroshi, Sumi, Kazuhiko · 2018 · Proceedings of the 18th International Conference on Intelligent Virtual Agents

Abstract

We present a novel framework to automatically generate natural gesture motions accompanying speech from audio utterances. Based on a Bi-Directional LSTM Network, our deep network learns speech-gesture relationships with both backward and forward consistencies over a long period of time. Our network regresses a full 3D skeletal pose of a human from perceptual features extracted from the input audio in each time step. Then, we apply combined temporal filters to smooth out the generated pose sequences. We utilize a speech-gesture dataset recorded with a headset and marker-based motion capture to train our network. We validated our approach with a subjective evaluation and compared it against "original" human gestures and "mismatched" human gestures taken from a different utterance. The evaluation result shows that our generated gestures are significantly better than the "mismatched" gestures with respect to time consistency. The generated gesture also shows marginally significant improvement in terms of semantic consistency when compared to "mismatched" gestures.

Overview

Integrity signals

Registry

Crossref · 10.1145/3267851.3267878

Cites

26 of 26 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 26 references

Research works printed but not linked: 7

  1. 10.5555/2041666.2041683 · doi:10.5555/2041666.2041683
  2. Han Cheolho, 2017, Proceedings of the Linguistic And Cognitive Approaches To Dialog Agents (LaCATODA), Workshop of the 26th International Joint Conference on Artificial Intelligence (IJCAI
  3. Awni Hannun Carl Case Jared Casper Bryan Catanzaro Greg Diamos Erich Elsen Ryan Prenger Sanjeev Satheesh Shubho Sengupta Adam Coates etal 2014. Deep speech: Scaling up end-to-end speech recognition. arXiv preprint arXiv:1412.5567. Awni Hannun Carl Case Jared Casper Bryan Catanzaro Greg Diamos Erich Elsen Ryan Prenger Sanjeev Satheesh Shubho Sengupta Adam Coates et al. 2014. Deep speech: Scaling up end-to-end speech recognition. arXiv preprint arXiv:1412.5567. · arXiv
  4. Ioffe Sergey, 2015, Proceedings of the International Conference on Machine Learning(ICML). 448--456
  5. Kingma Diederik, 2015, Proceedings of the International Conference on Learning Representations (ICLR).
  6. Nair Vinod, 2010, Proceedings of the International Conference on Machine Learning(ICML). 807--814
  7. 10.5555/2627435.2670313 · doi:10.5555/2627435.2670313

Cited by

See all 131

Metadata

Authors
Hasegawa, D., Kaneko, N., Shirakawa, S., Sakuta, H., & Sumi, K.
Affiliations
  • Hokkai Gakuen University (JP)
  • Aoyama Gakuin University (JP)
  • Yokohama National University (JP)
Venue
Proceedings of the 18th International Conference on Intelligent Virtual Agents
Publisher
ACM
Published
2018-11-05
Pages
79-86
OpenAlex
W2901872500
Cited by
131
References
26
Citations by year
2026: 5 · 2025: 17 · 2024: 21 · 2023: 20 · 2022: 23 · 2021: 20 · 2020: 13 · 2019: 12
FWCI
16.89
Type
conference paper
Language
en
Open access
closed
Locations
ACM International Conference on Intelligent Virtual Agents (IVA) (publishedVersion)
Topics
  • Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Human Motion and Animation
  • Physical Sciences › Computer Science › Human-Computer Interaction › Hand Gesture Recognition Systems
  • Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
Keywords
gesture synthesis, bidirectional LSTM, temporal smoothing, marker-based motion capture, time consistency, semantic consistency

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Evaluation of Speech-to-Gesture Generation Using Bi-Directional LSTM Network. https://app.cochat.ai/papers/10.1145/3267851.3267878
The paper
Hasegawa, D., Kaneko, N., Shirakawa, S., Sakuta, H., & Sumi, K. (2018). Evaluation of Speech-to-Gesture Generation Using Bi-Directional LSTM Network. Proceedings of the 18th International Conference on Intelligent Virtual Agents. https://doi.org/10.1145/3267851.3267878

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.