CoChat
Sign Up
PreprintCited by 3Artificial Intelligence in GamesSpeech and dialogue systemsTopic Modeling

VScript: Controllable Script Generation with Visual Presentation

Open access
Ji, Ziwei, Xu, Yan, Cheng, I-Tsun, Cahyawijaya, Samuel, Frieske, Rita, Ishii, Etsuko, Zeng, Min, Madotto, Andrea +1 more·2022·arXiv
DOI 10.48550/arxiv.2203.00314arXiv 2203.00314
Open this paper in CoChat — free accountOpen PDF

Abstract

In order to offer a customized script tool and inspire professional scriptwriters, we present VScript. It is a controllable pipeline that generates complete scripts, including dialogues and scene descriptions, as well as presents visually using video retrieval. With an interactive interface, our system allows users to select genres and input starting words that control the theme and development of the generated script. We adopt a hierarchical structure, which first generates the plot, then the script and its visual presentation. A novel approach is also introduced to plot-guided dialogue generation by treating it as an inverse dialogue summarization. The experiment results show that our approach outperforms the baselines on both automatic and human evaluations, especially in genre control.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2203.00314

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Tolerance towards hallucinations is relatively higher in dialogue systems because desired characteristics include user engagement as well as faithfulness, especially in open-domain dialogue.

    “In contrast, the tolerance is relatively higher in dialogue systems because the desired characteristics are not only faithfulness but also user engagement, especially in open-domain dialogue systems [103, 109].”

    From Survey of Hallucination in Natural Language Generation (2023) · cites this paper as [109]

    Section 3.1.2 describes plot-guided dialogue as natural and relevant to the plot, while Section 5.2.2 evaluates script preference and consistency. Neither addresses tolerance for hallucinations in dialogue systems or a trade-off between faithfulness and user engagement, including in open-domain dialogue. Searches for “hallucination” and “user engagement” found no passage stating that comparison. (LLM, conf=0.74)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Still being checked.

Research works printed but not linked: 26

  1. Nantheera Anantrasirichai and David Bull. 2021. Arti- ficial intelligence in the creative industries: a review. Artificial Intelligence Review, pages 1-68.
  2. Eden Bensaid, Mauro Martino, Benjamin Hoover, and Hendrik Strobelt. 2021. Fairytailor: A multi- modal generative framework for storytelling. arXiv preprint arXiv:2108.04324. · arXiv
  3. Shizhe Chen, Bei Liu, Jianlong Fu, Ruihua Song, Qin Jin, Pingping Lin, Xiaoyu Qi, Chunting Wang, and Jin Zhou. 2019. Neural storyboard artist: Visualiz- ing stories with coherent image sequences. In Pro- ceedings of the 27th ACM International Conference on Multimedia, pages 2236-2244.
  4. Yulong Chen, Yang Liu, Liang Chen, and Yue Zhang. 2021. Dialogsum: A real-life scenario dialogue summarization dataset. In Findings of the Associ- ation for Computational Linguistics: ACL-IJCNLP 2021.
  5. Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, et al. 2019. Plug and play language models: A simple approach to controlled text generation. In ICLR.
  6. Hao-Wen Dong, Wen-Yi Hsiao, Li-Chia Yang, and Yi- Hsuan Yang. 2018. Musegan: Multi-track sequen- tial generative adversarial networks for symbolic music generation and accompaniment. In AAAI.
  7. Angela Fan, Mike Lewis, and Yann Dauphin. 2018. Hi- erarchical neural story generation. In ACL.
  8. Bogdan Gliwa, Iwona Mochol, Maciej Biesek, and Aleksander Wawer. 2019. Samsum corpus: A human-annotated dialogue dataset for abstractive summarization. EMNLP-IJCNLP 2019, page 70.
  9. Jinyi Hu and Maosong Sun. 2020. Generating ma- jor types of chinese classical poetry in a uniformed framework. In Proceedings of the 12th Language Resources and Evaluation Conference.
  10. Oana Ignat, Santiago Castro, Hanwen Miao, Weiji Li, and Rada Mihalcea. 2021. Whyact: Identifying ac- tion reasons in lifestyle vlogs. In EMNLP-IJCNLP.
  11. Nitish Shirish Keskar, Bryan McCann, Lav R Varshney, Caiming Xiong, and Richard Socher. 2019. Ctrl: A conditional transformer language model for control- lable generation. arXiv preprint arXiv:1909.05858. · arXiv
  12. Dmytro Kotovenko, Artsiom Sanakoyeu, Pingchuan Ma, Sabine Lang, and Bjorn Ommer. 2019. A con- tent transformation block for image style transfer. In Proceedings of the IEEE Conference on Com- puter Vision and Pattern Recognition, pages 10032- 10041.
  13. Ben Krause, Akhilesh Deepak Gotmare, Bryan Mc- Cann, Nitish Shirish Keskar, Shafiq Joty, Richard Socher, and Nazneen Fatema Rajani. 2020. Gedi: Generative discriminator guided sequence genera- tion. arXiv preprint arXiv:2009.06367. · arXiv
  14. Holy Lovenia, Bryan Wilie, Romain Barraud, Samuel Cahyawijaya, Willy Chung, and Pascale Fung. 2022. Every picture tells a story: Image-grounded con- trollable stylistic story generation. arXiv preprint arXiv:2209.01638. · arXiv
  15. Andrea Madotto, Etsuko Ishii, Zhaojiang Lin, Sumanth Dathathri, and Pascale Fung. 2020. Plug-and-play conversational models. In Findings of EMNLP 2020.
  16. Lara Martin, Prithviraj Ammanabrolu, Xinyu Wang, William Hancock, Shruti Singh, Brent Harrison, and Mark Riedl. 2018. Event representations for auto- mated story generation with deep neural nets. In AAAI.
  17. Nedjma Ousidhoum, Xinran Zhao, Tianqing Fang, Yangqiu Song, and Dit-Yan Yeung. 2021. Probing toxic content in large pre-trained language models. In ACL-IJCNLP 2021.
  18. Jim Owens and Gerald Millerson. 2012. Video produc- tion handbook. Routledge.
  19. Hannah Rashkin, Eric Michael Smith, Margaret Li, and Y-Lan Boureau. 2019. Towards empathetic open- domain conversation models: A new benchmark and dataset. In ACL.
  20. Marilyn A Walker, Grace I Lin, Jennifer Sawyer, et al. 2012. An annotated corpus of film dialogue for learning and characterizing character style. In LREC.
  21. Sean Welleck, Ilia Kulikov, Stephen Roller, Emily Di- nan, Kyunghyun Cho, et al. 2019. Neural text gener- ation with unlikelihood training. In ICLR.
  22. Hu Xu, Gargi Ghosh, Po-Yao Huang, Dmytro Okhonko, Armen Aghajanyan, Florian Metze, Luke Zettlemoyer, and Christoph Feichtenhofer. 2021. Videoclip: Contrastive pre-training for zero-shot video-text understanding. In EMNLP-IJCNLP.
  23. Lili Yao, Nanyun Peng, Ralph Weischedel, Kevin Knight, Dongyan Zhao, and Rui Yan. 2019. Plan- and-write: Towards better automatic storytelling. In AAAI.
  24. Wenpeng Yin, Jamaal Hay, and Dan Roth. 2019. Benchmarking zero-shot text classification: Datasets, evaluation and entailment approach. In EMNLP-IJCNLP.
  25. Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan. 2020. Dialogpt: Large-scale generative pre-training for conversational response generation. In ACL (demo).
  26. Yutao Zhu, Ruihua Song, Zhicheng Dou, Jian-Yun Nie, and Jin Zhou. 2020. Scriptwriter: Narrative-guided script generation. In ACL 2020, pages 8647-8657.

Cited by

Sorted by citations

Related

Authors
Ji, Z., Xu, Y., Cheng, I.-T., Cahyawijaya, S., Frieske, R., Ishii, E., Zeng, M., Madotto, A., & Fung, P.
Affiliations
Hong Kong University of Science and Technology (HK)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2022
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2203.00314
arXiv
2203.00314
OpenAlex
W4226453333
Cited by
2
References
0
Citations by year
2024: 1 · 2023: 1
Type
posted content
Language
en
Open access
Licence
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords
script generation, dialogue generation, scene description generation, video retrieval, visual presentation, plot generation, interactive user interface

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review