CoChat

Storyfier: Exploring Vocabulary Learning Support with Text Generation Models

Open access
Peng, Zhenhui, Wang, Xingbo, Han, Qiushi, Zhu, Junkai, Ma, Xiaojuan, Qu, Huamin · 2023 · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology
Cited by 27

The DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.

Abstract

Vocabulary learning support tools have widely exploited existing materials, e.g., stories or video clips, as contexts to help users memorize each target word. However, these tools could not provide a coherent context for any target words of learners’ interests, and they seldom help practice word usage. In this paper, we work with teachers and students to iteratively develop Storyfier, which leverages text generation models to enable learners to read a generated story that covers any target words, conduct a story cloze test, and use these words to write a new story with adaptive AI assistance. Our within-subjects study (N=28) shows that learners generally favor the generated stories for connecting target words and writing assistance for easing their learning workload. However, in the read-cloze-write learning sessions, participants using Storyfier perform worse in recalling and using target words than learning with a baseline tool without our AI features. We discuss insights into supporting learning tasks with generative models.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3586183.3606786

Cites

30 of 30 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 63

  1. 10.1145/3491102.3501839 · DOI
  2. Gordon H Bower. 1970. Analysis of a mnemonic device: Modern psychology uncovers the powerful components of an ancient system for improving memory. American Scientist 58, 5 (1970), 496–510.
  3. Patricia L Carrell. 1984. Schema theory and ESL reading: Classroom implications and applications. The modern language journal 68, 4 (1984), 332–343.
  4. 10.1016/j.compedu.2007.06.011 · DOI · Paper page
  5. John Joon Young Chung, Wooseok Kim, Kang Min Yoo, Hwaran Lee, Eytan Adar, and Minsuk Chang. 2022. TaleBrush: Sketching Stories with Generative Pretrained Language Models. In CHI Conference on Human Factors in Computing Systems. 1–19.
  6. 10.1145/3526113.3545672 · DOI · Paper page
  7. 10.1145/3173574.3173885 · DOI · Paper page
  8. Chris Donahue, Mina Lee, and Percy Liang. 2020. Enabling Language Models to Fill in the Blanks. ArXiv abs/2005.05339 (2020). · arXiv
  9. Rita Stafford Dunn and Kenneth J Dunn. 1972. Practical approaches to individualizing instructions: contracts and other effective teaching strategies. Parker Publishing.
  10. Pablo Gervás, Belén Díaz-Agudo, Federico Peinado, and Raquel Hervás. 2004. Story plot generation based on CBR. In International Conference on Innovative Techniques and Applications of Artificial Intelligence. Springer, 33–46.
  11. 10.18653/v1/P17-4008 · DOI · Paper page
  12. 10.4135/9781412983419 · DOI · Paper page
  13. Seraphina Goldfarb-Tarrant, Tuhin Chakrabarty, Ralph Weischedel, and Nanyun Peng. 2020. Content planning for neural story generation with aristotelian rescoring. arXiv preprint arXiv:2009.09870 (2020). · arXiv
  14. Seraphina Goldfarb-Tarrant Haining Feng and Nanyun Peng. 2019. Plan Write and Revise: an Interactive System for Open-Domain Story Generation. In NAACL. · DOI · Paper page
  15. 10.1111/j.1467-1770.1996.tb01355.x · DOI · Paper page
  16. Yongqi Gu and Robert Keith Johnson. 1996. Vocabulary learning strategies and language learning outcomes. Language learning 46, 4 (1996), 643–679.
  17. Brent Harrison, Christopher Purdy, and Mark O Riedl. 2017. Toward automated story generation with markov chain monte carlo methods and deep neural networks. In Thirteenth Artificial Intelligence and Interactive Digital Entertainment Conference.
  18. 10.1145/3108242 · DOI · Paper page
  19. Min-Chai Hsieh and Hao-Chiang Koong Lin. 2006. Interaction design based on augmented reality technologies for English vocabulary learning. In Proceedings of the 18th International Conference on Computers in Education, Vol. 1. 663–666.
  20. 10.1016/j.compedu.2011.08.008 · DOI · Paper page
  21. 10.1145/2556288.2557256 · DOI
  22. Mirella Lapata Regina Barzilay 2005. Automatic evaluation of text coherence: Models and representations. In IJCAI Vol. 5. Citeseer 1085–1090.
  23. Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.08691 (2021). · arXiv
  24. Jiwei Li, Will Monroe, and Dan Jurafsky. 2017. Learning to decode for future success. arXiv preprint arXiv:1701.06549 (2017). · arXiv
  25. 10.1609/aaai.v32i1.12233 · DOI · Paper page
  26. 10.1609/aaai.v32i1.11430 · DOI · Paper page
  27. Paul Nation and Teresa Chung. 2009. Teaching and testing vocabulary. The handbook of language teaching (2009), 543–559.
  28. 10.1145/3397481.3450696 · DOI
  29. 10.1145/3397481.3450696 · DOI
  30. OpenAI. 2023. GPT-4 Technical Report. arxiv:2303.08774 [cs.CL] · arXiv
  31. Rebecca Oxford and David Crookall. 1990. Vocabulary learning: A critical analysis of techniques. TESL Canada Journal (1990), 09–30.
  32. 10.1111/j.1944-9720.1989.tb03139.x · DOI
  33. 10.1016/0346-251X(94)90059-0 · DOI
  34. 10.18653/v1/W18-1505 · DOI · Paper page
  35. 10.1007/978-3-642-10643-9_28 · DOI
  36. 10.3102/00346543052001061 · doi:10.3102/00346543052001061 · Paper page
  37. 10.1145/3231848.3231851 · DOI
  38. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9.
  39. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2019. Exploring the limits of transfer learning with a unified text-to-text transformer. arXiv preprint arXiv:1910.10683 (2019). · arXiv
  40. 10.5555/1946417.1946422 · doi:10.5555/1946417.1946422
  41. Melissa Roemmele, Andrew S Gordon, and Reid Swanson. 2017. Evaluating story generation systems using automated linguistic analyses. In SIGKDD 2017 Workshop on Machine Learning for Creativity. 13–17.
  42. 10.1145/3397481.3450648 · DOI · Paper page
  43. Nathan Sakunkoo and Pattie Sakunkoo. 2013. Gliflix: Using movie subtitles for language learning. In Proceedings of the 26th Symposium on User Interface Software and Technology. ACM.
  44. 10.1186/s41039-016-0028-2 · DOI · Paper page
  45. Norbert Schmitt. 2008. Instructed second language vocabulary learning. Language teaching research 12, 3 (2008), 329–363.
  46. Abigail See, Aneesh Pappu, Rohun Saxena, Akhila Yerukola, and Christopher D Manning. 2019. Do massively pretrained language models make better storytellers?arXiv preprint arXiv:1909.10705 (2019). · arXiv
  47. 10.1145/3459637.3481964 · DOI
  48. 10.1145/3526113.3545617 · DOI · Paper page
  49. 10.1111/j.1540-4781.1988.tb04175.x · DOI
  50. Višnja Pavii Taka. 2008. Vocabulary learning strategies and foreign language acquisition. Multilingual Matters. · DOI · Paper page
  51. Preecha Tangworakitthaworn, Preeyapol Owatsuwan, Nutsima Nongyai, and Nongnapas Arayapong. 2019. An Image-Based Vocabulary Learning System Based on Multi-Agent System. In 2019 16th International Joint Conference on Computer Science and Software Engineering (JCSSE). IEEE, 324–329.
  52. Scott R Turner. 2014. The creative process: A computer model of storytelling and creativity. Psychology Press.
  53. 10.1145/3411764.3445781 · DOI · Paper page
  54. Lu Wang, Munif Ishad Mujib, Jake Williams, George Demiris, and Jina Huh-Yoo. 2021. An Evaluation of Generative Pre-Training Model-based Therapy Chatbot for Caregivers. arXiv preprint arXiv:2107.13115 (2021). · arXiv
  55. 10.1145/3313831.3376726 · DOI
  56. 10.1109/TCIAIG.2013.2277051 · DOI · Paper page
  57. 10.1145/3313831.3376324 · DOI
  58. 10.1145/3555210 · DOI
  59. Keiko Yamamoto, Jesus Rodriguez, and Yoshihiro Tsujino. 2019. FinDo: A Foreign Language Vocabulary Learning System Based on Location-Context. In 2019 20th IEEE/ACIS International Conference on Software Engineering, Artificial Intelligence, Networking and Parallel/Distributed Computing (SNPD). IEEE, 302–307.
  60. 10.1609/aaai.v33i01.33017378 · DOI · Paper page
  61. 10.1109/DMDCM.2011.71 · DOI
  62. 10.1145/3491102.3501914 · DOI · Paper page
  63. 10.1145/3491102.3517479 · DOI · Paper page

Cited by

See all 27

Metadata

Authors
Peng, Z., Wang, X., Han, Q., Zhu, J., Ma, X., & Qu, H.
Venue
Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology
Publisher
ACM
Published
2023-10-29
Pages
1-16
OpenAlex
W4387801070
Cited by
27
References
30
FWCI
29.15
Type
conference paper
Language
en
Open access
bronze
Funders
The Young Scientists Fund of the National Natural Science Foundation of China (62202509)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.